Jill Lepore (2020) If Then: How One Data Company Invented the Future. John Murray. ISBN: 978-1-529-38617-2 (2021 pbk edition). [Link to book]
This is a most frustrating book. The company referred to in the subtitle is the Simulmatics Corporation, which collected and analysed data on public attitudes for politicians, retailers and the US Department of Defence between 1959 and 1970. Lepore says it carried out “simulation”, but is never very clear about what “simulation” meant to the founders of Simulmatics, what algorithms were involved, or how these algorithms used data. The history of Simulmatics is narrated along with that of US politics and the Vietnam War during its period of operation; the company worked for John Kennedy’s presidential campaign in 1960, although the campaign was shy about admitting this. There is much of interest in this historical context, but the book is marred by the apparent limitations of Lepore’s technical knowledge, her prejudices against the social and behavioural sciences (and in particular the use of computers within them), and irritating “tics” such as the frequent repetition of “If/Then”. There are copious notes, and an index, but no bibliography.
Lepore insists that human behaviour is not predictable, whereas both everyday observation and the academic study of human sciences and history show that on both individual and collective levels it is partially predictable – if it were not, social life would be impossible – and partially unpredictable; she also claims that there is a general repudiation of the importance of history among social and behavioural scientists and in “Silicon Valley”, and seems unaware that many historians and other humanities researchers use mathematics and even computers in their work.
Information about Simulmatics’ uses of computers is in fact available from contemporary documents which its researchers published. In the case of Kennedy’s presidential campaign (de Sola Pool and Abelson 1961, de Sola Pool 1963), the “simulation” involved was the construction of synthetic populations in order to amalgamate polling data from past (1952, 1954, 1956, 1958) American election campaigns. Americans were divided into 480 demographically defined “voter types” (e.g. “Eastern, metropolitan, lower-income, white, Catholic, female Democrats”), and the favourable/unfavourable/neither polling responses of members of these types to 52 specific “issues” (examples given include civil rights, anti-Communism, anti-Catholicism, foreign aid) were tabulated. Attempts were then made to “simulate” 32 of the USA’s 50 states by calculating the proportions of the 480 types in those states and assuming the frequency of responses within a voter type would be the same across states. This produced a ranking of how well Kennedy could be expected to do across these states, which matched the final results quite well. On top of this work an attempt was made to assess the impact of Kennedy’s Catholicism if it became an important issue in the election, but this required additional assumptions on how members of nine groups cross-classified by political and religious allegiance would respond. It is not clear that Kennedy’s campaign actually made any use of Simulmatics’ work, and there is no sense in which political dynamics were simulated. By contrast, in later Simulmatics work not dealt with by Lepore, on local referendum campaigns about water fluoridation (Abelson and Bernstein 1963), an approach very similar to current work in agent-based modelling was adopted. Agents based on the anonymised survey responses of individuals both responded to external messaging, and interacted with each other, to produce a dynamically simulated referendum campaign. It is unclear why Lepore does not cover this very interesting work. She does cover Simulmatics’ involvement in the Vietnam War, where their staff interviewed Vietnamese civilians and supposed “defectors” from the National Liberation Front of South Vietnam (“Viet Cong”) – who may in fact simply have gone back to their insurgent activity afterwards; but this work does not appear to have used computers for anything more than data storage.
In its work on American national elections (which continued through 1964) Simulmatics appears to have wildly over-promised given the data that it would have had available, subsequently under-performed, and failed as a company as a result; from this, indeed, today’s social simulators might take warning. Its leaders started out as “liberals” in American terms, but appear to have retained the colonialist mentality generally accompanying this self-identification, and fell into and contributed to the delusions of American involvement in the Vietnam War – although it is doubtful whether the history of this involvement would have been significantly different if the company had never existed. The fact that Simulmatics was largely forgotten, as Lepore recounts, hints that it was not, in fact, particularly influential, although interesting as the venue of early attempts at data analytics of the kind which may indeed now threaten what there is of democracy under capitalism (by enabling the “microtargeting” of specific lies to specific portions of the electorate), and at agent-based simulation of political dynamics. From a personal point of view, I am grateful to Lepore for drawing my attention to contemporary papers which contain far more useful information than her book about the early use of computers in the social sciences.
References
Abelson, R.P. and Bernstein, A. (1963) A Computer Simulation Model of Community Referendum Controversies. The Public Opinion Quarterly Vol. 27, No. 1 (Spring, 1963), pp. 93-122. Stable URL http://www.jstor.com/stable/2747294.
de Sola Pool, I. (1963) AUTOMATION: New Tool For Decision Makers. Challenge Vol. 11, No. 6 (MARCH 1963), pp. 26-27. Stable URL https://www.jstor.org/stable/40718664.
de Sola Pool, I. and Abelson, R.P. (1961) The Simulmatics Project. The Public Opinion Quarterly, Vol. 25, No. 2 (Summer, 1961), pp. 167-183. Stable URL https://www.jstor.org/stable/2746702.
Gotts, N. (2023) Yes, but what did they actually do? Review of: Jill Lepore (2020) "If Then: How One Data Company Invented the Future". Review of Artificial Societies and Social Simulation, 9 Mar 2023. https://rofasss.org/2023/03/09/ReviewofJillLepore
ETH Zürich – Department of Humanities, Social and Political Sciences (GESS)
The big mystery
Opinion dynamics (OD) is field dedicated to studying the dynamic evolution of opinions and is currently facing some extremely cryptic mysteries. Since 2009 there have been multiple calls for OD models to be strongly grounded in empirical data (Castellano et al., 2009, Valori et al., 2012, Flache et al., 2017; Dong et al., 2018), however the number of articles moving in this direction is still extremely limited. This is especially puzzling when compared with the increase in the number of publications in this field (see Fig 1). Another surprising issue, which extends also beyond OD, is that validated models are not cited as often as we would expect them to be (Chattoe-Brown, 2022; Kejjzer, 2022).
Some may argue that this could be explained by a general lack of people interested in the empirical side of opinion dynamics. However, the World seems in desperate need of empirically grounded OD models that could help us in shape policies on topics such as vaccination and climate change. Thus, it is very surprising to see that almost nobody is interested in meeting such a big and pressing demand.
In this short piece, I will share my experience both as a writer and as a reviewer for empirical OD papers, as well as the information I gathered from discussions with other researchers in similar roles. This will help us understand much better what is going on in the world of empirical OD and, more in general, in the empirical parts of agent-based modelling (ABM) related to psychological phenomena.
Publications containing the term “opinion dynamics” in abstract or title. Total 2,527. Obtained from dimensions.ai
Theoretical versus empirical OD
The main issue I have noticed with works in empirical OD is that these papers do not conform to the standard framework of ABM papers. Indeed, in “classical” ABM we usually try to address research questions like:
Can we develop a toy model to show how variables X and Y are linked?
Can we explain some macroscopic phenomenon as the result of agents’ interaction?
What happens to the outputs of a popular model if we add a new variable?
However, empirical papers do not fit into this framework. Indeed, empirical ABM papers ask questions such as:
How accurate are the predictions made by a certain model when compared with data?
How close is the micro-dynamic to the experimental data?
How can we refine previous models to improve their predicting ability?
Unfortunately, many reviewers do not view the latter questions as genuine research inquiries, ending up in pushing the authors to modify their papers to meet the first set of questions.
For instance, my empirical works often receive the critique that “the research question is not clear”, even though the question was explicitly stated in the main text, abstract and even in the title (See, for example “Deriving An Opinion Dynamics Model From Experimental Data”, Carpentras et al. 2022). Similarly, once a reviewer acknowledged that the experiment presented in the paper was an interesting addition to it, but they requested me to demonstrate why it was useful. Notice that, also in this case, the paper was on developing a model from the dynamical behavior observed in an experiment; therefore, the experiment was not just “an add on”, but core of the paper. I also have reviewed some empirical OD papers where the authors are asked, by other reviewers, to showcase how their model informs us about the world in a novel way.
As we will see in a moment, this approach does not just make authors’ life harder, but it also generates a cascade of consequences on the entire field of opinion dynamics. But to better understand our world, let us move first to a fictitious scenario.
A quick tale of natural selection of researcher
Let us now imagine a hypothetical world where people have almost no knowledge of the principles of physics. However, to keep the thought experiment simple, let us also suppose they have already developed the peer-review process. Of course, this fictious scenario is far from being realistic, but it should still help us understand what is going on with empirical OD.
In this world, a scientist named Alice writes a paper suggesting that there is an upward force when objects enter water. She also shows that many objects can float on water, therefore “validating” her model. The community is excited about this new paper which took Alice 6 months to write.
Now, consider another scientist named Bob. Bob, inspired by Alice’s paper, in 6 months conducts a series of experiments demonstrating that when an object is submerged in water, it experiences an upward force that is proportional to its submerged volume. This pushes knowledge forward as Bob does not just claim that this force exists, but he shows how this force has some clear quantitative relationship to the volume of the object.
However, when reviewers read Bob’s work, they are unimpressed. They question the novelty of his research and fail to see the specific research question he is attempting to address. After all, Alice already showed that this force exists, so what is new in this paper? One of the reviewers suggests that Bob should show how his study may impact their understanding of the world.
As a result, Bob spends an additional six months to demonstrate that he could technically design a floating object made out of metal (i.e. a ship). He also describes the advantages for society if such an object was invented. Unfortunately, one of the reviewers is extremely skeptical as metal is known to be extremely heavy and should not float in water, and requests additional proof.
After multiple revisions, Bob’s work is eventually published. However, the publication process takes significantly longer than Alice’s work, and the final version of the paper addresses a variety of points, including empirical validation, the feasibility of constructing a metal boat, and evidence to support this claim. Consequently, the paper becomes densely technical, making it challenging for most people to read and understand.
At the end, Bob is left with a single paper which is hardly readable (and therefore citable), while Alice, in the meanwhile, published many other easier-to-read papers having a much bigger impact.
Solving the mystery of empirical opinion dynamics
The previous sections helped us in understanding the following points: (1) validation and empirical grounding are often not seen as a legitimate research goal by many members of the ABM community. (2) This leads to bigger struggle when trying to publish this kind of research, and (3) reviewers often try to push the paper into the more classic research questions, possibly resulting in a monster-paper which tries to address multiple points all at once. (4) This also generates lower readability and so less impact.
So to sum it up: empirical OD gives you the privilege of working much more to obtain way less. This, combined with the “natural selection” of the “publish or perish” explains the scarcity of publications in this field, as authors need either to adapt to more standard ABM formulas or to “perish.” I also personally know an ex-researcher who tried to publish empirical OD until he got fed up and left the field.
Some clarifications
Let me make clear that this is a bit of a simplification and that, of course, it is definitely possible to publish empirical work in opinion dynamics even without “perishing.” However, choosing this instead of the traditional ABM approach strongly enhances the difficulty. This is a little like running while carrying extra weight: it is still possible that you will win the race, but the weight strongly decreases the probability of this happening.
I also want to say that while here I am offering an explanation of the puzzles I presented, I do not claim that this is the only possible explanation. Indeed, I am sure that what I am offering here is only part of the full story.
Finally, I want to clarify that I do not believe anyone in the system has bad intentions. Indeed, I think reviewers are in good faith when suggesting empirically-oriented papers to take a more classical approach. However, even with good intentions, we are creating a lot of useless obstacles for an entire research field.
Trying to solve the problem
To address this issue, in the past I have suggested dividing ABM researchers into theoretical and empirically oriented (Carpentras, 2020). The division of research into two streams could help us in developing better standards for both developing toy models and for empirical ABMs.
To give you a practical example, my empirical ABM works usually receive long and detailed comments about the model properties and almost no comment on the nature of the experiment or data analysis. Am I that good in these last two steps? Or maybe reviewers in ABM focus very little on the empirical side of empirical ABMs? While the first explanation would be flattering for me, I am afraid that the reality is better depicted by the second option.
With this in mind, together with other members of the community, we have created a special interest group for Experimental ABM (see http://www.essa.eu.org/sig/sig-experimental-abm/). However, for this to be successful, we really need people to recognize the distinction between these two fields. We need to acknowledge that empirically-related research questions are still valid and not push papers towards the more classical approach.
I really believe empirical OD will raise, but how this will happen is still to decide. Will it be at the cost of many researchers facing bigger struggle or will we develop a more fertile environment? Or maybe some researchers will create an entire new niche outside of the ABM community? The choice is up to us!
References
Carpentras, D., Maher, P. J., O’Reilly, C., & Quayle, M. (2022). Deriving An Opinion Dynamics Model From Experimental Data. Journal of Artificial Societies & Social Simulation, 25(4). https://www.jasss.org/25/4/4.html
Carpentras, D. (2020) Challenges and opportunities in expanding ABM to other fields: the example of psychology. Review of Artificial Societies and Social Simulation, 20th December 2021. https://rofasss.org/2021/12/20/challenges/
Castellano, C., Fortunato, S., & Loreto, V. (2009). Statistical physics of social dynamics. Reviews of modern physics, 81(2), 591. DOI: 10.1103/RevModPhys.81.591
Chattoe-Brown, E. (2022). If You Want To Be Cited, Don’t Validate Your Agent-Based Model: A Tentative Hypothesis Badly In Need of Refutation. Review of Artificial Societies and Social Simulation, 1 Feb 2022. https://rofasss.org/2022/02/01/citing-od-models/
Dong, Y., Zhan, M., Kou, G., Ding, Z., & Liang, H. (2018). A survey on the fusion process in opinion dynamics. Information Fusion, 43, 57-65. DOI: 10.1016/j.inffus.2017.11.009
Flache, A., Mäs, M., Feliciani, T., Chattoe-Brown, E., Deffuant, G., Huet, S., & Lorenz, J. (2017). Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation, 20(4). https://www.jasss.org/20/4/2.html
Valori, L., Picciolo, F., Allansdottir, A., & Garlaschelli, D. (2012). Reconciling long-term cultural diversity and short-term collective social behavior. Proceedings of the National Academy of Sciences, 109(4), 1068-1073. DOI: 10.1073/pnas.1109514109
Carpentras, D. (2023) Why we are failing at connecting opinion dynamics to the empirical world. Review of Artificial Societies and Social Simulation, 8 Mar 2023. https://rofasss.org/2023/03/08/od-emprics
*All authors contributed equally – author order determined by a pseudo-random number generator and does not reflect their respective contributions.
Introduction
Since its inception in 2010, ESSA@work has been a mainstay at the annual Social Simulation Conference (SSC). It continues as a forum where beginners in individual- and agent-based modelling (hereon, ABM) present a work-in-progress model, along with specific problems and questions, to a community of practitioners to get feedback, suggestions, and tips for specific aspects of their modelling projects. During the session, participants present their model to an audience and two experts, the latter of whom are chosen for their constructive style of feedback and broad expertise. Participants are not required to answer questions or defend their work, as might be the case in a more traditional setting. Instead, experts enter into a dialogue with each other with the explicit goal of providing constructive feedback towards the progress of the project. After the expert discussion, the audience can also add constructive ideas and questions.
Each ESSA@work session is organised by a team of volunteers, who were often introduced to the format by being participants themselves. In the weeks prior to the SSC, this group drafts all necessary documents to elicit participation, selects participants, contacts experts, and distributes information via mailing lists and social media channels. During the sessions, they serve as chairs and provide outreach via social media. In between conferences and other events with ESSA@work sessions, organisers serve as points of contact for anyone who might want to organise a local ESSA@work session engage with the management of the broader European Social Simulation Association (ESSA), maintain information on the ESSA@work website (http://www.essa.eu.org/essawork/), and recruit the next generation of volunteers. Organisers typically stay on for a number of years, so that a continuity of knowledge on the processes is secured.
Over the years, a few themes that characterise ESSA@work have crystallised and indicate the importance of the track. In this contribution, we outline these themes: how ESSA@work provides a learning experience to participants and the audience, as well as the organisers; how it fosters interdisciplinarity; and how it builds upon a community of practice. We conclude with our wishes for its future.
Themes
Learning experience
The participants in ESSA@work tend to be early-career researchers, such as masters students, doctoral candidates, or post-doctoral researchers, but we have also had participants who are experienced academics, but new to ABM. For early-career researchers, participating holds additional benefits, as the SSC where they participate in ESSA@work may be their first (on-site) conference. For instance, this was the case for the SSC2022, which was held in a hybrid format after a long period of restrictions and uncertainty due to COVID-19. This deeply affected the career of young researchers: for some, most of their PhD has been spent online with no or little opportunity to participate in events such as annual conferences.
While the learning experience is focused on the participants and their contributions, it extends beyond them to include audience members and organisers as well, so that ESSA@work sessions present different learning channels. The first learning channel is focused on presentation and social skills. These general skills apply to any career path and are facilitated and supported by the friendly environment and specific format that ESSA@work implements. The practice of presenting unfinished work fosters collaboration, open conversations, and reflection, among peers and more senior academics alike, rather than an environment where participants must ‘defend’ their work from reviewers. Participants must adapt their presentation to a specific format, where they clearly address their doubts and issues. This requires them to put together a clear, concise presentation aligned with the non-standard focus of the track. We have one-page guidance, detailed guidance, and Frequently Asked Questions (FAQs) covering these aspects online (http://www.essa.eu.org/essawork/how-to-participate/ and http://www.essa.eu.org/essawork/faq/). The track then also facilitates and encourages the development of social skills by bringing together members of the ESSA community of all experience levels, allowing participants to develop their network of contacts and collaborations based on shared experiences and mentorship.
Secondly, there is the specific feedback from experts, including literature and data recommendations, and references to existing models or other contacts. This adds to or complements the feedback that participants (especially PhD students and postdocs) receive from their supervisors. Participants can find diverse, enriching suggestions with respect to the line of work that they were following, and new perspectives. For cases in which relationships with supervisors and mentors are proving to be difficult, or where supervisors are less familiar with ABM, this can be a crucial source of motivation and support for researchers who find themselves stuck in the process. This can also be a useful source of ideas for audience members with similar questions or challenges.
There is also the organiser’s experience. This usually starts with being involved as a participant in the track. As speakers, participants begin to familiarise themselves with the specific ESSA@work format, as well as with the steps, timing, and process that lead to the conference events. This is also a way to get in touch with former and current team members before officially joining the organisers’ team. After being introduced to ESSA@work as a participant or audience member, new members of the organising team receive training by current and/or former members in a process of knowledge transfer guided by prior experiences. This is put in place with the goal of sharing, improving from the past, and creating a community.
Once researchers have fully joined the team and start helping to prepare the next edition of ESSA@work, the learning opportunities are numerous. From building and strengthening their network of contacts across ESSA, to practising organisation and chairing (which would otherwise often come at a later career stage), reviewing submitted manuscripts, improving communication and coordination skills, and project and time management. Last but not least, organisers work in a team. This exercise of coordination is a fantastic occasion for learning-by-doing of how to adapt and organise heterogeneous skills, schedules, and expertise towards continuous improvement. On the one hand, this mimics co-authorship, and thus offers an opportunity to familiarise oneself with a frequent pattern in academic work; on the other hand, it emphasises and strengthens the feeling of community that characterises ESSA, and that is even stronger in the ESSA@work family.
Interdisciplinarity
Through our time as organisers, we have seen first-hand how diverse the ABM community is. The background of participants can include physics, ecology, computer science, economics, or psychology, just to name a few. This is a double-edged sword, as it both allows researchers to produce work connecting multiple disciplines, but can also result in work that is not accessible to the different audiences who may otherwise be interested in it.
For example, people from statistical physics may be very interested in solving the mean-field approximation of a model, while psychologists may be more interested in the qualitative interpretation of such a model. Similar problems also regard the use of technical terms. For example, terms like “experiment” are used by some to mean “computer simulation” and by others to mean “empirical experiment with real people.” Similarly, Edmund Chattoe-Brown found 5 different uses of the term “validation” (Chattoe-Brown, 2021). Therefore, while ABM can connect multiple different fields, research content can still be very hard to understand by multiple scientists. This can paradoxically result in more difficulty in reaching out or communicating results to some communities or fields (Carpentras, 2022).
ESSA@work can have a unique role in tackling this problem, as it allows people who have recently begun working with ABM to get an “inside view” of the ABM community. By presenting their work and research questions to experts in ABM, and receiving feedback from them, participants can have a smoother process to publishing their models, for example by avoiding common mistakes and pitfalls, and gain more insights on typical research questions, problems and jargon of ABM. This allows participants to get more acquainted with the ABM community and mindsets (as discussed in the next section), allowing for a better integration and long-term connection with the field.
Community building
When speaking of communities of practice, we ask how practitioners of a certain profession or discipline both shape and are shaped by their profession. ESSA@work has a role to play in both, but perhaps more heavily in the latter.
As a whole, ESSA seems to be actively shaping a community of practice in social simulation, more specifically ABM, through shared standards and protocols, regular exchanges, and collaborations across disciplines, all rallying around a specific method. For many members, this is a community separate to the one that they belong to on a day-to-day basis in their departments or organisations. There is active communal support in jointly shaping the rules that should govern the community and the method, as well as a continuous (and friendly) negotiation of who or what is included and excluded from the community and where overlaps with other communities might be (see recent discussion in the SIMSOC mailing list; https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=ind2211&L=SIMSOC&O=D&P=19269). This is how the community shapes the practice – both actively and passively.
The other side of the coin is how the practice shapes the community. As discussed, ESSA@work sessions are often the place and time where new members are introduced to the community, and where their future outlook on the community and the method is significantly shaped. Through useful, tactful, and constructive feedback, new members are introduced to the core texts that at least partially constitute the collective imaginary of the community of practice, to the protocols that govern what constitutes good practice, and – perhaps most importantly – to the tone that the community uses in interacting with one another. ESSA@work therefore not only provides a forum for constructive feedback on work-in-progress, but also an experience which is useful to decide whether someone wants to be part of this community. With ‘alumni’ often coming back as organisers or panellists, and recommending the track to their peers and students, there is a sense that ESSA@work – and the attitude it embodies – is passed on through academic generations. It therefore becomes very much part of what we do, and how we do things, in the agent-based modelling community.
Future themes
As we look to the future for ESSA@work, we have considered both its continuing role in providing a multi-faceted learning experience and central point for the ESSA community, as well as how it can continue to contribute to the future of both ESSA and the field of agent-based modelling more broadly. Specifically, as agent-based modelling has become more accepted as a method for simulating and analysing complex systems, and therefore taken a more empirical turn, ESSA@work can have a unique role in fostering and maintaining the diversity of modelling purposes, which may otherwise become less valued in the rest of the scientific community.
Most participants have questions related to specific stages of their modelling journey. If you think of an ABM journey being roughly divided into the following stages: (1) conceptualisation and design, (2) development, (3) verification and calibration, (4) validation, and (5) simulations, uncertainty analysis and results, most ESSA@work participants are somewhere between steps 2 and 4 in their modelling journey. As step 1 presents several possibilities and needs longer for background work (like literature review, brainstorming, stakeholder consultation, etc.), we intentionally encourage participation in the forum from step 2 onwards, when the purpose, scope and objectives of models become clearer. This in turn enables specific modelling questions being put forth that can be usefully addressed within the time and space of an ESSA@work session. Over the years, we have received submissions from across disciplines and mostly focusing on issues in steps 2 to 4 of the modelling journey. More recently, we also started receiving submissions with questions about running simulation experiments, calibration and validation with empirical data, interpreting results, and conducting uncertainty analysis. We believe this speaks to ABM becoming more mainstream as a microsimulation approach during this period, enabled also by the availability and accessibility to powerful computing resources.
We find that when questions fall under modelling stages 2, 3 and 5, participants receive more direct answers as questions tend to be specific, which our practitioner community addresses based on their own work, or on wider references. On the other hand, questions about model validation (stage 4) could be quite broad and open-ended to attract a useful response in the time available. ‘How can I validate my model?’ – or the essence of this question worded differently – is a popular question in this category. A practical and straightforward answer to validate a model is to collect or use data on the modelled phenomenon, and use them as test data to check if the model replicates patterns of the test data. Often though, participants indicate that the test data do not exist or are difficult to obtain. This would then raise questions about the purpose of the model: specifically, whether it’s intended as a toy model to generate plausible explanations about an observed phenomenon (historically the realm of ABM), or as a specialised model to allow meaningful forecasts. Having the latter objective would mean that the model needs good quality data at every stage of model development, and lacking those data would raise concerns about the suitability of ABM in the first place to address the proposed research questions. Without validation, as robust as a model may be, it may not be trusted to generate valid predictions or forecasts.
On the other hand, where models are intended as ‘toy models’, lack of validation is less of a problem. These models are meant to inspire more informed research questions about observed phenomena, which can subsequently be explored through further targeted real-world experiments, data collection, modelling, or a combination thereof. These models also provide clear entry points to the discipline for someone just beginning to explore complex systems, ABM, or both – many of us can point to reading texts such as Growing Artificial Societies (Epstein and Axtell, 1996) as the first time we truly understood and connected with ABM. But somehow there appear to be fewer takers for developing toy models in recent years. This could be due to perceptions that toy models risk being dismissed as vague (or at least harder to publish), because practitioners are on tight timelines and thus experience a lack of time or room to experiment with toy models, or because of a need to deliver model-based predictions (forecasts or projections) to satisfy specific project requirements.
We fear that any such bias against toy models might incur a cost in the form of compromised quality of models, or discourage new entrants and sponsors for ABM. The former is likely to occur when modellers try to build an overly complicated or specific model based on minimal, poor, or fragmented data, and thus possibly relying on too many assumptions that lack sound evidence. The latter could happen when ABM is solely intended as a means to an end rather than as a means to experiment. Reflecting on our journey and thinking ahead, we believe ESSA@work could avoid these outcomes by providing an unbiased, supportive, and well-connected incubatory forum to encourage the development and housing of toy models, which have sound methodological and modelling rigour, despite being unsuitable for prediction due to the lack of validation using empirical data. We could then expect that a growing bank of model examples and modellers would pave the way for ABM practice to flourish, alongside guiding data confidentiality, data collection, sharing, and management practices that allow turning toy models into specialised models in methodical, reusable, and reproducible ways. The prominence of ESSA@work in the ESSA network could allow us to take on such a role in the future if more ABM practitioners (at all stages of their modelling career) volunteer to support with running the forum and its activities.
Conclusion
ESSA@work offers a valuable learning experience for participants, audience members, and organisers alike. It has become an integral part of the SSC annual conference and especially of the ABM community. While this is the result of past efforts and activities, our current work looks to the future and aims at continuity with the past but also renovation and further development.
We strive to improve and make our team and community grow. For this reason, we always welcome new organisers to contribute in this joint effort to grow both the spectrum and the reach of our activities. To guarantee continuity of this track and continue to improve it, we believe that diversity in participation could play a major role in innovation and better identifying early career researchers’ and other participants’ needs in the coming years.
The COVID era has confronted us, among others, with different professional and academic challenges. We all transferred our work from on-site to remote or hybrid, and likewise we adapted to new formats to guarantee that the ESSA community could continue to meet. While originally the result of needs and adaptation, online and hybrid formats have proved to be effective in ensuring a wide reach and increased accessibility. The SSC2022 and SocSimFesT past editions showed the possibility and success of a plurality of formats and ways to meet, discuss, and progress our research. These formats are now integrated in our working life and they represent a possibility for ESSA@work to get in touch with new cohorts of international modellers.
ESSA@work is a friendly space for in-depth discussion and learning, and as such, it extends beyond the boundaries of the annual conference or on-site events. We aim to continue offering online or hybrid events in the hope that they will make participation more accessible and provide additional feedback to anyone who needs it. In addition, we encourage the organisation of local ESSA@work sessions. In order to do so, the ambition and priority of ESSA@work is preserving its function as a community-builder and ensuring that participants are supported and able to self-organise according to the challenges and needs arising from their research.
References
Carpentras, D. (2020) Challenges and opportunities in expanding ABM to other fields: the example of psychology. Review of Artificial Societies and Social Simulation, 20th December 2021. https://rofasss.org/2021/12/20/challenges/
Chattoe-Brown, E. (2022) Today We Have Naming Of Parts: A Possible Way Out Of Some Terminological Problems With ABM. Review of Artificial Societies and Social Simulation, 11th January 2022. https://rofasss.org/2022/01/11/naming-of-parts/
Epstein, J. M., & Axtell, R. (1996). Growing artificial societies: social science from the bottom up. Brookings Institution Press.
Narasimhan, K., Leoni, S., Luckner, K., Carpentras, D. and Davis, N. (2022) ESSA@work: Reflections and looking ahead. Review of Artificial Societies and Social Simulation, 20 Feb 2023. https://rofasss.org/2022/02/20/essawork
“Just as physical tools and machines extend our physical abilities, models extend our mental abilities, enabling us to understand and control systems beyond our direct intellectual reach” (Calder & al. 2018)
Motivation
There is a modelling norm that one should be able to completely understand one’s own model. Whilst acknowledging there is a trade-off between a model’s representational adequacy and its simplicity of formulation, this tradition assumes there will be a “sweet spot” where the model is just tractable but also good enough to be usefully informative about the target of modelling – in the words attributed to Einstein, “Everything should be made as simple as possible, but no simpler”1. But what do we do about all the phenomena where to get an adequate model2 one has to settle for a complex one (where by “complex” I mean a model that we do not completely understand)? Despite the tradition in Physics to the contrary, it would be an incredibly strong assumption that there are no such phenomena, i.e. that an adequate simple model is always possible (Edmonds 2013).
There are three options in these difficult cases.
Do not model the phenomena at all until we can find an adequate model we can fully understand. Given the complexity of much around us this would mean to not model these for the foreseeable future and maybe never.
Accept inadequate simpler models and simply hope that these are somehow approximately right3. This option would allow us to get answers but with no idea whether they were at all reliable. There are many cases of overly simplistic models leading policy astray (Adoha & Edmonds 2017; Thompson 2022), so this is dangerous if such models influence decisions with real consequences.
Use models that are good for our purpose but that we only partially understand. This is the option examined in this paper.
When the purpose is empirical the last option is equivalent to preferring empirical grounding over model simplicity (Edmonds & Moss 2005).
Partially Understood Models
In practice this argument has already been won – we do not completely understand many computer simulations that we use and rely on. For example, due to the chaotic nature of the dynamics of the weather, forecasting models are run multiple times with slightly randomised inputs and the “ensemble” of forecasts inspected to get an idea of the range of different outcomes that could result (some of which might be qualitatively different from the others)4. Working out the outcomes in each case requires the computational tracking of a huge numbers of entities in a way that is far beyond what the human mind can do5. In fact, the whole of “Complexity Science” can be seen as different ways to get some understanding of systems for which there is no analytic solution6.
Of course, this raises the question of what is meant by “understand” a model, for this is not something that is formally defined. This could involve many things, including the following.
That the micro-level – the individual calculations or actions done by the model each time step – is understood. This is equivalent to understanding each line of the computer code.
That some of the macro-level outcomes that result from the computation of the whole model is understood in terms of partial theories or “rules of thumb”.
That all the relevant macro-level outcomes can be determined to a high degree of accuracy without simulating the model (e.g. by a mathematical model).
Clearly, level (1) is necessary for most modelling purposes in order to know the model is behaving as intended. The specification of this micro-level is usually how such models are made, so if this differs from what was intended then this would be a bug. Thus this level would be expected of most models7. However, this does not necessarily mean that this is at the finest level of detail possible – for example, we usually do not bother about how random number generators work, but simply rely on its operation, but in this case we have very good level (3) of understanding for these sub-routines.
At the other extreme, a level (3) understanding is quite rare outside the realm of physics. In a sense, having this level of understanding makes the model redundant, so would probably not be the case for most working models (those used regularly)8. As discussed above, there will be many kinds of phenomena for which this level of understanding is not feasible.
Clearly, what many modelers find useful is a combination of levels (1) & (2) – that is, the detailed, micro-level steps that the model takes are well understood and the outcomes understood well enough for the intended task. For example, when using a model to establish a complex explanation9 (of some observed pattern in data using certain mechanisms or structures) then one might understand the implementation of the candidate mechanisms and verify that the outcomes fit the target pattern for a range of parameters, but not completely understand the detail of the causation involved. There might well be some understanding, for example how robust this is to minor variations in the initial conditions or the working of the mechanisms involved (e.g. by adding some noise to the processes). A complete understanding might not be accessible but this does not stop an explanation being established (although a better understanding is an obvious goal for future research or avenue for critiques of the explanation).
Of course, any lack of a complete, formal understanding leaves some room for error. The argument here is not deriding the desirability of formal understanding, but is against prioritising that over model adequacy. Also the lack of a formal, level (3), understanding of a model does not mean we cannot take more pragmatic routes to checking it. For example: performing a series of well-designed simulation experiments that intend to potentially refute the stated conclusions, systematically comparing to other models, doing a thorough sensitivity analysis and independently reproducing models can help ensure their reliability. These can be compared with engineering methods – one may not have a proof that a certain bridge design is solid over all possible dynamics, but practical measures and partial modelling can ensure that any risk is so low as to be negligible. If we had to wait until bridge designs were proven beyond doubt, we would simply have to do without them.
Layering Models to Leverage some Understanding
As a modeller, if I do not understand something my instinct is to model it. This instinct does not change if what I do not understand is, itself, a model. The result is a model of the original model – a meta-model. This is, in fact, common practice. I may select certain statistics summarising the outcomes and put these on a graph; I might analyse the networks that have emerged during model runs; I may use maths to approximate or capture some aspect of the dynamics; I might cluster and visualise the outcomes using Machine Learning techniques; I might make a simpler version of the original and compare them. All of these might give me insights into the behaviour of the original model. Many of these are so normal we do not think of this as meta-modelling. Indeed, empirically-based models are already, in a sense, meta-models, since the data that they represent are themselves a kind of descriptive model of reality (gained via measurement processes).
This meta-modelling strategy can be iterated to produce meta-meta-models etc. resulting in “layers” of models, with each layer modelling some aspect of the one “below” until one reaches the data and then what the data measures. Each layer should be able to be compared and checked with the layer “below”, and analysed by the layer “above”.
An extended example of such layering was built during the SCID (Social Complexity of Immigration and Diversity) project10 and illustrated in Figure 1. In this a complicated simulation (Model 1) was built to incorporate some available data and what was known concerning the social and behavioural processes that lead people to bother to vote (or not). This simulation was used as a counter-example to show how assumptions about the chaining effect of interventions might be misplaced (Fieldhouse et al. 2016). A much simpler simulation was then built by theoretical physicists (Model 2), so that it produced the same selected outcomes over time and aa range of parameter values. This allowed us to show that some of the features in the original (such as dynamic networks) were essential to get the observed dynamics in it (Lafuerza et al. 2016a). This simpler model was in turn modelled by an even simpler model (Model 3) that was amenable to an analytic model (Model 4) that allowed us to obtain some results concerning the origin of a region of bistability in the dynamics (Lafuerza et al. 2016b).
Figure 1. The Layering of models that were developed in part of the SCID project
Although there are dangers in such layering – each layer could introduce a new weakness – there are also methodological advantages, including the following. (A) Each model in the chain (except model 4) is compared and checked against both the layer below and that above. Such multiple model comparisons are excellent for revealing hidden assumptions and unanticipated effects. (B) Whilst previously what might have happened was a “heroic” leap of abstraction from evidence and understanding straight to Model 3 or 4, here abstraction happens over a series of more modest steps, each of which is more amenable to checking and analysis. When you stage abstraction the introduced assumptions are more obvious and easier to analyse.
One can imagine such “layering” developing in many directions to leverage useful (but indirect) understanding, for example the following.
Using an AI algorithm to learn patterns in some data (e.g. medical data for disease diagnosis) but then modelling its working to obtain some human-accessible understanding of how it is doing it.
Using a machine learning model to automatically identify the different “phase spaces” in model results where qualitatively different model behaviour is exhibited, so one can then try to simplify the model within each phase.
Automatically identifying the processes and structures that are common to a given set of models to facilitate the construction of a more general, ‘umbrella’ model that approximates all the outcomes that would have resulted from the set, but within a narrower range of conditions.
As the quote at the top implies, we are used to settling for partial control of what machines do because it allows us to extend our physical abilities in useful ways. Each time we make their control more indirect, we need to check that this is safe and adequate for purpose. In the cars we drive there are ever more layers of electronic control between us and the physical reality it drives through which we adjust to – we are currently adjusting to more self-drive abilities. Of course, the testing and monitoring of these systems is very important but that will not stop the introduction of layers that will make them safer and more pleasant to drive.
The same is true of our modelling, which we will need to apply in ever more layers in order to leverage useful understanding which would not be accessible otherwise. Yes, we will need to use practical methods to test their fitness for purpose and reliability, and this might include the complete verification of some components (where this is feasible), but we cannot constrain ourselves to only models we completely understand.
Concluding Discussion
If the above seems obvious, then why am I bothering to write this? I think for a few reasons. Firstly, to answer the presumption that understanding one’s model must have priority over all other considerations (such as empirical adequacy) so that sometimes we must accept and use partially understood models. Secondly, to point out that such layering has benefits as well as difficulties – especially if it can stage abstraction into more verifiable steps and thus avoid huge leaps to simple but empirically-isolated models. Thirdly, because such layering will become increasingly common and necessary.
In order to extend our mental reach further, we will need to develop increasingly complicated and layered modelling. To do this we will need to accept that our understanding is leveraged via partially understood models, but also to develop the practical methods to ensure their adequacy for purpose.
Notes
[1] These are a compressed version of his actual words during a 1933 lecture, which were: “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” (Robinson 2018) [2] Adequate for whatever our purpose for it is (Edmonds & al. 2019). [3]The weasel words I once heard from a Mathematician excusing an analytic model he knew to be simplistic were: that, although he knew it was wrong, it was useful for “capturing core dynamics” (though how he knew that they were not completely wrong eludes me). [4] For an introduction to this approach read the European Centre for Medium-Range Weather Forecasts’ fact sheet on “Ensemble weather forecasting” at: https://www.ecmwf.int/en/about/media-centre/focus/2017/fact-sheet-ensemble-weather-forecasting [5] In principle, a person could do all the calculations involved in a forecast but only with the aid of exterior tools such as pencil and paper to keep track of it all so it is arguable whether the person doing the individual calculations has an “understanding” of the complete picture. Lewis Fry Richardson, who pioneered the idea of numerical forecasting of weather in the 1920s, did a 1-day forecast by hand to illustrate his method (Lynch 2008), but this does not change the argument. [6] An analytic solution is when one can obtain a closed-form equation that characterises all the outcomes by manipulating the mathematical symbols in a proof. If one has to numerically calculate outcomes for different initial conditions and parameters this is a computational solution. [7] For purely predictive models, whose purpose is only to anticipate an unknown value to a useful level of accuracy, this is not strictly necessary. For example, how some AI/Machine learning models work may not clear at the micro-level, but as long as it works (successfully predicts) this does not matter – even if its predictive ability is due to a bug. [8] Models may still be useful in this case, for example to check the assumptions made in the matching mathematical or other understanding. [9] For more on this use see (Edmonds et al. 2019). [10] For more about this project see http://cfpm.org/scid
Acknowledgements
Bruce Edmonds is supported as part of the ESRC-funded, UK part of the “ToRealSim” project, 2019-2023, grant number ES/S015159/1 and was supported as part of the EPSRC-funded “SCID” project 2010-2016, grant number EP/H02171X/1.
References
Calder, M., Craig, C., Culley, D., de Cani, R., Donnelly, C.A., Douglas, R., Edmonds, B., Gascoigne, J., Gilbert, N. Hargrove, C., Hinds, D., Lane, D.C., Mitchell, D., Pavey, G., Robertson, D., Rosewell, B., Sherwin, S., Walport, M. and Wilson, A. (2018) Computational modelling for decision-making: where, why, what, who and how. Royal Society Open Science, DOI:10.1098/rsos.172096.
Edmonds, B. (2013) Complexity and Context-dependency. Foundations of Science, 18(4):745-755. DOI:10.1007/s10699-012-9303-x
Edmonds, B. and Moss, S. (2005) From KISS to KIDS – an ‘anti-simplistic’ modelling approach. In P. Davidsson et al. (Eds.): Multi Agent Based Simulation 2004. Springer, Lecture Notes in Artificial Intelligence, 3415:130–144. DOI:10.1007/978-3-540-32243-6_11
Edmonds, B., le Page, C., Bithell, M., Chattoe-Brown, E., Grimm, V., Meyer, R., Montañola-Sales, C., Ormerod, P., Root H. & Squazzoni. F. (2019) Different Modelling Purposes. Journal of Artificial Societies and Social Simulation, 22(3):6. DOI:10.18564/jasss.3993
Fieldhouse, E., Lessard-Phillips, L. & Edmonds, B. (2016) Cascade or echo chamber? A complex agent-based simulation of voter turnout. Party Politics. 22(2):241-256. DOI:10.1177/1354068815605671
Lafuerza, LF, Dyson, L, Edmonds, B & McKane, AJ (2016a) Simplification and analysis of a model of social interaction in voting, European Physical Journal B, 89:159. DOI:10.1140/epjb/e2016-70062-2
Lynch, P. (2008). The origins of computer weather prediction and climate modeling. Journal of Computational Physics, 227(7), 3431-3444. DOI:10.1016/j.jcp.2007.02.034
Thompson, E. (2022) Escape from Model Land. Basic Books. ISBN-13: 9781529364873
Edmonds, B. (2023) The inevitable “layering” of models to extend the reach of our understanding. Review of Artificial Societies and Social Simulation, 9 Feb 2023. https://rofasss.org/2023/02/09/layering
Information and Computational Sciences Department, The James Hutton Institute, Aberdeen AB15 8QH, UK.
High-performance computing (HPC) clusters are increasingly being used for agent-based modelling (ABM) studies. There are reasons why HPC provides a significant benefit for ABM work, and to expect a growth in HPC/ABM applications:
ABMs typically feature stochasticity, which require multiple runs using the same parameter settings and initial conditions to ascertain the scope of the behaviour of the model. The ODD protocol has stipulated the explicit specification of this since it was first conceived (Grimm et al. 2006). Some regard stochasticity as ‘inelegant’ and to be avoided in models, but asynchrony in agents’ actions can avoid artefacts (results being a ‘special case’ rather than a ‘typical case’) and introduces an extra level of complexity affecting the predictability of the system even when all data are known (Polhill et al. 2021).
ABMs often have high-dimensional parameter spaces, which need to be sampled for sensitivity analyses and, in the case of empirical ABMs, for calibration and validation. The so-called ‘curse of dimensionality’ means that the problem of exploring parameter space grows exponentially with the number of parameters. While ABMs’ parameters may not all be ‘orthogonal’ (i.e. each point in parameter space does not uniquely specify model behaviour – a situation sometimes referred to as ‘equifinality’), diminishing the ‘curse’, the exponential growth means the challenge of parameter search does not need many dimensions before it becomes intractable exhaustively.
Both the above points are exacerbated in empirical applications of ABMs given Sun et al.’s (2016) observations about the ‘medawar zone’ of model complicatedness in relation to that of theoretical models. In empirical applications, we also may be more interested in knowing that an undesirable outcome cannot occur, or has a very low probability of occurring, requiring more runs with the same conditions. Further, the additional complicatedness of empirical ABM will entail more parameters, and the empirical application will place greater emphasis on searching parameter space for calibrating and validating to data.
HPC clusters are shared computing resources, and it is now commonplace for research organizations and universities to have them. There can be few academic disciplines without some sort of scientific computing requirement – typical applications include particle physics, astronomy, meteorology, materials, chemistry, neuroscience, medicine and genetics. And social science. As a shared resource, an HPC cluster is subject to norms and institutions frequently observed in common-pool resource dilemmas. Users of HPC clusters are asked to request allocations of computing time, memory and long-term storage space to accommodate their needs. The requests are made in advance of the runs being executed; sometimes so far in advance that the calculations form part of the research project proposal. Hence, as a user, if you do not know, or cannot calculate, the resources you will require, you have a dilemma: ask for more than it turns out you really need and risk normative sanctions; or ask for less than it turns out you really need and impair the scientific quality of your research. Normative sanctions are in the job description of the HPC cluster administrator. This can lead to emails such as those in Figure 1.
Figure 1: Example email and accompanying visualization from an HPC cluster administrator reminding users that it is antisocial to request more resources than you will use when submitting jobs.
The ‘managerialist’ turn in academia has been lamented in various articles. Kolsaker (2008), while presenting a nuanced view of the relationship between managerialist and academic modes of working, says that “managerialism represents a distinctive discourse based upon a set of values that justify the assumed right of one group to monitor and control the activities of others.” Steinþórsdóttir et al. (2019) note in the abstract to their article that their results from a case study in Iceland support arguments that managerialism discriminates against women and early-career researchers, in part because of a systemic bias towards natural sciences. Both observations are relevant in this context.
Measurement and control as the tools of managerialist conduct renders Goodhart’s Law (the principle that when a metric becomes a target, the metric is useless) relevant. Goodhart’s Law has been found to have led to bibliometrics now being useless for comparing researchers’ performance – both within and between departments (Fire and Guestrin 2019). We may therefore expect that if an HPC cluster’s administrator has the accurate prediction of computing resource as a target for their own performance assessment, or if they give it as a target for users – e.g. by prioritizing jobs submitted by users on the basis of the accuracy of their predicted resource use, or denying access to those consistently over-estimating requirements – this accuracy will be useless. To give a concrete example, programming languages such as C give the programmer direct control over memory allocation. Hence, were access to an HPC conditional on the accurate prediction of memory allocation requirements, a savvy C programmer would have the (excessive) memory allowance in the batch job submission as a command-line argument to their program, which on execution would immediately request that allocation from the server’s operating system. The rest of the program would use bespoke memory allocation functions that allocated the memory the program actually needed from the memory initially reserved. Similar principles can be used for CPU cycles – if the program runs too quickly, then calculate digits of π until the predicted CPU time has elapsed; and disk space – if too much disk space has been requested, then pad files with random data. These activities waste the programmer’s time, and entail additional use of computing resources with energy cost implications for the cluster administrator.
With respect to the normative statements such as those in Figure 1, Griesemer (2020, p. 77), discussing the use of metrics leading to ‘gaming the system’ in academia generally (the savvy C programmer’s behaviour being an example in the context of HPC usage) claims that “it is … problematic to moralize and shame [such] practices as if it were clear what constitutes ethical … practice in social worlds where Goodhart’s law operates” [emphasis mine]. In computer science, however, there are theoretical (in the mathematical sense of the term) reasons why such norms are problematic over-and-above the social context of measurement-and-control.
The theory of computer science is founded in mathematics and logic, and the work of notable thinkers such as Gödel, Turing, Hilbert, Kolmogorov, Chomsky, Shannon, Tarski, Russell and von Neumann. The growth in areas of computer science (e.g. artificial intelligence, internet-of-things) means that undergraduate degrees have increasingly less space to devote to teaching this theory. Blumenthal (2021, p. 46), comparing computer science curricula in 2014 and 2021, found that the proportion of courses with required modules on computational theory had dropped from 46% to 40%, though the sample size meant this result was not significant (P = 0.09 under a two-population z-test). Similarly, the time dedicated to algorithmics and complexity in CS2013 fell to 28 (of which 19 are ‘tier-1’ – required of every curriculum; and 9 are ‘tier-2’ – in which 80% topic coverage is the stipulated minimum) from 31 in CS2008 (Joint Task Force on Computing Curricula 2013).
One of the most critical theoretical results in computer science is the so-called Halting Problem (Turing 1937), which proves that it is impossible to write a computer program that (in the general case) takes as input another computer program and its input data and gives as output whether the latter program will halt or run forever. The halting problem is ‘tier-1’ in CS2013, and so should be taught to every computer scientist. Rice (1953) generalized Turing’s finding to prove that any ‘non-trivial’ properties of computer programs could not be decided algorithmically. These results mean that the automated job scheduling and resource allocation algorithms in HPC, such as SLURM (Yoo et al. 2003), cannot take a user’s submitted job as input and calculate the computing resources it will need. Any requirement for such prediction is thus pushed to the user. In the general case, this means users of HPC clusters are being asked to solve formally undecidable problems when submitting jobs. Qualified computer scientists should know this – but possibly not all cluster administrators, and certainly not all cluster users, are qualified computer scientists. The power dynamic implied by Kolsaker’s (2008) characterization of a managerialist working culture puts users as a disadvantage, while Steinþórsdóttir et al.’s (2019) observations suggest this practice may be indirectly discriminatory on the basis of age and gender; the latter particularly when social scientists are seeking access to shared HPC facilities.
I emphasized ‘in the general case’ above because in many specific cases, computing resources can be accurately estimated. Sorting a list of strings in alphabetical order, for example is known to grow in execution time with as a function of n log n, where n is the length of the list. Integers can even be sorted in linear time, but with demands on memory that are exponential in the number of bits used to store an integer (Andersson et al. 1998).
However, agent-based modellers should not expect to be so lucky. There are various features that ABMs may implement that make their computing resources difficult (perhaps impossible) to predict:
Birth and death of agents can render computing time and memory requirements difficult to predict. Indeed, the size of the population and any fluctuation in it may be the purpose of the simulation study. With each agent having memory needed to store its attributes, and execution time for its behaviour, if the maximum population size of a specific run is not predictable from its initial conditions and parameter settings without first running the model, then computing resources cannot be predicted for HPC job submission.
A more dramatic corollary of birth and death is the question of extinction – i.e. where all agents die before they can reproduce. At this point, a run would typically terminate – far sooner than the computing time budgeted.
Interactions among agents, where the set of other agents with which one agent interacts is not predetermined, will also typically result in unpredictable computing times, even if the time needed for any one interaction is known. In some cases, agents’ social networks may be formally represented using data structures (‘links’ in NetLogo), and if these connections can be created or destroyed as a result of the model’s dynamics, then the memory requirements will typically be unpredictable.
Memories of agents, where implemented, are most trivially stored in lists that may have arbitrary length. The algorithms implementing the agents’ behaviours that use their memories will have computing times that are a function of the list length at any one time. These lists may not have a predictable length (e.g. if the agent ‘forgets’ some memories) and hence their behavioural algorithms won’t have predictable execution time.
Gotts and Polhill (2010) have shown that running a specific model with larger spaces led to qualitatively different results than with smaller spaces. This suggests that smaller (personal) computers (such as desktops and laptops) cannot necessarily be used to accurately estimate execution times and memory requirements prior to submitting larger-scale simulations requiring resources only available on HPC clusters.
Worse, a job will typically comprise several runs in a ‘batch’ covering multiple parameter settings and/or initial conditions. Even if the maximum time and memory requirements of any of the runs in a batch were known, there is no guarantee that all of the other runs will use anything like as much. These matters combine to make agent-based modellers ‘antisocial’ users of HPC clusters where the ‘performance’ of the clusters’ users is measured by their ability to accurately predict resource requirements, or there isn’t an ‘accommodating’ relationship between the administrator and researcher. Further, the social environment in which researchers access these resources put early-career and female researchers at a potential systemic disadvantage
The main purpose of making these points is to lay down the foundations for more equitable access to HPC for social scientists, and provide tentative users of these facilities with the arguments they need to develop constructive working arrangements with cluster administrators for them to run their agent-based models on shared HPC equipment.
Acknowledgements
This work was supported by the Scottish Government Rural and Environment Science and Analytical Services Division (project reference JHI-C5-1)
References
Andersson, A., Hagerup, T., Nilsson, S. and Raman, R. (1998) Sorting in linear time? Journal of Computer and System Sciences57, 74-93. https://doi.org/10.1006/jcss.1998.1580
Blumenthal, R. (2021) Walking the curricular talk: a longitudinal study of computer science departmental course requirements. In Lu, B. and Smallwood, P. (eds.) The Journal of Computing Sciences in Colleges: Papers of the 30th Annual CCSC Rocky Mountain Conference, October 15th-16th, 2021, Utah Valley University (virtual), Orem, UT. Volume 37, Number 2, pp. 40-50.
Fire, M. and Guestrin, C. (2019) Over-optimization of academic publishing metrics: observing Goodhart’s Law in action. GigaScience8 (6), giz053. https://doi.org/10.1093/gigascience/giz053
Gotts, N. M. and Polhill, J. G. (2010) Size matters: large-scale replications of experiments with FEARLUS. Advances in Complex Systems13 (4), 453-467. https://doi.org/10.1142/S0219525910002670
Griesemer, J. (2020) Taking Goodhart’s Law meta: gaming, meta-gaming, and hacking academic performance metrics. In Kippmann, A. and Biagioli, M. (eds.) Gaming the Metrics: Misconduct and Manipulation in Academic Research. Cambridge, MA, USA: The MIT Press, pp. 77-87.
Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S. K., Huse, G., Huth, A., Jepsen, J. U., Jørgensen, C., Mooij, W. M., Müller, B., Pe’er, G., Piou, C., Railsback, S. F., Robbins, A. M., Robbins, M. M., Rossmanith, E., Rüger, N., Strand, E., Souissi, S., Stillman, R. A., Vabø, R., Visser, U. and DeAngelis, D. L. (2006) A standard protocol for describing individual-based and agent-based models. Ecological Modelling198, 115-126. https://doi.org/10.1016/j.ecolmodel.2006.04.023
(The) Joint Task Force on Computing Curricula, Association for Computing Machinery (ACM) IEEE Computer Society (2013) Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. https://doi.org/10.1145/2534860
Kolsaker, A. (2008) Academic professionalism in the managerialist era: a study of English universities. Studies in Higher Education33 (5), 513-525. https://doi.org/10.1080/03075070802372885
Polhill, J. G., Hare, M., Bauermann, T., Anzola, D., Palmer, E., Salt, D. and Antosz, P. (2021) Using agent-based models for prediction in complex and wicked systems. Journal of Artificial Societies and Social Simulation24 (3), 2. https://doi.org/10.18564/jasss.4597
Rice, H. G. (1953) Classes of recursively enumerable sets and their decision problems. Transactions of the American Mathematical Society74, 358-366. https://doi.org/10.1090/S0002-9947-1953-0053041-6
Steinþórsdóttir, F. S., Brorsen Smidt, T., Pétursdóttir, G. M., Einarsdóttir, Þ, and Le Feuvre, N. (2019) New managerialism in the academy: gender bias and precarity. Gender, Work & Organization26 (2), 124-139. https://doi.org/10.1111/gwao.12286
Sun, Z., Lorscheid, I., Millington, J. D., Lauf, S., Magliocca, N. R., Groeneveld, J., Balbi, S., Nolzen, H., Müller, B., Schulze, J. and Buchmann, C. M. (2016) Simple or complicated agent-based models? A complicated issue. Environmental Modelling & Software86, 56-67. https://doi.org/10.1016/j.envsoft.2016.09.006
Turing, A. M. (1937) On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Societys2-42 (1), 230-265. https://doi.org/10.1112/plms/s2-42.1.230
Yoo, A. B., Jette, M. A. and Grondona, M. (2003) SLURM: Simple Linux utility for resource management. In Feitelson, D., Rudolph, L. and Schwiegelshohn, U. (eds.) Job Scheduling Strategies for Parallel Processing. 9th International Workshop, JSSPP 2003, Seattle, WA, USA, June 2003, Revised Papers. Lecture Notes in Computer Science2862, pp. 44-60. Berlin, Germany: Springer. https://doi.org/10.1007/10968987_3
Polhill, G. (2022) Antisocial simulation: using shared high-performance computing clusters to run agent-based models. Review of Artificial Societies and Social Simulation, 14 Dec 2022. https://rofasss.org/2022/12/14/antisoc-sim
By Nanda Wijermans, Geeske Scholz, Rocco Paolillo, Tobias Schröder, Emile Chappin, Tony Craig, and Anne Templeton
Introduction
Understanding how individual or group behaviour are influenced by the presence of others is something both social psychology and agent-based social simulation are concerned with. However, there is only limited overlap between these two research communities, which becomes clear when terms such as “variable”, “prediction”, or “model” come into play, and we build on their different meanings. This situation challenges us when working together, since it complicates the uptake of relevant work from each community and thus hampers the potential impact that we could have when joining forces.
We[1] – a group of social psychologists and social simulation modellers – sought to clarify the meaning of models and modelling from an interdisciplinary perspective involving these two communities. This occurred while starting our collaboration to formalise ‘social identity approaches’ (SIA). It was part of our journey to learn how to communicate and understand each other’s work, insights, and arguments during our discussions.
We present a summary of our reflections on what we learned from and with each other in this paper, which we intend to be part of a conversation, complementary to existing readings on ABM and social psychology (e.g., Lorenz, Neumann, & Schröder, 2021; Smaldino, 2020; Smith & Conrey, 2007). Complementary, because one comes to understand things differently when engaging directly in conversation with people from other communities, and we hope to extend this from our network to the wider social simulation community.
What are variable- and agent-based models?
We started the discussion by describing to each other what we mean when we talk about “a model” and distinguishing between models in the two communities as variable-based models in social psychology and agent-based modelling in social simulation.
Models in social psychology generally come in two interrelated variants. Theoretical models, usually stated verbally and typically visualised with box-and-arrow diagrams as in Figure 1 (left), reflect assumptions of causal (but also correlational) relations between a limited number of variables. Statistical models are often based in theory and fitted to empirical data to test how well the explanatory variables predict the dependent variables, following the causal assumptions of the corresponding theoretical model. We therefore refer to social-psychological models as variable-based models (VBM). Core concepts are prediction and effect size. A prediction formulates whether one variable or combination of more variables causes an effect on an outcome variable. The effect size is the result of testing a prediction by indicating the strength of that effect, usually in statistical terms, the magnitude of variance explained by a statistical model.
It is good to realise that many social psychologists strive for a methodological gold standard using controlled behavioural experiments. Ideally, one predicts data patterns based on a theoretical model, which is then tested with data. However, observations of the real world are often messier. Inductive post hoc explanations emerge when empirical findings are unexpected or inconclusive. The discovery that much experimental work is not replicable has led to substantial efforts to increase the rigour of the methods, e.g., through the preregistration of experiments (Eberlen, Scholz & Gagliolo, 2017).
Models in Social Simulation come in different forms – agent-based models, mathematical models, microsimulations, system dynamic models etc – however here we focus on agent-based modelling as it is the dominant modelling approach within our SIAM network. Agent-based models reflect heterogeneous and autonomous entities (agents) that interact with each other and their environments over time (Conte & Paolucci, 2014; Gilbert & Troitzsch, 2005). Relationships between variables in ABMs need to be stated formally (equations or logical statements) in order to implement theoretical/empirical assumptions in a way that is understandable by a computer. An agent-based model can reflect assumptions about causal relations between as many variables as the modeller (team) intends to represent. Agent-based models are often used to help understand[2]why and how observed (macro) patterns arise by investigating the (micro/meso) processes underlying them (see Fig 1, right).
The extent to which social simulation models relate to data ranges from ‘no data used whatsoever’ to ‘fitting every variable value’ to empirical data. Put differently, the way one uses data does not define the approach. Note that assumptions based on theory and/or empirical observations do not suffice but require additional assumptions to make the model run.
Fig. 1: Visualisation of what a variable-based model in social psychology is (left) and what an agent-based model in social simulation is (right).
Comparing models
The discussion then moved from describing the meaning of “a model” to comparing similarities and differences between the concepts and approaches, but also what seems similar but is not…
Similar. The core commonalities of models in social psychology (VBM) and agent-based social simulation (ABM) are 1) the use of models to specify, test and/or explore (causal) relations between variables and 2) the ability to perform systematic experiments, surveys, or observations for testing the model against the real world. This means that words like ‘experimental design’, ‘dependent, independent and control variables’ have the same meaning. At the same time some aspects that are similar are labelled differently. For instance, the effect size in VBMs reflects the magnitude of the effect one can observe. In ABMs the analogy would be the sensitivity analysis, where one tests for the importance or role of certain variables on the emerging patterns in the simulation outcomes.
False Friends. There are several concepts that are given similar labels, but have different meanings. These are particularly important to be aware of in interdisciplinary settings as they can present “false friends”. The false friends we unpacked in our conversations are the following:
Model: whether the model is variable-based in social psychology (VBM) or agent-based in social simulation (ABM). The VBM focuses on the relation between two or a few variables typically in one snapshot of time, whereas the ABM focuses on the causal relations (mechanisms/processes) between (entities (agents) containing a number of) variables and simulates the resulting interactions over time.
Prediction: in VBMs a prediction is a variable-level claim, stating the expected magnitude of a relation between two or few variables. In ABMs prediction would instead be a claim about the future real-world system-level developments on the basis of observed phenomena in the simulation outcomes. In case such prediction is not the model purpose (which is likely), each future simulated system state is sometimes labelled nevertheless as a prediction, though it doesn’t mean to be necessarily accurate as a prediction to the real-world future. Instead, it can for example be a full explanation of the mechanisms required to replicate the particular phenomenon or a possible trajectory of which reality is just one.
Variable: here both types of models have variables (a label of some ‘thing’ that can have a certain ‘value’). In ABMs there can be many variables, some that have the same function as the variables in VBM (i.e., denoting a core concept and its value). Additionally, ABMs also have (many) variables to make things work.
Effect size: in VBM the magnitude of how much the independent variable can explain a dependent variable. In ABM the analogy would be sensitivity analysis, to determine the extent to which simulation outcomes are sensitive to changes in input settings. Note that, while effect size is critical in VBMs, in ABMs small effect sizes in micro interactions can lead toward large effects on the macro level.
Testing: VBMs usually test models using some form of hypothesis testing, whereas ABMs can be tested in very different ways (see David et al (2019)), depending on the purpose they have (e.g., explanation, theoretical exposition, prediction, see Edmonds et al. (2019)), and on different levels. For instance, testing can relate to the verification of the implementation of the model (software development specific), to make sure the model behaves as designed. However, testing can also relate to validation – checking whether the model lives up to its purpose – for instance testing the results produced by the ABM against real data if the aim is prediction of the real world-state.
Internal validity: in VBM this is to assure the causal relation between variables and their effect size. In ABMs it refers to the plausibility in assumptions and causal relations used in the model (design), e.g., by basing these on expert knowledge, empirical insights, or theory rather than on the modeller’s intuition only.
Differences. There are several differences when it comes to VBM and ABM. Firstly, there is a difference in what a model should replicate, i.e., the target of the model: in social psychology the focus tends to be on the relations between variables underlying behaviour, whereas in ABM it is usually on the macro-level patterns/structures that emerge. Also, the concept of causality differs in psychology, VBM models are predominantly built under the assumption of linear causality[3], with statistical models aiming to quantify the change in the dependent variable due to (associated) change in the independent variable. A causality or correlation often derived with “snapshot data”, i.e., one moment in time and one level of analysis. In ABMs, on the other hand, causality appears as a chain of causal relations that occur over time. Moreover, it can be non-linear (including multicausality, nonlinearity, feedback loops and/or amplifications of models’ outcomes). Lastly, the underlying philosophy can differ tremendously concerning the number of variables that are taken into consideration. By design, in social psychology one seeks to isolate the effects of variables, maintaining a high level of control to be confident about the effect of independent variables or the associations between variables. For example, by introducing control variables in regression models or assuring random allocation of participants in isolated experimental conditions. Whereas in ABMs, there are different approaches/preferences: KISS versus KIDS (Edmonds & Moss, 2004). KISS (Keep It Simple Stupid) advocates for keeping it simple as possible: only complexify if the simple model is not adequate. KIDS (Keep It Descriptive Stupid), on the other end of the spectrum, embraces complexity by relating to the target phenomenon as much as one can and only simplify when evidence justifies it. Either way, the idea of control in ABM is to avoid an explosion of complexity that impedes the understanding of the model, that can lead to e.g., causes misleading interpretations of emergent outcomes due to meaningless artefacts.
We summarise some core take-aways from our comparison discussions in Table 1.
Table 1. Comparing models in social psychology and agent-based social simulation
Social psychology (VBM)
Social Simulation (ABM)
Aim
Theory development and prediction (variable level)
Not predefined. Can vary widely purpose. (system level)
Model target
Replicate and test relations between variables
Reproduce and/or explain a social phenomenon – the macro level pattern
Composed of
Variables and relations between them
Agents, environment & interactions
Strive for
High control, (low number of variables and relations Replication
Purpose-dependent. Model complexity: represent what is needed, not more, not less.
Testing
Hypotheses testing using statistics, including possible measuring the effect size a relation to assess confidence in the variable’s importance’
Purpose-dependent. Can refer to verification, validation, sensitivity analysis or all of them. See text and refs under false friends.
Causality
(or correlation) between variables Linear representation
Between variables and/or model entities. Non-linear representation
Theory development
Critical reflection on theory through confirmation. Through hypothesis testing (a prediction) theory gets validated or (if not confirmed) input for reconsideration of the theory.
IFF aim of model, ways of doing is not predefined. It can be reproducing the theory prediction with or without internal validity. ABMs can further help to identify gaps in existing theory.
Dynamism
Little – often within snapshot causality
Core – within snapshot and over time causality
External validity(the ability to say something about the actual target/ empirical phenomenon)
VBM aims at generalisation and has predictive value for the phenomenon in focus. VBMs in lab experiments are often criticised for their weak external validity, considered high for field experiments.
ABMs insights are about the model, not directly about the real world. Without making predictive claims, they often do aim to say something about the real world.
Beyond blind spots, towards complementary powers
We shared the result of our discussions, the (seemingly) communalities and differences between models in social psychology and agent-based social simulation. We allowed for a peek into the content of our interdisciplinary journey as we invested time, allowed for trust to grow, and engaged in open communication. All of this was needed in the attempt to uncover conflicting ways of seeing and studying the social identity approach (SIA). This investment was crucial to be able to make progress in formalising SIA in ways that enable for deeper insights – formalisations that are in line with SIA theories, but also to push the frontiers in SIA theory. Joining forces allows for deeper insights, as VBM and ABM complement and challenge each other, thereby advancing the frontiers in ways that cannot be achieved individually (Eberlen, Scholz & Gagliolo, 2017; Wijermans et al. 2022,). SIA social psychologists bring to the table the deep understanding of the many facets of SIA theories and can engage in the negotiation dance of the formalisation process adding crucial understanding of the theories, placed in their theoretical context. Social psychology in general can point to empirically supported causal relations between variables, and thereby increase the realism of the assumptions of agents (Jager, 2017; Templeton & Neville 2020). Agent-based social simulation, on the other hand, pushes for over-time causality representation, bringing to light (logical) gaps of a theory and providing explicitness and thereby adding to the development of testable (extended) forms of (parts of) a theory, including the execution of those experiments that are hard or impossible in controlled experiments. We thus started our journey, hoping to shed some light on blind spots and releasing our complementary powers in the formalisation of SIA.
To conclude, we felt that having a conversation together led to a qualitatively different understanding than would have been the case had we all ‘just’ reading informative papers. These conversations reflect a collaborative research process (Schlüter et al. 2019). In this RofASSS paper, we strive for widening this conversation to the social simulation community, connecting with others about our thoughts as well as hearing your experiences, thoughts and learnings while being on an interdisciplinary journey with minds shaped by variable-based or agent-based models, or both.
Acknowledgements
The many conversations we had in this stimulating scientific network since 2020 were funded by the the Deutsche Forschungsgemeinschaft (DFG- 432516175)
References
Conte, R., & Paolucci, M. (2014). On agent-based modeling and computational social science. Frontiers in psychology, 5, 668. DOI:10.3389/fpsyg.2014.00668
David, N., Fachada, N., & Rosa, A. C. (2017). Verifying and validating simulations. In Simulating social complexity (pp. 173-204). Springer, Cham. DOI:10.1007/978-3-319-66948-9_9
Eberlen, J., Scholz, G., & Gagliolo, M. (2017). Simulate this! An introduction to agent-based models and their power to improve your research practice. International Review of Social Psychology, 30(1). DOI:10.5334/irsp.115/
Edmonds, B., & Moss, S. (2004). From KISS to KIDS–an ‘anti-simplistic’modelling approach. In International workshop on multi-agent systems and agent-based simulation (pp. 130-144). Springer, Berlin, Heidelberg. DOI:10.1007/978-3-540-32243-6_11
Edmonds, B., Le Page, C., Bithell, M., Chattoe-Brown, E., Grimm, V., Meyer, R., Montañola-Sales, C., Ormerod, P., Root, H. and Squazzoni, F. (2019) ‘Different Modelling Purposes’ Journal of Artificial Societies and Social Simulation 22 (3) 6 <http://jasss.soc.surrey.ac.uk/22/3/6.html>. doi: 10.18564/jasss.3993
Gilbert, N., & Troitzsch, K. (2005). Simulation for the social scientist. McGraw-Hill Education (UK).
Jager, W. (2017). Enhancing the realism of simulation (EROS): On implementing and developing psychological theory in social simulation. Journal of Artificial Societies and Social Simulation, 20(3). https://jasss.soc.surrey.ac.uk/20/3/14.html
Lorenz, J., Neumann, M., & Schröder, T. (2021). Individual attitude change and societal dynamics: Computational experiments with psychological theories. Psychological Review, 128(4), 623-642. https://doi.org/10.1037/rev0000291
Schlüter, M., Orach, K., Lindkvist, E., Martin, R., Wijermans, N., Bodin, Ö., & Boonstra, W. J. (2019). Toward a methodology for explaining and theorizing about social-ecological phenomena. Current Opinion in Environmental Sustainability, 39, 44-53. DOI:10.1016/j.cosust.2019.06.011
Smith, E.R. & Conrey, F.R. (2007): Agent-based modeling: a new approach for theory building in social psychology. Pers Soc Psychol Rev, 11:87-104. DOI:10.1177/1088868306294789
Templeton, A., & Neville, F. (2020). Modeling collective behaviour: insights and applications from crowd psychology. In Crowd Dynamics, Volume 2 (pp. 55-81). Birkhäuser, Cham. DOI:10.1007/978-3-030-50450-2_4
Wijermans, N., Schill, C., Lindahl, T., & Schlüter, M. (2022). Combining approaches: Looking behind the scenes of integrating multiple types of evidence from controlled behavioural experiments through agent-based modelling. International Journal of Social Research Methodology, 1-13. DOI:10.1080/13645579.2022.2050120
Notes
[1] Most VBMs are linear (or multilevel linear models), but not all. In the case of non-normally distributed data changes the tests that are used.
[2] We are researchers keen to use, extend, and test the social identity approach (SIA) using agent-based modelling. We started from interdisciplinary DFG network project (SIAM: Social Identity in Agent-based Models, https://www.siam-network.online/) and now form a continuous special-interest group at the European Social Simulation Association (ESSA) http://www.essa.eu.org/.
[3] ABMs can cater to diverse purposes, e.g., description, explanation, prediction, theoretical exploration, illustration, etc. (Edmonds et al., 2019).
Wijermans, N., Scholz, G., Paolillo, R., Schröder, T., Chappin, E., Craig, T. and Templeton, A. (2022) Models in Social Psychology and Agent-Based Social simulation - an interdisciplinary conversation on similarities and differences. Review of Artificial Societies and Social Simulation, 4 Oct 2022. https://rofasss.org/2022/10/04/models-in-spabss/
The Journal of Artificial Societies and Social Simulation (hereafter JASSS) retains a distinctive position amongst journals publishing articles on social simulation and Agent-Based Modelling. Many journals have published a few Agent-Based Models, some have published quite a few but it is hard to name any other journal that predominantly does this and has consistently done so over two decades. Using Web of Science on 25.07.22, there are 5540 hits including the search term <“agent-based model”> anywhere in their text. JASSS does indeed have the most of any single journal with 268 hits (5% of the total to the nearest integer). The basic search returns about 200 distinct journals and about half of these have 10 hits or less. Since this search is arranged by hit count, this means that the unlisted journals have even fewer hits than those listed i. e. less than 7 per journal. This supports the claim that the great majority of journals have very limited engagement with Agent-Based Modelling. Note that the point here is to evidence tendencies effectively and not to claim that this specific search term tells us the precise relative frequency of articles on the subject of Agent-Based Modelling in different journals.
This being so, it seems reasonable – and desirable for other practical reasons like being entirely open access, online and readily searchable – to use JASSS as a sample – though clearly not necessarily a representative sample – of what may be happening in Agent-Based Modelling more generally. This is the case study approach (Yin 2009) where smaller samples may be practically unavoidable to discuss richer or more complex phenomena like the actual structures of arguments rather than something quantitative like, say, the number of sources cited by each article.
This piece is motivated by the scepticism that some reviewers have displayed about such a case study approach focused on JASSS and conclusions drawn from it. It is actually quite strange to have the editors and reviewers of a journal argue against its ability to tell us anything useful about wider Agent-Based Modelling research even as a starting point (particularly since this approach has been used in articles previously published in the journal, see for example, Meyer et al. 2009 and Hauke et al. 2017). Of course, it is a given that different journals have unique editorial policies, distinct reviewer pools and so on. Though this may mean, for example, that journals only irregularly publishing Agent-Based Models are actually less typical because it is more arbitrary who reviews for them and there may therefore be less reviewing skill and consensus about the value of articles involved. Anecdotally, I have found this to be true in medical journals where excellent articles rub shoulders with much more problematic ones in a small overall pool. The point of my argument is not to claim that JASSS can really stand in for ABM research as a whole – which it plainly cannot – but that, if the case study approach is to be accepted at all, JASSS is one of the few journals that successfully qualifies for it on empirically justifiable grounds. Conversely, given the potentially distinctive character of journals and the wide spread of Agent-Based Modelling, attempts at representative sampling may be very challenging in resource terms.
Method and Results
Again, using Web of Science on 04.07.22, I searched for the most highly cited articles containing the string “opinion dynamics”. I am well aware that this will not capture all articles that actually have opinion dynamics as their subject matter but this is not the intention. The intention is to describe a reproducible and measurable procedure correlated with the importance of articles so my results can be checked, criticised and extended. Comparing results based on other search terms would be part of that process. Then I took the first ten distinct journals that could be identified from this set of articles in order of citation count. The idea here was to see what journals had published the most important articles in the field overall – at least as identified by this particular search term – and then follow up their coverage of opinion dynamics generally. In addition, for each journal, I accessed the top 50 most cited articles and then checked how many articles containing the string “opinion dynamics” featured in that top 50. The idea here was to assess the extent to which opinion dynamics articles were important to the impact of a particular journal. Table 1 shows the results of this analysis.
Journal Title
“opinion dynamics” Articles in the Top 50 Most Cited
Most Highly Cited “opinion dynamics” Article Citations
Number of Articles Containing the String “opinion dynamics”
Reviews of Modern Physics
0
2380
1
JASSS
6
1616
64
International Journal of Modern Physics C
4
376
72
Dynamic Games and Applications
1
338
5
Physical Review Letters
0
325
5
Global Challenges
1
272
1
IEEE Transactions on Automatic Control
0
269
38
SIAM Review
0
258
2
Central European Journal of Operations Research
1
241
1
Physica A: Statistical Mechanics and Its Applications
0
231
143
Table 1. The Coverage, Commitment and Importance of Different Journals in Regard to “opinion dynamics”: Top Ten by Citation Count of Most Influential Article.
This list attempts to provide two somewhat separate assessments of a journal with regard to “opinion dynamics”. The first is whether it has a substantial body of articles on the topic: Coverage. The second is whether, by the citation levels of the journal generally, “opinion dynamics” models are important to it: Commitment. These journals have been selected on a third dimension, their ability to contribute at least one very influential article to the literature as a whole: Importance.
The resulting patterns are interesting in several ways. Firstly, JASSS appears unique in this sample in being a clearly social science journal rather than a physical science journal or one dealing with instrumental problems like operations research or automatic control. It is an interesting corollary how many “opinion dynamics” models in a physics journal will have been reviewed by social scientists or modellers with a social science orientation at least. This is part of a wider question about whether, for example, physics journals are mainly interested in these models as formal systems rather than as having likely application to real societies. Secondly, 3 journals out of 10 have only a single “opinion dynamics” article – and a further journal has only 2 – which are nonetheless, extremely highly cited relative to such articles as a whole. It is unclear whether this “only one but what a one” pattern has any wider significance. It should also be noted that the most highly cited article in JASSS is four times more highly cited than the next most cited. Only 4 of these journals out of 10 could really be said to have a usable sample of such articles for case study analysis. Thirdly, only 2 journals out of 10 have a significant number of articles sufficiently important that they appear in the top 50 most cited and 5 journals have no “opinion dynamics” articles in their top 50 most cited at all. This makes the point that a journal can have good coverage of the topic and contain at least one highly cited article without “opinion dynamics” necessarily being a commitment of the journal.
Thus it seems that to be a journal contributing at least one influential article to the field as a whole, to have several articles that are amongst the most cited by that journal and to have a non-trivial number of articles overall is unusual. Only one other journal in the top 10 meets all three criteria (International Journal of Physics C). This result is corroborated in Table 2 which carries out the same analysis for all additional journals containing at least one highly cited “opinion dynamics” article (with an arbitrary cut off of at least 100 citations for that article). There prove to be fourteen such journals in addition to the ten above.
Journal Title
“opinion dynamics” Articles in the Top 50 Most Cited
Most Highly Cited “opinion dynamics” Article Citations
Number of Articles Containing the String “opinion dynamics”
Mathematics of Operations Research
1
215
2
Information Sciences
0
186
14
Physica D: Nonlinear Phenomena
0
182
4
Journal of Complex Networks
1
177
5
Annual Reviews in Control
2
165
4
Information Fusion
0
154
11
IEEE Transactions on Control of Network Systems
3
151
12
Automatica
0
141
32
Public Opinion Quarterly
0
132
5
Physical Review E
0
129
74
SIAM Journal on Control and Optimization
0
127
13
Europhysics Letters
0
116
3
Knowledge-Based Systems
0
112
5
Scientific Reports
0
111
26
Table 2. The Coverage, Commitment and Importance of Different Journals in Regard to “opinion dynamics”: All Remaining Distinct Journals whose most important “opinion dynamics” article receives at least 100 citations.
Table 2 confirms the dominance of physical science journals and those solving instrumental problems as opposed to those evidently dealing with the social sciences: A few terms like complex networks are ambivalent in this regard however. Further it confirms the scarcity of journals that simultaneously contribute at least one influential article to the wider field, have a sensibly sized sample of articles on this topic – so that provisional but nonetheless empirical hypotheses might be derived from a case study – and have “opinion dynamics” articles in their top 50 most cited articles as a sign of the importance of the topic to the journal and its readers. To some extent, however, the latter confirmation is an unavoidable artefact of the sampling strategy. As the most cited article becomes less highly cited, the chance it will appear in the top 50 most cited for a particular journal will almost certainly fall unless the journal is very new or generally not highly cited.
As a third independent check, I again used Web of Science to identify all journals which had – somewhat arbitrarily – at least 30 articles on “opinion dynamics”, giving some sense of their contribution. Only two more journals (see Table 3) not already occurring in the two tables above were identified. Generally, this analysis considers only journal articles and not conference proceedings and book chapter serials whose peer review status is less clear/comparable.
Journal Title
“opinion dynamics” Articles in the Top 50 Most Cited
Most Highly Cited “opinion dynamics” Article Citations
Number of Articles Containing the String “opinion dynamics”
Advances in Complex Systems
5
54
42
Plos One
0
53
32
Table 3. The Coverage, Commitment and Importance of Different Journals: All Journals with at Least 30 “opinion dynamics” hits not already listed in Tables 1 and 2.
This cross check shows that while the additional journals do have sample of articles large enough to form the basis for a case study, they either have not yet contributed a really influential article to the wider field – less than half the number of citations of the journals which qualify for Tables 1 and 2, do not have a high commitment to opinion dynamics – in terms of impact within the journal and among its readers – or both.
Before concluding this analysis, it is worth briefly reflecting on what these three criteria jointly tell us – though other criteria could also be used in further research. By sampling on highly cited articles we focus on journals that have managed to go beyond their core readership and influence the field as a whole. There is a danger that journals that have never done this are merely “talking to themselves” and may therefore form a less effective basis for a case study speaking to the field as a whole. By attending to the number of articles in the top 50 for the journal, we get a sense of whether the topic is central (or only peripheral) to that journal/its readership and, again, journals where the topic is central stand a chance of being better case studies than those where it is peripheral. The criteria for having enough articles is simply a practical one for conducting a meaningful case study. Researchers using different methods may disagree about how many instances you need to draw useful conclusions but there is general agreement that it is more than one!
Analysis and Conclusions
The present article was motivated by an attempt to evaluate the claim that JASSS may be parochial and therefore not constitute a suitable basis for provisional hypotheses generated by case study analysis of its articles. Although the argument presented here is clearly rough and ready – and could be improved on by subsequent researchers – it does not appear to support this claim. JASSS actually seems to be one of very few journals – arguably the only social science one – that simultaneously has made at least one really influential contribution to the wider field of opinion dynamics, has a large enough number of articles on the topic for plausible generalisation and has quite a few such articles in its top 50, which shows the importance of the topic to the journal and its wider readership. Unless one wishes to reject case study analysis altogether, there are – in fact – very few other journals on which it can effectively be done for this topic.
But actually, my main conclusion is a wider reflection on peer reviewing, sampling and scientific progress based on reviewer resistance to the case study approach. There are 1386 articles with the search term “opinion dynamics” in Web of Science as of 25.07.22. It is clearly not realistic for one article – or even one book – to analyse all that content, particularly qualitatively. This being so we have to consider what is practical and adequate to generate hypotheses suitable for publication and further development of research along these lines. Case studies of single journals are not the only strategy but do have a recognised academic tradition in methodology (Brown 2008). We could sample randomly from the population of articles but I have never yet seen such a qualitative analysis based on sampling and it is not clear whether it would be any better received by potential reviewers. (In particular, with many journals each having only a few examples of Agent-Based Models, realistically low sampling rates would leave many journals unrepresented altogether which would be a problem if they had distinctive approaches.) Most journals – including JASSS – have word limits and this restricts how much you can report. Qualitative analysis is more drawn-out than quantitative analysis which limits this research style further in terms of practical sample sizes. Both reading whole articles for analysis and writing up the resulting conclusions takes more resources of time and word count. As long as one does not claim that a qualitative analysis from JASSS can stand for all Agent-Based Modelling – but is merely a properly grounded hypothesis for further investigation – and shows ones working properly to support that further investigation, it isn’t really clear why that shouldn’t be sufficient for publication. Particularly as I have now shown that JASSS isn’t notably parochial along several potentially relevant dimensions. If a reviewer merely conjectures that your results won’t generalise, isn’t the burden of proof then on them to do the corresponding analysis and publish it? Otherwise the danger is that we are setting conjecture against actual evidence – however imperfect – and this runs the risk of slowing scientific progress by favouring research compatible with traditionally approved perspectives in publication. It might be useful to revisit the everyday idea of burden of proof in assessing the arguments of reviewers. What does it take in terms of evidence and argument (rather than simply power) for a comment by a reviewer to scientifically require an answer? It is a commonplace that a disproved hypothesis is more valuable to science than a mere conjecture or something that cannot be proven one way or another. One reason for this is that scientific procedure illustrates methodological possibility as well as generating actual results. A sample from JASSS may not stand for all research but it shows how a conclusion might ultimately be reached for all research if the resources were available and the administrative constraints of academic publishing could be overcome.
As I have argued previously (Chattoe-Brown 2022), and has now been pleasingly illustrated (Keijzer 2022), this situation may create an important and distinctive role for RofASSS. It may be valuable to get hypotheses, particularly ones that potentially go against the prevailing wisdom, “out there” so they can subsequently be tested more rigorously rather than having to wait until the framer of the hypothesis can meet what may be a counsel of perfection from peer reviewers. Another issue with reviewing is a tendency to say what will not do rather than what will do. This rather the puts the author at the mercy of reviewers during the revision process. RofASSS can also be used to hive off “contextual” analyses – like this one regarding what it might mean for a journal to be parochial – so that they can be developed in outline for the general benefit of the Agent-Based Modelling community – rather than having to add length to specific articles depending on the tastes of particular reviewers.
Finally, as should be obvious, I have only suggested that JASSS is not parochial in regard to articles involving the string “opinion dynamics”. However, I have also illustrated how this kind of analysis could be done systematically for different topics to justify the claim that a particular journal can serve as a reasonable basis for a case study.
Acknowledgements
This analysis was funded by the project “Towards Realistic Computational Models Of Social Influence Dynamics” (ES/S015159/1) funded by ESRC via ORA Round 5.
References
Brown, Patricia Anne (2008) ‘A Review of the Literature on Case Study Research’, Canadian Journal for New Scholars in Education/Revue Canadienne des Jeunes Chercheures et Chercheurs en Éducation, 1(1), July, pp. 1-13, https://journalhosting.ucalgary.ca/index.php/cjnse/article/view/30395.
Chattoe-Brown, E. (2022) ‘If You Want to Be Cited, Don’t Validate Your Agent-Based Model: A Tentative Hypothesis Badly in Need of Refutation’, Review of Artificial Societies and Social Simulation, 1st Feb 2022. https://rofasss.org/2022/02/01/citing-od-models
Hauke, Jonas, Lorscheid, Iris and Meyer, Matthias (2017) ‘Recent Development of Social Simulation as Reflected in JASSS Between 2008 and 2014: A Citation and Co-Citation Analysis’, Journal of Artificial Societies and Social Simulation, 20(1), 5. https://www.jasss.org/20/1/5.html. doi:10.18564/jasss.3238
Meyer, Matthias, Lorscheid, Iris and Troitzsch, Klaus G. (2009) ‘The Development of Social Simulation as Reflected in the First Ten Years of JASSS: A Citation and Co-Citation Analysis’, Journal of Artificial Societies and Social Simulation, 12(4), 12,. https://www.jasss.org/12/4/12.html.
Yin, R. K. (2009) Case Study Research: Design and Methods, fourth edition (Thousand Oaks, CA: Sage).
Chattoe-Brown, E. (2022) Is The Journal of Artificial Societies and Social Simulation Parochial? What Might That Mean? Why Might It Matter? Review of Artificial Societies and Social Simulation, 10th Sept 2022. https://rofasss.org/2022/09/10/is-the-journal-of-artificial-societies-and-social-simulation-parochial-what-might-that-mean-why-might-it-matter/
1Escuela de Ciencias Empresariales, Universidad Católica del Norte, Coquimbo, Chile, and CESIMO, Universidad de Los Andes, Mérid.
2CEMISID, Universidad de Los Andes, Merida, Venezuela; GIDITIC, Universidad EAFIT, Medellin, Colombia; and Universidad de Alcala, Dpto. Automatica, Alcala de Henares, Spain.
Abstract.
This work suggests to complementarily use Multi-Fuzzy Cognitive Maps (MFCM) and Multi-agent Based Simulation (MABS) for social simulation studies, to overcome deficiencies of MABS for contextually understanding social systems, including difficulties for considering the historical and political domains of the systems, variation of social constructs such as goals and interest, as well as modeler’s perspective and assumptions. MFCM are a construction much closer than MABS to natural language and narratives, used to model systems appropriately conceptualized, with support of data and/or experts in the modeled domains. Diverse domains of interest can be included in a MFCM, permitting to incorporate the history and context of the system, explicitly represent and vary agents’ social constructs, as well as take into account modeling assumptions and perspectives. We briefly describe possible forms of complementarily use these modeling paradigms, and exemplifies the importance of the approach by considering its relevance to investigate othering and polarization.
1. Introduction
In order to understand better issues such as othering and polarization, there is a claim in social simulation for research that includes the important domains of history, politics and game of power, as well as for greater use of social science data, make more explicit and conscious about the models’ assumptions, and be more cautious in relation to the interpretation of the simulations’ results (Edmonds et al., 2020). We describe a possible form of dealing with these difficulties: combining Multi-Agent based Simulation (MABS) and Multi Fuzzy Cognitive Maps (MFCM) (or other forms of cognitive maps), suggesting new forms of dealing with complexity of social behavior. By using MFCM an alternative modeling perspective to MABS is introduced, which facilitates expressing the context of the model, and the modelers’ assumptions, as suggested in Terán (2004). We will consider as a case studying othering and polarization, given the difficulties for modeling it via MABS (Edmonds et al., 2020). Our proposal permits to explicitly represent social constructs such as goals, interest and influence of powerful actors on, e.g., people’s othering and polarization, and so in better contextualizing the simulated model. Variations of social constructs (e.g., goals, othering, polarization, interests) can be characterized and modeled by using MFCM.
Combined use of MABS and Fuzzy Cognitive Maps (FCM) (MFCM, multi FCM, are an extension of FCM, see the Annex) has already been suggested, see for example Giabbanelli (2017). MABS develop models at the micro level, while FCM and MFCM permits us to create models at the macro or contextual level; the idea is to use one to complement the other, i.e., to generate rich feedback between them and enhance the modeling process. Additionally, Giabbanelli propose FCM as a representation closer than MABS to natural language, allowing more participatory models, and better representation of the decision making process. Giabbanelli recommend forms of combining these two modeling approaches, highlighting key questions modelers must be careful about. In this line, we also propose a combined usage of a MFCM and MABS to overcome deficiencies of MABS modelling in Social Simulation.
Initially (in section 2) we offer a description of human societies from a broad view point, which recognizes their deep complexities and clarifies the need for better contextualizing simulation models, allowing modeling of diverse agents’ constructs, and making explicit modelers’ assumptions and perspectives. Afterwards (in section 3), we briefly review the drawbacks of MABS for modeling some of these deep complexities. Then (in section 4), MFCM are briefly described, supported on a brief technical account in the Annex. Following (in section 5), we suggest to complementarily use MABS and MFCM for having a more comprehensive representation of human societies and their context, e.g., to better model problems such as othering and polarization. MFCM will model context and give a conceptual mark for MABS (allowing to model variation of context, e.g., changes of agents’ interests or goals, making explicit modelers’ perspective and assumptions, among other advantages), which, in turn, can be used to explore in detail specific configurations or scenarios of interest suggested by the MFCM. Finally (in section 6), some conclusions are given.
2. (A wide view of) Human societies and influence of communication media on actual culture
As humans and primates, we recognise the social groups within which we develop as people (e.g., family, the community where we grow up, partners at the school or at work) as part of our “large home”, in which its members develop a common identity, with strong rational and emotional links. Other groups beyond these close ones are “naturally” estrangers for us and its members “instinctively” seen as others. In large civilizations such as western society, we extend somewhat these limits to include nations, in certain respects. In groups we develop perspectives, follow certain myths and rites, and have common interests, viewpoints about problems, solutions for these, and give meaning to our life. Traditionally, human societies evolve from within groups by direct face to face interaction of its members, with diverse perspectives, goals, interest, and any other social construct with respect to other groups. Nowadays this evolution mainly from natural interaction has been importantly altered in some societies, especially western and western influenced societies, where social media has introduced a new form of communication and grouping: virtual grouping. Virtual grouping consists in the creation of groups, either formally or informally, by using the internet, and social networks such as Facebook, Instagram, etc. In this process, we access certain media sites, while discarding others, in accordance with our preferences, which in turn depends on our way of thinking and preferences created in social, both virtual and direct (face to face), interaction. Currently, social media, and traditional media (TV, newspapers, etc.) have a strong influence on our culture, impacting on ours myths, rites, perspectives, forms of life, goals, interests, opinion, reasoning, emotions, and othering.
Characteristics of virtual communication, e.g., anonymity, significantly impacts on all these constructs, which, differently from natural interaction, have a strong potential to promote polarization, because of several reasons, e.g., given that virtual environments usually create less reflexive groups, and emotional communication is poorer or lack deepness. Virtual interaction is poorer than direct social interaction: the lack of physical contact strongly reduces our emotional and reflexive connection. Virtual social interaction is “colder” than direct social interaction; e.g., lack of visual contact stops communication of many emotions that are transmitted via gestures, and prevents the call for attention from the other that visual contact and gestures demands.
Even more, many times sources and veracity of information, comments, ideas, and whatever is in social media, are not clear. Even more, fake news are common in social media, what generate false beliefs, and behavior of people influenced and somewhat controlled by those who promote fake news. Fake news can in this sense generate polarisation, as some groups in the society prefer certain media, and other groups choose a different one. As these media may promote different perspectives following interest of powerful actors (e.g., political parties), conflicting perspectives are induced in the different groups, what in turn generates polarization. Social media are highly sensitive to manipulation by powerful actors worldwide, including governments (because of, e.g., their geopolitical interests and strategies), corporations (in accordance with their economic goals), religious groups, political parties, among many others. Different groups of interests influence in direct and indirect, visible and hidden, forms the media, following a wide diversity of strategies, e.g., those of business marketing, which are supported by knowledge of people (e.g., psychology, sociology, games theory, etc.). Thus, the media can create and contribute to create visions of the word, or perspectives, in accordance with the interest of powerful international or national actors. For more about all this, see, e.g, Terán and Aguilar (2018).
As a consequence, people following media that promotes a world view, related with some powerful actor(s) (e.g., a political party or a group of governments) virtually group around media that support this world view, while other people do the same in relation to other media and powerful actor(s), who promote(s) a different perspective, which many times is in conflict with the first one. Thus, grouping following the media sometimes promotes groups with conflicting perspectives, goals, interests, etc., which generates polarization. We can find examples of this in diverse regions and countries of the world. The media has important responsibility for polarization in a diversity of issues such as regional integration in Europe, war in Ukraine, migrations from Middle East or Africa to Europe, etc. Consequently, media manipulation sometimes allow powerful actors to influence and somewhat control perspectives and social behavior. Even more, the influence of social media on people is sometimes stronger than the influence of direct social interaction. All these introduce deep complex issues in social human interaction and behavior. This is why we have chosen polarization as the case study for his essay.
Consequently, to comprehend actual human behavior, and in particular polarization, it is necessary to appropriately take into account the social context, what permits to understand better the actual complexity of social interaction, e.g., how powerful international, national, and local actors’ influence on media affects people perspectives, goals, interest, and polarization, as well as their strategies and actions in doing so. Contextualized modeling will help in determining social constructs (goals, interests, etc.) in certain situations, and their variation from situation to situation. For this, we suggest complementing MABS with MFCM. For more about the consequences of virtual interaction, see for example Prensky (2001a, 2001b). Prensky (2009) has also suggested forms to overcome such consequences: to promote digital wisdom. MABS and MFCM models will help in defining forms of dealing with the problems of high exposure to social networks, in line with Prensky’s concerns.
3. Weakness of the MABS approach for modeling context
Edmonds et al. (2020) recognize that MABS models assume a “state of the world” or “state of nature” that does not include the historical context of the agent, e.g., in such a way that they explicitly present goals, interests, etc., and pursue them via political actions, sometimes exerting power over others. For instance, the agents can not change their goals, interests or desires during the simulation, to show certain evolution, as a consequence of reflection and experience at the level of desires, allowing cognitive variations. The models are strongly limited in relation to representing the context of the social interaction, which in part determines variation of important factors of agents’ behavior, e.g., goals. This, to a good extent, is due to lack of representation of the agents’ context. For the same reason, it is difficult to represent modelers assumptions and perspectives, which might also be influenced by social media and powerful actors, as explained above.
The Special Issue of the Social Science Computer Review. (Volume 38, Issue 4, August 2020, see for instance Edmonds et al. (2020) and Rocco and Wander (2020)), presents several models aiming at dealing with some of these drawbacks of MABS, specifically, to relate models to social science data, be more aware about the models’ assumptions, and be more cautious in relation to the interpretation of the simulations’ results. However, in these works diverse difficulties are not addressed, e.g., having appropriate representation of the context in order to explicitly consider diverse constructs, e.g., goals and interests, as well as having a wide representation of modelers perspectives and assumptions so that diverse perspectives can be addressed and compared, among other important matters.
MABS represent social interaction, i.e., the interaction in a group, where the agent’s goal, and other social constructs are assumed given, not variable, and to understand the context where they appear is not of interest or is out of reach (too difficult). However, as explained above, agents are in diverse social groups, not only in the simulated one, and so goals, interests, and beliefs in the modeled group are shaped in accordance to their interactions in diverse groups, and the influence of multiple, virtual and natural groups in which they participate. In order to represent variations of such elements, the context must be taken into account, as well as to elaborate models from narratives. MFCM is naturally close to narratives, as it is elaborated from conceptual frameworks. In this sense, MFCM might represent an intermediate step towards MABS models. In a MFCM and in the steps towards elaborating the MABS, modelers’ perspectives and assumptions can be made explicit. In addition, MABS presents limitations to determine the conditions for which a certain behavior or tendency occurs (Terán, 2001; Terán et al. 2001), i.e., for making strong inferences and theorem proving of tendencies for subsets of the theory of the simulation, which could more easily be performed in the MFCM. Hopefully, exploring configurations of the MFCM the proof could be carried out indirectly, in a higher level than in MABS, as has already been suggested in previous papers (Aguilar et al., 2020; Perozo et al., 2013).
4. Multi-Fuzzy Cognitive Maps (MFCM)
We suggest conceptual or cognitive maps as a more flexible form than MABS to represent context of a social situation, and in particular MFCM, as implemented by Aguilar and others (see e.g., Kosko, 1986; Aguilar 2005, 2013, 2016; Aguilar et al., 2016, 2020; Contreras and Aguilar, 2010; and Sánchez et al., 2019; Puerto et al., 2019). A brief description of Fuzzy and Multi-fuzzy cognitive maps, following Sánchez et al. (2019), is given in the Annex.
Fuzzy cognitive maps help us in describing the context via qualitative (e.g., very low, low, medium, high, too high) and quantitative variables, as indicated in the annex. The system is represented by the network of concepts (variables) interrelated via weights (also given by variables). The high level of the MFCM paradigm, differently from a MABS, permits us to explicit different elements of the models such as the agents constructs (goals, interests, etc.), as well as the modelers assumptions and perspectives, as suggested in Terán (2004). MFCM will facilitate to explicit the accumulated set of assumptions (“abstraction-assumptions, design-assumptions, inference-assumptions, analysis-assumptions, interpretation-assumptions and application-assumptions”, as these are summarized in Terán, 2004).
In a MFCM, a particular situation of the system is given by a specific configuration of the weights (see, e.g., Sánchez et al., 2019). Suppose we are dealing with a model similar to that elaborated in Sánchez et al. (2019) to study the quality of opinion in a community. Sánchez et al. examine the capabilities of the MFCM for knowledge description and extraction about opinions presented in a certain topic, allowing the assessment of the quality of public opinion. Special attention is offered to the influence of the media on public opinion. The evolution of the concepts and relationships is presented. Concepts define the relevant aspects from which public opinion emerges, covering diverse domains, for instance, the social, technological and psycho-biological ones. The MFCM permits to identify the media preferred by the public in order to better understand several issues, including the high esteem that the new communication media hold.
In line with this, let us assume that we want to understand the quality of public opinion in a community of Europe about diverse issues during 2022. This network, with a certain configuration of the weights, but unspecified concepts, represents a social system with a certain structure (as the weights are given) that is in some sense general, as the values of the concepts can still vary. Variation of the concepts represents different scenarios of the social system (with a given structure, defined by the weights); e.g., the model of a European community considered in relation to three scenarios regarding the state of public opinion in relation to specific issues: 1: climate change, 2: situation of tourism in the community, 3: secondary effects of the COVID-19 vaccines. The weights of the network are determined by using a variety of scenarios; i.e., the network is trained with several scenarios, for which all possible values of the concepts are known. Once the network is trained, it can be used to infer unknown specific values of the concepts for other scenarios (following Aguilar et al. 2020; Sánchez et al. 2019; Terán y Aguilar, 2018); e.g., the state of public opinion in relation to the involvement of EU in the war in Ukraine. Even more, by exploring an appropriate set of scenarios, proofs about the state of certain concepts can be developed; e.g., that a majority of people in the community is against direct EU involvement in the war in Ukraine. The proof could be carried out for a subset of the possible configurations of a domain, several domains, or part of a domain, e.g., for the psycho-biological domain. Additionally, having an appropriate elaboration of the model would allow evaluating how polarized is the opinion of the community in relation to the involvement of EU in that war.
Diverse configurations of the MFCM can represent different modelers’ perspectives and assumptions, as well as various agents’ constructs, such as goals, interests, etc., allowing to deal with the above described drawbacks of MABS to cope with complexity of social systems.
5. Combined use of MABS and cognitive maps
The combined usage will give at least two levels of modeling: the inner, defined by the agents’ interaction concreted in a MABS, and the outer or contextual one, given by the MFCM. These will be the two last levels in the description given in 5.1. Interaction between these models occurs as the modeler interprets each model outputs and feedbacks the other. Ideally, we would have direct automatic feedback between these models.
5.1 Levels of description of the System
In order to contextually model a social system and investigate problems such as polarization, we suggest below five levels of description of the system. The first three levels are not directly associated to computational models, while levels four and five are descriptions that assist development of the computational models: MFCM and MABS, respectively. Each level gives context to the following one (the first gives context to the second, etc.). A lower level (e.g., 1. in relation to 2.) of description corresponds to a more general language, as suggested in Terán (2004). Each level must take into account the previous levels, especially the immediately superior level, which gives the most immediate context to it. This description is in line with the suggestions given in Terán (Idem). Each description makes certain assumptions and is shaped by the modeler’s perspective, which in part is coming from those actors given information to build the model. Assumptions and perspectives introduced in level of modeling i, i = 1, …, 6, can be called Assumptions-given in (i) and Perspectives-given in (i). At levels of description j, Assumption i = 1, 2, …, i are accumulated, and can be called Assumptions(j), as well as holistic Perspective(j) based on Perspectives-given in (i), i = 1, 2, .., i. These assumptions and perspectives correspond to those defined in Terán (2004) as “abstraction-assumptions, design-assumptions, inference-assumptions, analysis-assumptions, interpretation-assumptions and application-assumptions”.
Describe in natural language the system, including its relevant history, with emphasis in culture (practices, costumes, etc.) and behavior of individuals and groups relevant to the object of the study, e.g., from a historical-ontological perspective. Here, how the system has reached its actual situation is explained. This will give a global view of the society and the general form of behavior, problematics, conflicts, etc.
Describe the diverse relevant domains given context to the system of interest in accordance with the study, e.g., political, economical, dominant actors, etc., and the relationships among them. Concrete specifications of these domains sets scenarios for the real system, i.e., possible configurations of it.
Describe the particular social group of interest as part of the society explained above, in 1., and the domains given in 2., showing its particularities, e.g., culturally, in terms of interests, situation and social interaction of this group in relation with other groups in the whole society, in accordance to the problematic addressed in the study.
Elaborate a cognitive map of the situation of the social group of interest, following the description given in 2 and 3. This is a description to be represented in a computational language, such as the MFCM tool developed by Contreras and Aguilar (2010).
Describe the MABS model. The MABS model is then represented in a simulation language.
The computational MFCM (or other cognitive map) and the MABS developed following 4. and 5. are then used to generate the virtual outputs and simulation study.
5.2 Possible combined uses of MFCM and MABS.
The MFCM (in general a cognitive or conceptual map) gives context to the MABS model (modeler’s assumptions and perspectives are added in the process, as indicated above), while the MABS model represents in detail the interaction of the agents’ considered in the MFCM for a specific scenario of this, as indicated in the levels of modeling given above. With this idea in mind, among the specific forms a combined usage of ABM and MFCM, we have:
i) Offering feedback from the MFCM to the MABS. A MABS in a certain configuration can be used to generate either directly or indirectly (e.g., with additional verification or manipulations) input for a simulation model. For example, in parallel to the model presented in Sánchez et al. (2019), where the domains social, biological-psychological, technological and state of the opinion are displayed, a MABS model can be developed to represent the interaction between social entities, such as people who receive information from the media, the media itself, and powerful actors who design the agenda setting of the media. This MABS model might use diverse methodologies, e.g., endorsements or BDI, to represent social interaction, or a higher level of interaction where actors share resources, on which they have interest, as in a System of Organized Action (see, e.g., SocLab models Terán and Sibertin-Blanc, 2020; Sibertin-Blanc et al., 2013). Constructs required as inputs for the model, e.g., goals, interests, values to define the endorsements schema, etc. could be deducted from the MFCM, as direct values of some concepts or functions (mathematical, logical, etc.) of the concepts. These operations could be defined by experts in the modeled domains (e.g., media owners, academics working in the area, etc.). Ideally, we would have an isomorphic relation between some variables of the MABS and variables of the MFCM – however, this is not the usual case –. In this process, the MABS is contextualized by the MFCM, whose modeling level also permits to identify modeler’s assumptions and perspectives. Also, in a narrative, and then in the MFCM, goal, interest, and other constructs can be explicitly represented, then varied and their consequence understood in order to feedback the MABS model.
ii) Giving feedback to the MFCM from the MABS. Inputs and outputs values of the MABS simulation can be used as an input to the MFCM, e.g., as a set of scenarios to train the network and determine a certain structure of the MFCM, or to determine a specific scenario/ configuration (where both, the weights and the concepts are known).
iii) Determining conditions of correspondence among the models. By simulating the MABS associated to certain scenarios of the MFCM, or, vice-verse, by determining the scenarios of MFCM related to certain MABS, the consistency among the two models and possible errors, omissions, etc. in one of the models can be detected, and then the corrections applied. Even more, this exercise can hopefully determine certain rules or conditions of correspondence among the MABS and the MFCM.
iv) Using a model to verify properties in the other model. Once certain correspondence among the models has been determined, we can use one of the models to help in determining properties of the other. For instance, a proof of a tendency in an MABS (this has been an important area of research, Terán, 2001; Terán et al. 2001; Edmonds et al., 2006) could be developed in a much easier form in the corresponding MFCM. For this, we need to characterize the set of configurations of the MFCM corresponding to the set of configurations of the MABS for which we want to perform the proof.
These possible combined uses of MFCM and MABS do not exhaust all potentials, and diverse other alternatives could appear in accordance with the needs in a particular study. Even more, automatic feedback between MFCM (or other cognitive or conceptual map) and MABS could be implemented in the future, to facilitate the mutual contributions between the two modeling approaches. This would cover modeling requirements the MABS in itself does not support at present.
5.3 A case: Modeling othering and polarization,the case of “children with virtually mediated culture”.
We outline a possible model that considers othering and polarization. In section 2 we described a society. In a society, as virtual groups become homogeneous in beliefs, motivations, intentions and behavior, certain sort of endogamy of ideas and opinions appear, constraining the variety and richness of perspectives from which people observe and judge others, making them generally less tolerant to others, more restrictive in accepting opinions and behavior of others, so less inclusive. This has diverse additional effects, for instance, increase of polarization between virtual groups regarding a diversity of themes. Problems such as polarization occur also in children with strong usage of virtual social networks (see, Prensky, 2001a, 2001b, 2009).
To investigate this problem and support the MABS, as a case, we suggest a MFCM with four levels (see Figure 1). The goal of the models (MFCM and MABS) would be to better understand the differences between the communities of children whose interaction is basically virtually mediated and the community of children whose interaction is face to face, or direct, people to people. In general, it is of interest to determine the state of othering and polarization for diverse configurations of the MFCM. As we explained above, there are clear differences between virtual and face to face interaction, consequently the upper layers in Figure 1 are the two possible niches of cultural acquisition (costumes, points of view, etc.), during life of people (ontogenesis), namely, the virtual mediated culture and the direct, face to face, cultural acquisition. These two layers involve interaction among diverse actors (e.g., people, media and powerful actors are present in layer 2). Layer 2 represents technological actors, while layer 1 represents social interaction, but both of them might involve other elements, if required. The third level represent those biological aspects related with behavior, which are created via culture: the psycho-biological level. Both levels, 1 and 2, affect the third layer, as emotions, reasoning, etc., are founded on people interaction and have a cultural base. Constructs of behavior such as goals, interests, desires, polarization, etc., appear and can be explicitly represented at this level. This third level, in turn, impacts on the overall state of the community, e.g., on the auto-generative capacity of the society, finally affecting global society/community’s othering and polarization, as our emotions, reasoning, etc., impact on our view point, on othering, etc. These last are variables defined in terms of the previous levels. In this model, the definition of concepts such as “othering” and “polarization” is crucial, and indicates basic modeling assumptions and perspective. Finally, the overall state of society/community impacts back on the cultural niches (layers 1 and 2).
In a specific situation, the whole interaction (1) of the society or community is divided between the two niches given by layers 1 and 2, a proportion of interaction frequency occurs as virtual communication, and the compliment (one minus the proportion of virtual interaction), occurs as direct, face to face, contact. This is represented in Figure 3 by the variable “Proportion of virtual interaction type”. Changes of this variable allows us to explore diverse configurations or scenarios of interaction, ranging from total virtual interaction (null direct contact) (the variable takes the value 1), to null virtual interaction (total direct contact) (the variable takes the value 0).
Figure 1. The four levels of the MFCM. The two alternative niches structuring the psycho-biology of people are at the top of the process. The overall state represents general measures such as the auto-generative character of a social system, and attitudes including othering and polarization.
Example of possible variables in some levels (Figure 1) are:
i) Face to face interaction, and ii) virtual interaction or technological: The next variables are candidates to be at these levels. Degree of:
Coherence of the interaction (possible state: good, etc.);
Identification of the others in the interaction (good or clear, etc.);
Richness of the interaction (high or good, etc.);
Truthfulness of the messages (fairness) (e.g., good: messages and communication are fair);
Openness of the community (e.g., high: usually people is open to interact with others);
Speed of the interaction (e.g.: low, medium, …);
Intentioned influence and control of the communication by powerful actors (e.g., high, medium, low, ..);
iii) Psycho-biological level:
Reflection (state: good means that people question their experiences, and observed phenomena);
Closeness of interpretations, attitudes, desires, intentions, and plans (a high value means that people’s interpretations, etc., are not very different);
Emotion and mood;
Empathy;
Addiction to virtual interaction;
Goal;
Interest;
Immediatism (propensity to do things quickly and constantly change focus of reasoning).
iv) Overall state of people and society:
Auto-generative capacity of the society;
Capacity of people to reflect about social situations (and autonomously look for solutions);
Othering;
Polarization;
Concepts at each layer impacts concepts at the other layers. E.g., concepts of level three have a strong impact on concepts of the fourth layer, such as “polarization”, and “auto-generative character of the society”.
As indicated above, to understand the dynamics of the MFCM we can develop a wide range of scenarios, for instance, varying the switch “Proportion of virtual interaction” in the interval [0, 1], to explore a set of scenarios for which the degree of virtual interaction increases from 1 to 0, as the proportion of direct interaction decreases from 1 to 0 (the real case corresponds to an intermediate value between 1 and 0). These experiments will help us in understanding better the consequences of virtual mediated culture. Even when the outline of the model presented here might need some adjustments and improvements, the present proposal keeps its potential to reach this goal.
The MFCM will be useful to deal with many issues and questions of interest, for instance:
How social networks affect basic social attitudes such as: i) critical rationality (people’s habit for questioning and explaining their experience (issues/phenomena in their life)), ii) tolerance, iii) compromise with public well-being, and iv) othering and polarization
How social networks affect social feelings, such as empathy.
The MABS model will be elaborated in accordance with the description of the MFCM indicated above. In particular, different values of the social constructs at levels 1 and 2 (e.g., goals and interests of the actors), and the corresponding state of layer 3 (e.g., Polarization), imply diverse MABS models.
The whole network of concepts, the particular network of concepts at each level, and the definition of each concept, offers a perspective of the modelers. Different modelers can develop these elements of the model differently. Assumptions can be identified also at each level. Both, perspectives and assumptions come from the modelers as well as from the theories, consult to experts to create the model, etc.. An specific model is not part of this essay, but rather a subject of future work.
6. Conclusion
Social simulation has been widely recognized as an alternative to study social systems, using diverse modeling tools, including MABS, which, however, present some limitations, like any other research tool. One of the deficiencies of MABS is their limitations to contextually modeling social systems; e.g., to suitably include the historical and political contexts or domains; difficulties to represent variation of agents’ constructs, e.g., goals and interest; and drawbacks to made explicit modeler’s assumptions and perspectives. In this paper, we have suggested to mutually complement MABS and MFCM, to overcome MABS drawbacks, to potentiate the usefulness of MABS to represent social systems.
We argue that the high level of the MFCM paradigm permits us to express different elements of the models such as the agents’ constructs (goals, interests, etc.), as well as the modelers assumptions and perspectives, as suggested in Terán (2004). Thus, MFCM facilitates the identification of the accumulated set of assumptions during the modeling process. Even more, diverse configurations of a MFCM can represent diverse modelers’ perspectives and assumptions, as well as diverse agents constructs, such as goals, interests, etc., allowing to deal with the above described complexity. This permits us to more realistically elaborate models of a wide diversity of social problems, e.g., polarization, and consequences of the influence of social networks in culture.
Among the forms MFCM and MABS complement each other we have identified the followings: mutual feedbacking of variables and concepts between the MFCM and the MABS, determining conditions of correspondence among the models, what facilitate other modeling needs, e.g., using a model to verify properties in the other model (e.g., proofs required in a MABS could be carried out in a corresponding MFCM).
A case study was outlined to exemplify the problematic that can be addressed and the advantages of using MFCM to complement MABS: Modeling othering and polarization, the case of children with virtually mediated culture. Combined use of MFCM and MABS in this case will contribute to understand better the problems created by the high use of digital interaction, especially social networks, as described by Prensky (2001a, 2001b, 2009), given that virtual interaction has strong influence on our culture, impacting on ours myths, rites, perspectives, goals, interests, opinion, othering and polarization, etc. Characteristics of virtual communication, e.g., anonymity, significantly impacts on all these constructs, which, differently from natural interaction, have a strong potential to promote certain tendencies of such constructs, e.g., polarization or our opinions, because of several reasons, for instance, given that virtual environments usually create less reflexive groups, while emotional communication is poorer or lack deepness. It is difficult to represent all these dynamics in a MABS, but it can be alternatively expressed in a MFCM.
The purpose of the work was to give an outline of the proposal; future work will be conducted to wholly develop concrete study cases with complementary MABS and MFCM models.
References
Aguilar, J. (2005). A Survey about Fuzzy Cognitive Maps Papers. International journal of computational cognition 3 (2), pp. 27-33.
Aguilar, J. (2013) Different Dynamic Causal Relationship Approaches for Cognitive Maps”, Applied Soft Computing, Elsevier, 13(1), pp. 271–282.
Aguilar, J. (2016) “Multilayer Cognitive Maps in the Resolution of Problems using the FCM Designer Tool”, Applied Artificial Intelligence, 30 (7), pp. 720-743.
Aguilar J., Hidalgo J., Osuna F., Perez N.(2016) Multilayer Cognitive Maps to Model Problems”, Proc. IEEE World Congress on Computational Intelligence, pp. 1547-1553, 2016.
Aguilar Jose, Yolmer Romero y OswaldoTerán (2020). “Analysis of the effect on the marketing of the sport product from the “par Conditio” principle in Latin American football and baseball competitions”. Submitted to International Journal of Knowledge-Based and Intelligent Engineering Systems.
Contreras, J. Aguilar, J. (2010). “The FCM Designer Tool”. Fuzzy Cognitive Maps: Advances in Theory, Methodologies, Tools and Application (Ed. M. Glykas), Springer, pp. 71-88, 2010.
Edmonds, B. and Hales, D. and Lessard-Phillips, L.(2020). Simulation Models of Ethnocentrism and Diversity: An Introduction to the Special Issue. Social Science Computer Review. Volume 38 Issue 4, August 2020, 359–364, https://journals.sagepub.com/toc/ssce/38/4
Edmonds Bruce, Oswaldo Terán y Grary Polhill (2006). “To the Outer Limits and Beyond –characterising the envelope of social simulation trajectories”, Proceedings of theThe First World Congress on Social Simulation WCSS”, Kyoto, 21-25 Agosto, 2006.
Giabbanelli Philippe, Steven Gray and Payam Aminpour (2017), Combining fuzzy cognitive maps with agent-based modeling: Frameworks and pitfalls of a powerful hybrid modeling approach to understand human-environment interactions, Environmental Modeling and Software, September 2017, 95:320-325 DOI:10.1016/j.envsoft.2017.06.040
Kosko, B. (1986). Fuzzy Cognitive Maps. International Journal of Man-Machine Studies 24, pp. 65-75.
Perozo Niriaska, Jose Aguilar, Oswaldo Terán y Heidi Molina (2013). A Verification Method for MASOES, IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, Vol 43, num 1, February, pp. 64-76, ISBN: 1083-4419. DOI: 10.1109/TSMCB.2012.2199106
Prensky, Marc (2001a). Digital Natives, Digital Immigrants Part 1, On the Horizon, Vol. 9, No. 5, September, pp 1-6. DOI: 10.1108/10748120110424816
Prensky, Marc (2001b). Do They Really Think Differently? Digital Natives, Digital Immigrants Part 2, On the Horizon, Vol. 9, No. 6, October 2001, pp 1-6. DOI: 10.1108/10748120110424843
Prensky, Marc (2009). “H. Sapiens Digital: From Digital Immigrants and Digital Natives to Digital Wisdom,” Innovate: Journal of Online Education: Vol. 5 : Issue 3, Article 1. Available at: https://nsuworks.nova.edu/innovate/vol5/iss3/1
Puerto E., Aguilar J., Chávez D., López C. (2019) Using Multilayer Fuzzy Cognitive Maps to Diagnose Autism Spectrum Disorder, Applied Soft Computing Journal, 75, pp. 58–71.
Rocco Paolillo, and Wander Jager (2020). Simulating Acculturation Dynamics Between Migrants and Locals in Relation to Network Formation, Social Science Computer Review. Volume 38 Issue 4, August 2020, pp. 365–386. https://journals.sagepub.com/toc/ssce/38/4
Sánchez Hebert, Jose Aguilar, Oswaldo Terán, José Gutiérrez de Mesa (2019). “Modeling the process of shaping the public opinion through Multilevel Fuzzy Cognitive Maps”, Applied Soft Computing, Volume 85. https://doi.org/10.1016/j.asoc.2019.105756.
Sibertin-Blanc, C., Roggero, P., Adreit, F., Baldet, B., Chapron, P., El-Gemayel, J., Mailliard, M., and Sandri, S. (2013). “SocLab: A Framework for the Modeling, Simulation and Analysis of Power in Social Organizations”, Journal of Artificial Societies and Social Simulation (JASSS), 16(4). http://jasss.soc.surrey.ac.uk/
Terán Oswaldo (2001). Emergent Tendencies in Multi-Agent Based Simulations Using Constraint-Based Methods to Effect Practical Proofs Over Finite Subsets of Simulation Outcomes, Doctoral Thesis, Centre for Policy Modelling, Manchester Metropolitan University, 2001.
Terán Oswaldo (2004). Understanding MABS and Social Simulation: Switching Between Languages in a Hierarchy of Levels, Journal of Artificial Societies and Social Simulation vol. 7, no. 4. http://jasss.soc.surrey.ac.uk/7/4/5.html
Terán Oswaldo, Edmonds Bruce and Steve Wallis (2001) “Mapping the Envelope of Social Simulation Trajectories”, Lecture Notes in Artificial Intelligence (Subserie de Lecture Notes in Computer Science): MABS, Volume 1979, pp. 229-243, Springer, Alemania, 2001
Terán Oswaldo y Jose Aguilar (2018). “Modelo del proceso de influencia de los medios de comunicación social en la opinión pública”, EDUCERE, 21 (71), Universidad de Los Andes, Mérida, Venezuela. https://www.redalyc.org/toc.oa?id=356&numero=56002
Terán Oswaldo and Christophe Sibertin-Blanc (2020). Impact on Cooperation of altruism, tenacity, and the need of a resource, accepted in IEEE Transactions on Computational Social Systems. DOI: 10.1109/TCSS.2021.3136821.
Annex. Fuzzy Cognitive Maps (FCM) and Muti-Fuzzy Cognitive Maps (MFCM).
Cognitive map theory is based on symbolic representation for the description of a system. It uses information, knowledge and experience, to describe particular domains using concepts (variables, states, inputs, outputs), and the relationships between them (Aguilar 2005, 2013, 2016). Cognitive maps can be understood as directed graphs, whose arcs represent causal connections between the nodes (concepts), used to denote knowledge. An arc with a positive sign (alternatively, negative sign), going from node X to node Y means that X (causally) increases (alternatively, decreases) Y. Cognitive maps are graphically represented: concepts are connected by arcs through a connection matrix. In the connection matrix, the i-nth line represents the weight of the arc connections directed outside of the concept. The i-nth column lists the arcs directed toward , i.e., those affecting .
The conceptual development of FCMs rests on the definition and dynamic of concepts and relationships created by the theory of fuzzy sets (Kosko, 1986). FCM can describe any system using a causality-based model (that indicates positive or negative relationships), which takes fuzzy values and is dynamic (i.e., the effect of a change in one concept/node affects other nodes, which then affect further nodes). This structure establishes the forward and backward propagation of causality (Aguilar, 2005, 2013, 2016). Thus, the concepts and relations can be represented as fuzzy variables (expressed in linguistic terms), such as “Almost Always”, “Always”, “Normally”, “Some (see Figure 2).
The value of a concept depends on its previous iterations, following the equation (1):
Cm(i+1) stands for the value of the concept in the next iteration after the iteration i, N indicates the number of concepts, wm,k represents the value of the causal relationship between the concept Ck and the concept Cm, and S(y) is a function used to normalize the value of the concept.
Figure 2. Example of an FCM (taken from Sánchez et al., 2019).
MFCM is an extension of the FCM. It is a FCM with several layers where each layer represents a set of concepts that define a specific domain of a system. To construct a MFCM, the previous equation for calculating the current status of the concepts of a FCM is modified, to describe the relationships between different layers (Aguilar, 2016):
Where F(p) is the input function generated by the relationships among different layers, and p is the set of concepts of the other layers that impact this concept. Thus, the update function of the concepts has two parts. The first part, the classic, calculates the value of concept in iteration based on the values of concepts in the previous iteration . All these concepts belong to the same layer where the “m” concept belongs. The second part is the result of the causal relationship between the concepts in different levels of the MFCM.
Terán, O. & Aguilar, J. (2022) Towards contextualized social simulation: Complementary use of Multi-Fuzzy Cognitive Maps and MABS. Review of Artificial Societies and Social Simulation, 25th May 2022. https://rofasss.org/2022/05/25/MFCM-MABS
By Peer-Olaf Siebers, in collaboration with Kwabena Amponsah, James Hey, Edmund Chattoe-Brown and Melania Borit
Motivation
1.1: Some time ago, I had several discussions with my PhD students Kwabena Amponsah and James Hey (we are all computer scientists, with a research interest in multi-agent systems) on the topic of qualitative vs. quantitative data in the context of Agent-Based Social Simulation (ABSS). Our original goal was to better understand the role of qualitative vs. quantitative data in the life cycle of an ABSS study. But as you will see later, we conquered more ground during our discussions.
1.2: The trigger for these discussions came from numerous earlier discussions within the RAT task force (Sebastian Achter, Melania Borit, Edmund Chattoe-Brown, and Peer-Olaf Siebers) on the topic, while we were developing the Rigour and Transparency – Reporting Standard (RAT-RS). The RAT-RS is a tool to improve the documentation of data use in Agent-Based Modelling (Achter et al 2022). During our RAT-RS discussions we made the observation that when using the terms “qualitative data” and “quantitative data” in different phases of the ABM simulation study life cycle these could be interpreted in different ways, and we felt difficult to clearly state the definition/role of these different types of data in the different contexts that the individual phases within the life cycle represent. This was aggravated by the competing understandings of the terminology within different domains (from social and natural sciences) that form the field of social simulation.
1.3: As the ABSS community is a multi-disciplinary one, often doing interdisciplinary research, we thought that we should share the outcome of our discussions with the community. To demonstrate the different views that exist within the topic area, we ask some of our friends from the social simulation community to comment on our philosophical discussions. And we were lucky enough to get our RAT-RS colleagues Edmund Chattoe-Brown and Melania Borit on board who provided critical feedback and their own view of things. In the following we provide a summary of the overall discussion Each of the following paragraph contains summaries of the initial discussion outcomes, representing the computer scientists’ views, followed by some thoughts provided by our two friends from the social simulation community (Borit’s in {} brackets and italicand Chattoe-Brown’s in [] brackets and bold), both commenting on the initial discussion outcomes of the computer scientists. To see the diverse backgrounds of all the contributors and perhaps to better understand their way of thinking and their arguments, I have added some short biographies of all contributors at the end of this Research Note. To support further (public) discussions I have numbered the individual paragraphs to make it easier to refer back to them.
Terminology
2.1: As a starting point for our discussions I searched the internet for some terminology related to the topic of “data”. Following is a list of initial definitions of relevant terms [1]. First, the terms qualitative data and quantitative data, as defined by the Australian Bureau of Statistics: “Qualitative data are measures of ‘types’ and may be represented by a name, symbol, or a number code. They are data about categorical variables (e.g. what type). Quantitative data are measures of values or counts and are expressed as numbers. They are data about numeric variables (e.g. how many; how much; or how often).” (Australian Bureau of Statistics 2022) [Maybe don’t let a statistics unit define qualitative research? This has a topic that is very alien to us but argues “properly” about the role of different methods (Helitzer-Allen and Kendall 1992). “Proper” qualitative researchers would fiercely dispute this. It is “quantitative imperialism”.].
2.2: What might also help for this discussion is to better understand the terms qualitative data analysis and quantitative data analysis. Qualitative data analysis refers to “the processes and procedures that are used to analyse the data and provide some level of explanation, understanding, or interpretation” (Skinner et al 2021). [This is a much less contentious claim for qualitative data – and makes the discussion of the Australian Bureau of Statistics look like a distraction but a really “low grade” source in peer review terms. A very good one is Strauss (1987).] These methods include content analysis, narrative analysis, discourse analysis, framework analysis, and grounded theory and the goal is to identify common patterns. {These data analysis methods connect to different types of qualitative research: phenomenology, ethnography, narrative inquiry, case study research, or grounded theory. The goal of such research is not always to identify patterns – see (Miles and Huberman 1994): e.g., making metaphors, seeing plausibility, making contrasts/comparisons.}[In my opinion some of these alleged methods are just empire building or hot air. Do you actually need them for your argument?] These types of analysis must therefore use qualitative inputs, broadening the definition to include raw text, discourse and conceptual frameworks.
2.3 When it comes to quantitative data analysis “you are expected to turn raw numbers into meaningful data through the application of rational and critical thinking. Quantitative data analysis may include the calculation of frequencies of variables and differences between variables.” (Business Research Methodology 2022) {One does the same in qualitative data analysis – turns raw words (or pictures etc.) into meaningful data through the application of interpretation based on rational and critical thinking. In quantitative data analysis you usually apply mathematical/statistical models to analyse the data.}. While the output of quantitative data analysis can be used directly as input to a simulation model, the output of qualitative data analysis still needs to be translated into behavioural rules to be useful (either manually or through machine learning algorithms). {What is meant by “translated” in this specific context? Do we need this kind of translation only for qualitative data or also for quantitative data? Is there a difference between translation methods of qualitative and quantitative data?} [That seems pretty contentious too. It is what is often done, true, but I don’t think it is a logical requirement. I guess you could train a neural net using “cases” or design some other simple “cognitive architecture” from the data. Would this (Becker 1953), for example, best be modelled as “rules” or as some kind of adaptive process? But of course you have to be careful that “rule” is not defined so broadly that everything is one or it is true by definition. I wonder what the “rules” are in this: Chattoe-Brown (2009).]
2.4: Finally, let’s have a quick look at the difference between “data” and “evidence”. For this, we found the following distinction by Wilkinson (2022) helpful: “… whilst data can exist on its own, even though it is essentially meaningless without context, evidence, on the other hand, has to be evidence of or for something. Evidence only exists when there is an opinion, a viewpoint or an argument.”
Hypothesis
3.1: The RAT-RS divides the simulation life cycle into five phases, in terms of data use: model aim and context, conceptualisation, operationalisation, experimentation, and evaluation (Siebers et al 2019). We started our discussion by considering the following hypothesis: The outcome of qualitative data analysis is only useful for the purpose of conceptualisation and as a basis for producing quantitative data. It does not have any other roles within the ABM simulation study life cycle. {Maybe this hypothesis in itself has to be discussed. Is it so that you use only numbers in the operationalisation phase? One can write NetLogo code directly from qualitative data, without numbers.} [Is this inevitable given the way ABM works? Agents have agency and therefore can decide to do things (and we can only access this by talking to them, probably “open ended”). A statistical pattern – time series or correlation – has no agency and therefore cannot be accessed “qualitatively” – though we also sometimes mean by “qualitative” eyeballing two time series rather than using some formal measure of tracking. I guess that use would >>not<< be relevant here.]
Discussion
4.1: One could argue that qualitative data analysis provides causes for behaviour (and indications about their importance (ranking); perhaps also the likelihood of occurrence) as well as key themes that are important to be considered in a model. All sounds very useful for the conceptual modelling phase. The difficulty might be to model the impact (how do we know we model it correctly and at the right level), if that is not easily translatable into a quantitative value but requires some more (behavioural) mechanistic structures to represent the impact of behaviours. [And, of course, there is a debate in psychology (with some evidence on both sides) about the extent to which people are able to give subjective accounts we can trust (see Hastorf and Cantril (1954).] This might also provide issues when it comes to calibration – how does one calibrate qualitative data? {Triangulation.} One random idea we had was that perhaps fuzzy logic could help with this. More brainstorming and internet research is required to confirm that this idea is feasible and useful. [A more challenging example might be ethnographic observation of a “neighbourhood” in understanding crime. This is not about accessing the cognitive content of agents but may well still contribute to a well specified model. It is interesting how many famous models – Schelling, Zaller-Deffuant – actually have no “real” environment.]
4.2: One could also argue that what starts (or what we refer to initially) as qualitative data always ends up as quantitative data, as whatever comes out of the computer are numbers. {This is not necessarily true. Check the work on qualitative outputs using Grounded Theory by Neumann (2015).} Of course this is a question related to the conceptual viewpoint. [Not convinced. It sounds like all sociology is actually physics because all people are really atoms. Formally, everything in computers is numbers because it has to be but that isn’t the same as saying that data structures or whatever don’t constitute a usable and coherent level of description: We “meet” and “his” opinion changes “mine” and vice versa. Somewhere, that is all binary but you can read the higher level code that you can understand as “social influence” (whatever you may think of the assumptions). Be clear whether this (like the “rules” claim) is a matter of definition – in which case it may not be useful (even if people are atoms we have no idea of how to solve the “atomic physics” behind the Prisoner’s Dilemma) or an empirical one (in which case some models may just prove it false). This (Beltratti et al 1996) contains no “rules” and no “numbers” (except in the trivial sense that all programming does).]
4.3: Also, an algorithm is expressed in code and can only be processed numerically, so it can only deliver quantitative data as output. These can perhaps be translated into qualitative concepts later. A way of doing this via the use of grounded theory is proposed in Neumann and Lotzmann (2016). {This refers to the same idea as my previous comment.} [Maybe it is “safest” to discuss this with rules because everyone knows those are used in ABM. Would it make sense to describe the outcome of a non trivial set of rules – accessed for example like this: Gladwin (1989) – as either “quantitative” or “numbers?”]
4.4: But is it true that data delivered as output is always quantitative? Let’s consider, for example, a consumer marketing scenario, where we define stereotypes (shopping enthusiast; solution demander; service seeker; disinterested shopper; internet shopper) that can change over time during a simulation run (Siebers et al 2010). These stereotypes are defined by likelihoods (likelihood to buy, wait, ask for help, and ask for refund). So, during a simulation run an agent could change its stereotype (e.g. from shopping enthusiast to disinterested shopper), influenced by the opinion of others and their own previous experience. So, at the beginning of the simulation run the agent can have a different stereotype compared to the end. Of course we could enumerate the five different stereotypes, and claim that the outcome is numeric, but the meaning of the outcome would be something qualitative – the stereotype related to that number. To me this would be a qualitative outcome, while the number of people that change from one stereotype to another would be a quantitative outcome. They would come in a tandem. {So, maybe the problem is that we don’t yet have the right ways of expressing or visualising qualitative output?} [This is an interesting and grounded example but could it be easily knocked down because everything is “hard coded” and therefore quantifiable? You may go from one shopper type to another – and what happens depends on other assumptions about social influence and so on – but you can’t “invent” your own type. Compare something like El Farol (Arthur 1994) where agents arguably really can “invent” unique strategies (though I grant these are limited to being expressed in a specified “grammar”).]
4.5: In order to define someone’s stereotype we would use numerical values (likelihood = proportion). However, stereotypes refer to nominal data (which refers to data that is used for naming or labelling variables, without any quantitative value). The stereotype itself would be nominal, while the way one would derive the stereotype would be numerical. Figure 1 illustrates a case in which the agent moves from the disinterested stereotype to the enthusiast stereotype. [Is there a potential confusion here between how you tell an agent is a type – parameters in the code just say so – and how you say a real person is a type? Everything you say about the code still sounds “quantitative” because all the “ingredients” are.]
Figure 1: Enthusiastic and Disinterested agent stereotypes
4.6: Let’s consider a second example, related to the same scenario: The dynamics over time to get from an enthusiastic shopper (perhaps via phases) to a disinterested shopper. This is represented as a graph where the x-axis represents time and the y-axis stereotypes (categorical data). If you want to take a quantitative perspective on the outcome you would look at a specific point in time (state of the system) but to take a qualitative perspective of the outcome, you would look at the pattern that the curve represents over the entire simulation runtime. [Although does this shade into the “eyeballing” sense of qualitative rather than the “built from subjective accounts” sense? Another way to think of this issue is to imagine “experts” as a source of data. We might build an ABM based on an expert perception of say, how a crime gang operates. That would be qualitative but not just individual rules: For example, if someone challenges the boss to a fight and loses they die or leave. This means the boss often has no competent potential successors.]
4.7: So, the inputs (parameters, attributes) to get the outcome are numeric, but the outcome itself in the latter case is not. The outcome only makes sense once it’s put into the qualitative context. And then we could say that the simulation produces some qualitative outputs. So, does the fact that data needs to be seen in a context make it evidence, i.e. do we only have quantitative and qualitative evidence on the output side? [Still worried that you may not be happy equating qualitative interview data with qualitative eyeballing of graphs. Mixes up data collection and analysis? And unlike qualitative interviews you don’t have to eyeball time series. But the argument of qualitative research is you can’t find out some things any other way because, to run a survey say, or an experiment, you already have to have a pretty good grasp of the phenomenon.]
4.8: If one runs a marketing campaign that will increase the number of enthusiastic shoppers. This can be seen as qualitative data as it is descriptive of how the system works rather than providing specific values describing the performance of a system. You could also equally express this algebraic terms which would make it quantitative data. So, it might be useful to categorise quantitative data to make the outcome easier to understand. [I don’t think this argument is definitely wrong – though I think it may be ambiguous about what “qualitative” means – but I think it really needs stripping down and tightening. I’m not completely convinced as a new reader that I’m getting at the nub of the argument. Maybe just one example in detail and not two in passing?]
Outcome
5.1: How we understand things and how the computer processes things are two different things. So, in fact qualitative data is useful for the conceptualisation and for describing experimentation and evaluation output, and needs to be translated into numerical data or algebraic constructs for the operationalisation. Therefore, we can reject our initial hypothesis, as we found more places where qualitative data can be useful. [Yes, and that might form the basis for a “general” definition of qualitative that was not tied to one part of the research process but you would have to be clear that’s what you were aiming at and not just accidentally blurring two different “senses” of qualitative.]
5.2: In the end of the discussion we picked up the idea of using Fuzzy Logic. Could perhaps fuzzy logic be used to describe qualitative output, as it describes a degree of membership to different categories? An interesting paper to look at in this context would be Sugeno and Yasukawa (1993). Also, a random idea that was mentioned is if there is potential in using “fuzzy logic in reverse”, i.e. taking something that is fuzzy, making it crisp for the simulation, and making it fuzzy again for presenting the result. However, we decided to save this topic for another discussion. [Devil will be in the detail. Depends on exactly what assumptions the method makes. Devil’s advocate: What if qualitative research is only needed for specification – not calibration or validation – but it doesn’t follow from this that that use is “really” quantitative? How intellectually unappealing is that situation and why?]
Conclusion
6.1: The purpose of this Research Note is really to stimulate you to think about, talk about, and share your ideas and opinions on the topic! What we present here is a philosophical impromptu discussion of our individual understanding of the topic, rather than a scientific debate that is backed up by literature. We still thought it is worthwhile to share this with you, as you might stumble across similar questions. Also, we don’t think we have found the perfect answers to the questions yet. So we would like to invite you to join the discussion and leave some comments in the chat, stating your point of view on this topic. [Is the danger of discussing these data types “philosophically”? I don’t know if it is realistic to use examples directly from social simulation but for sure examples can be used from social science generally. So here is a “quantitative” argument from quantitative data: “The view that cultural capital is transmitted from parents to their children is strongly supported in the case of pupils’ cultural activities. This component of pupils’ cultural capital varies by social class, but this variation is entirely mediated by parental cultural capital.” (Sullivan 2001). As well as the obvious “numbers” (social class by a generally agreed scheme) there is also a constructed “measure” of cultural capital based on questions like “how many books do you read in a week?” Here is an example of qualitative data from which you might reason: “I might not get into Westbury cos it’s siblings and how far away you live and I haven’t got any siblings there and I live a little way out so I might have to go on a waiting list … I might go to Sutton Boys’ instead cos all my mates are going there.” (excerpt from Reay 2002). As long as this was not just a unique response (but was supported by several other interviews) one would add to one’s “theory” of school choice: 1) Awareness of the impact of the selection system (there is no point in applying here whatever I may want) and 2) The role of networks in choice: This might be the best school for me educationally but I won’t go because I will be lonely.]
Biographies of the authors
Peer-Olaf Siebers is an Assistant Professor at the School of Computer Science, University of Nottingham, UK. His main research interest is the application of Computer Simulation and Artificial Intelligence to study human-centric and coupled human-natural systems. He is a strong advocate of Object-Oriented Agent-Based Social Simulation and is advancing the methodological foundations. It is a novel and highly interdisciplinary research field, involving disciplines like Social Science, Economics, Psychology, Geography, Operations Research, and Computer Science.
Kwabena Amponsah is a Research Software Engineer working for the Digital Research Service, University of Nottingham, UK. He completed his PhD in Computer Science at Nottingham in 2019 by developing a framework for evaluating the impact of communication on performance in large-scale distributed urban simulations.
James Hey is a PhD student at the School of Computer Science, University of Nottingham, UK. In his PhD he investigates the topic of surrogate optimisation for resource intensive agent based simulation of domestic energy retrofit uptake with environmentally conscious agents. James holds a Bachelor degree in Economics as well as a Master degree in Computer Science.
Edmund Chattoe-Brown is a lecturer in Sociology, School of Media, Communication and Sociology, University of Leicester, UK. His career has been interdisciplinary (including Politics, Philosophy, Economics, Artificial Intelligence, Medicine, Law and Anthropology), focusing on the value of research methods (particularly Agent-Based Modelling) in generating warranted social knowledge. His aim has been to make models both more usable generally and particularly more empirical (because the most rigorous social scientists tend to be empirical). The results of his interests have been published in 17 different peer reviewed journals across the sciences to date. He was funded by the project “Towards Realistic Computational Models Of Social Influence Dynamics” (ES/S015159/1) by the ESRC via ORA Round 5.
Melania Borit is an interdisciplinary researcher and the leader of the CRAFT Lab – Knowledge Integration and Blue Futures at UiT The Arctic University of Norway. She has a passion for knowledge integration and a wide range of interconnected research interests: social simulation, agent-based modelling; research methodology; Artificial Intelligence ethics; pedagogy and didactics in higher education, games and game-based learning; culture and fisheries management, seafood traceability; critical futures studies.
References
Achter S, Borit M, Chattoe-Brown E, and Siebers PO (2022) RAT-RS: a reporting standard for improving the documentation of data use in agent-based modelling. International Journal of Social Research Methodology, DOI: 10.1080/13645579.2022.2049511
Chattoe-Brown E (2009) The social transmission of choice: a simulation with applications to hegemonic discourse. Mind & Society, 8(2), pp.193-207. DOI: 10.1007/s11299-009-0060-7
Gladwin CH (1989) Ethnographic Decision Tree Modeling. SAGE Publications.
Hastorf AH and Cantril H (1954) They saw a game; a case study. The Journal of Abnormal and Social Psychology, 49(1), pp.129–134.
Helitzer-Allen DL and Kendall C (1992) Explaining differences between qualitative and quantitative data: a study of chemoprophylaxis during pregnancy. Health Education Quarterly, 19(1), pp.41-54. DOI: 10.1177%2F109019819201900104
Miles MB and Huberman AM (1994) . Qualitative Data Analysis: An Expanded Sourcebook. Sage
Neumann M (2015) Grounded simulation. Journal of Artificial Societies and Social Simulation, 18(1)9. DOI: 10.18564/jasss.2560
Neumann M and Lotzmann U (2016) Simulation and interpretation: a research note on utilizing qualitative research in agent based simulation. International Journal of Swarm Intelligence and Evolutionary Computing 5/1.
Reay D (2002) Shaun’s Story: Troubling discourses of white working-class masculinities. Gender and Education, 14(3), pp.221-234. DOI: 10.1080/0954025022000010695
Siebers PO, Achter S, Palaretti Bernardo C, Borit M, and Chattoe-Brown E (2019) First steps towards RAT: a protocol for documenting data use in the agent-based modeling process (Extended Abstract). Social Simulation Conference 2019 (SSC 2019), 23-27 Sep, Mainz, Germany.
Siebers PO, Aickelin U, Celia H and Clegg C (2010) Simulating customer experience and word-of-mouth in retail: a case study. Simulation: Transactions of the Society for Modeling and Simulation International, 86(1) pp. 5-30. DOI: 10.1177%2F0037549708101575
Skinner J, Edwards A and Smith AC (2021) Qualitative Research in Sport Management – 2e, p171. Routledge.
Strauss AL (1987). Qualitative Analysis for Social Scientists. Cambridge University Press.
Sugeno M and Yasukawa T (1993) A fuzzy-logic-based approach to qualitative modeling. IEEE Transactions on Fuzzy Systems, 1(1), pp.7-31.
Sullivan A (2001) Cultural capital and educational attainment. Sociology 35(4), pp.893-912. DOI: 10.1017/S0038038501008938
[1] An updated set of the terminology, defined by the RAT task force in 2022, is available as part of the RAT-RS in Achter et al (2022) Appendix A1.
Peer-Olaf Siebers, Kwabena Amponsah, James Hey, Edmund Chattoe-Brown and Melania Borit (2022) Discussions on Qualitative & Quantitative Data in the Context of Agent-Based Social Simulation. Review of Artificial Societies and Social Simulation, 16th May 2022. https://rofasss.org/2022/05/16/Q&Q-data-in-ABM
1Department of Computing Science, Faculty of Science and Technology, Umeå University, frank.dignum@umu.se 2Centre for Policy Modelling, Manchester Metropolitan University, bruce@edmonds.name 3Department of Psychology, University of Limerick, dino.carpentras@gmail.com
In this position paper we argue for the creation of a new ‘field’: Socio-Cognitive Systems. The point of doing this is to highlight the importance of a multi-levelled approach to understanding those phenomena where the cognitive and the social are inextricably intertwined – understanding them together.
What goes on ‘in the head’ and what goes on ‘in society’ are complex questions. Each of these deserves serious study on their own – motivating whole fields to answer them. However, it is becoming increasingly clear that these two questions are deeply related. Humans are fundamentally social beings, and it is likely that many features of their cognition have evolved because they enable them to live within groups (Herrmann et al. 20007). Whilst some of these social features can be studied separately (e.g. in a laboratory), others only become fully manifest within society at large. On the other hand, it is also clear that how society ‘happens’ is complicated and subtle and that these processes are shaped by the nature of our cognition. In other words, what people ‘think’ matters for understanding how society ‘is’ and vice versa. For many reasons, both of these questions are difficult to answer. As a result of these difficulties, many compromises are necessary in order to make progress on them, but each compromise also implies some limitations. The main two types of compromise consist of limiting the analysis to only one of the two (i.e. either cognition or society)[1]. To take but a few examples of this.
Neuro-scientists study what happens between systems of neurones to understand how the brain does things and this is so complex that even relatively small ensembles of neurones are at the limits of scientific understanding.
Psychologists see what can be understood of cognition from the outside, usually in the laboratory so that some of the many dimensions can be controlled and isolated. However, what can be reproduced in a laboratory is a limited part of behaviour that might be displayed in a natural social context.
Economists limit themselves to the study of the (largely monetary) exchange of services/things that could occur under assumptions of individual rationality, which is a model of thinking not based upon empirical data at the individual level. Indeed it is known to contradict a lot of the data and may only be a good approximation for average behaviour under very special circumstances.
Ethnomethodologists will enter a social context and describe in detail the social and individual experience there, but not generalise beyond that and not delve into the cognition of those they observe.
Other social scientists will take a broader view, look at a variety of social evidence, and theorise about aspects of that part of society. They (almost always) do not include individual cognition into account in these and do not seek to integrate the social and the cognitive levels.
Each of these in the different ways separate the internal mechanisms of thought from the wider mechanisms of society or limits its focus to a very specific topic. This is understandable; what each is studying is enough to keep them occupied for many lifetimes. However, this means that each of these has developed their own terms, issues, approaches and techniques which make relating results between fields difficult (as Kuhn, 1962, pointed out).
Figure 1: Schematic representation of the relationship between the individual and society. Individuals’ cognition is shaped by society, at the same time, society is shaped by individuals’ beliefs and behaviour.
This separation of the cognitive and the social may get in the way of understanding many things that we observe. Some phenomena seem to involve a combination of these aspects in a fundamental way – the individual (and its cognition) being part of society as well as society being part of the individual. Some examples of this are as follows (but please note that this is far from an exhaustive list).
Norms. A social norm is a constraint or obligation upon action imposed by society (or perceived as such). One may well be mistaken about a norm (e.g. whether it is ok to casually talk to others at a bus stop), thus it is also a belief – often not told to one explicitly but something one needs to infer from observation. However, for a social norm to hold it also needs to be an observable convention. Decisions to violate social norms require that the norm is an explicit (referable) object in the cognitive model. But the violation also has social consequences. If people react negatively to violations the norm can be reinforced. But if violations are ignored it might lead to a norm disappearing. How new norms come about, or how old ones fade away, is a complex set of interlocking cognitive and social processes. Thus social norms are a phenomena that essentially involves both the social and the cognitive (Conte et al. 2013).
Joint construction of social reality. Many of the constraints on our behaviour come from our perception of social reality. However, we also create this social reality and constantly update it. For example, we can invent a new procedure to select a person as head of department or exit a treaty and thus have different ways of behaving after this change. However, these changes are not unconstrained in themselves. Sometimes the time is “ripe for change”, while at other times resistance is too big for any change to take place (even though a majority of the people involved would like to change). Thus what is socially real for us depends on what people individually believe is real, but this depends in complex ways on what other people believe and their status. And probably even more important: the “strength” of a social structure depends on the use people make of it. E.g. a head of department becomes important if all decisions in the department are deferred to the head. Even though this might not be required by university or law.
Identity. Our (social) identity determines the way other people perceive us (e.g. a sports person, a nerd, a family man) and therefore creates expectations about our behaviour. We can create our identities ourselves and cultivate them, but at the same time, when we have a social identity, we try to live up to it. Thus, it will partially determine our goals and reactions and even our feeling of self-esteem when we live up to our identity or fail to do so. As individuals we (at least sometimes) have a choice as to our desired identity, but in practice, this can only be realised with the consent of society. As a runner I might feel the need to run at least three times a week in order for other people to recognize me as runner. At the same time a person known as a runner might be excused from a meeting if training for an important event. Thus reinforcing the importance of the “runner” identity.
Social practices. The concept already indicates that social practices are about the way people habitually interact and through this interaction shape social structures. Practices like shaking hands when greeting do not always have to be efficient, but they are extremely socially important. For example, different groups, countries and cultures will have different practices when greeting and performing according to the practice shows whether you are part of the in-group or out-group. However, practices can also change based on circumstances and people, as it happened, for example, to the practice of shaking hands during the covid-19 pandemic. Thus, they are flexible and adapting to the context. They are used as flexible mechanisms to efficiently fit interactions in groups, connecting persons and group behaviour.
As a result, this division between cognitive and the social gets in the way not only of theoretical studies, but also in practical applications such as policy making. For example, interventions aimed at encouraging vaccination (such as compulsory vaccination) may reinforce the (social) identity of the vaccine hesitant. However, this risk and its possible consequences for society cannot be properly understood without a clear grasp of the dynamic evolution of social identity.
Computational models and systems provide a way of trying to understand the cognitive and the social together. For computational modellers, there is no particular reason to confine themselves to only the cognitive or only the social because agent-based systems can include both within a single framework. In addition, the computational system is a dynamic model that can represent the interactions of the individuals that connect the cognitive models and the social models. Thus the fact that computational models have a natural way to represent the actions as an integral and defining part of the socio-cognitive system is of prime importance. Given that the actions are an integral part of the model it is well suited to model the dynamics of socio-cognitive systems and track changes at both the social and the cognitive level. Therefore, within such systems we can study how cognitive processes may act to produce social phenomena whilst, at the same time, as how social realities are shaping the cognitive processes. Caarley and Newell (1994) discusses what is necessary at the agent level for sociality, Hofested et al. (2021) talk about how to understand sociality using computational models (including theories of individual action) – we want to understand both together. Thus, we can model the social embeddedness that Granovetter (1985) talked about – going beyond over- or under-socialised representations of human behaviour. It is not that computational models are innately suitable for modelling either the cognitive or the social, but that they can be appropriately structured (e.g. sets of interacting parts bridging micro-, meso- and macro-levels) and include arbitrary levels of complexity. Lots of models that represent the social have entities that stand for the cognitive, but do not explicitly represent much of that detail – similarly much cognitive modelling implies the social in terms of the stimuli and responses of an individual that would be to other social entities, but where these other entities are not explicitly represented or are simplified away.
Socio-Cognitive Systems (SCS) are: those models and systems where both cognitive and social complexity are represented with a meaningful level of processual detail.
A good example of an application where this appeared of the biggest importance was in simulations for the covid-19 crisis. The spread of the corona virus on macro level could be given by an epidemiological model, but the actual spreading depended crucially on the human behaviour that resulted from individuals’ cognitive model of the situation. In Dignum (2021) it was shown how the socio-cognitive system approach was fundamental to obtaining better insights in the effectiveness of a range of covid-19 restrictions.
Formality here is important. Computational systems are formal in the sense that they can be unambiguously passed around (i.e. unlike language, it is not differently re-interpreted by each individual) and operate according to their own precisely specified and explicit rules. This means that the same system can be examined and experimented on by a wider community of researchers. Sometimes, even when the researchers from different fields find it difficult to talk to one another, they can fruitfully cooperate via a computational model (e.g. Lafuerza et al. 2016). Other kinds of formal systems (e.g. logic, maths) are geared towards models that describe an entire system from a birds eye view. Although there are some exceptions like fibred logics Gabbay (1996), these are too abstract to be of good use to model practical situations. The lack of modularity and has been addressed in context logics Giunchiglia, F., & Ghidini, C. (1998). However, the contexts used in this setting are not suitable to generate a more general societal model. It results in most typical mathematical models using a number of agents which is either one, two or infinite (Miller and Page 2007), while important social phenomena happen with a “medium sized” population. What all these formalisms miss is a natural way of specifying the dynamics of the system that is modelled, while having ways to modularly describe individuals and the society resulting from their interactions. Thus, although much of what is represented in Socio-Cognitive Systems is not computational, the lingua franca for talking about them is.
The ‘double complexity’ of combining the cognitive and the social in the same system will bring its own methodological challenges. Such complexity will mean that many socio-cognitive systems will be, themselves, hard to understand or analyse. In the covid-19 simulations, described in (Dignum 2021), a large part of the work consisted of analysing, combining and representing the results in ways that were understandable. As an example, for one scenario 79 pages of graphs were produced showing different relations between potentially relevant variables. New tools and approaches will need to be developed to deal with this. We only have some hints of these, but it seems likely that secondary stages of analysis – understanding the models – will be necessary, resulting in a staged approach to abstraction (Lafuerza et al. 2016). In other words, we will need to model the socio-cognitive systems, maybe in terms of further (but simpler) socio-cognitive systems, but also maybe with a variety of other tools. We do not have a view on this further analysis, but this could include: machine learning, mathematics, logic, network analysis, statistics, and even qualitative approaches such as discourse analysis.
An interesting input for the methodology of designing and analysing socio-cognitive systems is anthropology and specifically ethnographical methods. Again, for the covid-19 simulations the first layer of the simulation was constructed based on “normal day life patterns”. Different types of persons were distinguished that each have their own pattern of living. These patterns interlock and form a fabric of social interactions that overall should satisfy most of the needs of the agents. Thus we calibrate the simulation based on the stories of types of people and their behaviours. Note that doing the same just based on available data of behaviour would not account for the underlying needs and motives of that behaviour and would not be a good basis for simulating changes. The stories that we used looked very similar to the type of reports ethnographers produce about certain communities. Thus further investigating this connection seems worthwhile.
For representing the output of the complex socio-cognitive systems we can also use the analogue of stories. Basically, different stories show the underlying (assumed) causal relations between phenomena that are observed. E.g. seeing an increase in people having lunch with friends can be explained by the fact that a curfew prevents people having dinner with their friends, while they still have a need to socialize. Thus the alternative of going for lunch is chosen more often. One can see that the explaining story uses both social as well as cognitive elements to describe the results. Although in the covid-19 simulations we have created a number of these stories, they were all created by hand after (sometimes weeks) of careful analysis of the results. Thus for this kind of approach to be viable, new tools are required.
Although human society is the archetypal socio-cognitive system, it is not the only one. Both social animals and some artificial systems also come under this category. These may be very different from the human, and in the case of artificial systems completely different. Thus, Socio-Cognitive Systems is not limited to the discussion of observable phenomena, but can include constructed or evolved computational systems, and artificial societies. Examination of these (either theoretically or experimentally) opens up the possibility of finding either contrasts or commonalities between such systems – beyond what happens to exist in the natural world. However, we expect that ideas and theories that were conceived with human socio-cognitive systems in mind might often be an accessible starting point for understanding these other possibilities.
In a way, Socio-Cognitive Systems bring together two different threads in the work of Herbert Simon. Firstly, as in Simon (1948) it seeks to take seriously the complexity of human social behaviour without reducing this to overly simplistic theories of individual behaviour. Secondly, it adopts the approach of explicitly modelling the cognitive in computational models (Newell & Simon 1972). Simon did not bring these together in his lifetime, perhaps due to the limitations and difficulty of deploying the computational tools to do so. Instead, he tried to develop alternative mathematical models of aspects of thought (Simon 1957). However, those models were limited by being mathematical rather than computational.
To conclude, a field of Socio-Cognitive Systems would consider the cognitive and the social in an integrated fashion – understanding them together. We suggest that computational representation or implementation might be necessary to provide concrete reference between the various disciplines that are needed to understand them. We want to encourage research that considers the cognitive and the social in a truly integrated fashion. If by labelling a new field does this it will have achieved its purpose. However, there is the possibility that completely new classes of theory and complexity may be out there to be discovered – phenomena that are denied if either the cognitive or the social are not taken together – a new world of a socio-cognitive systems.
Notes
[1] Some economic models claim to bridge between individual behaviour and macro outcomes, however this is traditionally notional. Many economists admit that their primary cognitive models (varieties of economic rationality) are not valid for individuals but are what people on average do – i.e. this is a macro-level model. In other economic models whole populations are formalised using a single representative agent. Recently, there are some agent-based economic models emerging, but often limited to agree with traditional models.
Acknowledgements
Bruce Edmonds is supported as part of the ESRC-funded, UK part of the “ToRealSim” project, grant number ES/S015159/1.
References
Carley, K., & Newell, A. (1994). The nature of the social agent. Journal of mathematical sociology, 19(4): 221-262. DOI: 10.1080/0022250X.1994.9990145
Conte R., Andrighetto G. and Campennì M. (eds) (2013) Minding Norms – Mechanisms and dynamics of social order in agent societies. Oxford University Press, Oxford.
Dignum, F. (ed.) (2021) Social Simulation for a Crisis; Results and Lessons from Simulating the COVID-19 Crisis. Springer.
Herrmann E., Call J, Hernández-Lloreda MV, Hare B, Tomasello M (2007) Humans have evolved specialized skills of social cognition: The cultural intelligence hypothesis. Science317(5843): 1360-1366. DOI: 10.1126/science.1146282
Hofstede, G.J, Frantz, C., Hoey, J., Scholz, G. and Schröder, T. (2021) Artificial Sociality Manifesto. Review of Artificial Societies and Social Simulation, 8th Apr 2021. https://rofasss.org/2021/04/08/artsocmanif/
Gabbay, D. M. (1996). Fibred Semantics and the Weaving of Logics Part 1: Modal and Intuitionistic Logics. The Journal of Symbolic Logic, 61(4), 1057–1120.
Ghidini, C., & Giunchiglia, F. (2001). Local models semantics, or contextual reasoning= locality+ compatibility. Artificial intelligence, 127(2), 221-259. DOI: 10.1016/S0004-3702(01)00064-9
Granovetter, M. (1985) Economic action and social structure: The problem of embeddedness. American Journal of Sociology91(3): 481-510. DOI: 10.1086/228311
Kuhn, T,S, (1962) The structure of scientific revolutions. University of Chicago Press, Chicago
Miller, J. H., Page, S. E., & Page, S. (2009). Complex adaptive systems. Princeton university press.
Newell A, Simon H.A. (1972) Human problem solving. Prentice Hall, Englewood Cliffs, NJ
Simon, H.A. (1948) Administrative behaviour: A study of the decision making processes in administrative organisation. Macmillan, New York
Simon, H.A. (1957) Models of Man: Social and rational. John Wiley, New York
Dignum, F., Edmonds, B. and Carpentras, D. (2022) Socio-Cognitive Systems – A Position Statement. Review of Artificial Societies and Social Simulation, 2nd Apr 2022. https://rofasss.org/2022/04/02/scs
Review of Artificial Societies and Social Simulation 