Tag Archives: Agent-based Social Simulation

Content

Agent-based Modelling as a Method for Prediction for Complex Social Systems – a review of the special issue

Leave a comment

International Journal of Social Research Methodology, Volume 26, Issue 2.

By Oswaldo Terán

Escuela de Ciencias Empresariales, Universidad Católica del Norte, Coquimbo, Chile

This special issue appeared following a series of articles in RofASSS regarding the polemic around Agent-Based Modelling (ABM) prediction (https://rofasss.org/tag/prediction-thread/).  As expected, the articles in the special issue complement and expand upon the initial RofASSS’s discussion.

The goal of the special issue is to explore a wide range of positions regarding ABM prediction, encompassing methodological, epistemic and pragmatic issues. Contributions go from moderately sceptic and pragmatic positions to strongly sceptic positions. Moderately sceptic views argue that ABM can cautiously be employed for prediction, sometimes as a complement to other approaches, acknowledging its somewhat peripheral role in social research. Conversely, strongly sceptic positions contend that, in general, ABM can not be utilized for prediction. Several factors are instrumental in distinguishing and understanding these positions with respect to ABM prediction, especially the following:

  • the conception of prediction.
  • the complexity of modelled systems and models: this encompasses factors such as multiple views (or perspectives), uncertainty, auto-organization, self-production, emergence, structural change, and data incompleteness. These complexities are associated with the limitations of our language and tools to comprehend and symmetrically model complex systems.

Considering these factors, we will summarize the diverse positions presented in this special issue. Then, we will delve into the notions of prediction and complexity and briefly situate each position within the framework provided by these definitions

Elsebroich and Polhill (2023) (Editorial) summarizes the diverse positions in the special issue regarding prediction, categorizing them into three groups: 1) Positive, a position that assumes that “all we need for prediction is to have the right data,  methods and mechanism” (p. 136); 2) pragmatic, a position advocate to for cautious use of ABM to attempt prediction, often to compliment other approaches and avoid exclusive reliance on them; and 3) sceptic, a position arguing that ABM can not be used for prediction but can serve other purposes.  The authors place this discussion in a broader context, considering other relevant papers on ABM prediction. The authors acknowledge the challenge of prediction in complex systems, citing factors such as multiple perspectives, asynchronous agent actions, emergence, nonlinearity, non-ergodicity, evolutionary dynamics and heterogeneity. They indicate that some of these factors are well managed in ABM, but not others, noticeably “multiple perspectives/views”. Uncertainty is another critical element affecting ABM prediction, along with the relationship between prediction and explanation. The authors proved a summary of the debate surrounding the possibilities of prediction and its relation with explanation, incorporating insightful views from external sources (e.g., Thompson & Derr, 2009; Troitzsch, 2009). They also highlight recent developments in this debate, noticing that ABM has evolved into a more empirical and data-driven approach, deeply focused on modelling complex social and ecological systems, including Geographical Information Systems data and real time data integration, leading to a more contentious discussion regarding empirical data-driven ABM prediction.

Chattoe-Brown (2023) supports the idea that ABM prediction is possible. He argues for the utility of using AMB not only to predict real world outcomes but also to predict models. He also advocates for using prediction for predictive failure and assessing predictions. His notion of prediction finds support on by key elements of prediction in social science derived from real research across disciplines. For instance, the need of adopting a conceptual approach to enhance our comprehension of the various facets of prediction, the functioning of diverse prediction approaches, and the need for clear thinking about temporal logic. Chattoe-Brown argues that he attempts to make prediction intelligible rather than seen if it is successful. He support the idea that ABM prediction is useful for coherent social science. He contrasts ABM to other modelling methods that predict on trend data alone, underscoring the advantages of ABM. From his position, ABM prediction can add value to other research, taking a somewhat secondary role.

Dignum (2023) defends the ability of ABM to make prediction while distinguishing the usefulness of a prediction from the truth of a prediction. He argues in favour of limited prediction in specific cases, especially when human behaviour is involved. He shows prediction alongside explanations of the predicted behaviour, which arise under specific constrains that define particular scenarios. His view is moderately positive, suggesting that prediction is possible under certain specific conditions, including a stable environment and sufficient available data.

Carpentras and Quayle (2023) call for improved agent specification to reduce distortions when using psychometric instruments, particularly in measurements of political opinion within ABM. They contend that the quality of prediction and validation depends on the scale of the system but acknowledges the challenges posed by the high complexity of the human brain, which is central to their study. Furthermore, they raise concerns about representativeness, especially considering the discrepancy between certain theoretical frameworks (e.g., opinion dynamics) and survey data.

Anzola and García-Díaz (2023) advocate for better criteria to judge prediction and a more robust framework for the practice of prediction to better coordinate efforts within the research community (helping to better contextualize needs and expectations). They hold a somewhat sceptic position, suggesting that prediction typically serve an instrumental role in scientific practices, subservient to other epistemic goals.

Elsenbroich and Badham (2023) adopt a somewhat negative and critical stance toward using ABM for prediction, asserting that ABM can improve forecasting but not provide definite predictions of specific future events. ABM can only generate coherent extrapolations from a certain initialization of the ABM and a set of assumptions. They argue that ABM generates “justified stories” based on internal coherence, mechanisms and consistency  with empirical evidence, but these can not be confused with precise predictions. They ask for the combined support of ABM on theoretical developments and external data.

Edmonds (2023) is the most sceptical regarding the use of ABM for prediction, contending that the motivation for prediction in ABM is a desire without evidence of its achievability. He highlights inherent reasons for preventing prediction in complex social and ecological systems, including incompleteness, chaos, context specificity, and more. In his perspective, it is essential to establish the distinction between prediction and explanation. He advocates for recognizing the various potential applications of AMB beyond prediction, such as description, explanation, analogy, and more. For Edmonds, prediction should entail generating data that is unknown to the modellers. To address the ongoing debate and the weakness of the practices in ABM prediction, Edmonds proposes a process of iterative and independent verification. However, this approach faces limitations due to the incomplete understanding of the underlying process that should be included into the requirement for high-quality, relevant data. Despite these challenges, Edmonds suggest that prediction could prove valuable in meta-modelling, particularly to comprehend better our own simulation models.

The above summarized diverse positions on ABM prediction within the reviewed articles can be better understood through the lenses of Troitzsch’s notion of prediction and McNabb’s descriptions of complex and complicated systems. Troitzsch (2009) distinguishes the difference between prediction and explanation by using three possible conceptions of predictions. The typical understanding of ABM prediction closely aligns with Troitzsch’s third definition of prediction, which answer to the following question:

“Which state will the target system reach in the near future, again given parameters and previous states which may or may not have been precisely measured?”

The answer to this question results in a prediction, which can be either stochastic or deterministic. In our view, explanations encompass broader range of statements than predictions. An explanation entails a wider scope, including justifications, descriptions, and reasons for various real or hypothetical scenarios. Explanation is closely tied to a fundamental aspect of human communication capacity signifying the act of making something plain, clear or comprehensible by elaborating its meaning. But, what precisely does it expand or elaborate?. It expands a specific identification, opinion, judgement or belief. In general, a prediction implies a much narrower and more precise statement than an explanation, often hinting at possibilities regarding future events.

Several factors influence complex systems, including self-organization, multiple views, and dynamic complexity as defined by McNabb (2023a-c). McNabb contend that in complex systems the interaction among components and between the system as a whole and its environment transcend the insights derived from a mere components analysis. Two central characteristics of complex systems are self-organization and emergence. It is important to distinguish between complex systems and complicated systems: complex systems are organic systems (comprising biological, psychological and social systems), whereas complicated systems are mechanical systems (e.g., air planes, a computer, and ABM models). The challenge of agency arises primarily in complex systems, marked by highly uncertain behaviour. Relationships within self-organized system exhibit several noteworthy properties, although, given the need for a concise discussion regarding ABM prediction, we will consider here only a few of them (McNabb, 2023a-c):

  1. Multiple views,
  2. Dynamic interactions (connexion among components changes over time),
  3. Non-linear interaction (small causes can lead to unpredictable effects),
  4. The system lacks static equilibrium (instead, it maintains a dynamic equilibrium and remains unstable),
  5. Understanding the current state necessitates examining Its history (a diachronic, not synchronic study, is essential)

Given the possibility of multiple views, a complex systems are prone to significant structural change due to  dynamic and non-linear interactions, dynamic equilibrium  and diachronic evolution. Additionally, the probability of possessing both the right change mechanism (the logical process) and complete data (addressing the challenge of data incompleteness) required to initialize the model and establish necessary assumptions is excessively low. Consequently, predicting outcomes in complex systems (defined as organic systems) whether using AMB or alternative mechanisms, becomes nearly impossible. If such prediction does occur, it typically happens under highly specific conditions, such as within a brief time frame and controlled settings, often amounting to a form of coincidental success. Only after the expected event or outcomes materializes can we definitely claim that it was predicted. Although prediction remains a challenging endeavour in complex systems, it remains viable in complicated systems. In complicated systems, prediction serves as an answer to Troitzsch’s aforementioned question.

Taking into account Troitzsch’s notion of prediction and McNabb’s ideas on complex systems and complicated systems, let’s briefly revisit the various positions presented in this special issue.

Chattoe-Brown (2023) suggests using models to predict models. Models are considered complicated rather than complex systems, so it this case, we would be predicting a complicated system rather than a complex one. This represents a significant reduction.

Dignum (2023) argues that prediction is possible in cases where there is a stable environment (conditions) and sufficient available data. However, this generally is not the case, making it challenging to meet the requirements for prediction when considering complex (organic) systems.

Carpentras and Quayle (2023) themselves acknowledge the difficulties of prediction in ABM when studying issues related to psychological systems involving psychometric measures, which are a type of organic system, aligning with our argument.

Elsenbroich and Badham (2023), Elsebroich and Polhill (2023), and Edmonds (2023) maintain a strongly sceptic position regarding ABM prediction. They argue that AMBs yield coherent extrapolations based on a specific initialization of the model and a set of assumptions, but these extrapolations are not necessarily grounded in reality. According to them, complex systems exhibit properties such as information incompleteness, multiple perspectives, emergence, evolutionary dynamics, and context specificity. In this respect, their position aligns with the stance we are presenting here.

Finally, Anzola and García-Díaz (2023) advocate for a more robust framework for prediction and recognizes the ongoing debate on prediction, an stance that closely resonates with our own.

In conclusion, Troitzsch notion of prediction and McNabb descriptions of complex systems and complicated systems have helped us better understand the diverse positions on ABM prediction in the reviewed issue. This exemplifies how a good conceptual framework, in this
case offered by appropriate notions of prediction and complexity, can
contribute to reducing the controversy surrounding ABM prediction.

References

Anzola D. and García-Díaz C. (2023). What kind of prediction? Evaluating different facets of prediction in agent-based social simulation International Journal of Social Research Methodology, 26(2), pp. 171-191. https://doi.org/10.1080/13645579.2022.2137919

Carpentras D. and Quayle M. (2023). The psychometric house-of-mirrors: the effect of measurement distortions on agent-based models’ predictions. International Journal of Social Research Methodology, 26(2), pp. 215-231. https://doi.org/10.1080/13645579.2022.2137938

Chattoe-Brown E. (2023). Is agent-based modelling the future of prediction International Journal of Social Research Methodology, 26(2), pp. 143-155. https://doi.org/10.1080/13645579.2022.2137923

Dignum F. (2023). Should we make predictions based on social simulations?}. International Journal of Social Research Methodology, 26(2), pp. 193-206. https://doi.org/10.1080/13645579.2022.2137925

Edmonds B. (2023). The practice and rhetoric of prediction – the case in agent-based modelling. International Journal of Social Research Methodology, 26(2), pp. 157-170. https://doi.org/10.1080/13645579.2022.2137921

Edmonds, B., Polhill, G., & Hales, D. (2019). Predicting Social Systems – A Challenge. https://rofasss.org/2019/11/04/predicting-social-systems-a-challenge/

Elsenbroich C. and Polhill G. (2023) Editorial: Agent-based modelling as a method for prediction in complex social systems. International Journal of Social Research Methodology, 26/2, 133-142. https://doi.org/10.1080/13645579.2023.2152007

Elsenbroich C. and Badham J. (2023). Negotiating a Future that is not like the Past. International Journal of Social Research Methodology, 26(2), pp. 207-213. https://doi.org/10.1080/13645579.2022.2137935

McNabb D. (2023a, September 20). El Paradigma de la complejidad (1/3) [Video]. YouTube. https://www.youtube.com/watch?app=desktop&v=Uly1n6tOOlA&ab_channel=DarinMcNabb

McNabb D. (2023b, September 20). El Paradigma de la complejidad (2/3) [Video]. YouTube. https://www.youtube.com/watch?v=PT2m9lkGhvM&ab_channel=DarinMcNabb

McNabb D. (2023c, September 20). El Paradigma de la complejidad (3/3) [Video]. YouTube. https://www.youtube.com/watch?v=25f7l6jzV5U&ab_channel=DarinMcNabb

Troitzsch, K. G. (2009). Not all explanations predict satisfactorily, and not all good predictions explain. Journal of Artificial Societies and Social Simulation, 12(1), 10. https://www.jasss.org/12/1/10.html

Terán, O. (2023) Agent-based Modelling as a Method for Prediction for Complex Social Systems - a review of the special issue. Review of Artificial Societies and Social Simulation, 28 Sep 2023. https://rofasss.org/2023/09/28/review-ABM-for-prediction

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

An Institute for Crisis Modelling (ICM) – Towards a resilience center for sustained crisis modeling capability

Leave a comment

By Fabian Lorig1*, Bart de Bruin2, Melania Borit3, Frank Dignum4, Bruce Edmonds5, Sinéad M. Madden6, Mario Paolucci7, Nicolas Payette8, Loïs Vanhée4

*Corresponding author
1 Internet of Things and People Research Center, Malmö University, Sweden
2 Delft University of Technology, Netherlands
3 CRAFT Lab, Arctic University of Norway, Tromsø, Norway
4 Department of Computing Science, Umeå University, Sweden
5 Centre for Policy Modelling, Manchester Metropolitan University Business School, UK
6 School of Engineering, University of Limerick, Ireland
7 Laboratory of Agent Based Social Simulation, ISTC/CNR, Italy
8 Complex Human-Environmental Systems Simulation Laboratory, University of Oxford, UK

The Need for an ICM

Most crises and disasters do occur suddenly and hit the society while it is unprepared. This makes it particularly challenging to react quick to their occurrence, to adapt to the resulting new situation, to minimize the societal impact, and to recover from the disturbance. A recent example was the Covid-19 crisis, which revealed weak points of our crisis preparedness. Governments were trying to put restrictions in place to limit the spread of the virus while ensuring the well-being of the population and at the same time preserving economic stability. It quickly became clear that interventions which worked well in some countries did not seem to have the intended effect in other countries and the reason for this is that the success of interventions to a great extent depends on individual human behavior.

Agent-based Social Simulations (ABSS) explicitly model the behavior of the individuals and their interactions in the population and allow us to better understand social phenomena. Thus, ABSS are perfectly suited for investigating how our society might be affected by different crisis scenarios and how policies might affect the societal impact and consequences of these disturbances. Particularly during the Covid-19 crisis, a great number of ABSS have been developed to inform policy making around the globe (e.g., Dignum et al. 2020, Balkely et al. 2021, Lorig et al. 2021). However, weaknesses in creating useful and explainable simulations in a short time also became apparent and there is still a lack of consistency to be better prepared for the next crisis (Squazzoni et al. 2020). Especially, ABSS development approaches are, at this moment, more geared towards simulating one particular situation and validating the simulation using data from that situation. In order to be prepared for a crisis, instead, one needs to simulate many different scenarios for which data might not yet be available. They also typically need a more interactive interface where stake holders can experiment with different settings, policies, etc.

For ABSS to become an established, reliable, and well-esteemed method for supporting crisis management, we need to organize and consolidate the available competences and resources. It is not sufficient to react once a crisis occurs but instead, we need to proactively make sure that we are prepared for future disturbances and disasters. For this purpose, we also need to systematically address more fundamental problems of ABSS as a method of inquiry and particularly consider the specific requirements for the use of ABSS to support policy making, which may differ from the use of ABSS in academic research. We therefore see the need for establishing an Institute for Crisis Modelling (ICM), a resilience center to ensure sustained crisis modeling capability.

The vision of starting an Institute for Crisis Modelling was the result of the discussions and working groups at the Lorentz Center workshop on “Agent Based Simulations for Societal Resilience in Crisis Situations” that took place in Leiden, Netherlands from 27 February to 3 March 2023**.

Vision of the ICM

“To have tools suitable to support policy actors in situations that are of
big uncertainty, large consequences, and dependent on human behavior.”

The ICM consists of a taskforce for quickly and efficiently supporting policy actors (e.g., decision makers, policy makers, policy analysts) in situations that are of big uncertainty, large consequences, and dependent on human behavior. For this purpose, the taskforce consists of a larger (informal) network of associates that contribute with their knowledge, skills, models, tools, and networks. The group of associates is composed of a core group of multidisciplinary modeling experts (ranging from social scientists and formal modelers to programmers) as well as of partners that can contribute to specific focus areas (like epidemiology, water management, etc.). The vision of ICM is to consolidate and institutionalize the use of ABSS as a method for crisis management. Although physically ABSS competences may be distributed over a variety of universities, research centers, and other institutions, the ICM serves as a virtual location that coordinates research developments and provides a basic level of funding and communication channel for ABSS for crisis management. This does not only provide policy actors with a single point of contact, making it easier for them to identify who to reach when simulation expertise is needed and to develop long-term trust relationships. It also enables us to jointly and systematically evolve ABSS to become a valuable and established tool for crisis response. The center combines all necessary resources, competences, and tools to quickly develop new models, to adapt existing models, and to efficiently react to new situations.

To achieve this goal and to evolve and establish ABSS as a valuable tool for policy makers in crisis situations, research is needed in different areas. This includes the collection, development, critical analysis, and review of fundamental principles, theories, methods, and tools used in agent-based modeling. This also includes research on data handling (analysis, sharing, access, protection, visualization), data repositories, ontologies, user-interfaces, methodologies, documentation, and ethical principles. Some of these points are concisely described in (Dignum, 2021, Ch. 14 and 15).

The ICM shall be able to provide a wide portfolio of models, methods, techniques, design patterns, and components required to quickly and effectively facilitate the work of policy actors in crisis situations by providing them with adequate simulation models. For the purpose of being able to provide specialized support, the institute will coordinate the human effort (e.g., the modelers) and have specific focus areas for which expertise and models are available. This might be, for instance, pandemics, natural disasters, or financial crises. For each of these focus areas, the center will develop different use cases, which ensures and facilitates rapid responses due to the availability of models, knowledge, and networks.

Objectives of the ICM

To achieve this vision, there are a series of objectives that a resilience center for sustained crisis modeling capability in crisis situations needs to address:

1) Coordinate and promote research

Providing quick and appropriate support for policy actors in crisis situations requires not only a profound knowledge on existing models, methods, tools, and theories but also the systematic development of new approaches and methodologies. This is to advance and evolve ABSS for being better prepared for future crises and will serve as a beacon for organizing the ABSS research oriented towards practical applications.

2) Enable trusted connections with policy actors

Sustainable collaborations and interactions with decision-makers and policy analysts as well as other relevant stakeholders is a great challenge in ABSS. Getting in contact with the right actors, “speaking the same language”, and having realistic expectations are only some of the common problems that need to be addressed. Thus, the ICM should not only connect to policy actors in times of crises, but have continuous interactions, provide sample simulations, develop use cases, and train the policy actors wherever possible.

3) Enable sustainability of the institute itself

Classic funding schemes are unfit for responding in crises, which require fast responses with always-available resources as well as the continuous build-up of knowledge, skills, network, and technological buildup requires long-term. Sustainable funding is needed that for enabling such a continuity, for which the IBM provides a demarked, unifying frame.

4) Actively maintain the network of associates

Maintaining a network of experts is challenging because it requires different competences and experiences. PhD candidates, for instance, might have a great practical experience in using different simulation frameworks, however, after their graduation, some might leave academia and others might continue to other positions where they do not have the opportunity to use their simulation expertise. Thus, new experts need to be acquired continuously to form a resilient and balanced network.

5) Inform policy actors

The most advanced and profound models cannot do any good in crisis situations in case of a lacking demand from policy actors. Many modelers perceive a certain hesitation from policy actors regarding the use of ABSS which might be due to them being unfamiliar with the potential benefits and use-cases of ABSS, lacking trust in the method itself, or simply due to a lack of awareness that ABSS actually exists. Hence, the center needs to educate policy makers and raise awareness as well as improve trust in ABSS.

6) Train the next generation of experts

To quickly develop suitable ABSS models in critical situations requires a variety of expertise. In addition to objective 4, the acquisition of associates, it is also of great importance to educate and train the next generation of experts. ABSS research is still a niche and not taught as an inherent part of the spectrum of methods of most disciplines. The center shall promote and strengthen ABSS education to ensure the training of the next generation of experts.

7) Engage the general public

Finally, the success of ABSS does not only depend on the trust of policy actors but also on how it is perceived by the general public. When developing interventions during the Covid-19 crisis and giving recommendations, the trust in the method was a crucial success factor. Also, developing realistic models requires the active participation of the general public.

Next steps

For ABSS to become a valuable and established tool for supporting policy actors in crisis situations, we are convinced that our efforts need to be institutionalized. This allows us to consolidate available competences, models, and tools as well as to coordinate research endeavors and the development of new approaches required to ensure a sustained crisis modeling capability.

To further pursue this vision, a Special Interest Group (SIG) on Building ResilienCe with Social Simulations (BRICSS) was established at the European Social Simulation Association (ESSA). Moreover, Special Tracks will be organized at the 2023 Social Simulation Conference (SSC) to bring together interested experts.

However, for this vision to become reality, the next steps towards establishing an Institute for Crisis Modelling consist of bringing together ambitious and competent associates as well as identifying core funding opportunities for the center. If the readers feel motivated to contribute in any way to this topic, they are encouraged to contact Frank Dignum, Umeå University, Sweden or any of the authors of this article.

Acknowledgements

This piece is a result of discussions at the Lorentz workshop on “Agent Based Simulations for Societal Resilience in Crisis Situations” at Leiden, NL in earlier this year! We are grateful to the organisers of the workshop and to the Lorentz Center as funders and hosts for such a productive enterprise. The final report of the workshop as well as more information can be found on the webpage of the Lorentz Center: https://www.lorentzcenter.nl/agent-based-simulations-for-societal-resilience-in-crisis-situations.html

References

Blakely, T., Thompson, J., Bablani, L., Andersen, P., Ouakrim, D. A., Carvalho, N., Abraham, P., Boujaoude, M.A., Katar, A., Akpan, E., Wilson, N. & Stevenson, M. (2021). Determining the optimal COVID-19 policy response using agent-based modelling linked to health and cost modelling: Case study for Victoria, Australia. Medrxiv, 2021-01. doi: 10.1101/2021.01.11.21249630

Dignum, F., Dignum, V., Davidsson, P., Ghorbani, A., van der Hurk, M., Jensen, M., Kammler C., Lorig, F., Ludescher, L.G., Melchior, A., Mellema, R., Pastrav, C., Vanhee, L. & Verhagen, H. (2020). Analysing the combined health, social and economic impacts of the coronavirus pandemic using agent-based social simulation. Minds and Machines, 30, 177-194. doi: 10.1007/s11023-020-09527-6

Dignum, F. (ed.). (2021) Social Simulation for a Crisis; Results and Lessons from Simulating the COVID-19 Crisis. Springer.

Lorig, Fabian, Johansson, Emil and Davidsson, Paul (2021) ‘Agent-Based Social Simulation of the Covid-19 Pandemic: A Systematic Review’ Journal of Artificial Societies and Social Simulation 24(3), 5. http://jasss.soc.surrey.ac.uk/24/3/5.html. doi: 10.18564/jasss.4601

Squazzoni, F. et al. (2020) ‘Computational Models That Matter During a Global Pandemic Outbreak: A Call to Action‘ Journal of Artificial Societies and Social Simulation 23(2), 10. http://jasss.soc.surrey.ac.uk/23/2/10.html. doi: 10.18564/jasss.4298

Lorig, F., de Bruin, B., Borit, M., Dignum, F., Edmonds, B., Madden, S.M., Paolucci, M., Payette, N. and Vanhée, L. (2023) An Institute for Crisis Modelling (ICM) –
Towards a resilience center for sustained crisis modeling capability. Review of Artificial Societies and Social Simulation, 22 May 2023. https://rofasss.org/2023/05/22/icm

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)