Tag Archives: ABM

Content

A Reminder – computability limits on “vibe coding” ABMs (using LLMS to do the programming for us)

Leave a comment

By Bruce Edmonds

Introduction

Machine-learning systems, including Large Language Models (LLMs), are algorithms trained on large datasets rather than something categorically different. Consequently, they inherit the standard theoretical and practical limitations that apply to all algorithmic methods. Here we look at the computational limits in terms of “Vibe coding” – where the LLM writes some of the code for an Agent-Based Model (ABM) in response to a descriptive prompt. Firstly, we review a couple of fundamental theorems in terms of producing or checking code relative to its descriptive specification. In general, this is shown to be impossible (Edmonds & Bryson 2004). However, this does not rule out the possibility that this could work with simpler classes of code that fall short of full Turing completeness (being able to mimic any conceivable computation). Thus, I recap a result that shows how simple a class of ABMs can be and still be Turing complete (and thus subject to the previous results) (Edmonds 2004). If you do not like discussion of proofs, I suggest you skip to the concluding discussion. More formal versions of the proofs can be found in the Appendix.

Descriptions of Agent-Based Models

When we describe code, the code should be consistent with that description. That is, the code should be one of the possible codes which are right for the code. In this paper we are crucially concerned with the relationship between code and its description and the difficulty of passing from one to the other.

When we describe an ABM, we can do so in a number of different ways. We can do this with different degrees of formality: from chatty natural language to pseudo code to UML diagrams. The more formal the method of description, the less ambiguity there is concerning its meaning. We can also describe what we want at low or high levels. A low-level description specifies the detail of what bit of code does at each time – an imperative description. A high-level description specifies what the system should do as a whole or what the results should be like. These tend to be declarative descriptions.

Whilst a compiler takes a formal, low-level description of code, an LLM takes a high-level, informal description – in the former case the description is very prescriptive with the same description always producing the same code, but in the case of LLMs there are usually a great many sets of code that are consistent with the input prompt. In other words, the LLM makes a great many decisions for the user, saving time – decisions that a programmer would be forced to confront if using a compiler (Keles 2026).

Here, we are focusing on the latter case, when we use an LLM to do some or all of our ABM programming for us. We use high-level, informal natural language in order to direct the LLM as to what ABM (or aspect of ABM) we would like. Of course, one can be more precise in one’s use of language, but it will tend to remain at a fairly high level (if we are going to do a complete low-level description then we might as well write the code ourselves).

In the formal results below, we restrict ourselves to formal descriptions as this is necessary to do any proofs. However, what is true below for formal descriptions is also true for the wider class of any description as one can always use natural language in a precise manner. For a bit more detail in what we mean here by a formal language, see the Appendix.

The impossibility of a general “specification compiler”

The dream is that one could write a description of what one would like and an algorithm, T, would produce code that fitted that description. However, to enable proofs to be used we need to formalize the situation so that the description, is in some suitably expressive but formal language (e.g. a logic with enumerable expressions). This situation is illustrated in figure 1.

Figure 1. Automatically generating code from a specification.

Obviously, it is easy to write an impossible formal specification – one for which no code exists –so the question is whether there could be such an algorithm, T, that would give us code that fitted a specification when it does exist. The proof is taken from (Edmonds & Bryson 2004) and given in more formal detail in the Appendix.

The proof uses a version of Turing’s “halting problem” (Turing 1937). This is the problem of checking some code (which takes a number as an input parameter) to see if would come to a halt (the program finishes) or go on for ever. The question here is whether there is any effective and systematic way of doing this. In other words, whether there an “automatic program checker” is possible – a program, H, which takes two inputs: the program number, x, and a possible input, y and then works out if the computation Px(y) ever ends.  Whilst in some cases spotting this is easy – e.g. trivial infinite loops – other cases are hard (e.g. testing the even numbers to find one that is not the sum of two prime numbers1).

For our purposes, let us consider a series of easier problems – what I call “limited halting problems”. This is the problem of checking whether programs, x, applied to inputs y ever come to an end, but only for x, y ≤ n, where n is a fixed upper limit. Imagine a big n ´ n table with the columns being the program numbers and the rows being the inputs. Each element is 0 if the combination never stops and 1 if it does. A series of simpler checking programs, Hn, would just look up the answers in the table as long as they had been filled in correctly. We know that these programs exist, since programs that implement simple look up tables always exist and that one of the possible n x´n tables will be the right one for Hn. For each limited halting problem, we can write a formal specification for this, giving us a series of specifications (one for each n).

Now imagine that we had a general-purpose specification compiling program, T, as described above and illustrated in Figure 1. Then, we could do the following:

  1. work out max(x,y)
  2. given any computation Px(y) we could construct the specification for the limited halting problem with index, max(x,y)
  3. then we could use T to construct some code for Hn and
  4. use that code to see if Px(y) ever halted.

Taken together, these steps (a-d) can be written as a new piece of computer code that would solve the general halting problem. However, this we know is impossible (Turing 1937), therefore there is not a general compiling program like T above, it is impossible.

The impossibility of a general “code checker”

The checking problem is apparently less ambitious than the programming problem – here we are given a program and a specification and have ‘only’ to check whether they correspond.  That is whether the code satisfies the specification.

Figure 2. Algorithmically checking if some code satisfies a descriptive specification.

Again, the answer is in the negative.  The proof to this is similar. If there were a checking program C that, given a program and a formal specification would tell us whether the program met the specification, we could again solve the general halting problem.  We would be able to do this as follows:

  1. work out the maximum of x and y (call this m);
  2. construct a sequence of programs implementing all possible finite lookup tables of type: mxm→{0,1};
  3. test these programs one at a time using C to find one that satisfies SHn (we know there is at least one);
  4. use this program to compute whether Px(y) halts. 

Thus, there is no general specification checking program, like C.

Thus, we can see that there are some, perfectly well-formed specifications, where we know code exists that would comply with the specification but where there is no such algorithm, however clever, that will always take us from a specification to the code. Since trained neural nets are a kind of clever algorithm, they cannot do this either.

What about simple Agent-Based Models?

To illustrate how simple such systems can be, I defined a particular class of particularly simple multi-agent system, called “GASP” systems (Giving Agent System with Plans).  These are defined as follows.  There are n agents, labelled: 1, 2, 3, etc., each of which has an integer store which can change and a finite number of simple plans (which do not change).  Each time interval the store of each agent is incremented by one.  Each plan is composed of: a (possibly empty) sequence of ‘give instructions’ and finishes with a single ‘test instruction’.  Each ‘give instruction’, Ga, has the effect of giving 1 unit to agent a (if the store is non-zero).  The ‘test instruction’ is of the form JZa,p,q, which has the effect of jumping (i.e. designating the plan that will be executed next time period) to plan p if the store of agent a is zero and plan q otherwise.  This class is described more in (Edmonds 2004). This is illustrated in Figure 3.

Figure 3. An Illustration of a “GASP” multi-agent system.

Thus ‘all’ that happens in this class of GASP systems is the giving of tokens with value 1 to other agents and the testing of other agents’ store to see if they are zero to determine the next plan.  There is no fancy communication, learning or reasoning done by agents. Agents have fixed and very simple plans and only one variable. However, this class of agent can be shown to be Turing Complete. The proof is taken from (Edmonds & Bryson 2004).

The detail of this proof is somewhat tedious, but basically involves showing that any computation (any Turing Machine) can be mapped to a GASP machine using a suitable effective and systematic mapping. This is done in three stages. That is for any particular Turing Machine:

  1. Create an equivalent “Unlimited Register Machine” (URM), with an indefinitely large (but finite) number of integer variables and four basic kinds of instruction (add one to a variable, set a variable to 0, copy the number in a variable to another, jump to a set instruction if two specified variables are equal. This is known to be possible (Cutland page 57).
  2. Create an equivalent “AURA” machine for this URM machine (Moss & Edmonds 1994)
  3. Create an equivalent “GASP” ABM for this AURA system.

This is unsurprising – many systems that allow for an indefinite storage, basic arithmetic operations and a kind of “IF” statement are Turing Complete (see any textbook on computability, e.g. Cutland 1980).

This example of a class of GASP agents shows just how simple an ABM can be and still be Turing Complete, and subject to the impossibility of a general compiler (like T above) or checker (like C above), however clever these might be.

Discussion

What the above results show is that:

  1. There is no algorithm that will take any formal specification and give you code that satisfies it. This includes trained LLMs.
  2. There is no algorithm that will take any formal specification and some code and then check whether the code satisfies that specification. This includes trained LLMs.
  3. Even apparently simple classes of agent-based model are capable of doing any computation and so there will be examples where the above two negative results hold.

These general results do not mean that there are not special cases where programs like T or C are possible (e.g. compilers). However, as we see from the example ABMs above, it does not take much in the way of its abilities to make this impossible for high level descriptions. Using informal, rather than formal, language does not escape these results, but merely adds more complication (such as vagueness).

In conclusion, this means that there will be kinds of ABMs for which no algorithm can turn descriptions into the correct, working code2. This does not mean that LLMs can’t be very good at producing working code from the prompts given to them. They might (in some cases) be better than the best humans at this but they can never be perfect. There will always be specifications where they either cannot produce the code or produce the wrong code.

The underlying problem is that coding is a very hard, in general. There are no practical, universal methods that always work – even when it is known to be possible. Suitably-trained LLMs, human ingenuity, various methodologies can help but none will be a panacea.

Notes

1. Which would disprove “Goldbach’s conjecture”, whose status is still unknown despite centuries of mathematical effort. If there is such a number it is known to be more than 4×1017.

2. Of course, if humans are limited to effective procedures – ones that could be formally written down as a program (which seems likely) – then humans are similarly limited.

Acknowledgements

Many thanks for the very helpful comments on earlier drafts of this by Peer-Olaf Siebers, Luis Izquierdo and other members of the LLM4ABM SIG. Also, the participants of AAMAS 2004 for their support and discussion on the formal results when they were originally presented.

Appendix

Formal descriptions

The above proofs rely on the fact that the descriptions are “recursively enumerable”, as in the construction of Gödel (1931). That is, one can index the descriptions (1, 2, 3…) in such a way that once can reconstruct the description from the index. Most formal languages, including those compilers take, computer code, formal logic expressions, are recursively enumerable since they can be constructed from an enumerable set of atoms (e.g. variable names) using a finite number of formal composition rules (e.g. if A and B are allowed expressions, then so are A → B, A & B etc. Any language that can be specified using syntax diagrams (e.g. using Backus–Naur form) will be recursively enumerable in this sense.

Producing code from a specification

The ‘halting problem’ is an undecidable problem (Turing 1937), (that is it is a question for which there does not exist a program that will answer it, say outputting 1 for yes and 0 for no).  This is the problem of whether a given program will eventually come to a halt with a given input.  In other words, whether Px(y), program number x applied to input y, ever finishes with a result or whether it goes on for ever.  Turing proved that there is no such program (Turing 1937).

Define a series of problems, LH1, LH2, etc., which we call ‘limited halting problems’.  LHn is the problem of ‘whether a program with number £n and an input £n will ever halt’.  The crucial fact is that each of these is computable, since each can be implemented as a finite lookup table.  Call the programs that implement these lookup tables: PH1, PH2, etc. respectively.  Now if the specification language can specify each such program, one can form a corresponding enumeration of formal specifications: SH1, SH2,etc. 

The question now is whether there is any way of computationally finding PHn from the specification SHn.  But if there were such a way we could solve Turing’s general halting problem in the following manner: first find the maximum of x and y (call this m); then compute PHm from SHm; and finally use PHm to compute whether Px(y) halts.  Since we know the general halting problem is not computable, we also know that there is no effective way of discovering PHn from SHn even though for each SHn we know an appropriate PHn exists!

Thus, the only question left is whether the specification language is sufficiently expressive to enable SH1, SH2, etc. to be formulated.  Unfortunately, the construction in Gödel’s famous incompleteness proof (Gödel 1931) guarantees that any formal language that can express even basic arithmetic properties will be able to formulate such specifications.

Checking code meets a specification

To demonstrate this, we can reuse the limited halting problems defined in the last subsection.  The counter-example is whether one can computationally check (using C) that a given program P meets the specification SHn.  In this case we will limit ourselves to programs, P, that implement n´n finite lookup tables with entries: {0,1}. 

Now we can see that if there were a checking program C that, given a program and a formal specification would tell us whether the program met the specification, we could again solve the general halting problem.  We would be able to do this as follows: first find the maximum of x and y (call this m); then construct a sequence of programs implementing all possible finite lookup tables of type: mxm→{0,1}; then test these programs one at a time using C to find one that satisfies SHn (we know there is at least one: PHm);and finally use this program to compute whether Px(y) halts.  Thus, there is no such program, C.

Showing GASP ABMs are Turning Complete

The class of Turing machines is computationally equivalent to that of unlimited register machines (URMs) (Cutland page 57).  That is the class of programs with 4 types of instructions which refer to registers, R1, R2, etc. which hold positive integers.  The instruction types are: Sn, increment register Rn by one; Zn, set register Rn to 0; Cn,m, copy the number from Rn to Rm (erasing the previous value); and Jn,m,q, if Rn=Rm jump to instruction number q.  This is equivalent to the class of AURA programs which just have two types of instruction: Sn, increment register Rn by one; and DJZn,q, decrement Rn if this is non-zero then if the result is zero jump to instruction step q (Moss & Edmonds 1994).   Thus we only need to prove that given any AURA program we can simulate its effect with a suitable GASP system.  Given an AURA program of m instructions: i1, i2,…, im which refers to registers R1, …, Rn, we construct a GASP system with n+2 agents, each of which has m plans.   Agent An+1 is basically a dump for discarded tokens and agent An+2 remains zero (it has the single plan: (Gn+1, Ja+1,1,1)). Plan s (sÎ{1,…,m}) in agent number a (aÎ{1,…,n}) is determined as follows: there are four cases depending on the nature of instruction number s:

1.        is is Sa: plan s is (Ja,s+1,s+1);

2.        is is Sb where b¹a: plan s is (Gn+1, Ja,s+1,s+1);

3.        is is DJZa,q: plan s is (Gn+1, Gn+1, Ja,q,s+1);

4.        is is DJZb,q where b¹a: plan s is (Gn+1, Ja,q,s+1).

Thus, each plan s in each agent mimics the effect of instruction s in the AURA program with respect to the particular register that the agent corresponds to.

References

Cutland, N. (1980) Computability: An Introduction to Recursive Function Theory. Oxford University Press.

Edmonds, B. (2004) Using the Experimental Method to Produce Reliable Self-Organised Systems. In Brueckner, S. et al. (eds.) Engineering Self Organising Sytems: Methodologies and Applications, Springer, Lecture Notes in Artificial Intelligence, 3464:84-99. http://cfpm.org/cpmrep131.html

Edmonds, B. & Bryson, J. (2004) The Insufficiency of Formal Design Methods – the necessity of an experimental approach for the understanding and control of complex MAS. In Jennings, N. R. et al. (eds.) Proceedings of the 3rd Internation Joint Conference on Autonomous Agents & Multi Agent Systems (AAMAS’04), July 19-23, 2004, New York. ACM Press, 938-945. http://cfpm.org/cpmrep128.html

Gödel, K. (1931), Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme, I, Monatshefte für Mathematik und Physik, 38(1):173–198. http://doi.org/10.1007/BF01700692

Keles, A. (2026) LLMs could be, but shouldn’t be compilers. Online https://alperenkeles.com/posts/llms-could-be-but-shouldnt-be-compilers/ (viewed 11 Feb 2026)

Moss, S. and Edmonds, B. (1994) Economic Methodology and Computability: Some Implications for Economic Modelling, IFAC Conf. on Computational Economics, Amsterdam, 1994. http://cfpm.org/cpmrep01.html

Turing, A.M. (1937), On Computable Numbers, with an Application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, s2-42: 230-265. https://doi.org/10.1112/plms/s2-42.1.230

Edmonds, B. (2026) A Reminder – computability limits on “vibe coding” ABMs (using LLMS to do the programming for us). Review of Artificial Societies and Social Simulation, 12 Feb 2026. https://rofasss.org/2026/02/12/vibe

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Why Object-Oriented Programming is not the best method to implement Agent-Based Models

Leave a comment

By Martin Hinsch

Research Department of Genetics, Evolution and Environment
University College London

Introduction

A considerable part of the history of software engineering consists of attempts to make the complexity of software systems manageable in the sense of making them easier to implement, understand, modify, and extend. An important aspect of this is the separation of concerns (SoC, Dijkstra 1982). SoC reduces complexity by dividing the implementation of a system into (presumably simpler) problems that can be solved without having to spend too much thought on other aspects of the system. Architecturally, SoC is accomplished through modularity and encapsulation. This means that parts of the system that have strong inter-dependencies are put together into a “module” (in the widest sense) that presents only aspects of itself to the outside that are required to interact with other modules. This is based on the fundamental assumption that the visible behaviour of a component (its interface) is simpler than its potentially complex inner workings which can be ignored when interacting with it.

The history of Object-Oriented Programming (OOP) is complicated and there are various flavours and philosophies of OOP (Black 2013). However, one way to see Object Orientation (OO) is as a coherent, formalised method to ensure modularisation and encapsulation. In OOP data and functions to create and manipulate that data are combined in objects. Depending on programming language, some object functions (methods) and properties can be made inaccessible to users of the object, thereby hiding internal complexity and presenting a simplified behaviour to the outside. Many OO languages furthermore allow for polymorphism, i.e. different types of objects can have the same interface (but different internal implementations) and can therefore be used interchangeably.

After its inception in the 1960s (Dahl and Nygaard 1966) OOP gained popularity throughout the 80s and 90s, to the point that many established programming languages were retrofitted with language constructs that enabled OOP (C++, Delphi, OCaml, CLOS, Visual Basic, …) and new languages were designed based on OOP principles (Smalltalk, Python, Ruby, Java, Eiffel,…). By the mid-90s many computer science departments taught OOP not as one, or even just a useful paradigm, but as the paradigm that would make all other methods obsolete.

This is the climate in which agent-based or individual-based modelling (ABM) emerged as a new modelling methodology. In ABM the behaviour of a system is not modelled directly but instead the model consists of (many, similar or identical) individual components. The interactions between these components leads to the emergence of global behaviour.

While the origins of the paradigm reach further back, it only started to become popular in the 90s (Bianchi and Squazzoni 2015). As the method requires programming expertise, which in academia was rare outside of computer science, the majority of early ABMs were created by or with the help of computer scientists, which in turn applied the at the time most popular programming paradigm. At first glance OOP also seems to be an excellent fit for ABM – agents are objects, their state is represented by object variables, and their behaviour by methods. It is therefore no surprise that OOP has become and remained the predominant way to implement ABMs (even after the enthusiasm for OOP has waned to some degree in mainstream computer science).

In the following I will argue that OOP is not only not necessarily the best method to write ABMs, but that it has, in fact, some substantial drawbacks. More specifically, I think that the claim that OOP is uniquely suited for ABM is based on a conceptual confusion that can lead to a number of bad modelling habits. Furthermore the specific requirements of ABM implementations do not mesh well with an OOP approach.

Sidenote: Strictly speaking, for most languages we have to distinguish between objects (the entities holding values) and classes (the types that describe the makeup and functionality of objects). This distinction is irrelevant for the point I am making, therefore I will only talk about objects.

Conceptual confusion

About every introduction to OOP I have come across starts with a simple toy example that demonstrates core principles of the methods. Usually a few classes corresponding to everyday objects from the same category are declared (e.g. animal, cat, dog or vehicle, car, bicycle). These classes have methods that usually correspond to activities of these objects (bark, meow, drive, honk).

Beyond introducing the basic syntax and semantics of the language constructs involved, these introductions also transport a message: OOP is easy and intuitive because OOP objects are just representations of objects from the real world (or the problem domain). OOP is therefore simply the process of translating objects in the problem domain into software objects.

OOP objects are not representations of real-world objects

While this approach makes the concept of OOP more accessible, it is misleading. At its core the motivation behind OOP is the reduction of complexity by rigorous application of some basic tenets of software engineering (see Introduction). OOP objects therefore are not primarily defined by their representational relationship to real-world objects, but by their functionality as modules in a complicated machine.

For programmers, this initial misunderstanding is harmless as they will undergo continued training. For nascent modellers without computer science background, however, these simple explanations often remain the extent of their exposure to software engineering principles, and the misunderstanding sticks. This is unfortunately further reinforced by many ABM tutorials. Similar to introductions to OOP they present the process of the implementation of an ABM as simply consisting of defining agents as objects, with object properties that represent the real-world entities’ state and methods that implement their behaviour.

At this point a second misunderstanding almost automatically follows. By emphasising a direct correspondence between real-world entities and OOP objects, it is often implied (and sometimes explicitly stated) that modelling is, in fact, the process of translating from one to the other.

OOP is not modelling

As mentioned above, this is a misinterpretation of the intent behind OOP – to reduce software complexity. Beyond that, however, it is also a misunderstanding of the process of modelling. Unfortunately, it connects very well with a common “lay theory of modelling” that I have encountered many times when talking to domain experts with no or little experience with modelling: the idea that a model is a representation of a real system where a “better” or “more correct” representation is a better model.

Models are not (simply) representations

There are various ways to use a model and reasons to do it (Epstein 2008), but put in the most general terms, a (simulation) model is an artificial (software) system that in one way or other teaches us something about a real system that is similar in some aspects (Noble 1997). Importantly, however, the question or purpose for which the model was built determines which aspects of the real system will be part of the model. As a corollary, even given the same real-world system, two models with different questions can look very different, to the point that they use different modelling paradigms (Hinsch and Bijak 2021).

Experienced modellers are aware of all this, of course, and will not be confused by objects and methods. For novices and domain experts without that experience, however, OOP and the way it is taught in connection with ABM can lead to a particular style of modelling where first, all entities in the system are captured as agents, and second, these agents are being equipped with more and more properties and methods, “because it is more realistic”. 

An additional issue with this is that it puts the focus of the modelling process on entities. The direct correspondence between nouns in our (natural language-based) description of the model and classes in our object-oriented implementation makes it very tempting to think about the model solely in terms of entities and their properties.

ABMs are not (just) collections of entities

There are other reasons to build a simulation model, but in most cases the dynamic behaviour of the finished model will be crucial. The reason to use an ABM as opposed to, say, a differential equation model, is not that the system is composed of entities, but that the behaviour of the system depends in such a way on interactions between entities that it cannot be reduced to aggregate population behaviour. The “interesting” part of the model is therefore not the agents per se, but their behaviour and the interactions between them. It is only possible to understand the model’s macroscopic behaviour (which is often the goal of ABM) by thinking about it in terms of microscopic interactions. When creating the model it is therefore crucial to think not (only) about which entities are part of the system, but primarily which entity-level interactions and behaviours are likely to affect the macroscopic behaviour of interest.

To summarise the first part, OOP is a software engineering methodology, not a way to create models. This unfortunately often gets lost in the way it is commonly taught (in particular in connection with ABM), so that OOP can easily lead to a mindset that sees models as representations, emphasises “realism”, and puts the focus on entities rather than the more important interactions.

Practical considerations

But assuming a seasoned modeller who understands all this – surely there would be no harm in choosing an OOP implementation?

At first approximation this is certainly true. The points discussed above apply to the modelling process, so assuming all of the mentioned pitfalls are avoided, the implementation should only be a matter of translating a formal structure into working program code. As long as the code is exactly functionally equivalent to the formal model, it should not matter which programming paradigm is used.

In reality things are a little bit more complicated, however. For a number of reasons model code has different properties and requirements to “normal” code. These combine to make OOP not very suitable for the implementation of ABMs.

OOP does not reduce complexity of an ABM

Any non-trivial piece of software is too complicated to understand all at once. At the same time, we usually want its behaviour to be well-defined, well-understood, and predictable. OOP is a way to accomplish this by partitioning the complexity into manageable pieces. By composing the program of simple(r) modules, which in turn have well-defined, well-understood, and predictable behaviour and which interact in a simple, predictable manner, the complexity of the system remains manageable and understandable.

An ABM has parts that we want to be well-understood and predictable as well, such as parameterisation, data output, visualisation, etc. For these “technical” parts of the simulation program, the usual rules of software engineering apply, and OOP can be a helpful technique. The “semantic” part, i.e. the implementation of the model itself is different, however. By definition, the behaviour of a model is unpredictable and difficult to understand. Furthermore, in an ABM the complexity of the model behaviour is the result of the (non-linear) interactions between its components – the agents – which themselves are often relatively simple. The benefit of OOP – a reduction in complexity by hiding it behind simple object interfaces – therefore does not apply for the semantic part of the implementation of an ABM.

OOP makes ABMs more difficult to read and understand

There is more, however. Making code easy to read and understand is an important part of good practice in programming. This holds even more so for ABM code.

First, most ordinary application code is constructed to produce very specific runtime behaviour. To put it very simply – if the program does not show that behaviour, we have found an error; if it does, our program is by definition correct. For ABM code the behaviour can not be known in advance (otherwise we would not need to simulate). Some of it can be tested by running edge cases with known behaviour, but to a large degree making sure that the simulation program is implemented correctly has to rely on inspection of the source code.

Second, for more complicated models such as ABMs the simulation program is very rarely just the translation of a formal specification. Language is inherently ambiguous and to my knowledge there is no practical mathematical notation for ABMs (or software in general). Given further factors such as turnaround times of scientific work, ambiguity of language and documentation drift, it is often unavoidable that the code remains the ultimate authority on what the model does. In fact, good arguments have been made to embrace this reality and its potential benefits (Meisser 2016), but even so we have to live with the reality that for most ABMs, most of the time, the code is the model.

Finally, an important part of the modelling process is working out the mechanisms that lead to the observed behaviour. This involves trying to relate the observed model behaviour to the effect of agent interactions, often by modifying parameter values or making small changes to the model itself. During this process, being able to understand at a glance what a particular piece of the model does can be very helpful.

For all of these reasons, readability and clarity are paramount for ABM code. Implementing the model in an OO manner directly contradicts this requirement. We would try to implement most functionality as methods of an object. The processes that make up the dynamic behaviour of the model – the interactions between the agents – are then split into methods belonging to various objects. Someone who tries to understand – or modify – a particular aspect of the behaviour then has to jump between these methods, often distributed over different files, having to assemble the interactions that actually take place in their mind. Furthermore, encapsulation, i.e. the hiding of complexity behind simple interfaces, can make ABM code more difficult to understand by giving the misleading impression of simplicity. If we encounter agent.get_income() for example, we might access a simple state variable or we might get the result of a complex calculation. For normal code this would not make a difference since the potential complexity hidden behind that function call should not affect the caller. For ABM code, however, the difference might be crucial.

To sum up the second part – due to the way complexity arises in ABMs an OOP implementation does not lead to simplification, but on the contrary can make the code more difficult to understand and maintain.

Conclusion and discussion

Obviously none of the points mentioned above are absolutes and excellent models have been created using object-oriented languages and implementation principles. However, I would like to argue that the current state of affairs where object-orientation is uncritically presented as the best or even only way to implement agent-based models does on average lead to worse models and worse model implementations. I think that in the future any beginner’s course on agent-based modelling should at least:

  • Clarify the difference between model and implementation.
  • Show examples of the same model implemented according to a number of different paradigms.
  • Emphasise that ABMs are about interactions, not entities.

Concerning best practices for implementation, I think readability is the best guideline. Personally, I have found it useful to implement agents as “shallow” objects with the rule of thumb that only functions that a) have an obvious meaning and b) only affect the agent in question become methods implemented at the same place as the agent definition. Everything else is implemented as free functions, which can then be sorted into files by processes, e.g. ’reproduction’ or ’movement’. This avoids philosophical problems – does infection in a disease model, for example, belong to the agents, some environment object or maybe even a disease object? But above all it makes it easy to quickly find and understand a specific aspect of the model.

If at the same time the model code is kept as independent of the parts of the code that manages technical infrastructure (such as parameter loading or gui) as possible, we can maintain the implementation of the model (and only the model) as a self-contained entity in a form that is optimised for clarity and readability.

References

Bianchi, Federico, and Flaminio Squazzoni. 2015. “Agent-Based Models in Sociology.” Wiley Interdisciplinary Reviews: Computational Statistics 7 (4): 284–306. https://doi.org/10.1002/wics.1356.

Black, Andrew P. 2013. “Object-Oriented Programming: Some History, and Challenges for the Next Fifty Years.” Information and Computation, Fundamentals of Computation Theory, 231 (October): 3–20. https://doi.org/10.1016/j.ic.2013.08.002.

Dahl, Ole-Johan, and Kristen Nygaard. 1966. “SIMULA: An ALGOL-Based Simulation Language.” Commun. ACM 9 (9): 671–78. https://doi.org/10.1145/365813.365819.

Dijkstra, Edsger W. 1982. “On the Role of Scientific Thought.” In Selected Writings on Computing: A Personal Perspective, edited by Edsger W. Dijkstra, 60–66. New York, NY: Springer. https://doi.org/10.1007/978-1-4612-5695-3_12.

Epstein, Joshua M. 2008. “Why Model?” Jasss-the Journal of Artificial Societies and Social Simulation 11 (4): 12. https://doi.org/10.13140/2.1.5032.9927.

Hinsch, Martin, and Jakub Bijak. 2021. “Principles and State of the Art of Agent-Based Migration Modelling.” In Towards Bayesian Model-Based Demography: Agency, Complexity and Uncertainty in Migration Studies. Methodos Series 17, 33-49.

Meisser, Luzius. 2016. “The Code Is the Model.” International Journal of Microsimulation 10 (3): 184–201. https://doi.org/10.34196/ijm.00169.

Noble, Jason. 1997. “The Scientific Status of Artificial Life.” In Poster Presented at the Fourth European Conference on Artificial Life (ECAL97), Brighton, UK.

Hinsch, M.(2025) Why Object-Oriented Programming is not the best method to implement Agent-Based models. Review of Artificial Societies and Social Simulation, 3 Feb 2026. https://rofasss.org/2026/02/03/oop

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Modelling Deep Structural Change in Agent-Based Social Simulation

Leave a comment

By Thorid Wagenblast1, Nicholas Roxburgh2 and Alessandro Taberna3

1 Delft University of Technology, 0009-0003-5324-3778
2 The James Hutton Institute, 0000-0002-7821-1831
3 CMCC Foundation – Euro-Mediterranean Center on Climate Change, RFF-CMCC European Institute on Economics and the Environment, 0000-0002-0207-4148

Introduction

Most agent-based models (ABMs) are designed around the assumption of a broadly stable system architecture. Whether exploring emergent dynamics or testing the effects of external interventions or stressors, such models typically operate with a fixed ontology – predefined agent types, attribute classes, behavioural repertoires, processes, and social and institutional structures. While this can allow rich exploration of dynamics within the given configuration, it limits the model’s possibility space by excluding forms of change that would require the structure itself to evolve.

Some of the most consequential forms of real-world change involve shifts in the system architecture itself. These forms of change – what we refer to here as deep structural change – reconfigure the underlying logic and potentialities of the system. This may involve, for example, dramatic shifts in the environment in which agents operate, the introduction of novel technologies, or reshaping of the roles and categories through which agents understand and act in the world. Such transformations pose a fundamentally different challenge from those typically addressed in most agent-based modelling studies to date – one that pushes beyond parameter tuning or rule adjustment, and calls for new approaches to ontology design, model construction, and the conceptualisation of structural transformation and uncertainty in simulation.

Various theoretical lenses can be applied to this topic. The concepts of transformations or regime shifts seem particularly pertinent. Transformations, in contrast to incremental or minor changes, are changes that are large-scale and significant, but apart from that do not seem to consist of any specific features (Feola, 2015). The changes we explore here are more closely linked to regime shifts, which are characterised by structural changes, but with a notion of abruptness. Methods to detect and understand these regime shifts and the structural changes in relation to social simulation have been discussed for some time (Filatova, Polhill & van Ewijk, 2016). Nonetheless, there is still a lack of understanding around what this structural change entails and how this applies in social simulation, particularly ABMs.

To explore these issues, the European Social Simulation Association (ESSA) Special Interest Group on Modelling Transformative Change (SIG-MTC) organised a dedicated session at the Social Simulation Fest 2025. The session aimed to elicit experiences, ideas, and emerging practices from the modelling community around how deep structural change is understood and approached in agent-based simulation. Participants brought perspectives from a wide range of modelling contexts – including opinion dynamics, energy systems, climate adaptation, food systems, and pandemic response – with a shared interest in representing deep structural change. A majority of participants (~65%) reported that they were already actively working on, or thinking about, aspects of deep structural change in their modelling practice.

The session was framed as an opportunity to move beyond static ontologies and explore how models might incorporate adaptive structures or generative mechanisms capable of capturing deep structural shifts. As described in the session abstract:

“We will discuss what concepts related to deep structural change we observe and how models can incorporate adaptive ontologies or generative mechanisms to capture deep structural shifts. Furthermore, we want to facilitate discussion on the challenges we face when trying to model these deep changes and what practices are currently used to overcome these.”

This article reflects on key insights from that session, offering a synthesis of participant definitions, identified challenges, and promising directions for advancing the modelling of deep structural change in agent-based social simulation.

Defining deep structural change

Participant perspectives


To explore how participants understood deep structural change and its characteristics, we used both a pre-workshop survey (N=20) and live group discussion activities (N ≈ 20; divided into six discussion groups). The survey asked participants to define “deep structural change” in the context of social systems or simulations, and to explain how it differs from incremental change. During the workshop, groups expanded on these ideas using a collaborative Miro board, where they responded to three prompts: “What is deep structural change?”, “How does it differ from incremental change?”, and they were asked to come up with a “Group definition”. The exercises benefited from the conceptual and disciplinary diversity of participants. Individuals approached the prompts from different angles – shaped by their academic backgrounds and modelling traditions – resulting in a rich and multifaceted view of what deep structural change can entail.

Across the different exercises, a number of common themes emerged. One of the most consistent themes was the idea that deep structural change involves a reconfiguration of the system’s architecture – a shift in its underlying mechanisms, causal relationships, feedback loops, or rules of operation. This perspective goes beyond adjusting parameters; it points to transformations in what the system is, echoing the emphasis in our introductory framing on changes to the system’s underlying logic and potentialities. Participants described this in terms such as “change in causal graph”, “drastic shift in mechanisms and rules”, and “altering the whole architecture”. Some also emphasised the outcomes of such reconfigurations – the emergence of a new order, new dominant feedbacks, or a different equilibrium. As one participant put it, deep structural change is “something that brings out new structure”; others described “profound, systemic shifts that radically reshape underlying structures, processes and relationships”.

Another frequently discussed theme was the role of social and behavioural change in structural transformation – particularly shifts in values, norms, and decision-making. Several groups suggested that changes in attitudes, awareness, or shared meanings could contribute to or signal deeper structural shifts. In some cases, these were framed as indicators of transformation; in others, as contributing factors or intended outcomes of deliberate change efforts. Examples included evolving diets, institutional reform, and shifts in collective priorities. Participants referred to “behavioural change coming from a change in values and/or norms” and “a fundamental shift in values and priorities”.
Furthermore, participants discussed how deep structural change differs from incremental change. They described deep structural change as difficult to reverse and characterised by discontinuities or thresholds that shift the system into a new configuration, compared to slow, gradual incremental change. While some noted that incremental changes might accumulate and contribute to structural transformation, deep structural change was more commonly seen as involving a qualitative break from previous patterns. Several responses highlighted periods of instability or disruption as part of this process, in which the system may reorder around new structures or priorities.

Other topics emerging in passing included the distinction between scale and depth, the role of intentionality, and the extent to which a change must be profound or radical to qualify as deeply structural. This diversity of thought reflects both the complexity of deep structural change as a phenomenon and the range of domains in which it is seen as relevant. Rather than producing a single definition, the session surfaced multiple ways in which change can be considered structural, opening up productive space for further conceptual and methodological exploration.

A distilled definition

Drawing on both existing literature and the range of perspectives shared by participants, we propose the following working definition. It aims to clarify what is meant by deep structural change from the standpoint of agent-based modelling, while acknowledging its place within broader discussions of transformative change.

Deep structural change is a type of transformative change: From an agent-based modelling perspective, it entails an ontological reconfiguration. This reconfiguration is related to the emergence, disappearance, or transformation of entities, relationships, structures, and contextual features. While transformative change can occur within a fixed model ontology, deep structural change entails a revision of the ontology itself.

Challenges in modelling deep structural change

To understand the challenges modellers face when trying to incorporate deep structural change in ABMs or social simulations in general, we again asked participants in the pre-conference survey and had them brainstorm using a Miro board. We asked them about the “challenges [they] have encountered in this process” and “how [they] would overcome these challenges”. The points raised by the participants can roughly be grouped into: theory and data, model complexity, definition and detection.

The first challenge relates to availability of data on deep structural change and formalisation of related theory. Social simulations are increasingly based on empirical data to be able to model real-world phenomena more realistically. However, the data is often not good at capturing structural system changes, reflecting the status quo rather than the potential. While there are theories describing change, formalising this qualitative process comes with its own challenges, leading to hypothesising of the mechanisms and large uncertainties about model accuracy.

Second, a fine line has to be struck between keeping the model simple and understandable, while making it complex enough to allow for ontologies to shift and deep structural change to emerge. Participants highlighted the need for flexibility in the model structures, to allow new structures to develop. On the other hand, there is a risk of imposing transformation paths, so basically “telling” the model how to transform. In other words, it is often unclear how to make sure the necessary conditions for modelling deep structural change are there, without imposing the pathway of change.

The final challenge concerns the definition and detection of deep structural change. This article begins to address the question of definition, but detection remains difficult — even with greater conceptual clarity. How can one be confident that an observed change is genuinely deep and structural, and that the system has entered a new regime? This question touches on our ability to characterise system states, dominant feedbacks, necessary preconditions, and the timescales over which change occurs.

Closing remarks

Understanding transformative change in general, but increasingly so with the use of social simulation, is gaining attention to provide insights into complex issues. For social simulation modellers, it is therefore important to model deep structural changes. This workshop serves as a starting point for hopefully a wider discussion within the ESSA community on how to model transformative change. Bringing together social simulation researchers showed us that this is tackled from different angles. The definition provided above is a first attempt to combine these views, but key challenges remain. Thus far, people have approached this in a case-by-case manner; it would be useful to have a set of more systematic approaches.

The SIG-MTC will continue to examine questions around how we might effectively model deep structural change over the coming months and years, working with the ABM community to identify fruitful routes forward. We invite readers to comment  below on any further approaches to modelling deep structural change that they view as promising and to provide their own reflections on the topics discussed above. If you are interested in this topic and would like to engage further, please check out our ESSA Special Interest Group on Modelling Transformative Change or reach out to any one of us.

Acknowledgements

We would like to thank the participants of the SimSocFest 2025 Workshop on Modelling Deep Structural Change for their engagement in the workshop and the willingness to think along with us.

References

Feola, G. (2015). Societal transformation in response to global environmental change: A review of emerging concepts. Ambio, 44(5), 376–390. https://doi.org/10.1007/s13280-014-0582-z

Filatova, T., Polhill, J. G., & van Ewijk, S. (2016). Regime shifts in coupled socio-environmental systems: Review of modelling challenges and approaches. Environmental Modelling & Software, 75, 333–347. https://doi.org/10.1016/j.envsoft.2015.04.003

Wagenblast, T., Roxburgh, N. and Taberna, A. (2025) Modelling Deep Structural Change in Agent-Based Social Simulation. Review of Artificial Societies and Social Simulation, 8 Aug 2025. https://rofasss.org/2025/08/08/structch

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Nigel Gilbert

Leave a comment

By Corinna Elsenbroich & Petra Ahrweiler

The first piece on winners of the European Social Simulation Association’s Rosaria Conte Outstanding Contribution Award for Social Simulation.

“Gilbert, a former sociologist of science, has been one of the chief links in Britain between computer scientists and sociologists of science” [1, p. 294]

Nigel has always been and still is a sociologist – not only of science, but also of technology, innovation, methods and many other subfields of sociology with important contributions in theory, empirical research and sociological methods.

He has pioneered a range of sociological areas such as Sociology of Scientific Knowledge, Secondary Analysis of Government Datasets, Access to Social Security Information, Social Simulation, and Complexity Methods of Policy Evaluation.

Collins is right, however, that Nigel is one of the chief links between sociologists and computer scientists in the UK and beyond. This earned him to be elected as the first practising social scientist elected as a Fellow of the Royal Academy of Engineering (1999). As the principal founding father of agent-based modelling as a method for the social sciences in Europe, he initiated, promoted and institutionalised a completely novel way of doing social sciences through the Centre for Research in Social Simulation (CRESS) at the University of Surrey, the Journal of Artificial Societies and Social Simulation (JASSS), founded Sociological Research Online (1993) and Social Research Update. Nigel has 100s of publications on all aspects of social simulation and seminal books like: Simulating societies: the computer simulation of social phenomena (Gilbert & Doran 1994), Artificial Societies: The Computer Simulation of Social Phenomena (Gilbert & Conte 1995), Simulation for the Social Scientist (Gilbert &Troitzsch 2005), and Agent-based Models (Gilbert 2019). His entrepreneurial spirit and acumen resulted in over 25 large project grants (across the UK and Europe), often in close collaboration with policy and decision makers to ensure real life impact, a simulation platform on innovation networks called SKIN, and a spin off company CECAN Ltd, training practitioners in complexity methods and bringing their use to policy evaluation projects.

Nigel is a properly interdisciplinary person, turning to the sociology of scientific knowledge in his PhD under Michael Mulkay after graduating in Engineering from Cambridge’s Emmanuel College. He joined the Sociology Department at the University of Surrey in 1976 where he became professor of sociology in 1991. Nigel was appointed Commander of the Order of the British Empire (CBE) in 2016 for contributions to engineering and social sciences.

He was the second president of the European Social Simulation Association ESSA, the originator of the SIMSOC mailing list, launched and edited the Journal of Artificial Societies and Social Simulation from 1998-2014 and he was the first holder of the Rosaria Conte Outstanding Contribution Award for Social Simulation in 2016, an unanimous decision by the ESSA Management Committee.

Despite all of this, all these achievements and successes, Nigel is the most approachable, humble and kindest person you will ever meet. In any peril he is the person that will bring you a step forward when you need a helping hand. On asking him, after getting a CBE etc. what is the recognition that makes him most happy, he said, with the unique Nigel Gilbert twinkle in his eye, “my Rosaria Conte Award”.

References

Collins, H. (1995). Science studies and machine intelligence. In Handbook of Science and Technology Studies, Revised Edition (pp. 286-301). SAGE Publications, Inc., https://doi.org/10.4135/9781412990127

Gilbert, N., & Doran, R. (Eds.). (1994). Simulating societies: the computer simulation of social phenomena. Routledge.

Gilbert, N. & Conte, R. (1995) Artificial Societies: the computer simulation of social life. Routeledge. https://library.oapen.org/handle/20.500.12657/24305

Gilbert, N. (2019). Agent-based models. Sage Publications.

Gilbert, N., & Troitzsch, K. (2005). Simulation for the social scientist. Open University Press; 2nd edition.

Elsenbroich, C. & Ahrweiler, P. (2025) Nigel Gilbert. Review of Artificial Societies and Social Simulation, 3 Mar 2025. https://rofasss.org/2025/04/03/nigel-gilbert

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Outlining some requirements for synthetic populations to initialise agent-based models

Leave a comment

By Nick Roxburgh1, Rocco Paolillo2, Tatiana Filatova3, Clémentine Cottineau3, Mario Paolucci2 and Gary Polhill1

1  The James Hutton Institute, Aberdeen AB15 8QH, United Kingdom {nick.roxburgh,gary.polhill}@hutton.ac.uk

2  Institute for Research on Population and Social Policies, Rome, Italy {rocco.paolillo,mario.paolucci}@cnr.it

3 Delft University of Technology, Delft, The Netherlands {c.cottineau,t.filatova}@tudelft.nl

Abstract. We propose a wish list of features that would greatly enhance population synthesis methods from the perspective of agent-based modelling. The challenge of synthesising appropriate populations is heightened in agent-based modelling by the emphasis on complexity, which requires accounting for a wide array of features. These often include, but are not limited to: attributes of agents, their location in space, the ways they make decisions and their behavioural dynamics. In the real-world, these aspects of everyday human life can be deeply interconnected, with these associations being highly consequential in shaping outcomes. Initialising synthetic populations in ways that fail to respect these covariances can therefore compromise model efficacy, potentially leading to biased and inaccurate simulation outcomes.

1 Introduction

With agent-based models (ABMs), the rationale for creating ever more empirically informed, attribute-rich synthetic populations is clear: the closer agents and their collectives mimic their  real-world counterparts, the more accurate the models can be and the wider the range of questions they can be used to address (Zhou et al., 2022). However, while many ABMs would benefit from synthetic populations that more fully capture the complexity and richness of real-world populations – including their demographic and psychological attributes, social networks, spatial realms, decision making, and behavioural dynamics – most efforts are stymied by methodological and data limitations. One reason for this is that population synthesis methods have predominantly been developed with microsimulation applications in mind (see review by Chapuis et al. (2022)), rather than ABM. We therefore argue that there is a need for improved population synthesis methods, attuned to support the specific requirements of the ABM community, as well as commonly encountered data constraints. We propose a wish list of features for population synthesis methods that could significantly enhance the capability and performance of ABMs across a wide range of application domains, and we highlight several promising approaches that could help realise these ambitions. Particular attention is paid to methods that prioritise accounting for covariance of characteristics and attributes.

2 The interrelationships among aspects of daily life

2.1 Demographic and psychological attributes

To effectively replicate real-world dynamics, ABMs must realistically depict demographic and psychological attributes at both individual and collective levels. A critical aspect of this realism is accounting for the covariance of such attributes. For instance, interactions between race and income levels significantly influence spatial segregation patterns in the USA, as demonstrated in studies like Bruch (2014).

Several approaches to population synthesis have been developed over the years, often with a specific focus on assignment of demographic attributes. That said, where psychological attributes are collected in surveys alongside demographic data, they can be incorporated into synthetic populations just like other demographic attributes (e.g., Wu et al. (2022)). Among the most established methods is Iterative Proportional Fitting (IPF). While capable of accounting for covariances, it does have significant limitations. One of these is that it “matches distributions only at one demographic level (i.e., either household or individual)” (Zhou et al., 2022 p.2). Other approaches have sought to overcome this – such as Iterative Proportional Updating, Combinatorial Optimisation, and deep learning methods – but they invariably have their own limitations and downsides, though the extent to which these will matter depends on the application. In their overview of the existing population synthesis landscape, Zhou et al., (2022) suggest that deep learning methods appear particularly promising for high-dimensional cases. Such approaches tend to be data hungry, though – a potentially significant barrier to exploitation given many studies already face challenges with survey availability and sample size.

2.2 Social networks

Integrating realistic social networks into ABMs during population synthesis is crucial for effectively mimicking real-world social interactions, such as those underlying epidemic spread, opinion dynamics, and economic transactions (Amblard et al., 2015). In practice, this means generating networks that link agents by edges that represent particular associations between them. These networks may need to be weighted, directional, or multiplex, and potentially need to account for co-dependencies and correlations between layers. Real-world social networks emerge from distinct processes and tendencies. For example, homophily preferences strongly influence the likelihood of friendship formation, with connections more likely to have developed in cases where agents share attributes like age, gender, socio-economic context, and location (McPherson et al., 2001). Another example is personality which can strongly influence the size and nature of an individual’s social network (Zell et al., 2014). For models where social interactions play an important role, it is therefore critical that consideration be given to the underlying factors and mechanisms that are likely to have influenced the development of social networks historically, if synthetic networks are to have any chance of reasonably depicting real world network structures.

Generating synthetic social networks is challenging due to often limited or unavailable data. Consequently, researchers tend to use simple models like regular lattices, random graphs, small-world networks, scale-free networks, and models based on spatial proximity. These models capture basic elements of real-world social networks but can fall short in complex scenarios. For instance, Jiang et al. (2022) describes a model where agents, already assigned to households and workplaces, form small-world networks based on employment or educational ties. While this approach accounts for spatial and occupational similarities, it overlooks other factors, limiting its applicability for networks like friendships that rely on personal history and intangible attributes.

To address these limitations, more sophisticated methods have been proposed, including Exponential Random Graph Models (ERGM) (Robins et al., 2007) and Yet Another Network Generator (YANG) (Amblard et al., 2015). However, they also come with their own challenges; for example, ERGMs sometimes misrepresent the likelihood of certain network structures, deviating from real-world observations.

2.3 Spatial locations

The places where people live, work, take their leisure and go to school are critically interlinked and interrelated with social networks and demographics. Spatial location also affects options open to people, including transport, access to services, job opportunities and social encounters. ABMs’ capabilities in representing space explicitly and naturally is a key attraction for geographers interested in social simulation and population synthesis (Cottineau et al., 2018). Ignoring the spatial concentration of agents with common traits, or failing to account for the effects that space has on other aspects of everyday human existence, risks overlooking a critical factor that influences a wide range of social dynamics and outcomes.

Spatial microsimulation generates synthetic populations tailored to defined geographic zones, such as census tracts (Lovelace and Dumont, 2017). However, many ABM applications require agents to be assigned to specific dwellings and workplaces, not just aggregated zones. While approaches to dealing with this have been proposed, agreement on best practice is yet to cohere. Certain agent-location assignments can be implemented using straightforward heuristic methods without greatly compromising fidelity, if heuristics align well with real-world practices. For example, children might be allocated to schools simply based on proximity, such as in Jiang et al., (2022). Others use rule-based or stochastic methods to account for observed nuances and random variability, though these often take the form of crude approximations. One of the more well-rounded examples is detailed by Zhou et al. (2022). They start by generating a synthetic population, which they then assign to specific dwellings and jobs using a combination of rule-based matching heuristic and probabilistic models. Dwellings are assigned to households by considering factors like household size, income, and dwelling type jointly. Meanwhile, jobs are assigned to workers using a destination choice model that predicts the probability of selecting locations based on factors such as sector-specific employment opportunities, commuting costs, and interactions between commuting costs and individual worker attributes. In this way, spatial location choices are more closely aligned with the diverse attributes of agents. The challenge with such an approach is to obtain sufficient microdata to inform the rules and probabilities.

2.4 Decision-making and behavioural dynamics

In practice, peoples’ decision-making and behaviours are influenced by an array of factors, including their individual characteristics such as wealth, health, education, gender, and age, their social network, and their geographical circumstances. These factors shape – among other things – the information agents’ are exposed to, the choices open to them, the expectations placed on them, and their personal beliefs and desires (Lobo et al., 2023). Consequently, accurately initialising such factors is important for ensuring that agents are predisposed to make decisions and take actions in ways that reflect how their real world counterparts might behave. Furthermore, the assignment of psychographic attributes to agents necessitates the prior establishment of these foundational characteristics as they are often closely entwined.

Numerous agent decision-making architectures have been proposed (see Wijermans et al. (2023)). Many suggest that a range of agent state attributes could, or even should, be taken into consideration when evaluating information and selecting behaviours. For example, the MoHub Framework (Schlüter et al., 2017) proposes four classes of attributes as potentially influential in the decision-making process: needs/goals, knowledge, assets, and social. In practice, however, the factors taken into consideration in decision-making procedures tend to be much narrower. This is understandable given the higher data demands that richer decision-making procedures entail. However, it is also regrettable given we know that decision-making often draws on many more factors than are currently accounted for, and the ABM community has worked hard to develop the tools needed to depict these richer processes.

3 Practicalities

Our wish list of features for synthetic population algorithms far exceeds their current capabilities. Perhaps the main issue today is data scarcity, especially concerning less tangible aspects of populations, such as psychological attributes and social networks, where systematic data collection is often more limited. Another significant challenge is that existing algorithms struggle to manage the numerous conditional probabilities involved in creating realistic populations, excelling on niche measures of performance but not from a holistic perspective. Moreover, there are accessibility issues with population synthesis tools. The next generation of methods need to be made more accessible to non-specialists through developing easy to use stand-alone tools or plugins for widely used platforms like NetLogo, else they risk not having their potential exploited.

Collectively, these issues may necessitate a fundamental rethink of how synthetic populations are generated. The potential benefits of successfully addressing these challenges are immense. By enhancing the capabilities of synthetic population tools to meet the wish list set out here, we can significantly improve model realism and expand the potential applications of social simulation, as well as strengthen credibility with stakeholders. More than this, though, such advancements would enhance our ability to draw meaningful insights, respecting the complexities of real-world dynamics. Most critically, better representation of the diversity of actors and circumstances reduces the risk of overlooking factors that might adversely impact segments of the population – something there is arguably a moral imperative to strive for.

Acknowledgements

MP & RP were supported by FOSSR (Fostering Open Science in Social Science Research), funded by the European Union – NextGenerationEU under NPRR Grant agreement n. MUR IR0000008. CC was supported by the ERC starting Grant SEGUE (101039455).

References

Amblard, F., Bouadjio-Boulic, A., Gutiérrez, C.S. and Gaudou, B. 2015, December. Which models are used in social simulation to generate social networks? A review of 17 years of publications in JASSS. In 2015 Winter Simulation Conference (WSC) (pp. 4021-4032). IEEE. https://doi.org/10.1109/WSC.2015.7408556

Bruch, E.E., 2014. How population structure shapes neighborhood segregation. American Journal of Sociology, 119(5), pp.1221-1278. https://doi.org/10.1086/675411

Chapuis, K., Taillandier, P. and Drogoul, A., 2022. Generation of synthetic populations in social simulations: a review of methods and practices. Journal of Artificial Societies and Social Simulation, 25(2). https://doi.org/10.18564/jasss.4762

Cottineau, C., Perret, J., Reuillon, R., Rey-Coyrehourcq, S. and Vallée, J., 2018, March. An agent-based model to investigate the effects of social segregation around the clock on social disparities in dietary behaviour. In CIST2018-Représenter les territoires/Representing territories (pp. 584-589). https://hal.science/hal-01854398v1

Jiang, N., Crooks, A.T., Kavak, H., Burger, A. and Kennedy, W.G., 2022. A method to create a synthetic population with social networks for geographically-explicit agent-based models. Computational Urban Science, 2(1), p.7. https://doi.org/10.1007/s43762-022-00034-1

Lobo, I., Dimas, J., Mascarenhas, S., Rato, D. and Prada, R., 2023. When “I” becomes “We”: Modelling dynamic identity on autonomous agents. Journal of Artificial Societies and Social Simulation, 26(3). https://doi.org/10.18564/jasss.5146

Lovelace, R. and Dumont, M., 2017. Spatial microsimulation with R. Chapman and Hall/CRC. https://spatial-microsim-book.robinlovelace.net

McPherson, M., Smith-Lovin, L. and Cook, J.M., 2001. Birds of a feather: Homophily in social networks. Annual review of sociology, 27(1), pp.415-444. https://doi.org/10.1146/annurev.soc.27.1.415

Robins, G., Pattison, P., Kalish, Y. and Lusher, D., 2007. An introduction to exponential random graph (p*) models for social networks. Social networks, 29(2), pp.173-191. https://doi.org/10.1016/j.socnet.2006.08.002

Schlüter, M., Baeza, A., Dressler, G., Frank, K., Groeneveld, J., Jager, W., Janssen, M.A., McAllister, R.R., Müller, B., Orach, K. and Schwarz, N., 2017. A framework for mapping and comparing behavioural theories in models of social-ecological systems. Ecological economics, 131, pp.21-35. https://doi.org/10.1016/j.ecolecon.2016.08.008

Wijermans, N., Scholz, G., Chappin, É., Heppenstall, A., Filatova, T., Polhill, J.G., Semeniuk, C. and Stöppler, F., 2023. Agent decision-making: The Elephant in the Room-Enabling the justification of decision model fit in social-ecological models. Environmental Modelling & Software, 170, p.105850. https://doi.org/10.1016/j.envsoft.2023.105850

Wu, G., Heppenstall, A., Meier, P., Purshouse, R. and Lomax, N., 2022. A synthetic population dataset for estimating small area health and socio-economic outcomes in Great Britain. Scientific Data, 9(1), p.19. https://doi.org/10.1038/s41597-022-01124-9

Zell, D., McGrath, C. and Vance, C.M., 2014. Examining the interaction of extroversion and network structure in the formation of effective informal support networks. Journal of Behavioral and Applied Management, 15(2), pp.59-81. https://jbam.scholasticahq.com/article/17938.pdf

Zhou, M., Li, J., Basu, R. and Ferreira, J., 2022. Creating spatially-detailed heterogeneous synthetic populations for agent-based microsimulation. Computers, Environment and Urban Systems, 91, p.101717. https://doi.org/10.1016/j.compenvurbsys.2021.101717

Roxburgh, N., Paolillo, R., Filatova, T., Cottineau, C., Paolucci, M. and Polhill, G. (2025) Outlining some requirements for synthetic populations to initialise agent-based models. Review of Artificial Societies and Social Simulation, 27 Jan 2025. https://rofasss.org/2025/01/29/popsynth

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Quantum computing in the social sciences

1 Comment

By Emile Chappin and Gary Polhill

The dream

What could quantum computing mean for the computational social sciences? Although quantum computing is at an early stage, this is the right time to dream about precisely that question for two reasons. First, we need to keep the computational social sciences ‘in the conversation’ about use cases for quantum computing to ensure our potential needs are discussed. Second, thinking about how quantum computing could affect the way we work in the computational social sciences could lead to interesting research questions, new insights into social systems and their uncertainties, and form the basis of advances in our area of work.

At first glance, quantum computing and the computational social sciences seem unrelated. Computational social science uses computer programs written in high-level languages to explore the consequences of assumptions as macro-level system patterns based on coded rules for micro-level behaviour (e.g., Gilbert, 2007). Quantum computing is in an early phase, with the state-of-the-art being in the order of 100s of qubits [1],[2], and a wide range of applications are envisioned (Hassija, 2020), e.g., in the areas of physics (Di Meglio et al., 2024) and drug discovery (Blunt et al., 2022). Hence, the programming of quantum computers is also in an early phase. Major companies (e.g., IBM, Microsoft, Alphabet, Intel, Rigetti Computing) are investing heavily and have put out high expectations – though how much of this is hyperbole to attract investors and how much it is backed up by substance remains to be seen. This means it is still hard to comprehend what opportunities may come from scaling up.

Our dream is that quantum computing enables us to represent human decision-making on a much larger scale, do more justice to how decisions come about, and embrace the influences people have on each other. It would respect that people’s actual choices are undetermined until they have to show behaviour. On a philosophical level, these features are consistent with how quantum computation operates. Applying quantum computing to decision-making with interactions may help us inform or discover behavioural theory and contribute to complex systems science.

The mysticism around quantum computing

There is mysticism around what qubits are. To start thinking about how quantum computing could be relevant for computational social science, there is no direct need to understand the physics of how qubits are physically set up. However, it is necessary to understand the logic and how quantum computers operate. At the logical level, there are similarities between quantum and traditional computers.

The main similarity is that the building blocks are bits and that they are either 0 or 1, but only when you measure them. A second similarity is that quantum computers work with ‘instructions’. Quantum ‘processors’ alter the state of the bits in a ‘memory’ using programs that comprise sequences of ‘instructions’ (e.g., Sutor, 2019).

There are also differences. They are: 1) qubits are programmed to have probabilities of being a zero or a one, 2) qubits have no determined value until they are measured, and 3) multiple qubits can be entangled. The latter means the values (when measured) depend on each other.

Operationally speaking, quantum computers are expected to augment conventional computers in a ‘hybrid’ computing environment. This means we can expect to use traditional computer programs to do everything around a quantum program, not least to set up and analyse the outcomes.

Programming quantum computers

Until now, programming languages for quantum computing are low-level; like assembly languages for regular machines. Quantum programs are therefore written very close to ‘the hardware’. Similarly, in the early days of electronic computers, instructions for processors to perform directly were programmed directly: punched cards contained machine language instructions. Over time, computers got bigger, more was asked of them, and their use became more widespread and embedded in everyday life. At a practical level, different processors, which have different instruction sets, and ever-larger programs became more and more unwieldy to write in machine language. Higher-level languages were developed, and reached a point where modellers could use the languages to describe and simulate dynamic systems. Our code is still ultimately translated into these lower-level instructions when we compile software, or it is interpreted at run-time. The instructions now developed for quantum computing are akin to the early days of conventional computing, but development of higher-level programming languages for quantum computers may happen quickly.

At the start, qubits are put in entangled states (e.g., Sutor, 2019); the number of qubits at your disposal makes up the memory. A quantum computer program is a set of instructions that is followed. Each instruction alters the memory, but only by changing the probabilities of qubits being 0 or 1 and their entanglement. Instruction sets are packaged into so-called quantum circuits. The instructions operate on all qubits at the same time, (you can think of this in terms of all probabilities needing to add up to 100%). This means the speed of a quantum program does not depend on the scale of the computation in number of qubits, but only depends on the number of instructions that one executes in a program. Since qubits can be entangled, quantum computing can do calculations that take too long to run on a normal computer.

Quantum instructions are typically the inverse of themselves: if you execute an instruction twice, you’re back at the state before the first operation. This means you can reverse a quantum program simply by executing the program again, but now in reverse order of the instructions. The only exception to this is the so-called ‘read’ instruction, by which the value is determined for each qubit to either be 1 or 0. This is the natural end of the quantum program.

Recent developments in quantum computing and their roadmaps

Several large companies such as Microsoft, IBM and Alphabet are investing heavily in developing quantum computing. The route currently is to move up in the scale of these computers with respect to the number of qubits they have and the number of gates (instructions) that can be run. IBM’s roadmap they suggest growing to 7500 instructions, as quickly as 2025[3]. At the same time, programming languages for quantum computing are being developed, on the basis of the types of instructions above. At the moment, researchers can gain access to actual quantum computers (or run quantum programs on simulated quantum hardware). For example, IBM’s Qiskit[4] is one of the first open-source software developing kit for quantum computing.

A quantum computer doing agent-based modelling

The exponential growth in quantum computing capacity (Coccia et al., 2024) warrants us to consider how it may be used in the computational social sciences. Here is a first sketch. What if there is a behavioural theory that says something about ‘how’ different people decide in a specific context on a specifical behavioural action. Can we translate observed behaviour into the properties of a quantum program and explore the consequences of what we can observe? Or, in contrast, can we unravel the assumptions underneath our observations? Could we look at alternative outcomes that could also have been possible in the same system, under the same conceptualization? Given what we observe, what other system developments could have had emerged that also are possible (and not highly unlikely)? Can we unfold possible pathways without brute-forcing a large experiment? These questions are, we believe, different when approached from a perspective of quantum computing. For one, the reversibility of quantum programs (until measuring) may provide unique opportunities. This also means, doing such analyses may inspire new kinds of social theory, or it may give a reflection on the use of existing theory.

One of the early questions is how we may use qubits to represent modelled elements in social simulations. Here we sketch basic alternative routes, with alternative ideas. For each strain we include a very rudimentary application to both Schelling’s model of segregation and the Traffic Basic model, both present in NetLogo model library.

Qubits as agents

A basic option could be to represent an agent by a qubit. Thinking of one type of stylized behaviour, an action that can be taken, then a quantum bit could represent whether that action is taken or not. Instructions in the quantum program would capture the relations between actions that can be taken by the different agents, interventions that may affect specific agents. For Schelling’s model, this would have to imply to show whether segregation takes place or not. For Traffic Basic, this would be what the probability is for having traffic jams. Scaling up would mean we would be able to represent many interacting agents without the simulation to slow down. This is, by design, abstract and stylized. But it may help to answer whether a dynamic simulation on a quantum computer can be obtained and visualized.

Decision rules coded in a quantum computer

A second option is for an agent to perform a quantum program as part of their decision rules. The decision-making structure should then match with the logic of a quantum computer. This may be a relevant ontological reference to how brains work and some of the theory that exists on cognition and behaviour. Consider a NetLogo model with agents that have a variety of properties that get translated to a quantum program. A key function for agents would be that the agent performs a quantum calculation on the basis of a set of inputs. The program would then capture how different factors interact and whether the agent performs specific actions, i.e., show particular behaviour. For Schelling’s segregation model, it would be the decision either to move (and in what direction) or not. For Traffic Basic it would lead to a unique conceptualization of heterogeneous agents. But for such simple models it would not necessarily take benefit of the scale-advantage that quantum computers have, because most of the computation occurs on traditional computers and the limited scope of the decision logic of these models. Rather, it invites to developing much more rich and very different representations of how decisions are made by humans. Different brain functions may all be captured: memory, awareness, attitudes, considerations, etc. If one agent’s decision-making structure would fit in a quantum computer, experiments can already be set up, running one agent after the other (just as it happens on traditional computers). And if a small, reasonable number of agents would fit, one could imagine group-level developments. If not of humans, this could represent companies that function together, either in a value chain or as competitors in a market. Because of this, it may be revolutionary:  let’s consider this as quantum agent-based modelling.

Using entanglement

Intuitively one could consider the entanglement if qubits to be either represent the connection between different functions in decision making, the dependencies between agents that would typically interact, or the effects of policy interventions on agent decisions. Entanglement of qubits could also represent the interaction of time steps, capturing path dependencies of choices, limiting/determining future options. This is the reverse of memory: what if the simulation captures some form of anticipation by entangling future options in current choices. Simulations of decisions may then be limited, myopic in their ability to forecast. By thinking through such experiments, doing the work, it may inspire new heuristics that represent bounded rationality of human decision making. For Schelling’s model this could be the local entanglement restricting movement, it could be restricting movement because of future anticipated events, which contributes to keep the status quo. For Traffic Basic, one could forecast traffic jams and discover heuristics to avoid them which, in turn may inspire policy interventions.

Quantum programs representing system-level phenomena

The other end of the spectrum can also be conceived. As well as observing other agents, agents could also interact with a system in order to make their observations and decisions where the system with which they interact with itself is a quantum program. The system could be an environmental, or physical system, for example. It would be able to have the stochastic, complex nature that real world systems show. For some systems, problems could possibly be represented in an innovative way. For Schelling’s model, it could be the natural system with resources that agents benefit from if they are in the surroundings; resources having their own dynamics depending on usage. For Traffic Basic, it may represent complexities in the road system that agents can account for while adjusting their speed.

Towards a roadmap for quantum computing in the social sciences

What would be needed to use quantum computation in the social sciences? What can we achieve by taking the power of high-performance computing combined with quantum computers when the latter scale up? Would it be possible to reinvent how we try to predict the behaviour of humans by embracing the domain of uncertainty that also is essential in how we may conceptualise cognition and decision-making? Is quantum agent-based modelling at one point feasible? And how do the potential advantages compare to bringing it into other methods in the social sciences (e.g. choice models)?

A roadmap would include the following activities:

  • Conceptualise human decision-making and interactions in terms of quantum computing. What are promising avenues of the ideas presented here and possibly others?
  • Develop instruction sets/logical building blocks that are ontologically linked to decision-making in the social sciences. Connect to developments for higher-level programming languages for quantum computing.
  • Develop a first example. One could think of reproducing one of the traditional models. Either an agent-based model, such as Schelling’s model of segregation or Basic Traffic, or a cellular automata model, such as game-of-life. The latter may be conceptualized with a relatively small number of cells and could be a valuable demonstration of the possibilities.
  • Develop quantum computing software for agent-based modelling, e.g., as a quantum extension for NetLogo, MESA, or for other agent-based modelling packages.

Let us become inspired to develop a more detailed roadmap for quantum computing for the social sciences. Who wants to join in making this dream a reality?

Notes

[1] https://newsroom.ibm.com/2022-11-09-IBM-Unveils-400-Qubit-Plus-Quantum-Processor-and-Next-Generation-IBM-Quantum-System-Two

[2] https://www.fastcompany.com/90992708/ibm-quantum-system-two

[3] https://www.ibm.com/roadmaps/quantum/

[4] https://github.com/Qiskit/qiskit-ibm-runtime

References

Blunt, Nick S., Joan Camps, Ophelia Crawford, Róbert Izsák, Sebastian Leontica, Arjun Mirani, Alexandra E. Moylett, et al. “Perspective on the Current State-of-the-Art of Quantum Computing for Drug Discovery Applications.” Journal of Chemical Theory and Computation 18, no. 12 (December 13, 2022): 7001–23. https://doi.org/10.1021/acs.jctc.2c00574.

Coccia, M., S. Roshani and M. Mosleh, “Evolution of Quantum Computing: Theoretical and Innovation Management Implications for Emerging Quantum Industry,” in IEEE Transactions on Engineering Management, vol. 71, pp. 2270-2280, 2024, https://doi: 10.1109/TEM.2022.3175633.

Di Meglio, Alberto, Karl Jansen, Ivano Tavernelli, Constantia Alexandrou, Srinivasan Arunachalam, Christian W. Bauer, Kerstin Borras, et al. “Quantum Computing for High-Energy Physics: State of the Art and Challenges.” PRX Quantum 5, no. 3 (August 5, 2024): 037001. https://doi.org/10.1103/PRXQuantum.5.037001.

Gilbert, N., Agent-based models. SAGE Publications Ltd, 2007. ISBN 978-141-29496-44

Hassija, V., Chamola, V., Saxena, V., Chanana, V., Parashari, P., Mumtaz, S. and Guizani, M. (2020), Present landscape of quantum computing. IET Quantum Commun., 1: 42-48. https://doi.org/10.1049/iet-qtc.2020.0027

Sutor, R. S. (2019). Dancing with Qubits: How quantum computing works and how it can change the world. Packt Publishing Ltd.

Chappin, E. & Polhill, G (2024) Quantum computing in the social sciences. Review of Artificial Societies and Social Simulation, 25 Sep 2024. https://rofasss.org/2024/09/24/quant

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Delusional Generality – how models can give a false impression of their applicability even when they lack any empirical foundation

Leave a comment

By Bruce Edmonds1, Dino Carpentras2, Nick Roxburgh3, Edmund Chattoe-Brown4 and Gary Polhill3

  1. Centre for Policy Modelling, Manchester Metropolitan University
  2. Computational Social Science, ETH Zurich
  3. James Hutton Institute, Aberdeen
  4. University of Leicester

“Hamlet: Do you see yonder cloud that’s almost in shape of a camel?
Polonius: By the mass, and ‘tis like a camel, indeed.
Hamlet: Methinks it is like a weasel.
Polonius: It is backed like a weasel.
Hamlet: Or like a whale?
Polonius: Very like a whale.”

Models and Generality

The essence of a model is that it represents – if it is not a model of something it is not a model at all (Zeigler 1976, Wartofsky 1979). A random bit of code or set of equations is not a model. The point of a model is that one can use the model to infer or understand some aspects about what it represents. However, models can represent a variety of kinds of things in a variety of ways (Edmonds & al. 2019) – it can represent ideas, correspond to data, or aspects of other models and it can represent each of these in either a vague or precise manner. To completely understand a model – its construction, properties and working – one needs to understand how it does this mapping. This piece focuses attention on this mapping, rather than the internal construction of models.

What a model reliably represents may be a single observed situation, but it might satisfactorily represent more than one such situation. The range of situations that the model satisfactorily represents is called the “scope” of the model (what is “satisfactory” depending on the purpose for which the model is being used). The more extensive the scope, the more “general” we say the model is. A model that only represents one case has no generality at all and may be more in the nature of a description.

There is a hunger for general accounts of social phenomena (let us call these ‘theories’). However, this hunger is often frustrated by the sheer complexity and ‘messiness’ involved in such phenomena. If every situation we observe is essentially different, then no such theory is possible. However, we hope that this is not the case for the social world and, indeed, informal observation suggests that there is, at least some, commonality between situations – in other words, that some kind of reliable generalisation about social phenomena might be achievable, however modest (Merton 1968). This piece looks at two kinds of applicability – analogical applicability and empirical applicability – and critiques those that conflate them. Although the expertise of the authors is in the agent-based modelling of social phenomena, and so we restrict our discussion to this, we strongly suspect that our arguments are true for many kinds of modelling across a range of domains.

In the next sections we contrast two uses for models: as analogies (ways of thinking about observed systems) and those that intend to represent empirical data in a more precise way. There are, of course, other uses of model such as that of exploring theory which have nothing to do with anything observed.

Models used as analogies

Analogical applicability comes from the flexibility of the human mind in interpreting accounts in terms of the different situations. When we encounter a new situation, the account is mapped onto it – the account being used as an analogy for understanding this situation. Such accounts are typically in the form of a narrative, but a model can also be used as an analogy (which is the case we are concerned with here). The flexibility with which this mapping can be constructed means that such an account can be related to a wide range of phenomena. Such analogical mapping can lead to an impression that the account has a wide range of applicability. Analogies are a powerful tool for thinking since it may give us some insights into otherwise novel situations. There are arguments that analogical thinking is a fundamental aspect of human thought (Hofstadter 1995) and language (Lakoff 2008). We can construct and use analogical mappings so effortlessly that they seem natural to us. The key thing about analogical thinking is that the mapping from the analogy to the situation to which it is applied is re-invented each time – there is no fixed relationship between the analogy and what it might be applied to. We are so good at doing this that we may not be aware of how different the constructed mapping is each time. However, its flexibility comes at a cost, namely that because there is no well-defined relationship with what it applies to, the mapping tends to be more intuitive than precise. An analogy can give insights but analogical reasoning suggests rather than establishes anything reliably and you cannot empirically test it (since analogical mappings can be adjusted to avoid falsification). Such “ways of thinking” might be helpful, but equally might be misleading [note ‎1].

Just because the content of an analogy might be expressed formally does not change any of this (Edmonds 2018), in fact formally expressed analogies might give the impression of being applicable, but often are only related to anything observed via ideas – the model relates to some ideas, and the ideas relate to reality (Edmonds 2000). Using models as analogies is a valid use of models but this is not an empirically reliable one (Edmonds et al. 2019). Arnold (2013) makes a powerful argument that many of the more abstract simulation models are of this variety and simply not relatable to empirically observed cases and data at all – although these give the illusion of wide applicability, that applicability is not empirical. In physics the ways of thinking about atomic or subatomic entities have changed over time whilst the mathematically-expressed, empirically-relevant models have not (Hartman 1997). Although Thompson (2022) concentrates on mathematically formulated models, she also distinguishes between well-validated empirical models and those that just encapsulate the expertise/opinion of the modeller. She gives some detailed examples of where the latter kind had disproportionate influence, beyond that of other expertise, just because it was in the form of a model (e.g. the economic impact of climate change).

An example of an analogical model is described in Axelrod (1984) – a formalised tournament where algorithmically-expressed strategies are pitted against each other, playing the iterated prisoner’s dilemma game. It is shown how the ‘tit for tat’ strategy can survive against many other mixes of strategies (static or evolving).  In the book, the purpose of the model is to suggest a new way of thinking about the evolution of cooperation. The book claims the idea ‘explains’ many observed phenomena, but this in an analogical manner – no precise relationship with any observed measurements is described. There is no validation of the model here or in the more academic paper that described these results (Axelrod & Hamilton 1981).

Of course, researchers do not usually call their models “analogies” or “analogical” explicitly but tend to use other phrasings that imply a greater importance. An exception is Epstein (2008) where it is explicitly listed as one of the 15 modelling purposes, other than prediction, that he discusses. Here he says such models are “…more than beautiful testaments to the unifying power of models: they are headlights in dark unexplored territory.” (ibid.) thus suggesting their use in thinking about phenomena where we do not already have reliable empirical models. Anything that helps us think about such phenomena could be useful, but that does not mean they are at all reliable. As Herbert Simon said: “Metaphor and analogy can be helpful, or they can be misleading. ” (Simon 1968, p. 467).

Another purpose listed in Epstein (2008) is to “Illuminate core dynamics”. After raising the old chestnut that “All models are wrong”, he goes on to justify them on the grounds that “…they capture qualitative behaviors of overarching interest”. This is fine if the models are, in fact, known to be useful as more than vague analogies [Note 2] – that they do, in some sense, approximate observed phenomena – but this is not the case with novel models that have not been empirically tested. This phrase is more insidious, because it implies that the dynamics that have been illuminated by the model are “core” – some kind of approximation of what is important about the phenomena, allowing for future elaborations to refine the representation. This implies a process where an initially rough idea is iteratively improved. However, this is premature because we do not know if what has been abstracted away in the abstract model was essential to the dynamics of the target phenomena or not without empirical testing – this is just assumed or asserted based on the intuitions of the modeller.

This idea of the “core dynamics” leads to some paradoxical situations – where a set of competing models are all deemed to be core. Indeed, the literature has shown how the same phenomenon can be modelled in many contrasting ways. For instance, political polarisation has been modelled through models with mechanisms for repulsion, bounded confidence, reinforcement, or even just random fluctuations, to name a few (Flache et al., 2017; Banisch & Olbrich 2019; Carpentras et al. 2022). However, it is likely that only a few of them contribute substantially to the political polarisation we observe in the real world, and so that all the others are not a real “core dynamic” but until we have more empirical work we do not know which are core and which not.

A related problem with analogical models is that, even when relying on parsimony principles [Note 3], it is not possible to decide which model is better. This aspect, combined with the constant production of new models, can makes the relevant literature increasingly difficult to navigate as models proliferate without any empirical selection, especially for researchers new to ABM. Furthermore, most analogical models define their object of study in an imprecise manner so that it is hard to evaluate whether they are even intended to capture element of any particular observed situation. For example, opinion dynamics models rarely define the type of interaction they represent (e.g. in person vs online) or even what an opinion is. This has led to cases where even knowledge of facts has been studied as “opinions” (e.g. Chacoma & Zanette, 2015).

In summary, analogical models can be a useful tool to start thinking about complex phenomena. However, the danger with them is that they give an impression of progress but result in more confusion than clarity, possibly slowing down scientific progress. Once one has some possible insights, one needs to confront these with empirical data to determine which are worth further investigation.

Models that relate directly to empirical data

An empirical model, in contrast, has a well-defined way of mapping to the phenomena it represents. For example, the variables of the gas laws (volume, temperature and pressure) are measured using standard methods developed over a long period of time, one does not invent a new way of doing this each time the laws are applied. In this case, the ways of measuring these properties have developed alongside the mathematical models of the laws so that these work reliably under broad (and well known) conditions and cannot be adjusted at the whim of a modeller. Empirical generality comes from when a model applies reliably to many different situations – in the case of the gas laws, to a wide range of materials in gaseous form to a high degree of accuracy.

Empirical models can be used for different purposes, including: prediction, explanation and description (Edmonds et al. 2019). Each of these uses how the model is mapped to empirical data in different ways, to reflect these purposes. With a descriptive model the mapping is one-way from empirical data to the model to justify the different parts. In a predictive model, the initial model setup is determined from known data and the model is then run to get its results. These results are then mapped back to what we might expect as a prediction, which can be later compared to empirically measured values to check the model’s validity. An explanatory model supports a complex explanation of some known outcomes in terms of a set of processes, structures and parameter values. When it is shown that the outcomes of such a model sufficiently match those from the observed data – the model represents a complex chain of causation that would result in that data in terms of the processes, structures and parameter values it comprised. It thus supports an explanation in terms of the model and its input of what was observed. In each of these three cases the mapping from empirical data to the model happens in a different order and maybe in a different direction, however they all depend upon the mapping being well defined.

Cartwright (1983), studying how physics works, distinguished between explanatory and phenomenological laws – the former explains but does not necessary relate exactly to empirical data (such as when we fit a line to data using regression), whilst the latter fits the data but does not necessarily explain (like the gas laws). Thus the jobs of theoretical explanation and empirical prediction are done by different models or theories (often calling the explanatory version “theory” and the empirical versions “models”). However, in physics the relationship between the two is, itself, examined so that the “bridging laws” between them are well understood, especially in formal terms. In this case, we attribute reliable empirical meaning to the explanatory theories to the extent that the connection to the data is precise, even though it is done via the intermediary of an “phenomenological” model because both mappings (explanatory↔phenomenological and phenomenological↔empirical data) are precise and well established. The point is that the total mapping from model or theory to empirical data is not subject to interpretation or post-hoc adjustment to improve its fit.

ABMs are often quite complicated and require many parameters or other initialising input to be specified before they can be run. If some of these are not empirically determinable (even in principle) then these might be guessed at using a process of “calibration”, that is searching the space of possible initialisations for some values for which some measured outcomes of the results match other empirical data. If the model has been separately shown to be empirically reliable then one could do such a calibration to suggest what these input values might have been. Such a process might establish that the model captures a possible explanation of the fitted outcomes (in terms of the model plus those backward-inferred input values), but this is not a very strong relationship, since many models are very flexible and so could fit a wide range of possible outcomes. The reliability of such a suggested explanation, supported by the model, is only relative to (a) the empirical reliability of any theory or other assumptions the model is built upon (b) how flexibly the model outcomes can be adjusted to fit the target data and (c) how precisely the choice of outcome measures and fit are. Thus, calibration does not provide strong evidence of the empirical adequacy of an ABM and any explanation supported by such a procedure is only relative to the ‘wiggle room’ afforded by free parameters and unknown input data as well as any assumptions used in the making of the model. However, empirical calibration is better than none and may empirically fix the context in which theoretical exploration occurs – showing that the model is, at least, potentially applicable to the case being considered [Note 4].

An example of a model that is strongly grounded in empirical data is the “538” model of the US electoral college for presidential elections (Silver 2012). This is not an ABM but more like a micro-simulation. It aggregates the uncertainty from polling data to make probabilistic predictions about what this means for the outcomes. The structure of the model comes directly from the rules of the electoral college, the inputs are directly derived from the polling data and it makes predictions about the results that can be independently checked. It does a very specific, but useful job, in translating the uncertainty of the polling data into the uncertainty about the outcome.

Why this matters

If people did not confuse the analogical and empirical cases, there would not be a problem. However, researchers seem to suffer from a variety of “Kuhnian Spectacles” (Kuhn 1962) – namely that because they view their target systems through an analogical model, they tend to think that this is how that system actually is – i.e. that the model has not just analogical but also empirical applicability. This is understandable, we use many layers of analogy to navigate our world and in many every-day cases it is practical to conflate our models with the reality we deal with (when they are very reliable). However, people who claim to be scientists are under an obligation to be more cautious and precise than this, since others might wish to rely upon our theories and models (this is, after all, why they support us in our privileged position). However, such caution is not always followed. There are cases where modellers declare their enterprise a success even after a long period without any empirical backing, making a variety of excuses instead of coming clean about this lack (Arnold 2015).

Another fundamental aspect is that agent-based models can be very interdisciplinary and, because of that, they can be used also by researchers in different fields. However, many fields do not consider models as simple analogies, especially when they provide precise mathematical relationship among variables. This can easily result in confusions where the analogical applicability of ABMs is interpreted as empirical in another field.

Of course, we may be hopeful that, sometime in the future, our vague or abstract analogical model maybe developed into something with proven empirical abilities, but we should not suggest such empirical abilities until these have been established. Furthermore, we should be particularly careful to ensure that non-modellers understand that this possibility is only a hope and not imply anything otherwise (e.g. imply that it is likely to have empirical validity). However, we suspect that in many cases this confusion goes beyond optimistic anticipation and that some modellers conflate analogical with empirical applicability, assuming that their model is basically right just because it seems that way to them. This is what we call “delusional generality” – that a researcher is under the impression that their model has a wide applicability (or potentially wide applicability) due to the attractiveness of the analogy it presents. In other words, unaware of the unconscious process of re-inventing the mapping to each target system, they imagine (without further justification) that it has some reliable empirical (or potentially empirical) generality at its core [Note 5].

Such confusion can have severe real-world consequences if a model with only analogical validity is assumed to also have some empirical reliability. Thompson (2022) discusses how abstract economic models of the cost of future climate change did affect the debate about the need for prevention and mitigation, even though they had no empirical validity. However, agent-based modellers have also made the same mistake, with a slew of completely unvalidated models about COVID affecting public debate about policy (Squazzoni et al 2021).

Conclusion

All of the above discussion raises the question of how we might achieve reliable models with even a moderate level of empirical generality in the social sciences. This is a tricky question of scientific strategy, which we are not going to answer here [Note 6]. However, we question whether the approach of making “heroic” jumps from phenomena to abstract non-empirical models on the sole basis of its plausibility to its authors will be a productive route when the target is complex phenomena, such as socio-cognitive systems (Dignum, Edmonds and Carpentras 2022). Certainly, that route has not yet been empirically demonstrated.

Whatever the best strategy is, there is a lot of theoretical modelling in the field of social simulation that assumes or implies that it is the precursor for empirical applicability and not a lot of critique about the extent of empirical success achieved. The assumption seems to be that abstract theory is the way to make progress understanding social phenomena but, as we argue here, this is largely wishful thinking – the hope that such models will turn out to have empirical generality being a delusion.  Furthermore, this approach has substantive deleterious effects in terms of encouraging an explosion of analogical models without any process of selection (Edmonds 2010). It seems that the ‘famine’ of theory about social phenomena with any significant level of generality is so severe, that many seem to give credence to models they might otherwise reject – constructing their understanding using models built on sand.

Notes

1. There is some debate about the extent to which analogical reasoning works, what kind of insights it results in and under what circumstances (Hofstede 1995). However, all we need for our purposes is that: (a) it does not reliably produce knowledge, (b) the human mind is exceptionally good at ‘fitting’ analogies to new situations (adjusting the mapping to make it ‘work’ somehow) and (c) due to this ability analogies can be far more convincing that the analogical reasoning warrants.

2. In pattern-oriented modelling (Grimm & al 2005) models are related to empirical evidence in a qualitative (pattern-based) manner, for example to some properties of a distribution of numeric outcomes. In this kind of modelling, a precise numerical correspondence is replaced by a set of qualitative correspondences in many different dimensions. In this the empirical relevance of a model is established on the basis that it is too hard to simultaneously fit a model to evidence in this way, thus ruling that out as a source of its correspondence with that evidence.

3. So-called “parsimony principles” are a very unreliable manner of evaluating competing theories on grounds other than convenience or that of using limited data to justify the values of parameters (Edmonds 2007).

4. In many models a vague argument for its plausibility is often all that is described to show that it is applicable to the cases being discussed. At least calibration demonstrates its empirical applicability, rather than simply assuming it.

5. We are applying the principle of charity here, assuming that such conflations are innocent and not deliberate. However, there is increasing pressure from funding agencies to demonstrate ‘real life relevance’ so some of these apparent confusions might be more like ‘spin’ – trying to give an impression of empirical relevance even when this is merely an aspiration, in order to suggest that their model has more significant than they have reliably established.

6. This has been discussed elsewhere, e.g. (Moss & Edmonds 2005).

Acknowledgements

Thanks to all those we have discussed these issues with, including Scott Moss (who was talking about these kinds of issue more than 30 years ago), Eckhart Arnold (who made many useful comments and whose careful examination of the lack of empirical success of some families of model demonstrates our mostly abstract arguments), Sven Banisch and other members of the ESSA special interest group on “Strongly Empirical Modelling”.

References

Arnold, E. (2013). Simulation models of the evolution of cooperation as proofs of logical possibilities. How useful are they? Ethics & Politics, XV(2), pp. 101-138. https://philpapers.org/archive/ARNSMO.pdf

Arnold, E. (2015) How Models Fail – A Critical Look at the History of Computer Simulations of the Evolution of Cooperation. In Misselhorn, C. (Ed.): Collective Agency and Cooperation in Natural and Artificial Systems. Explanation, Implementation and Simulation, Philosophical Studies Series, Springer, pp. 261-279. https://eckhartarnold.de/papers/2015_How_Models_Fail

Axelrod, R. (1984) The Evolution of Cooperation, Basic Books.

Axelrod, R.  & Hamilton, W.D. (1981) The evolution of cooperation. Science, 211, 1390-1396. https://www.science.org/doi/abs/10.1126/science.7466396

Banisch, S., & Olbrich, E. (2019). Opinion polarization by learning from social feedback. The Journal of Mathematical Sociology, 43(2), 76-103. https://doi.org/10.1080/0022250X.2018.1517761

Carpentras, D., Maher, P. J., O’Reilly, C., & Quayle, M. (2022). Deriving An Opinion Dynamics Model From Experimental Data. Journal of Artificial Societies & Social Simulation, 25(4).http://doi.org/10.18564/jasss.4947

Cartwright, N. (1983) How the Laws of Physics Lie. Oxford University Press.

Chacoma, A. & Zanette, D. H. (2015). Opinion formation by social influence: From experiments to modelling. PloS ONE, 10(10), e0140406.https://doi.org/10.1371/journal.pone.0140406

Dignum, F., Edmonds, B. and Carpentras, D. (2022) Socio-Cognitive Systems – A Position Statement. Review of Artificial Societies and Social Simulation, 2nd Apr 2022. https://rofasss.org/2022/04/02/scs

Edmonds, B. (2000). The Use of Models – making MABS actually work. In. S. Moss and P. Davidsson. Multi Agent Based Simulation. Berlin, Springer-Verlag. 1979: 15-32. http://doi.org/10.1007/3-540-44561-7_2

Edmonds, B. (2007) Simplicity is Not Truth-Indicative. In Gershenson, C.et al. (eds.) Philosophy and Complexity. World Scientific, pp. 65-80.

Edmonds, B. (2010) Bootstrapping Knowledge About Social Phenomena Using Simulation Models. Journal of Artificial Societies and Social Simulation, 13(1), 8. http://doi.org/10.18564/jasss.1523

Edmonds, B. (2018) The “formalist fallacy”. Review of Artificial Societies and Social Simulation, 11th June 2018. https://rofasss.org/2018/07/20/be/

Edmonds, B., le Page, C., Bithell, M., Chattoe-Brown, E., Grimm, V., Meyer, R., Montañola-Sales, C., Ormerod, P., Root H. & Squazzoni. F. (2019) Different Modelling Purposes. Journal of Artificial Societies and Social Simulation, 22(3):6. http://doi.org/10.18564/jasss.3993

Epstein, J. M. (2008). Why Model?. Journal of Artificial Societies and Social Simulation, 11(4),12. https://www.jasss.org/11/4/12.html

Flache, A., Mäs, M., Feliciani, T., Chattoe-Brown, E., Deffuant, G., Huet, S. & Lorenz, J. (2017). Models of social influence: Towards the next frontiers. Journal of Artificial Societies and Social Simulation, 20(4), 2. http://doi.org/10.18564/jasss.4298

Grimm, V., Revilla, E., Berger, U., Jeltsch, F., Mooij, W.M., Railsback, S.F., et al. (2005). Pattern-oriented modeling of agent-based complex systems: lessons from ecology. Science, 310 (5750), 987–991. https://www.jstor.org/stable/3842807

Hartman, S. (1997) Modelling and the Aims of Science. 20th International Wittgenstein Symposium, Kirchberg am Weshsel.

Hofstadter, D. (1995) Fluid Concepts and Creative Analogies. Basic Books.

Kuhn, T. S. (1962). The structure of scientific revolutions. University of Chicago Press.

Lakoff, G. (2008). Women, fire, and dangerous things: What categories reveal about the mind. University of Chicago Press.

Merton, R.K. (1968). On the Sociological Theories of the Middle Range. In Classical Sociological Theory, Calhoun, C., Gerteis, J., Moody, J., Pfaff, S. and Virk, I. (Eds), Blackwell, pp. 449–459.

Meyer, R. & Edmonds, B. (2023). The Importance of Dynamic Networks Within a Model of Politics. In: Squazzoni, F. (eds) Advances in Social Simulation. ESSA 2022. Springer Proceedings in Complexity. Springer. (Earlier, open access, version at: https://cfpm.org/discussionpapers/292)

Moss, S. and Edmonds, B. (2005). Towards Good Social Science. Journal of Artificial Societies and Social Simulation, 8(4), 13. https://www.jasss.org/8/4/13.html

Squazzoni, F. et al. (2020) ‘Computational Models That Matter During a Global Pandemic Outbreak: A Call to Action’ Journal of Artificial Societies and Social Simulation 23(2):10. http://doi.org/10.18564/jasss.4298

Silver, N, (2012) The Signal and the Noise: Why So Many Predictions Fail – But Some Don’t. Penguin.

Simon, H. A. (1962). The architecture of complexity. Proceedings of the American philosophical society, 106(6), 467-482.https://www.jstor.org/stable/985254

Thompson, E. (2022). Escape from Model Land: How mathematical models can lead us astray and what we can do about it. Basic Books.

Wartofsky, M. W. (1979). The model muddle: Proposals for an immodest realism. In Models (pp. 1-11). Springer, Dordrecht.

Zeigler, B. P. (1976). Theory of Modeling and Simulation. Wiley Interscience, New York.

Edmonds, B., Carpentras, D., Roxburgh, N., Chattoe-Brown, E. and Polhill, G. (2024) Delusional Generality – how models can give a false impression of their applicability even when they lack any empirical foundation. Review of Artificial Societies and Social Simulation, 7 May 2024. https://rofasss.org/2024/05/06/delusional-generality

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

How Agent-based Models Offer Insights on Strategies to Mitigate Soil Degradation in North Korea: A Conversation with Dr. Yoosoon An

Leave a comment

By Hyesop Shin1 and Yoosoon An2

  1. Interviewer: (HS), University of Glasgow, UK
  2. Interviewee: (YA), Institute for Korean Regional Studies, Seoul National University, S.Korea.

Introduction

While there’s limited knowledge about North Korea’s farming system and food chain, it’s evident that soil degradation has been an ongoing concern for the nation. To gain deeper insights, I spoke with Dr. Yoosoon An, a renowned agent-based modeller from South Korea. His PhD research delved into land degradation and declining food production in North Korea during the 1990s using Agent-Based Modelling (ABM).

HS: Can you introduce yourself?

YA: Certainly. I’m Dr. Yoosoon An, a research fellow at the Institute for Korean Regional Studies at Seoul National University. My primary research interests are North Korea, Agent-Based Modelling, and the relationship between soil health and food security. I can’t believe I’ve been modelling ABM for nearly a decade!

HS: Can you give a brief overview of your research?

YA: During my academic journey, I was deeply intrigued by issues related to land degradation and landslides. But what really caught my attention was reading about the North Korean famine in the 1990s. It’s heartbreaking to think about it. Basically, in the mid-90s, North Korea faced this huge famine. It wasn’t just because of natural disasters like droughts, but also due to the economic chaos after the Soviet Union collapsed, and some big problems in their farming systems. This just destroyed their land, and so many people almost starved. You can find more details on its Wikipedia page.

HS: What part of social simulation would you like to introduce to the community?

YA: Well for ABM right?  I’d like to introduce my PhD research that explored strategies to combat land degradation and food shortages in North Korea, with a special emphasis on the devastating famine of the 1990s (An 2020). Although there’s a clear connection between land degradation and famine, both issues are intricate and there’s limited information available, both in North Korea and globally. Through agent-based modelling (ABM), my study examined the interplay between land degradation and the decline in food production as a pivotal factor behind North Korea’s major famine in the 1990s. This “vicious cycle of land degradation”, where agricultural productivity drops because of worsening land conditions, and then the land degrades further as people intensively cultivate it to compensate, plays a central role in the broader challenges of devastation, famine, and poverty.

I utilised ABM to look at land cover changes and posited scenarios to hypothesise the potential outcomes, given alternate policies during the 1990s. Through this research, I aimed to unravel the intricacies of the relationship between land degradation and food production, providing insights that may pave the way for future policy development and intervention strategies in analogous situations.

HS: So, you’re focusing on the famine from the ’90s, but what made you decide to simulate from the 1960s?

YA: The 1960s hold significance for several key reasons. After North Korea adopted the “shared ownership system” in 1946, private land ownership was permitted. But by 1960, following the Korean War, these private lands had been integrated into collective farms. Most of today’s agricultural practices in North Korea can be traced back to that period. Furthermore, my research pointed out a noticeable increase in documentation and data collection beginning in the 1960s, underscoring its importance. From a socio-ecological perspective, I believe that the famine was a culmination of multiple intersecting crises including the one that took place in 1995. Starting the simulation from the 1960s, and tracking land cover changes up to 2020, seemed the most comprehensive approach to understanding the intricate dynamics at play.

The Agent-based Model: the “North Korean Collective Farm”

HS: Let’s delve deeper into your model. The incorporation of both land use and human agents is particularly fascinating. Could you break down this concept figure for us before we discuss the simulation?

YA: Of course. If you refer to Figure 1, it visually represents the farm’s layout and topography. We’ve chosen to represent it through simplified square and ski-slope shapes. The model also integrates the initial forest cover to demonstrate the degradation that occurred when forests were converted into farmland. When setting the model, we positioned different land uses based on the environmental adaptation strategies traditional to the Korean people. So, you’ll notice the steeper forests situated to the north, the flatter rice fields to the south, and the villages strategically placed along the mountain edge.

YA: To give you a broader picture, the model we’ve termed the “North Korean Collective Farm” (as shown in Figure 1) is a composite representation of collective farms. In this model, a collective farm is visualised as a community where several farmers either co-own their land, reflecting cooperative farming practices (akin to the “Kolkhoze” in the Soviet Union) or as part of a state-owned agricultural entity (resembling the state farm or “Sovkhozy” from the Soviet Union). North Korea embraced this model in 1954 and by 1960 had fully transitioned all its farms into this system. While there’s a dearth of comprehensive data about North Korean collective farms, a few studies offer some general insights. Typically, a farm spans between 550 and 750 hectares, roughly equivalent to ‘Ri’, North Korea’s smallest administrative unit. On average, each of these farms accommodates 300-400 households, which translates to 700-900 active workers and a total of 1900-2000 residents. These farms are further segmented into 5-10 workgroups, serving as the foundational unit for both farming activities and the distribution of yield.

HS: So, in areas where there’s a lack of specific data or where details are too diverse to be standardised, you’ve employed abstraction and summarisation. This approach to modelling seems pragmatic. When you mention setting the initial agricultural land cover to 30% rice fields and 70% other farmland, is this a reflection of the general agricultural makeup in North Korea? Would this distribution be typical or is it an average derived from various sources?

YA: Exactly. Given the limited and sometimes ambiguous data regarding North Korea, abstraction and summarization become invaluable tools for our model. The 30% rice fields and 70% other farmland distribution is a generalised representation derived from an aggregate of available North Korean land use data. While it might not precisely mirror any specific farm, it provides a reasonable approximation of the agricultural landscape across the region. This method allows us to capture the essential features and dynamics without getting mired in the specifics of any one location.

Fig 1

Figure 1. Conceptual model of the Artificial North Korean Collective Farm: integrating land use and human agents to build an agent-based model for mitigating famine risk in North Korea.

HS: Okay so let’s talk about agents. So, you’ve focused on the ‘cooperative farm’ as a representative agent in your model. This is essentially to capture the intricacies of the North Korean agricultural landscape. Can you expand a bit more on how the ‘cooperative farm’ reflects the realities of North Korean agriculture and how the LUDAS framework enhances this?

YA: Certainly. The ‘cooperative farm’ or ‘cooperative household’ is more than just a symbolic entity. It encapsulates the very essence of North Korean agricultural practices. Beginning in the 1960s and persisting to the present day, these cooperative structures are foundational to the nation’s farming landscape. Notably, their geographical boundaries often align with administrative units, making them not just agricultural but also socio-political entities. When we employ broader system dynamics models that span the entirety of North Korea, often the granularity and the subtleties can get lost. Hence, zooming into the cooperative farm level provides us with the precision and detail needed to observe intricate dynamics and interactions.

YA: Another important reason is to apply the Land-use dynamic simulator (LUDAS) framework for the case of North Korea. Now, speaking of LUDAS – this framework was chosen for its ability to seamlessly bridge biophysical and socio-economic parameters. It’s a holistic approach that factors in long-term land use/cover changes against a backdrop of varied management, planning, and policy scenarios. The strength of LUDAS lies in its capability to encapsulate the intertwined dynamics of human-environment interactions. Through a multi-agent simulation process, LUDAS effectively mirrors real-world causal mechanisms, feedback loops, and interactions. By integrating this framework into our model, we aimed to offer a comprehensive portrayal of North Korea’s agricultural landscape, rich in both depth and breadth.

HS: How do the agents decide their actions and movements?

YA: Agent decisions are based on a simple principle: when they require more food, they change their work strategy and land use. Their decisions are divided into two categories: labour allocation and land-use changes. If their labour-to-food demand ratio exceeds 1, they redirect their labour and change their land use. If this ratio is less than one, they will stick to their previous strategies.

YA: In terms of labour allocation, we assume that a worker is available 300 days per year, working 8 hours per day. The minimum labour required to cultivate an average crop on a 100m2 rice field is 36 hours per year and 48 hours for other crops. These figures are based on South Korean farming data because North Korean data is unavailable. Our model initially used 6 hours for rice and 8 hours for other crops, but these settings had no effect. As a result, we changed the hours to better reflect conditions in North Korea.

YA: Agents with a food demand ratio of less than one will allocate their labour time based on our initial assumption (if this is the first year) or on the previous year’s allocation. If the ratio exceeds one, they adjust their time allocation based on soil productivity. They will first reduce or eliminate investment in less productive lands, then devote more time to more fertile areas. The labour efficiency metric is determined by comparing the current labour time to the initially assumed time. If you have time you can take a look at Equation (3) mentioned in the paper (An & Park 2023).

HS: So, in essence, how does this environment shape the behaviour and choices of the agents?

YA: The agents operate within the landscape-environmental system, which is a subsystem influenced by the LUDAS framework. This system offers a detailed insight into land degradation and food production processes specific to North Korea. Comprising five unique submodules, it considers the biological, physical, and chemical properties of the soil, coupled with a quality index for the soil and a final metric that evaluates potential food yield by integrating these factors. All these elements together determine how agents adapt and make decisions based on the changing environment.

HS: How did you decide on a one-year interval for your simulation, especially in the context of Discrete Event Simulation?

YA: In places with a temperate to cold climate like North Korea, farming activities primarily follow an annual rhythm. Apart from this agricultural reasoning, my decision was, in part, based on the data availability. The datasets I had access to didn’t provide more detailed time frames. However, considering that many nations’ agricultural practices revolve around an annual cycle, it made sense to align both environmental and socioeconomic indicators with this timeframe. Still, I’m eager to eventually incorporate more granular data, such as monthly datasets, to explore the nuanced seasonal changes in land cover.

HS: Can you explain this loop diagram for us?

YA: The diagram presents a feedback loop related to land use happening every year in the simulation. When land productivity goes down because of overuse, there’s a greater demand for food. This greater demand then causes people to use the land more, further decreasing its quality. This continuous cycle results in ongoing harm to the land, and thus increases the food pressure for the agents also known as cooperative farms.

YA: Essentially, the loop demonstrates that “lower land productivity leads to more demand for food, which then causes even more intensive land use, further reducing the land’s quality.” In our study, we noticed that as the quality of the land decreased steadily, the decrease in the food it produced was much faster. This suggests that the effects get stronger with each cycle due to the feedback loop.

Fig 2

Figure 2. A feedback loop that connects land degradation and soil quality, subsequently inducing food pressure on agents. Within this loop, two critical points are identified: “E,” representing an early warning signal, and “T,” representing a threshold. Crossing this threshold can lead to a systematic collapse.

HS: Given the challenges associated with gathering information on North Korea, how did you ensure the validity of your model’s results?

YA: Validating the outcomes, especially for North Korea, was indeed challenging. For the environmental aspects, we relied on satellite imagery and referenced previous research data to validate our variables. When it came to the human agents, we tapped into an extensive array of literature and data on North Korean cooperative farms. We kept the behavioural rules for these agents straightforward, for instance, they’d modify their behaviours when faced with hunger, prioritise maximising land productivity, and turn to inter-mountain cultivation if they encountered continued food shortages. As for variables like labour hours and land potential, we began with South Korean data due to the absence of precise data from the North. Then, based on the outcomes of our iterative simulations, we made necessary adjustments to ensure the model aligned as closely as possible with reality.

HS: Before we dive into the findings, I just wanted to hear your opinion on Proof-Of-Concept (POC) models because you employed POC for your simulation. Can you discuss the advantages and limitations of using such models?

YA: POC models are particularly effective in scenarios with limited data availability. Despite the data constraints from North Korea, the consistency in their reports allowed me to simulate the progression of land degradation over time. POC models often have an intuitive interface, enabling easy adjustments and scenario applications. Debugging is also straightforward. However, the results can sometimes lack precise real-world applicability. Adding data or algorithms necessitates an abstraction process, which can introduce inaccuracies. For instance, equating one grid pixel to a household can oversimplify the model. Additionally, the interface might sometimes be less intuitive.

YA: I aimed to represent the food sustainability and socio-ecological systems in northeastern Asia, encompassing both China and the Korean peninsula. However, due to the lack of data for North Korea, I used a Proof-of-Concept model instead.

Findings

HS: From your simulations, what were the main insights or conclusions you drew?

YA: Our baseline simulation of the North Korean cooperative farm model painted a concerning picture. It revealed a vicious cycle where land degradation led to decreased food production, eventually culminating in a famine. Beginning the simulation from 1960, our model anticipated a famine occurring approximately 35 years later, which aligns with the real-world famine of 1995 in North Korea. You can take a look at Figure 3.

YA: On introducing the additional food supply scenario, we observed a delay in the onset of the famine. This finding highlights the significance of addressing the isolated nature of North Korea when aiming to prevent famine. However, it’s imperative to understand that merely making the food system more accessible isn’t a silver bullet. Comprehensive solutions must also focus on various other interventions.

HS: Based on your research, what are the potential solutions to address the future food crisis in North Korea?

YA: Our model highlights a feedback loop that intensifies food scarcity as land quality degrades. One approach we tested was enhancing external food supply. The results showed that this strategy can slow down the threat of famine, but it doesn’t completely break the loop. Even with more food coming in, the core issue—deteriorating land quality—remains unresolved.

YA: Several alternatives to address this feedback loop include adopting sustainable agricultural practices, supplementing with external energy sources, or restructuring North Korea’s collective farming system. We’re still working on modelling these solutions effectively.

YA: Historically, the Korean Peninsula faced severe famines in the 1600s, attributed to factors like climatic changes, deforestation, and diplomatic isolation. These circumstances resemble North Korea’s recent famine in the 1990s. The underlying problem in both cases is a cycle where declining land productivity demands more food production, further harming the land.

YA: Considering this historical context, it’s possible to argue that the Korean Peninsula, by itself, might not sustain its population and environment without external help. Supplying food and energy from outside might be more of a temporary solution, giving us time to seek more permanent ones.

YA: To genuinely address the land and food problem, we need to explore and test alternatives further. This could involve sustainable farming methods, efficient agricultural systems, and broader diplomatic actions for international trade and cooperation. The ultimate goal is a sustainable future for both North Korea and the entire Korean Peninsula.

Fig 3

Figure 3. Summary of the results for replicability of the great famine in the 1990s: (a) Mean and standard deviation trends of land-use change (left) and food yield and soil quality (right); (b) Examples of land-use change in the model (NetLogo Interface)

Other Stories

HS: Can you share more stories from your research journey?

YA: When starting my PhD, the initial idea was to build upon my Master’s thesis about the North Korean land degradation-famine model, known as the “Pyong-an-do Model”. To note, Pyong-an-do (pronounced as doe a deer) is a province that encompasses Pyongyang, the capital, and the surrounding regions. However, data limitations made progress challenging. Around mid-2018, a visiting professor in ecological modelling suggested simplifying the model, sparking the concepts of “creating a virtual North Korea” and “establishing a virtual collective farm.”

YA: By July 2018, with a basic model ready, I applied to present at the Computational Social Science (CSS) 2018 conference. Unbeknownst to me, a full paper was required beyond just an abstract. Thankfully, the Computational Social Science Society of the Americas (CSSSA) provided an extra two weeks for submission due to the intriguing nature of the topic. That intense fortnight saw a majority of my thesis chapter being written!

YA: During the conference, a grad student from India pointed out that the results from my model, which predicted the collapse of North Korea’s farm system in around 35 years, had some eerie similarities to what happened in India and Ghana after the British messed around with their agriculture. They faced famines about 30-40 years later. He even mentioned maybe I should look into making a more general famine model, and brought up Dr. Amartya Sen’s thoughts on freedom, inequality, and development. I thought it was a cool idea, but more like a long-term idea for me.

YA: Fast forward to early 2021, I conducted interviews with experts and North Korean defectors about my model’s findings. While some feedback was beyond my thesis’ scope or challenging to incorporate, a comment from a defector with agricultural expertise stood out. He mentioned that, contrary to criticisms, the model’s depiction of nearly abandoned agricultural lands in North Korea during the early 1990s mirrored reality, further validating the accuracy of my work.

HS: For those interested in delving deeper, where can they access your model?

YA: You can find the model on my Github account (An 2023). Additionally, I’m considering publishing it on comses.net for broader accessibility and collaboration.

Date of Interview: Feb 2023, Translated into English: Sep 2023.

References

An. Y(2020), A Study on Land Degradation and Declining Food Production based on the Concept of Complex Adaptive System: Focusing on the North Korean Famine in the 1990s (Doctoral dissertation), Seoul National University (in Korean with English Abstract). link

An, Y and Park S.J (2023), Developing an Agent-Based Model to Mitigate Famine Risk in North Korea: Insights from the “Artificial North Korean Collective Farm” Model, Land, 12(4), 735 https://doi.org/10.3390/land12040735

An, Y (2023) Artificial_NK_cooperative_farm_model: https://github.com/newsoon8/Artificial_NK_cooperative_farm_model

Shin, H. & An, Y. (2023) How Agent-based Models Offer Insights on Strategies to Mitigate Soil Degradation in North Korea: A Conversation with Dr. Yoosoon An. Review of Artificial Societies and Social Simulation, 25 Oct 2023. https://rofasss.org/2023/10/25/Interview-Yoosoon-An

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Exascale computing and ‘next generation’ agent-based modelling

Leave a comment

By Gary Polhill, Alison Heppenstall, Michael Batty, Doug Salt, Ricardo Colasanti, Richard Milton and Matt Hare

Introduction

In the past decade we have seen considerable gains in the amount of data and computational power that are available to us as scientific researchers.  Whilst the proliferation of new forms of data can present as many challenges as opportunities (linking data sets, checking veracity etc.), we can now begin to construct models that are capable of answering ever more complex and interrelated questions.  For example, what happens to individual health and the local economy if we pedestrianize a city centre?  What is the impact of increasing travel costs on the price of housing? How can we divert economic investment to places in economic decline from prosperous cities and regions. These advances are slowly positioning agent-based modelling to support decision-makers to make informed evidence-based decisions.  However, there is still a lack of ABMs being used outside of academia and policy makers find it difficult to mobilise and apply such tools to inform real world problems: here we explore the background in computing that helps address the question why such models are so underutilised in practice.

Whilst reaching a level of maturity (defined as being an accepted tool) within the social sciences, agent-based modelling still has several methodological barriers to cross.  These were first highlighted by Crooks et al. (2008) and revisited by Heppenstall et al. (2020) and include robust validation, elicitation of behaviour from data and scaling up.  Whilst other disciplines, such as meteorology, are able to conduct large numbers of simulations (ensemble modelling) using high-performance computing, there is a relative absence of this capability within agent-based modelling. Moreover, many different kinds of agent-based models are being devised, and key issues concern the number and type of agents and these are reflected in the whole computational context in which such models are developed. Clearly there is potential for agent-based modelling to establish itself as a robust policy tool, but this requires access to large-scale computing.

Exascale high-performance computing is defined with respect to speed of calculation with orders of magnitude defined as 10^18 (a billion-billion) floating point operations per second (flops). That is fast enough to calculate the ratios of the ages of each of every possible pair of people in China in roughly a second. By comparison, modern-day personal computers are around 10^9 flops (gigascale) – a billion times slower. The same rather pointless calculation of age ratios of the Chinese would take just over thirty years on a standard laptop at the time of writing (2023). Though agent-based modellers are more interested in instructions incorporating the rules operated by each agent executed per second than in floating-point operations, the speed of the two is approximately the same.

Anecdotally, the majority of simulations of agent-based models are on personal computers operating on the desktop. However, there are examples of the use of high-performance computing environments such as computing clusters (terascale) and cloud services such as Microsoft’s Azure, Amazon’s AWS or Google Cloud (tera- to peta-scale). High-performance computing provides the capacity to do more of what we already do (more runs for calibration, validation and sensitivity analysis) and/or at a larger scale (regional or sub-national scale rather than local scale) with the number of agents scaled accordingly. As a rough guide, however, since terascale computing is a million times slower than exascale computing, an experiment that currently takes a few days or weeks in a high-performance computing environment could be completed in a fraction of a second at exascale.

We are all familiar with poor user interface design in everyday computing, and in particular the frustration of waiting for the hourglasses, spinning wheels and progress bars to finish so that we can get on with our work. In fact, the ‘Doherty Threshold’ (Yablonski 2020) stipulates 400ms interaction time between human action and computer response for best productivity. If going from 10^9 to 10^18 flops is simply a case of multiplying the speed of computation by a billion, the Doherty threshold is potentially feasible with exascale computing when applied to simulation experiments that now require very long wait times for completion.

The scale of performance of exascale computers means that there is scope to go beyond doing-more-of-what-we-already-do to thinking more deeply about what we could achieve with agent-based modelling. Could we move past some of these methodological barriers that are characteristic of agent-based modelling? What could we achieve if we had appropriate software support, and how this would affect the processes and practices by which agent-based models are built? Could we move agent-based models to having the same level of ‘robustness’ as climate models, for example? We can conceive of a productivity loop in which an empirical agent-based model is used for sequential experimentation with continual adaptation and change, continued experiment with perhaps a new model emerging from these workflows to explore tangential issues. But currently we need to have tools that help us build empirical agent-based models much more rapidly, and critically, to find, access and preprocess empirical data that the model will use for initialisation, then finding and affirming parameter values.

The ExAMPLER project

The ExAMPLER (Exascale Agent-based Modelling for PoLicy Evaluation in Real-time) project is an eighteen-month project funded by the Engineering and Physical Sciences Research Council to explore the software, data and institutional requirements to support agent-based modelling at exascale.

With high-performance computing use not being commonplace in the agent-based modelling community, we are interested in finding out what the state-of-the-art is in high-performance computing use by agent-based modellers, undertaking a systematic literature review to assess the community’s ‘exascale-readiness’. This is not just a question of whether the community has the necessary technical skills to use the equipment. It is also a matter that covers whether the hardware is appropriate to the computational demands that agent-based modellers have, whether the software in which agent-based models are built can take advantage of the hardware, and whether the institutional processes by which agent-based modellers access high-performance computing – especially with respect to information requested of applicants – is aware of their needs.

We will then benchmark the state-of-the-art against high-performance computing use in other domains of research: ecology and microsimulation, which are comparable to agent-based social simulation (ABSS); and fields such as transportation, land use and urban econometric  modelling that are  not directly comparable to ABSS, but have similar computational challenges (e.g. having to simulate many interactions, needing to explore a vast uncharted parameter space, containing multiple qualitatively different outcomes from the same initial conditions, and so on). Ecology might not simulate agents with decision-making algorithms as computationally demanding as some of those used by agent-based modellers of social systems, while a crude characterisation of microsimulation work is that it does not simulate interactions among heterogeneous agents, which affects the parallelisation of simulating them. Land use and transport models usually rely on aggregates of agents but increasingly there are being disaggregated to finer and fine spatial units with these units themselves being treated more like agents. The ‘discipline-to-be-decided’ might have a community with generally higher technical computing skills than would be expected among social scientists. Benchmarking would allow us to gain better insights into the specific barriers faced by social scientists in accessing high-performance computing.

Two other strands of work in ExAMPLER feature significant engagement with the agent-based modelling community. The project’s imaginary starting point is a computer powerful enough to experiment with an agent-based model which run in fractions of a second. With a pre-existing agent-based model, we could use such a computer in a one-day workshop to enable a creative discussion with decision-makers about how to handle problems and policies associated with an emerging crisis. But what if we had the tools at our disposal to gather and preprocess data and build models such that these activities could also be achievable in the same day? or even the same hour? Some of our land use and transportation models are already moving in this direction (Horni, Nagel, and Axhausen, 2016). Agent-based modelling would thus become a social activity that facilitates discussion and decision-making that is mindful of complexity and cascading consequences. The practices and procedures associated with building an agent-based model would then have evolved significantly from what they are now, as have the institutions built around accessing and using high-performance computing.

The first strand of work co-constructs with the agent-based modelling community various scenarios by which agent-based modelling is transformed by the dramatic improvements in computational power that exascale computing entails. These visions will be co-constructed primarily through workshops, the first of which is being held at the Social Simulation Conference in Glasgow – a conference that is well-attended by the European (and wider international) agent-based social simulation community. However, we will also issue a questionnaire to elicit views from the wider community of those who cannot attend one of our events. There are two purposes to these exercises: to understand the requirements of the community and their visions for the future, but also to advertise the benefits that exascale computing could have.

In a second series of workshops, we will develop a roadmap for exascale agent-based modelling that identifies the institutional, scientific and infrastructure support needed to achieve the envisioned exascale agent-based modelling use-cases. In essence, what do we need to have in place to make exascale a reality for the everyday agent-based modeller? This activity is underpinned by training ExAMPLER’s research team in the hardware, software and algorithms that can be used to achieve exascale computation more widely. That knowledge, together with the review of the state-of-the-art in high-performance computing use with agent-based models, can be used to identify early opportunities for the community to make significant gains (Macal, and North, 2008)

Discussion

Exascale agent-based modelling is not simply a case of providing agent-based modellers with usernames and passwords on an exascale computer and letting them run their models on it. There are many institutional, scientific and infrastructural barriers that need to be addressed.

On the scientific side, exascale agent-based modelling could be potentially revolutionary in transforming the practices, methods and audiences for agent-based modelling. As a highly diverse community, methodological development is challenged both by the lack of opportunity to make it happen, and by the sheer range of agent-based modelling applications. Too much standardization and ritualized behaviour associated with ‘disciplining’ agent-based modelling risks some of the creative benefits of having the cross-disciplinary discussions that agent-based modelling enables us to have. Nevertheless, it is increasingly clear that off-the-shelf methods for designing, implementing and assessing models are ill-suited to agent-based modelling, or – especially in the case of the last of these – fail to do it justice (Polhill and Salt 2017, Polhill et al. 2019). Scientific advancement in agent-based modelling is predicated on having the tools at our disposal to tell the whole story of its benefits, and enabling non-agent-based modelling colleagues to understand how to work with the ABM community.

Hence, hardware is only a small part of the story of the infrastructure supporting exascale agent-based modelling. Exascale computers are built using GPUs (Graphical Processing Units) – which, bluntly-speaking, are specialized computing engines for performing matrix calculations and ‘drawing millions of triangles as quickly as possible’ – they are, in any case, different from CPU-based computing. In Table 4 of Kravari and Bassiliades’ (2015) survey of agent-based modelling platforms, only two of the 24 platforms reviewed (Cormas – Bommel et al. 2016 and GAMA – Taillandier et al. 2019) are not listed as involving Java and/or the Java Virtual Machine. (As it turns out, GAMA does use Java.) TornadoVM (Papadimitriou et al. 2019) is one tool allowing Java Virtual Machines to run on GPUs. Even if we can then run NetLogo on a GPU, specialist GPU-based agent-based modelling platforms such as Richmond et al.’s (2010, 2022) FLAME GPU may be preferable in order to make best use of the highly parallelized computing environment on GPUs.

Such software simply achieves getting an agent-based model running on an exascale computer. Realizing some of the visions of future exascale-enabled agent-based modelling means rather more in the way of software support. For example, the one-day workshop in which an agent-based modelling is co-constructed with stakeholders asks either a great deal of the developers in terms of building a bespoke application in tens of minutes, or many stakeholders trusting pre-constructed modular components that can be brought together rapidly using a specialist software tool.

As has been noted (e.g. Alessa et al. 2006, para 3.4), agent-based modelling is already challenging for social scientists without programming expertise, and GPU programming is a highly specialized domain in the world of software environments. Exascale computing intersects GPU programming with high-performance computing; issues with the ways in which high-performance computing clusters are typically administered make access to them a significant obstacle for agent-based modellers (Polhill 2022). There are therefore institutional barriers that need to be broken down for the benefits of exascale agent-based modelling to be realized in a community primarily interested in the dynamics of social and/or ecological complexity, and rather less in the technology that enables them to pursue that interest. ExAMPLER aims to provide us with a voice that gets our requirements heard so that we are not excluded from taking best advantage of advanced development in computing hardware.

Acknowledgements

The ExAMPLER project is funded by the EPSRC under grant number EP/Y008839/1.  Further information is available at: https://exascale.hutton.ac.uk

References

Alessa, L. N., Laituri, M. and Barton, M. (2006) An “all hands” call to the social science community: Establishing a community framework for complexity modeling using cyberinfrastructure. Journal of Artificial Societies and Social Simulation 9 (4), 6. https://www.jasss.org/9/4/6.html

Bommel, P., Becu, N., Le Page, C. and Bousquet, F. (2016) Cormas: An agent-based simulation platform for coupling human decisions with computerized dynamics. In Kaneda, T., Kanegae, H., Toyoda, Y. and Rizzi, P. (eds.) Simulation and Gaming in the Network Society. Translational Systems Sciences 9, pp. 387-410. doi:10.1007/978-981-10-0575-6_27

Crooks, A. T., C. J. E. Castle, and M. Batty. (2008). “Key Challenges in Agent-Based Modelling for Geo-spatial Simulation.” Computers, Environment and Urban Systems  32(6),  417– 30.

Heppenstall A, Crooks A, Malleson N, Manley E, Ge J, Batty M. (2020). Future Developments in Geographical Agent-Based Models: Challenges and Opportunities. Geographical Analysis. 53(1): 76 – 91 doi:10.1111/gean.12267

Horni, A, Nagel, K and Axhausen, K W. (eds)(2016) The Multi-Agent Transport Simulation MATSim, Ubiquity Press, London, 447–450

Kravari, K. and Bassiliades, N. (2015) A survey of agent platforms. Journal of Artificial Societies and Social Simulation 18 (1), 11. https://www.jasss.org/18/1/11.html

Macal, C. M., and North, M. J. (2008) Agent-Based Modeling And Simulation for EXASCALE Computing, http://www.scidac.org

Papadimitriou, M., Fumero, J., Stratikopoulos, A. and Kotselidis, C. (2019) Towards prototyping and acceleration of Java programs onto Intel FPGAs. Proceedings of the 2019 IEEE 27th Annueal International Symposium on Field-Programmable Custom Computing Machines (FCCM). doi:10.1109/FCCM.2019.00051

Polhill, G. (2022) Antisocial simulation: using shared high-performance computing clusters to run agent-based models. Review of Artificial Societies and Social Simulation, 14 Dec 2022. https://rofasss.org/2022/12/14/antisoc-sim

Polhill, G. and Salt, D. (2017) The importance of ontological structure: why validation by ‘fit-to-data’ is insufficient. In Edmonds, B. and Meyer, R. (eds.) Simulating Social Complexity (2nd edition), pp. 141-172. doi:10.1007/978-3-319-66948-9_8

Polhill, J. G., Ge, J., Hare, M. P., Matthews, K. B., Gimona, A., Salt, D. and Yeluripati, J. (2019) Crossing the chasm: a ‘tube-map’ for agent-based simulation of policy scenarios in spatially-distributed systems. Geoinformatica 23, 169-199. doi:10.1007/s10707-018-00340-z

Richmond, P., Chisholm, R., Heywood, P., Leach, M. and Kabiri Chimeh, M. (2022) FLAME GPU (2.0.0-rc). Zenodo. doi:10.5281/zenodo.5428984

Richmond, P., Walker, D., Coakley, S. and Romano, D. (2010) High performance cellular level agent-based simulation with FLAME for the GPU. Briefings in Bioinformatics 11 (3), 334-347. doi:10.1093/bib/bbp073

Taillandier, P., Gaudou, B., Grignard, A.,Huynh, Q.-N., Marilleau, N., P. Caillou, P., Philippon, D. and Drogoul, A. (2019). Building, composing and experimenting complex spatial models with the GAMA platform. Geoinformatica 23 (2), 299-322, doi:10.1007/s10707-018-00339-6

Yablonski, J. (2020) Laws of UX. O’Reilly. https://www.oreilly.com/library/view/laws-of-ux/9781492055303/

Polhill, G., Heppenstall, A., Batty, M., Salt, D., Colasanti, R., Milton, R. and Hare, M. (2023) Exascale computing and ‘next generation’ agent-based modelling. Review of Artificial Societies and Social Simulation, 9 Mar 2023. https://rofasss.org/2023/09/29/exascale-computing-and-next-gen-ABM

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content

Agent-based Modelling as a Method for Prediction for Complex Social Systems – a review of the special issue

Leave a comment

International Journal of Social Research Methodology, Volume 26, Issue 2.

By Oswaldo Terán

Escuela de Ciencias Empresariales, Universidad Católica del Norte, Coquimbo, Chile

This special issue appeared following a series of articles in RofASSS regarding the polemic around Agent-Based Modelling (ABM) prediction (https://rofasss.org/tag/prediction-thread/).  As expected, the articles in the special issue complement and expand upon the initial RofASSS’s discussion.

The goal of the special issue is to explore a wide range of positions regarding ABM prediction, encompassing methodological, epistemic and pragmatic issues. Contributions go from moderately sceptic and pragmatic positions to strongly sceptic positions. Moderately sceptic views argue that ABM can cautiously be employed for prediction, sometimes as a complement to other approaches, acknowledging its somewhat peripheral role in social research. Conversely, strongly sceptic positions contend that, in general, ABM can not be utilized for prediction. Several factors are instrumental in distinguishing and understanding these positions with respect to ABM prediction, especially the following:

  • the conception of prediction.
  • the complexity of modelled systems and models: this encompasses factors such as multiple views (or perspectives), uncertainty, auto-organization, self-production, emergence, structural change, and data incompleteness. These complexities are associated with the limitations of our language and tools to comprehend and symmetrically model complex systems.

Considering these factors, we will summarize the diverse positions presented in this special issue. Then, we will delve into the notions of prediction and complexity and briefly situate each position within the framework provided by these definitions

Elsebroich and Polhill (2023) (Editorial) summarizes the diverse positions in the special issue regarding prediction, categorizing them into three groups: 1) Positive, a position that assumes that “all we need for prediction is to have the right data,  methods and mechanism” (p. 136); 2) pragmatic, a position advocate to for cautious use of ABM to attempt prediction, often to compliment other approaches and avoid exclusive reliance on them; and 3) sceptic, a position arguing that ABM can not be used for prediction but can serve other purposes.  The authors place this discussion in a broader context, considering other relevant papers on ABM prediction. The authors acknowledge the challenge of prediction in complex systems, citing factors such as multiple perspectives, asynchronous agent actions, emergence, nonlinearity, non-ergodicity, evolutionary dynamics and heterogeneity. They indicate that some of these factors are well managed in ABM, but not others, noticeably “multiple perspectives/views”. Uncertainty is another critical element affecting ABM prediction, along with the relationship between prediction and explanation. The authors proved a summary of the debate surrounding the possibilities of prediction and its relation with explanation, incorporating insightful views from external sources (e.g., Thompson & Derr, 2009; Troitzsch, 2009). They also highlight recent developments in this debate, noticing that ABM has evolved into a more empirical and data-driven approach, deeply focused on modelling complex social and ecological systems, including Geographical Information Systems data and real time data integration, leading to a more contentious discussion regarding empirical data-driven ABM prediction.

Chattoe-Brown (2023) supports the idea that ABM prediction is possible. He argues for the utility of using AMB not only to predict real world outcomes but also to predict models. He also advocates for using prediction for predictive failure and assessing predictions. His notion of prediction finds support on by key elements of prediction in social science derived from real research across disciplines. For instance, the need of adopting a conceptual approach to enhance our comprehension of the various facets of prediction, the functioning of diverse prediction approaches, and the need for clear thinking about temporal logic. Chattoe-Brown argues that he attempts to make prediction intelligible rather than seen if it is successful. He support the idea that ABM prediction is useful for coherent social science. He contrasts ABM to other modelling methods that predict on trend data alone, underscoring the advantages of ABM. From his position, ABM prediction can add value to other research, taking a somewhat secondary role.

Dignum (2023) defends the ability of ABM to make prediction while distinguishing the usefulness of a prediction from the truth of a prediction. He argues in favour of limited prediction in specific cases, especially when human behaviour is involved. He shows prediction alongside explanations of the predicted behaviour, which arise under specific constrains that define particular scenarios. His view is moderately positive, suggesting that prediction is possible under certain specific conditions, including a stable environment and sufficient available data.

Carpentras and Quayle (2023) call for improved agent specification to reduce distortions when using psychometric instruments, particularly in measurements of political opinion within ABM. They contend that the quality of prediction and validation depends on the scale of the system but acknowledges the challenges posed by the high complexity of the human brain, which is central to their study. Furthermore, they raise concerns about representativeness, especially considering the discrepancy between certain theoretical frameworks (e.g., opinion dynamics) and survey data.

Anzola and García-Díaz (2023) advocate for better criteria to judge prediction and a more robust framework for the practice of prediction to better coordinate efforts within the research community (helping to better contextualize needs and expectations). They hold a somewhat sceptic position, suggesting that prediction typically serve an instrumental role in scientific practices, subservient to other epistemic goals.

Elsenbroich and Badham (2023) adopt a somewhat negative and critical stance toward using ABM for prediction, asserting that ABM can improve forecasting but not provide definite predictions of specific future events. ABM can only generate coherent extrapolations from a certain initialization of the ABM and a set of assumptions. They argue that ABM generates “justified stories” based on internal coherence, mechanisms and consistency  with empirical evidence, but these can not be confused with precise predictions. They ask for the combined support of ABM on theoretical developments and external data.

Edmonds (2023) is the most sceptical regarding the use of ABM for prediction, contending that the motivation for prediction in ABM is a desire without evidence of its achievability. He highlights inherent reasons for preventing prediction in complex social and ecological systems, including incompleteness, chaos, context specificity, and more. In his perspective, it is essential to establish the distinction between prediction and explanation. He advocates for recognizing the various potential applications of AMB beyond prediction, such as description, explanation, analogy, and more. For Edmonds, prediction should entail generating data that is unknown to the modellers. To address the ongoing debate and the weakness of the practices in ABM prediction, Edmonds proposes a process of iterative and independent verification. However, this approach faces limitations due to the incomplete understanding of the underlying process that should be included into the requirement for high-quality, relevant data. Despite these challenges, Edmonds suggest that prediction could prove valuable in meta-modelling, particularly to comprehend better our own simulation models.

The above summarized diverse positions on ABM prediction within the reviewed articles can be better understood through the lenses of Troitzsch’s notion of prediction and McNabb’s descriptions of complex and complicated systems. Troitzsch (2009) distinguishes the difference between prediction and explanation by using three possible conceptions of predictions. The typical understanding of ABM prediction closely aligns with Troitzsch’s third definition of prediction, which answer to the following question:

“Which state will the target system reach in the near future, again given parameters and previous states which may or may not have been precisely measured?”

The answer to this question results in a prediction, which can be either stochastic or deterministic. In our view, explanations encompass broader range of statements than predictions. An explanation entails a wider scope, including justifications, descriptions, and reasons for various real or hypothetical scenarios. Explanation is closely tied to a fundamental aspect of human communication capacity signifying the act of making something plain, clear or comprehensible by elaborating its meaning. But, what precisely does it expand or elaborate?. It expands a specific identification, opinion, judgement or belief. In general, a prediction implies a much narrower and more precise statement than an explanation, often hinting at possibilities regarding future events.

Several factors influence complex systems, including self-organization, multiple views, and dynamic complexity as defined by McNabb (2023a-c). McNabb contend that in complex systems the interaction among components and between the system as a whole and its environment transcend the insights derived from a mere components analysis. Two central characteristics of complex systems are self-organization and emergence. It is important to distinguish between complex systems and complicated systems: complex systems are organic systems (comprising biological, psychological and social systems), whereas complicated systems are mechanical systems (e.g., air planes, a computer, and ABM models). The challenge of agency arises primarily in complex systems, marked by highly uncertain behaviour. Relationships within self-organized system exhibit several noteworthy properties, although, given the need for a concise discussion regarding ABM prediction, we will consider here only a few of them (McNabb, 2023a-c):

  1. Multiple views,
  2. Dynamic interactions (connexion among components changes over time),
  3. Non-linear interaction (small causes can lead to unpredictable effects),
  4. The system lacks static equilibrium (instead, it maintains a dynamic equilibrium and remains unstable),
  5. Understanding the current state necessitates examining Its history (a diachronic, not synchronic study, is essential)

Given the possibility of multiple views, a complex systems are prone to significant structural change due to  dynamic and non-linear interactions, dynamic equilibrium  and diachronic evolution. Additionally, the probability of possessing both the right change mechanism (the logical process) and complete data (addressing the challenge of data incompleteness) required to initialize the model and establish necessary assumptions is excessively low. Consequently, predicting outcomes in complex systems (defined as organic systems) whether using AMB or alternative mechanisms, becomes nearly impossible. If such prediction does occur, it typically happens under highly specific conditions, such as within a brief time frame and controlled settings, often amounting to a form of coincidental success. Only after the expected event or outcomes materializes can we definitely claim that it was predicted. Although prediction remains a challenging endeavour in complex systems, it remains viable in complicated systems. In complicated systems, prediction serves as an answer to Troitzsch’s aforementioned question.

Taking into account Troitzsch’s notion of prediction and McNabb’s ideas on complex systems and complicated systems, let’s briefly revisit the various positions presented in this special issue.

Chattoe-Brown (2023) suggests using models to predict models. Models are considered complicated rather than complex systems, so it this case, we would be predicting a complicated system rather than a complex one. This represents a significant reduction.

Dignum (2023) argues that prediction is possible in cases where there is a stable environment (conditions) and sufficient available data. However, this generally is not the case, making it challenging to meet the requirements for prediction when considering complex (organic) systems.

Carpentras and Quayle (2023) themselves acknowledge the difficulties of prediction in ABM when studying issues related to psychological systems involving psychometric measures, which are a type of organic system, aligning with our argument.

Elsenbroich and Badham (2023), Elsebroich and Polhill (2023), and Edmonds (2023) maintain a strongly sceptic position regarding ABM prediction. They argue that AMBs yield coherent extrapolations based on a specific initialization of the model and a set of assumptions, but these extrapolations are not necessarily grounded in reality. According to them, complex systems exhibit properties such as information incompleteness, multiple perspectives, emergence, evolutionary dynamics, and context specificity. In this respect, their position aligns with the stance we are presenting here.

Finally, Anzola and García-Díaz (2023) advocate for a more robust framework for prediction and recognizes the ongoing debate on prediction, an stance that closely resonates with our own.

In conclusion, Troitzsch notion of prediction and McNabb descriptions of complex systems and complicated systems have helped us better understand the diverse positions on ABM prediction in the reviewed issue. This exemplifies how a good conceptual framework, in this
case offered by appropriate notions of prediction and complexity, can
contribute to reducing the controversy surrounding ABM prediction.

References

Anzola D. and García-Díaz C. (2023). What kind of prediction? Evaluating different facets of prediction in agent-based social simulation International Journal of Social Research Methodology, 26(2), pp. 171-191. https://doi.org/10.1080/13645579.2022.2137919

Carpentras D. and Quayle M. (2023). The psychometric house-of-mirrors: the effect of measurement distortions on agent-based models’ predictions. International Journal of Social Research Methodology, 26(2), pp. 215-231. https://doi.org/10.1080/13645579.2022.2137938

Chattoe-Brown E. (2023). Is agent-based modelling the future of prediction International Journal of Social Research Methodology, 26(2), pp. 143-155. https://doi.org/10.1080/13645579.2022.2137923

Dignum F. (2023). Should we make predictions based on social simulations?}. International Journal of Social Research Methodology, 26(2), pp. 193-206. https://doi.org/10.1080/13645579.2022.2137925

Edmonds B. (2023). The practice and rhetoric of prediction – the case in agent-based modelling. International Journal of Social Research Methodology, 26(2), pp. 157-170. https://doi.org/10.1080/13645579.2022.2137921

Edmonds, B., Polhill, G., & Hales, D. (2019). Predicting Social Systems – A Challenge. https://rofasss.org/2019/11/04/predicting-social-systems-a-challenge/

Elsenbroich C. and Polhill G. (2023) Editorial: Agent-based modelling as a method for prediction in complex social systems. International Journal of Social Research Methodology, 26/2, 133-142. https://doi.org/10.1080/13645579.2023.2152007

Elsenbroich C. and Badham J. (2023). Negotiating a Future that is not like the Past. International Journal of Social Research Methodology, 26(2), pp. 207-213. https://doi.org/10.1080/13645579.2022.2137935

McNabb D. (2023a, September 20). El Paradigma de la complejidad (1/3) [Video]. YouTube. https://www.youtube.com/watch?app=desktop&v=Uly1n6tOOlA&ab_channel=DarinMcNabb

McNabb D. (2023b, September 20). El Paradigma de la complejidad (2/3) [Video]. YouTube. https://www.youtube.com/watch?v=PT2m9lkGhvM&ab_channel=DarinMcNabb

McNabb D. (2023c, September 20). El Paradigma de la complejidad (3/3) [Video]. YouTube. https://www.youtube.com/watch?v=25f7l6jzV5U&ab_channel=DarinMcNabb

Troitzsch, K. G. (2009). Not all explanations predict satisfactorily, and not all good predictions explain. Journal of Artificial Societies and Social Simulation, 12(1), 10. https://www.jasss.org/12/1/10.html

Terán, O. (2023) Agent-based Modelling as a Method for Prediction for Complex Social Systems - a review of the special issue. Review of Artificial Societies and Social Simulation, 28 Sep 2023. https://rofasss.org/2023/09/28/review-ABM-for-prediction

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)