Category Archives: Questions & Answers

Area for people to pose questions and get answers. Please reply using a comment or (if too shy) email the author directly.

Content, Questions & Answers

Query: How could we make Data Mining tools more useful for Agent-Based modelling?

Leave a comment

By Robin Faber

I am currently doing my master thesis in Computer Science at TU Delft on Data Mining (DM) in Agent-Based Simulation. The goal of this thesis is to provide model designers and analysts with DM tools to make the evaluation of models easier.

The main idea is to create a tool in Python that connects with NetLogo to run models, design experiments and obtain and present the output with visualisations. Because Python has many data analytic libraries, it provides tools that NetLogo lacks in terms of data analytics for the output of ABMs. From my understanding, there are some tools in NetLogo such as BehaviorSpace to run experiments, but this is quite basic and produces a text file which still has to be analysed elsewhere. What I would like to do is develop a library in Python that streamlines this whole process of “run model → get output → analyse output”, with a focus on the usability and ease of use to also make it available for people that are not experienced programmers. However, because my background is in Computer Science, I obviously lack some knowledge of what is needed in order for the tool to be useful and usable for an actual model designer.

The three main questions I would like to ask the RofASSS readers are these:

  • Which requirements would you define to make the tool easy to use for non-programmers? (e.g. documentation, GUI, lines of code, data structures)
  • What type of information is important to obtain from a simulation? (e.g variables, locations, agent counts)
  • How should the information obtained from the model output/experiments be presented? (e.g, types of graphs/tables/visualisation)

If you have any questions, comments or would like to schedule a call to discuss this topic, please contact me using the comments facility at the bottom of this post (for comments) or emailing me at: r.j.faber@student.tudelft.nl.

Faber, R.J. (2021) Query: How could we make Data Mining tools more useful for Agent-Based Modelling. Review of Artificial Societies and Social Simulation, 5th February 2021. https://rofasss.org/2021/02/3/dm4abm/

 

Content, Questions & Answers

What are the best journals for publishing re-validations of existing models?

Leave a comment

By Annamaria Berea

A few years ago I worked on an ABM that I eventually published in a book. Recently, I have conducted new experiments with the same model, re-analyzed the data and had a different dataset that I used for validation of the model. Where can I publish this new work on an older model? I submitted it to a special issue of a journal, but was rejected as “the model was not original”. While the model is not, the new data analysis and validation are and I think it is even more important within the current discussions about replication crises in science.

Berea, A. (2019) What are the best journals or publishers for reports of re-validations of existing models? Review of Artificial Societies and Social Simulation, 31st October 2019. https://rofasss.org/2019/10/30/best-journal/

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

Content, Questions & Answers

Query: What is the earliest example of a social science simulation (that is nonetheless arguably an ABM) and shows real and simulated data in the same figure or table?

Leave a comment

By Edmund Chattoe-Brown

On one level this is a straightforward request. The earliest convincing example I have found is Hägerstrand (1965, p. 381) an article that seems to be undeservedly neglected because it is also the earliest example of a simulation I have been able to identify that demonstrates independent calibration and validation (Gilbert and Troitzsch 2005, p. 17).1

However, my attempts to find the earliest examples are motivated two more substantive issues (which may help to focus the search for earlier candidates). Firstly, what is the value of a canon (and giving due intellectual credit) for the success of ABM? The Schelling model is widely known and taught but it is not calibrated and validated. If a calibrated and validated model already existed in 1965, should it not be more widely cited? If we mostly cite a non-empirical model, might we give the impression that this is all that ABM can do? Also, failing to cite an article means that it cannot form the basis for debate. Is the Hägerstrand model in some sense “better” or “more important” than the Schelling model? This is a discussion we cannot have without awareness of the Hägerstrand model in the first place.

The second (and related) point regards the progress made by ABM and how those outside the community might judge it. Looking at ABM research now, the great majority of models appear to be non-empirical (Angus and Hassani-Mahmooei 2015, Table 5 in section 4.5). Without citations of articles like Hägerstrand (and even Clarkson and Meltzer), the non-expert reader of ABM might be led to conclude that it is too early (or too difficult) to produce such calibrated and validated models. But if this was done 50 years ago, and is not being much publicised, might we be using up our credibility as a “new” field still finding its feet?) If there are reasons for not doing, or not wanting to do, what Hägerstrand managed, let us be obliged to be clear what they are and not simply hide behind widespread neglect of such examples2.)

Notes

  1. I have excluded an even earlier example of considerable interest (Clarkson and Meltzer 1960 which also includes an attempt at calibration and validation but has never been cited in JASSS) for two reasons. Firstly, it deals with the modelling of a single agent and therefore involves no interaction. Secondly, it appears that the validation may effectively be using the “same” data as the calibration in that protocols elicited from an investment officer regarding portfolio selection are then tested against choices made by that same investment officer.
  2. And, of course, this is a vicious circle because in our increasingly pressurised academic world, people only tend to read and cite what is already cited.

References

Angus, Simon D. and Hassani-Mahmooei, Behrooz (2015) ‘“Anarchy” Reigns: A Quantitative Analysis of Agent-Based Modelling Publication Practices in JASSS, 2001-2012’, Journal of Artificial Societies and Social Simulation, 18(4), October, article 16, .

Clarkson, Geoffrey P. and Meltzer, Allan H. (1960) ‘Portfolio Selection: A Heuristic Approach, The Journal of Finance, 15(4), December, pp. 465-480.

Gilbert, Nigel and Troitzsch, Klaus G. (2005) Simulation for the Social Scientist, 2nd edition (Buckingham: Open University Press).

Hägerstrand, Torsten (1965) ‘A Monte Carlo Approach to Diffusion’, Archives Européennes de Sociologie, 6(1), May, Special Issue on Simulation in Sociology, pp. 43-67.

Chattoe-Brown, E. (2018) What is the earliest example of a social science simulation (that is nonetheless arguably an ABM) and shows real and simulated data in the same figure or table? Review of Artificial Societies and Social Simulation, 11th June 2018. https://rofasss.org/2018/06/11/ecb/

© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)