By Kavin Narasimhan, Silvia Leoni, Katharina Luckner, Dino Carpentras, and Natalie Davis*
*All authors contributed equally – author order determined by a pseudo-random number generator and does not reflect their respective contributions.
Introduction
Since its inception in 2010, ESSA@work has been a mainstay at the annual Social Simulation Conference (SSC). It continues as a forum where beginners in individual- and agent-based modelling (hereon, ABM) present a work-in-progress model, along with specific problems and questions, to a community of practitioners to get feedback, suggestions, and tips for specific aspects of their modelling projects. During the session, participants present their model to an audience and two experts, the latter of whom are chosen for their constructive style of feedback and broad expertise. Participants are not required to answer questions or defend their work, as might be the case in a more traditional setting. Instead, experts enter into a dialogue with each other with the explicit goal of providing constructive feedback towards the progress of the project. After the expert discussion, the audience can also add constructive ideas and questions.
Each ESSA@work session is organised by a team of volunteers, who were often introduced to the format by being participants themselves. In the weeks prior to the SSC, this group drafts all necessary documents to elicit participation, selects participants, contacts experts, and distributes information via mailing lists and social media channels. During the sessions, they serve as chairs and provide outreach via social media. In between conferences and other events with ESSA@work sessions, organisers serve as points of contact for anyone who might want to organise a local ESSA@work session engage with the management of the broader European Social Simulation Association (ESSA), maintain information on the ESSA@work website (http://www.essa.eu.org/essawork/), and recruit the next generation of volunteers. Organisers typically stay on for a number of years, so that a continuity of knowledge on the processes is secured.
Over the years, a few themes that characterise ESSA@work have crystallised and indicate the importance of the track. In this contribution, we outline these themes: how ESSA@work provides a learning experience to participants and the audience, as well as the organisers; how it fosters interdisciplinarity; and how it builds upon a community of practice. We conclude with our wishes for its future.
Themes
Learning experience
The participants in ESSA@work tend to be early-career researchers, such as masters students, doctoral candidates, or post-doctoral researchers, but we have also had participants who are experienced academics, but new to ABM. For early-career researchers, participating holds additional benefits, as the SSC where they participate in ESSA@work may be their first (on-site) conference. For instance, this was the case for the SSC2022, which was held in a hybrid format after a long period of restrictions and uncertainty due to COVID-19. This deeply affected the career of young researchers: for some, most of their PhD has been spent online with no or little opportunity to participate in events such as annual conferences.
While the learning experience is focused on the participants and their contributions, it extends beyond them to include audience members and organisers as well, so that ESSA@work sessions present different learning channels. The first learning channel is focused on presentation and social skills. These general skills apply to any career path and are facilitated and supported by the friendly environment and specific format that ESSA@work implements. The practice of presenting unfinished work fosters collaboration, open conversations, and reflection, among peers and more senior academics alike, rather than an environment where participants must ‘defend’ their work from reviewers. Participants must adapt their presentation to a specific format, where they clearly address their doubts and issues. This requires them to put together a clear, concise presentation aligned with the non-standard focus of the track. We have one-page guidance, detailed guidance, and Frequently Asked Questions (FAQs) covering these aspects online (http://www.essa.eu.org/essawork/how-to-participate/ and http://www.essa.eu.org/essawork/faq/). The track then also facilitates and encourages the development of social skills by bringing together members of the ESSA community of all experience levels, allowing participants to develop their network of contacts and collaborations based on shared experiences and mentorship.
Secondly, there is the specific feedback from experts, including literature and data recommendations, and references to existing models or other contacts. This adds to or complements the feedback that participants (especially PhD students and postdocs) receive from their supervisors. Participants can find diverse, enriching suggestions with respect to the line of work that they were following, and new perspectives. For cases in which relationships with supervisors and mentors are proving to be difficult, or where supervisors are less familiar with ABM, this can be a crucial source of motivation and support for researchers who find themselves stuck in the process. This can also be a useful source of ideas for audience members with similar questions or challenges.
There is also the organiser’s experience. This usually starts with being involved as a participant in the track. As speakers, participants begin to familiarise themselves with the specific ESSA@work format, as well as with the steps, timing, and process that lead to the conference events. This is also a way to get in touch with former and current team members before officially joining the organisers’ team. After being introduced to ESSA@work as a participant or audience member, new members of the organising team receive training by current and/or former members in a process of knowledge transfer guided by prior experiences. This is put in place with the goal of sharing, improving from the past, and creating a community.
Once researchers have fully joined the team and start helping to prepare the next edition of ESSA@work, the learning opportunities are numerous. From building and strengthening their network of contacts across ESSA, to practising organisation and chairing (which would otherwise often come at a later career stage), reviewing submitted manuscripts, improving communication and coordination skills, and project and time management. Last but not least, organisers work in a team. This exercise of coordination is a fantastic occasion for learning-by-doing of how to adapt and organise heterogeneous skills, schedules, and expertise towards continuous improvement. On the one hand, this mimics co-authorship, and thus offers an opportunity to familiarise oneself with a frequent pattern in academic work; on the other hand, it emphasises and strengthens the feeling of community that characterises ESSA, and that is even stronger in the ESSA@work family.
Interdisciplinarity
Through our time as organisers, we have seen first-hand how diverse the ABM community is. The background of participants can include physics, ecology, computer science, economics, or psychology, just to name a few. This is a double-edged sword, as it both allows researchers to produce work connecting multiple disciplines, but can also result in work that is not accessible to the different audiences who may otherwise be interested in it.
For example, people from statistical physics may be very interested in solving the mean-field approximation of a model, while psychologists may be more interested in the qualitative interpretation of such a model. Similar problems also regard the use of technical terms. For example, terms like “experiment” are used by some to mean “computer simulation” and by others to mean “empirical experiment with real people.” Similarly, Edmund Chattoe-Brown found 5 different uses of the term “validation” (Chattoe-Brown, 2021). Therefore, while ABM can connect multiple different fields, research content can still be very hard to understand by multiple scientists. This can paradoxically result in more difficulty in reaching out or communicating results to some communities or fields (Carpentras, 2022).
ESSA@work can have a unique role in tackling this problem, as it allows people who have recently begun working with ABM to get an “inside view” of the ABM community. By presenting their work and research questions to experts in ABM, and receiving feedback from them, participants can have a smoother process to publishing their models, for example by avoiding common mistakes and pitfalls, and gain more insights on typical research questions, problems and jargon of ABM. This allows participants to get more acquainted with the ABM community and mindsets (as discussed in the next section), allowing for a better integration and long-term connection with the field.
Community building
When speaking of communities of practice, we ask how practitioners of a certain profession or discipline both shape and are shaped by their profession. ESSA@work has a role to play in both, but perhaps more heavily in the latter.
As a whole, ESSA seems to be actively shaping a community of practice in social simulation, more specifically ABM, through shared standards and protocols, regular exchanges, and collaborations across disciplines, all rallying around a specific method. For many members, this is a community separate to the one that they belong to on a day-to-day basis in their departments or organisations. There is active communal support in jointly shaping the rules that should govern the community and the method, as well as a continuous (and friendly) negotiation of who or what is included and excluded from the community and where overlaps with other communities might be (see recent discussion in the SIMSOC mailing list; https://www.jiscmail.ac.uk/cgi-bin/wa-jisc.exe?A2=ind2211&L=SIMSOC&O=D&P=19269). This is how the community shapes the practice – both actively and passively.
The other side of the coin is how the practice shapes the community. As discussed, ESSA@work sessions are often the place and time where new members are introduced to the community, and where their future outlook on the community and the method is significantly shaped. Through useful, tactful, and constructive feedback, new members are introduced to the core texts that at least partially constitute the collective imaginary of the community of practice, to the protocols that govern what constitutes good practice, and – perhaps most importantly – to the tone that the community uses in interacting with one another. ESSA@work therefore not only provides a forum for constructive feedback on work-in-progress, but also an experience which is useful to decide whether someone wants to be part of this community. With ‘alumni’ often coming back as organisers or panellists, and recommending the track to their peers and students, there is a sense that ESSA@work – and the attitude it embodies – is passed on through academic generations. It therefore becomes very much part of what we do, and how we do things, in the agent-based modelling community.
Future themes
As we look to the future for ESSA@work, we have considered both its continuing role in providing a multi-faceted learning experience and central point for the ESSA community, as well as how it can continue to contribute to the future of both ESSA and the field of agent-based modelling more broadly. Specifically, as agent-based modelling has become more accepted as a method for simulating and analysing complex systems, and therefore taken a more empirical turn, ESSA@work can have a unique role in fostering and maintaining the diversity of modelling purposes, which may otherwise become less valued in the rest of the scientific community.
Most participants have questions related to specific stages of their modelling journey. If you think of an ABM journey being roughly divided into the following stages: (1) conceptualisation and design, (2) development, (3) verification and calibration, (4) validation, and (5) simulations, uncertainty analysis and results, most ESSA@work participants are somewhere between steps 2 and 4 in their modelling journey. As step 1 presents several possibilities and needs longer for background work (like literature review, brainstorming, stakeholder consultation, etc.), we intentionally encourage participation in the forum from step 2 onwards, when the purpose, scope and objectives of models become clearer. This in turn enables specific modelling questions being put forth that can be usefully addressed within the time and space of an ESSA@work session. Over the years, we have received submissions from across disciplines and mostly focusing on issues in steps 2 to 4 of the modelling journey. More recently, we also started receiving submissions with questions about running simulation experiments, calibration and validation with empirical data, interpreting results, and conducting uncertainty analysis. We believe this speaks to ABM becoming more mainstream as a microsimulation approach during this period, enabled also by the availability and accessibility to powerful computing resources.
We find that when questions fall under modelling stages 2, 3 and 5, participants receive more direct answers as questions tend to be specific, which our practitioner community addresses based on their own work, or on wider references. On the other hand, questions about model validation (stage 4) could be quite broad and open-ended to attract a useful response in the time available. ‘How can I validate my model?’ – or the essence of this question worded differently – is a popular question in this category. A practical and straightforward answer to validate a model is to collect or use data on the modelled phenomenon, and use them as test data to check if the model replicates patterns of the test data. Often though, participants indicate that the test data do not exist or are difficult to obtain. This would then raise questions about the purpose of the model: specifically, whether it’s intended as a toy model to generate plausible explanations about an observed phenomenon (historically the realm of ABM), or as a specialised model to allow meaningful forecasts. Having the latter objective would mean that the model needs good quality data at every stage of model development, and lacking those data would raise concerns about the suitability of ABM in the first place to address the proposed research questions. Without validation, as robust as a model may be, it may not be trusted to generate valid predictions or forecasts.
On the other hand, where models are intended as ‘toy models’, lack of validation is less of a problem. These models are meant to inspire more informed research questions about observed phenomena, which can subsequently be explored through further targeted real-world experiments, data collection, modelling, or a combination thereof. These models also provide clear entry points to the discipline for someone just beginning to explore complex systems, ABM, or both – many of us can point to reading texts such as Growing Artificial Societies (Epstein and Axtell, 1996) as the first time we truly understood and connected with ABM. But somehow there appear to be fewer takers for developing toy models in recent years. This could be due to perceptions that toy models risk being dismissed as vague (or at least harder to publish), because practitioners are on tight timelines and thus experience a lack of time or room to experiment with toy models, or because of a need to deliver model-based predictions (forecasts or projections) to satisfy specific project requirements.
We fear that any such bias against toy models might incur a cost in the form of compromised quality of models, or discourage new entrants and sponsors for ABM. The former is likely to occur when modellers try to build an overly complicated or specific model based on minimal, poor, or fragmented data, and thus possibly relying on too many assumptions that lack sound evidence. The latter could happen when ABM is solely intended as a means to an end rather than as a means to experiment. Reflecting on our journey and thinking ahead, we believe ESSA@work could avoid these outcomes by providing an unbiased, supportive, and well-connected incubatory forum to encourage the development and housing of toy models, which have sound methodological and modelling rigour, despite being unsuitable for prediction due to the lack of validation using empirical data. We could then expect that a growing bank of model examples and modellers would pave the way for ABM practice to flourish, alongside guiding data confidentiality, data collection, sharing, and management practices that allow turning toy models into specialised models in methodical, reusable, and reproducible ways. The prominence of ESSA@work in the ESSA network could allow us to take on such a role in the future if more ABM practitioners (at all stages of their modelling career) volunteer to support with running the forum and its activities.
Conclusion
ESSA@work offers a valuable learning experience for participants, audience members, and organisers alike. It has become an integral part of the SSC annual conference and especially of the ABM community. While this is the result of past efforts and activities, our current work looks to the future and aims at continuity with the past but also renovation and further development.
We strive to improve and make our team and community grow. For this reason, we always welcome new organisers to contribute in this joint effort to grow both the spectrum and the reach of our activities. To guarantee continuity of this track and continue to improve it, we believe that diversity in participation could play a major role in innovation and better identifying early career researchers’ and other participants’ needs in the coming years.
The COVID era has confronted us, among others, with different professional and academic challenges. We all transferred our work from on-site to remote or hybrid, and likewise we adapted to new formats to guarantee that the ESSA community could continue to meet. While originally the result of needs and adaptation, online and hybrid formats have proved to be effective in ensuring a wide reach and increased accessibility. The SSC2022 and SocSimFesT past editions showed the possibility and success of a plurality of formats and ways to meet, discuss, and progress our research. These formats are now integrated in our working life and they represent a possibility for ESSA@work to get in touch with new cohorts of international modellers.
ESSA@work is a friendly space for in-depth discussion and learning, and as such, it extends beyond the boundaries of the annual conference or on-site events. We aim to continue offering online or hybrid events in the hope that they will make participation more accessible and provide additional feedback to anyone who needs it. In addition, we encourage the organisation of local ESSA@work sessions. In order to do so, the ambition and priority of ESSA@work is preserving its function as a community-builder and ensuring that participants are supported and able to self-organise according to the challenges and needs arising from their research.
References
Carpentras, D. (2020) Challenges and opportunities in expanding ABM to other fields: the example of psychology. Review of Artificial Societies and Social Simulation, 20th December 2021. https://rofasss.org/2021/12/20/challenges/
Chattoe-Brown, E. (2022) Today We Have Naming Of Parts: A Possible Way Out Of Some Terminological Problems With ABM. Review of Artificial Societies and Social Simulation, 11th January 2022. https://rofasss.org/2022/01/11/naming-of-parts/
Epstein, J. M., & Axtell, R. (1996). Growing artificial societies: social science from the bottom up. Brookings Institution Press.
Narasimhan, K., Leoni, S., Luckner, K., Carpentras, D. and Davis, N. (2022) ESSA@work: Reflections and looking ahead. Review of Artificial Societies and Social Simulation, 20 Feb 2023. https://rofasss.org/2022/02/20/essawork
© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)
Review of Artificial Societies and Social Simulation 