Category Archives: review

Yes, but what did they actually do? Review of: Jill Lepore (2020) “If Then: How One Data Company Invented the Future”

By Nick Gotts

ngotts@gn.apc.org

Jill Lepore (2020) If Then: How One Data Company Invented the Future. John Murray. ISBN: 978-1-529-38617-2 (2021 pbk edition). [Link to book]

This is a most frustrating book. The company referred to in the subtitle is the Simulmatics Corporation, which collected and analysed data on public attitudes for politicians, retailers and the US Department of Defence between 1959 and 1970. Lepore says it carried out “simulation”, but is never very clear about what “simulation” meant to the founders of Simulmatics, what algorithms were involved, or how these algorithms used data. The history of Simulmatics is narrated along with that of US politics and the Vietnam War during its period of operation; the company worked for John Kennedy’s presidential campaign in 1960, although the campaign was shy about admitting this. There is much of interest in this historical context, but the book is marred by the apparent limitations of Lepore’s technical knowledge, her prejudices against the social and behavioural sciences (and in particular the use of computers within them), and irritating “tics” such as the frequent repetition of “If/Then”. There are copious notes, and an index, but no bibliography.

Lepore insists that human behaviour is not predictable, whereas both everyday observation and the academic study of human sciences and history show that on both individual and collective levels it is partially predictable – if it were not, social life would be impossible – and partially unpredictable; she also claims that there is a general repudiation of the importance of history among social and behavioural scientists and in “Silicon Valley”, and seems unaware that many historians and other humanities researchers use mathematics and even computers in their work.

Information about Simulmatics’ uses of computers is in fact available from contemporary documents which its researchers published. In the case of Kennedy’s presidential campaign (de Sola Pool and Abelson 1961, de Sola Pool 1963), the “simulation” involved was the construction of synthetic populations in order to amalgamate polling data from past (1952, 1954, 1956, 1958) American election campaigns. Americans were divided into 480 demographically defined “voter types” (e.g. “Eastern, metropolitan, lower-income, white, Catholic, female Democrats”), and the favourable/unfavourable/neither polling responses of members of these types to 52 specific “issues” (examples given include civil rights, anti-Communism, anti-Catholicism, foreign aid) were tabulated. Attempts were then made to “simulate” 32 of the USA’s 50 states by calculating the proportions of the 480 types in those states and assuming the frequency of responses within a voter type would be the same across states. This produced a ranking of how well Kennedy could be expected to do across these states, which matched the final results quite well. On top of this work an attempt was made to assess the impact of Kennedy’s Catholicism if it became an important issue in the election, but this required additional assumptions on how members of nine groups cross-classified by political and religious allegiance would respond. It is not clear that Kennedy’s campaign actually made any use of Simulmatics’ work, and there is no sense in which political dynamics were simulated. By contrast, in later Simulmatics work not dealt with by Lepore, on local referendum campaigns about water fluoridation (Abelson and Bernstein 1963), an approach very similar to current work in agent-based modelling was adopted. Agents based on the anonymised survey responses of individuals both responded to external messaging, and interacted with each other, to produce a dynamically simulated referendum campaign. It is unclear why Lepore does not cover this very interesting work. She does cover Simulmatics’ involvement in the Vietnam War, where their staff interviewed Vietnamese civilians and supposed “defectors” from the National Liberation Front of South Vietnam (“Viet Cong”) – who may in fact simply have gone back to their insurgent activity afterwards; but this work does not appear to have used computers for anything more than data storage.

In its work on American national elections (which continued through 1964) Simulmatics appears to have wildly over-promised given the data that it would have had available, subsequently under-performed, and failed as a company as a result; from this, indeed, today’s social simulators might take warning. Its leaders started out as “liberals” in American terms, but appear to have retained the colonialist mentality generally accompanying this self-identification, and fell into and contributed to the delusions of American involvement in the Vietnam War – although it is doubtful whether the history of this involvement would have been significantly different if the company had never existed. The fact that Simulmatics was largely forgotten, as Lepore recounts, hints that it was not, in fact, particularly influential, although interesting as the venue of early attempts at data analytics of the kind which may indeed now threaten what there is of democracy under capitalism (by enabling the “microtargeting” of specific lies to specific portions of the electorate), and at agent-based simulation of political dynamics. From a personal point of view, I am grateful to Lepore for drawing my attention to contemporary papers which contain far more useful information than her book about the early use of computers in the social sciences.

References

Abelson, R.P. and Bernstein, A. (1963) A Computer Simulation Model of Community Referendum Controversies. The Public Opinion Quarterly Vol. 27, No. 1 (Spring, 1963), pp. 93-122. Stable URL http://www.jstor.com/stable/2747294.

de Sola Pool, I. (1963) AUTOMATION: New Tool For Decision Makers. Challenge Vol. 11, No. 6 (MARCH 1963), pp. 26-27. Stable URL https://www.jstor.org/stable/40718664.

de Sola Pool, I. and Abelson, R.P. (1961) The Simulmatics Project. The Public Opinion Quarterly, Vol. 25, No. 2 (Summer, 1961), pp. 167-183. Stable URL https://www.jstor.org/stable/2746702.


Gotts, N. (2023) Yes, but what did they actually do? Review of: Jill Lepore (2020) "If Then: How One Data Company Invented the Future". Review of Artificial Societies and Social Simulation, 9 Mar 2023. https://rofasss.org/2023/03/09/ReviewofJillLepore


© The authors under the Creative Commons’ Attribution-NoDerivs (CC BY-ND) Licence (v4.0)

RofASSS to encourage reproduction reports and reviews of old papers&books

Reproducing simulation models is essential for verifying them and critiquing them. This involves a lot more work than one would think (Axtell & al. 1996) and can reveal surprising flaws, even in the simplest of models (e.g. Edmonds & Hales 2003). Such reproduction is especially vital if the model outcomes are likely to affect people’s lives (Chattoe-Brown & al. 2021).

Whilst substantial pieces of work – where there is extensive analysis or extension – can be submitted to JASSS/CMOT, some such reports might be much simpler and not justify a full journal paper. Thus RofASSS has decided to encourage researchers to submit reports of reproductions here – however simple or complicated.

Similarly, JASSS, CMOT etc. do publish book reviews, but these tend to be of recent books. Although new books are of obvious interest to those at the cutting edge of research, it often happens that important papers & books are forgotten or overlooked. At RofASSS we would like to encourage reviews of any relevant book or paper, however old.

References

Axtell, R., Axelrod, R., Epstein, J. M., & Cohen, M. D. (1996). Aligning simulation models: A case study and results. Computational & Mathematical Organization Theory, 1, 123-141. DOI: 10.1007/BF01299065

Edmonds, B., & Hales, D. (2003). Replication, replication and replication: Some hard lessons from model alignment. Journal of Artificial Societies and Social Simulation, 6(4), 11. https://jasss.soc.surrey.ac.uk/6/4/11.html

Chattoe-Brown, E. Gilbert, N., Robertson, D. A. & Watts, C. (2021) Reproduction as a Means of Evaluating Policy Models: A Case Study of a COVID-19 Simulation. medRxiv 2021.01.29.21250743; DOI: 10.1101/2021.01.29.21250743

A Forgotten Contribution: Jean-Paul Grémy’s Empirically Informed Simulation of Emerging Attitude/Career Choice Congruence (1974)

By Edmund Chattoe-Brown

Since this is new venture, we need to establish conventions. Since JASSS has been running since 1998 (twenty years!) it is reasonable to argue that something un-cited in JASSS throughout that period has effectively been forgotten by the ABM community. This contribution by Grémy is actually a single chapter in a book otherwise by Boudon (a bibliographical oddity that may have contributed to its neglect. Grémy also appears to have published mostly in French, which may also have had an effect. An English summary of his contribution to simulation might be another useful item for RofASSS.) Boudon gets 6 hits on the JASSS search engine (as of 31.05.18), none of which mention simulation and Gremy gets no hits (as does Grémy: unfortunately it is hard to tell how online search engines “cope with” accents and thus whether this is a “real” result).

Since this book is still readily available as a mass-market paperback, I will not reprise the argument of the simulation here (and its limitations relative to existing ABM methodology could be a future RofASSS contribution). Nonetheless, even approximately empirical modelling in the mid-seventies is worthy of note and the article is early to say other important things (for example about simulation being able to avoid “technical assumptions” – made for solubility rather than realism).

The point of this contribution is to draw attention to an argument that I have only heard twice (and only found once in print) namely that we should look at the form of real data as an initial justification for using ABM at all (please correct me if there are earlier or better examples). Grémy (1974, p. 210) makes the point that initial incongruities between the attitudes that people hold (altruistic versus selfish) and their career choices (counsellor versus corporate raider) can be resolved in either direction as time passes (he knows this because Boudon analysed some data collected by Rosenberg at two points from US university students) as well as remaining unresolved and, as such, cannot readily be explained by some sort of “statistical trend” (that people become more selfish as they get older or more altruistic as they become more educated). He thus hypothesises (reasonably it seems to me) that the data requires a model of some sort of dynamic interaction process that Grémy then simulates, paying some attention to their survey results both in constraining the model and analysing its behaviour.

This seems to me an important methodological practice to rescue from neglect. (It is widely recognised anecdotally that people tend to use the research methods they know and like rather than the ones that are suitable.) Elsewhere (Chattoe-Brown 2014), inspired by this argument, I have shown how even casually accessed attitude change data really looks nothing like the output of the (very popular) Zaller-Deffuant model of opinion change (very roughly, 228 hits in JASSS for Deffuant, 8 for Zaller and 9 for Zaller-Deffuant though hyphens sometimes produce unreliable results for online search engines too.) The attitude of the ABM community to data seems to be rather uncomfortable. Perhaps support in theory and neglect in practice would sum it up (Angus and Hassani-Mahmooei 2015, Table 5 in section 4.5). But if our models can’t even “pass first base” with existing real data (let alone be calibrated and validated) should we be too surprised if what seems plausible to us does not seem plausible to social scientists in substantive domains (and thus diminishes their interest in ABM as a “real method?”) Even if others in the ABM community disagree with my emphasis on data (and I know that they do) I think this is a matter that should be properly debated rather than just left floating about in coffee rooms (as such this is what we intend RofASSS to facilitate). As W. C. Fields is reputed to have said (though actually the phrase appears to have been common currency), we would wish to avoid ABM being just “Another good story ruined by an eyewitness”.

References

Angus, Simon D. and Hassani-Mahmooei, Behrooz (2015) ‘“Anarchy” Reigns: A Quantitative Analysis of Agent-Based Modelling Publication Practices in JASSS, 2001-2012’, Journal of Artificial Societies and Social Simulation, 18(4):16.

Chattoe-Brown, Edmund (2014) ‘Using Agent Based Modelling to Integrate Data on Attitude Change’, Sociological Research Online, 19(1):16.

Gremy, Jean-Paul (1974) ‘Simulation Techniques’, in Boudon, Raymond, The Logic of Sociological Explanation (Harmondsworth: Penguin), chapter 11:209-227.


Chattoe-Brown, E. (2018) A Forgotten Contribution: Jean-Paul Grémy’s Empirically Informed Simulation of Emerging Attitude/Career Choice Congruence (1974). Review of Artificial Societies and Social Simulation, 1st June 2018. https://rofasss.org/2018/06/01/ecb/