What’s Missing in Pandemic Models

In the COVID-19 pandemic, numerous models are being used to predict the future. But as helpful as they are, they cannot make sense of themselves. They rely on epidemiologists and other modelers to interpret them. Trouble is, making predictions in a pandemic is also a philosophical exercise. We need to think about hypothetical worlds, causation, evidence, and the relationship between models and reality.^1,2

The value of philosophy in this crisis is that although the pandemic is unique, many of the challenges of prediction, evidence, and modeling are general problems. Philosophers like myself are trained to see the most general contours of problems—the view from the clouds. They can help interpret scientific results and claims and offer clarity in times of uncertainty, bringing their insights down to Earth. When it comes to predicting in an outbreak, building a model is only half the battle. The other half is making sense of what it shows, what it leaves out, and what else we need to know to predict the future of COVID-19.

Prediction is about forecasting the future, or, when comparing scenarios, projecting several hypothetical futures. Because epidemiology informs public health directives, predicting is central to the field. Epidemiologists compare hypothetical worlds to help governments decide whether to implement lockdowns and social distancing measures—and when to lift them. To make this comparison, they use models to predict the evolution of the outbreak under various simulated scenarios. However, some of these simulated worlds may turn out to misrepresent the real world, and then our prediction might be off.

In his book Philosophy of Epidemiology, Alex Broadbent, a philosopher at the University of Johannesburg, argues that good epidemiological prediction requires asking, “What could possibly go wrong?” He elaborated in an interview with Nautilus, “To predict well is to be able to explain why what you predict will happen rather than the most likely hypothetical alternatives. You consider the way the world would have to be for your prediction to be true, then consider worlds in which the prediction is false.” By ruling out hypothetical worlds in which they are wrong, epidemiologists can increase their confidence that they are right. For instance, by using antibody tests to estimate previous infections in the population, public health authorities could rule out the hypothetical possibility (modeled by a team at Oxford) that the coronavirus has circulated much more widely than we think.³

One reason the dynamics of an outbreak are often more complicated than a traditional model can predict is that they result from human behavior and not just biology.

Broadbent is concerned that governments across Africa are not thinking carefully enough about what could possibly go wrong, having for the most part implemented coronavirus policies in line with the rest of the world. He believes a one-size-fits-all approach to the pandemic could prove fatal.⁴ The same interventions that might have worked elsewhere could have very different effects in the African context. For instance, the economic impacts of social distancing policies on all-cause mortality might be worse because so many people on the continent suffer increased food insecurity and malnutrition in an economic downturn.⁵ Epidemic models only represent the spread of the infection. They leave out important elements of the social world.

Another limitation of epidemic models is that they model the effect of behaviors on the spread of infection, but not the effect of a public health policy on behaviors. The latter requires understanding how a policy works. Nancy Cartwright, a philosopher at Durham University and the University of California, San Diego, suggests that “the road from ‘It works somewhere’ to ‘It will work for us’ is often long and tortuous.”⁶ The kinds of causal principles that make policies effective, she says, “are both local and fragile.” Principles can break in transit from one place to the other. Take the principle, “Stay-at-home policies reduce the number of social interactions.” This might be true in Wuhan, China, but might not be true in a South African township in which the policies are infeasible or in which homes are crowded. Simple extrapolation from one context to another is risky. A pandemic is global, but prediction should be local.

Predictions require assumptions that in turn require evidence. Cartwright and Jeremy Hardie, an economist and research associate at the Center for Philosophy of Natural and Social Science at the London School of Economics, represent evidence-based policy predictions using a pyramid, where each assumption is a building block.⁷ If evidence for any assumption is missing, the pyramid might topple. I have represented evidence-based medicine predictions using a chain of inferences, where each link in the chain is made of an alloy containing assumptions.⁸ If any assumption comes apart, the chain might break.

An assumption can involve, for example, the various factors supporting an intervention. Cartwright writes that “policy variables are rarely sufficient to produce a contribution [to some outcome]; they need an appropriate support team if they are to act at all.” A policy is only one slice of a complete causal pie.⁹ Take age, an important support factor in causal principles of social distancing. If social distancing prevents deaths primarily by preventing infections among older individuals, wherever there are fewer older individuals there may be fewer deaths to prevent—and social distancing will be less effective. This matters because South Africa and other African countries have younger populations than do Italy or China.¹⁰

The lesson that assumptions need evidence can sound obvious, but it is especially important to bear in mind when modeling. Most epidemic modeling makes assumptions about the reproductive number, the size of the susceptible population, and the infection-fatality ratio, among other parameters. The evidence for these assumptions comes from data that, in a pandemic, is often rough, especially in early days. It has been argued that nonrepresentative diagnostic testing early in the COVID-19 pandemic led to unreliable estimates of important inputs in our epidemic modeling.¹¹

Epidemic models also don’t model all the influences of the pathogen and of our policy interventions on health and survival. For example, what matters most when comparing deaths among hypothetical worlds is how different the death toll is overall, not just the difference in deaths due to the direct physiological effects of a virus. The new coronavirus can overwhelm health systems and consume health resources needed to save non-COVID-19 patients if left unchecked. On the other hand, our policies have independent effects on financial welfare and access to regular healthcare that might in turn influence survival.

A surprising difficulty with predicting in a pandemic is that the same pathogen can behave differently in different settings. Infection fatality ratios and outbreak dynamics are not intrinsic properties of a pathogen; these things emerge from the three-way interaction among pathogen, population, and place. Understanding more about each point in this triangle can help in predicting the local trajectory of an outbreak.

In April, an influential data-driven model, developed by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington, which uses a curve-fitting approach, came under criticism for its volatile projections and questionable assumption that the trajectory of COVID-19 deaths in American states can be extrapolated from curves in other countries.^12,13 In a curve-fitting approach, the infection curve representing a local outbreak is extrapolated from data collected locally along with data regarding the trajectory of the outbreak elsewhere. The curve is drawn to fit the data. However, the true trajectory of the local outbreak, including the number of infections and deaths, depends upon characteristics of the local population as well as policies and behaviors adopted locally, not just upon the virus.

Predictions require assumptions that in turn require evidence.

Many of the other epidemic models in the coronavirus pandemic are SIR-type models, a more traditional modelling approach for infectious-disease epidemiology. SIR-type models represent the dynamics of an outbreak, the transition of individuals in the population from a state of being susceptible to infection (S) to one of being infectious to others (I) and, finally, recovered from infection (R). These models simulate the real world. In contrast to the data-driven approach, SIR models are more theory-driven. The theory that underwrites them includes the mathematical theory of outbreaks developed in the 1920s and 1930s, and the qualitative germ theory pioneered in the 1800s. Epidemiologic theories impart SIR-type models with the know-how to make good predictions in different contexts.

For instance, they represent the transmission of the virus as a factor of patterns of social contact as well as viral transmissibility, which depend on local behaviors and local infection control measures, respectively. The drawback of these more theoretical models is that without good data to support their assumptions they might misrepresent reality and make unreliable projections for the future.

One reason why the dynamics of an outbreak are often more complicated than a traditional model can predict, or an infectious-disease epidemiology theory can explain, is that the dynamics of an outbreak result from human behavior and not just human biology. Yet more sophisticated disease-behavior models can represent the behavioral dynamics of an outbreak by modeling the spread of opinions or the choices individuals make.^14,15 Individual behaviors are influenced by the trajectory of the epidemic, which is in turn influenced by individual behaviors.

“There are important feedback loops that are readily represented by disease-behavior models,” Bert Baumgartner, a philosopher who has helped develop some of these models, explains. “As a very simple example, people may start to socially distance as disease spreads, then as disease consequently declines people may stop social distancing, which leads to the disease increasing again.” These looping effects of disease-behavior models are yet another challenge to predicting.

It is a highly complex and daunting challenge we face. That’s nothing unusual for doctors and public health experts, who are used to grappling with uncertainty. I remember what that uncertainty felt like when I was training in medicine. It can be discomforting, especially when confronted with a deadly disease. However, uncertainty need not be paralyzing. By spotting the gaps in our models and understanding, we can often narrow those gaps or at least navigate around them. Doing so requires clarifying and questioning our ideas and assumptions. In other words, we must think like a philosopher.

Jonathan Fuller is an assistant professor in the Department of History and Philosophy of Science at the University of Pittsburgh. He draws on his dual training in philosophy and in medicine to answer fundamental questions about the nature of contemporary disease, evidence, and reasoning in healthcare, and theory and methods in epidemiology and medical science.

References

1. Walker, P., et al. The global impact of COVID-19 and strategies for mitigation and suppression. Imperial College London (2020).

2. Flaxman, S., et al. Estimating the number of infections and the impact of non-pharmaceutical interventions on COVID-19 in 11 European countries. Imperial College London (2020).

3. Lourenco, J., et al. Fundamental principles of epidemic spread highlight the immediate need for large-scale serological surveys to assess the stage of the SARS-CoV-2 epidemic. medRxiv:10.1101/2020.03.24.20042291 (2020).

4. Broadbent, A., & Smart, B. Why a one-size-fits-all approach to COVID-19 could have lethal consequences. TheConversation.com (2020).

5. United Nations. Global recession increases malnutrition for the most vulnerable people in developing countries. United Nations Standing Committee on Nutrition (2009).

6. Cartwright, N. Will this policy work for you? Predicting effectiveness better: How philosophy helps. Philosophy of Science 79, 973-989 (2012).

7. Cartwright, N. & Hardie, J. Evidence-Based Policy: A Practical Guide to Doing it Better Oxford University Press, New York, New York (2012).

8. Fuller, J., & Flores, L. The Risk GP Model: The standard model of prediction in medicine. Studies in History and Philosophy of Biological and Biomedical Sciences 54, 49-61 (2015).

9. Rothman, K., & Greenland, S. Causation and causal inference in epidemiology. American Journal Public Health 95, S144-S50 (2005).

10. Dowd, J. et al. Demographic science aids in understanding the spread and fatality rates of COVID-19. Proceedings of the National Academy of Sciences 117, 9696-9698 (2020).

11. Ioannidis, J. Coronavirus disease 2019: The harms of exaggerated information and non‐evidence‐based measures. European Journal of Clinical Investigation 50, e13222 (2020).

12. COVID-19 Projections. Healthdata.org. https://covid19.healthdata.org/united-states-of-america.

13. Jewell, N., et al. Caution warranted: Using the Institute for Health metrics and evaluation model for predicting the course of the COVID-19 pandemic. Annals of Internal Medicine (2020).

14. Nardin, L., et al. Planning horizon affects prophylactic decision-making and epidemic dynamics. PeerJ 4:e2678 (2016).

15. Tyson, R., et al. The timing and nature of behavioural responses affect the course of an epidemic. Bulletin of Mathematical Biology 82, 14 (2020).

Lead image: yucelyilmaz / Shutterstock

Get the Nautilus newsletter

Cutting-edge science, unraveled by the very brightest living thinkers.

Newsletter Signup – In Page Mobile

Email: *

Captcha

If you are human, leave this field blank.

Sign up for free