Learning with insufficient data: a multi-armed bandit perspective on covid-19 interventions

In February 2020, as covid-19 infections spread to more than fifty countries, public health officials needed to recommend how the public could protect themselves, balancing safety and urgency. But there was very little data since this novel virus had only been identified three months prior. How could public health officials decide with insufficient data? The multi-armed bandit problem of computer science offers adaptive decision-making procedures that can achieve both safety and urgency. These adaptive methods balance learning information (exploring) with using information (exploiting), adjusting the balance toward learning when uncertainty is high (March 1991; Kaelbling et al. 1996). Related methods are already used in adaptive clinical trials for pharmaceuticals (Pallmann et al. 2018). But we still need to develop these methods for non-pharmaceutical interventions, as I will illustrate with a case study of public mask-wearing to reduce the spread of covid-19. Public health pronouncements impact future learning.