Predicting facility-based delivery in Zanzibar: The vulnerability of machine learning algorithms to adversarial attacks

BACKGROUND: Community health worker (CHW)-led maternal health programs have contributed to increased facility-based deliveries and decreased maternal mortality in sub-Saharan Africa. The recent adoption of mobile devices in these programs provides an opportunity for real-time implementation of machine learning predictive models to identify women most at risk for home-based delivery. However, it is possible that falsified data could be entered into the model to get a specific prediction result – known as an “adversarial attack”. The goal of this paper is to evaluate the algorithm's vulnerability to adversarial attacks. METHODS: The dataset used in this research is from the Uzazi Salama (“Safer Deliveries”) program, which operated between 2016 and 2019 in Zanzibar. We used LASSO regularized logistic regression to develop the prediction model. We used “One-At-a-Time (OAT)” adversarial attacks across four different types of input variables: binary – access to electricity at home, categorical – previous delivery location, ordinal – educational level, and continuous – gestational age. We evaluated the percent of predicted classifications that change due to these adversarial attacks. RESULTS: Manipulating input variables affected prediction results. The variable with the greatest vulnerability was previous delivery location, with 55.65% of predicted classifications changing when applying adversarial attacks from previously delivered at a facility to previously delivered at home, and 37.63% of predicted classifications changing when applying adversarial attacks from previously delivered at home to previously delivered at a facility. CONCLUSION: This paper investigates the vulnerability of an algorithm to predict facility-based delivery when facing adversarial attacks. By understanding the effect of adversarial attacks, programs can implement data monitoring strategies to assess for and deter these manipulations. Ensuring fidelity in algorithm deployment secures that CHWs target those women who are actually at high risk of delivering at home.