Phone-based monitoring to evaluate health policy and program implementation in Kenya

Monitoring and evaluating policies and programs in low- and middle-income countries are often difficult because of the lack of routine data. High mobile phone ownership in these countries presents an opportunity for efficient data collection through telephone interviews. This study examined the feasibility of collecting data on medicines through telephone interviews in Kenya. Data on the availability and prices of medicines at 137 health facilities and 639 patients were collected in September 2016 via in-person interviews. Between December 2016 and December 2017, monthly telephone interviews were conducted with health facilities and patients. An unannounced in-person interview was conducted with respondents to validate the telephone interview within 24 h. A bottom-up itemization costing approach was used to estimate the costs of telephone and in-person data collection. In-depth interviews were conducted with data collectors and respondents to explore their perceptions on both modes of data collection. The level of agreement between data on medicines availability collected through phone and in-person interviews was strong at the health facility level [kappa = 0.90; confidence interval (CI) 0.88–0.92] and moderate at the household level (kappa = 0.50, CI 0.39–0.60). Price data from telephone and in-person interviews showed strong intra-class correlation at health facilities [intra-class correlation coefficient (ICC) = 0.96] and moderate intra-class correlation at households (ICC = 0.47). The cost per phone interview at health facilities and households were $19.73 and $16.86, respectively, compared to $186.20 for a baseline in-person interview. Participants considered telephone interviews to be more convenient. In countries with high cell phone penetration, telephone data collection should be considered in monitoring and evaluating public health programs especially at health facilities. Additional strategies may be needed to optimize this mode of data collection at the household level. Variations in cell phone ownership, telecommunication network and data collection costs across different settings may limit the generalizability of the findings from this study.