The predictive validity of the Living Goods selection tools for community health workers in Kenya: cohort study

BACKGROUND: Ensuring that selection processes for Community Health Workers (CHWs) are effective is important due to the scale and scope of modern CHW programmes. However they are relatively understudied. While community involvement in selection should never be eliminated entirely, there are other complementary methods that could be used to help identify those most likely to be high-performing CHWs. This study evaluated the predictive validity of three written tests and two individual sections of a one-to-one interview used for selection into CHW posts in eight areas of Kenya. METHODS: A cohort study of CHWs working for Living Goods in eight local areas of Kenya was undertaken. Data on the selection scores, post-training assessment scores and subsequent on-the-job performance (number of household and pregnancy registrations, number of child assessments, proportion of on-time follow-ups and value of goods sold) were obtained for 547 CHWs. Kendall’s tau-b correlations between each selection score and performance outcome were calculated. RESULTS: None of the correlations between selection scores and outcomes reached the 0.3 threshold of an “adequate” predictor of performance. Correlations were higher for the written components of the selection process compared to the interview components, with some small negative correlations found for the latter. CONCLUSIONS: If the measures of performance included in this study are considered critical, then further work to develop the CHW selection tools is required. This could include modifying the content of both tools or increasing the length of the written tests to make them more reliable, for if a test is not reliable then it cannot be valid. Other important outcomes not included in this study are retention in post and quality of care. Other CHW programme providers should consider evaluating their own selection tools in partnership with research teams.