To call or not to call: exploring the validity of telephone interviews to derive maternal self-reports of experiences with facility childbirth care in northern Nigeria

BACKGROUND: To institutionalise respectful maternity care, frequent data on the experience of childbirth care is needed by health facility staff and managers. Telephone interviews have been proposed as a low-cost alternative to derive timely and actionable maternal self-reports of experience of care. However, evidence on the validity of telephone interviews for this purpose is limited. METHODS: Eight indicators of positive maternity care experience and 18 indicators of negative maternity care experience were investigated. We compared the responses from exit interviews with women about their childbirth care experience (reference standard) to follow-up telephone interviews with the same women 14 months after childbirth. We calculated individual-level validity metrics including, agreement, sensitivity, specificity, area under the receiver operating characteristic curve (AUC). We compared the characteristics of women included in the telephone follow-up interviews to those from the exit interviews. RESULTS: Demographic characteristics were similar between the original exit interview group (n=388) and those subsequently reached for telephone interview (n=294). Seven of the eight positive maternity care experience indicators had reported prevalence higher than 50% at both exit and telephone interviews. For these indicators, agreement between the exit and the telephone interviews ranged between 50% and 92%; seven positive indicators met the criteria for validation analysis, but all had an AUC below 0.6. Reported prevalence for 15 of the 18 negative maternity care experience indicators was lower than 5% at exit and telephone interviews. For these 15 indicators, agreement between exit and telephone interview was high at over 80%. Just three negative indicators met the criteria for validation analysis, and all had an AUC below 0.6. CONCLUSIONS: The telephone interviews conducted 14 months after childbirth did not yield results that were consistent with exit interviews conducted at the time of facility discharge. Women’s reports of experience of childbirth care may be influenced by the location of reporting or changes in the recall of experiences of care over time.