Cargando…

Assessing the accuracy of automatic speech recognition for psychotherapy

Accurate transcription of audio recordings in psychotherapy would improve therapy effectiveness, clinician training, and safety monitoring. Although automatic speech recognition software is commercially available, its accuracy in mental health settings has not been well described. It is unclear whic...

Descripción completa

Detalles Bibliográficos
Autores principales: Miner, Adam S., Haque, Albert, Fries, Jason A., Fleming, Scott L., Wilfley, Denise E., Terence Wilson, G., Milstein, Arnold, Jurafsky, Dan, Arnow, Bruce A., Stewart Agras, W., Fei-Fei, Li, Shah, Nigam H.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7270106/
https://www.ncbi.nlm.nih.gov/pubmed/32550644
http://dx.doi.org/10.1038/s41746-020-0285-8
Descripción
Sumario:Accurate transcription of audio recordings in psychotherapy would improve therapy effectiveness, clinician training, and safety monitoring. Although automatic speech recognition software is commercially available, its accuracy in mental health settings has not been well described. It is unclear which metrics and thresholds are appropriate for different clinical use cases, which may range from population descriptions to individual safety monitoring. Here we show that automatic speech recognition is feasible in psychotherapy, but further improvements in accuracy are needed before widespread use. Our HIPAA-compliant automatic speech recognition system demonstrated a transcription word error rate of 25%. For depression-related utterances, sensitivity was 80% and positive predictive value was 83%. For clinician-identified harm-related sentences, the word error rate was 34%. These results suggest that automatic speech recognition may support understanding of language patterns and subgroup variation in existing treatments but may not be ready for individual-level safety surveillance.