Cargando…
Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations
During a psychotherapy session, the counselor typically adopts techniques which are codified along specific dimensions (e.g., ‘displays warmth and confidence’, or ‘attempts to set up collaboration’) to facilitate the evaluation of the session. Those constructs, traditionally scored by trained human...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8535177/ https://www.ncbi.nlm.nih.gov/pubmed/34679105 http://dx.doi.org/10.1371/journal.pone.0258639 |
_version_ | 1784587716181098496 |
---|---|
author | Flemotomos, Nikolaos Martinez, Victor R. Chen, Zhuohao Creed, Torrey A. Atkins, David C. Narayanan, Shrikanth |
author_facet | Flemotomos, Nikolaos Martinez, Victor R. Chen, Zhuohao Creed, Torrey A. Atkins, David C. Narayanan, Shrikanth |
author_sort | Flemotomos, Nikolaos |
collection | PubMed |
description | During a psychotherapy session, the counselor typically adopts techniques which are codified along specific dimensions (e.g., ‘displays warmth and confidence’, or ‘attempts to set up collaboration’) to facilitate the evaluation of the session. Those constructs, traditionally scored by trained human raters, reflect the complex nature of psychotherapy and highly depend on the context of the interaction. Recent advances in deep contextualized language models offer an avenue for accurate in-domain linguistic representations which can lead to robust recognition and scoring of such psychotherapy-relevant behavioral constructs, and support quality assurance and supervision. In this work, we propose a BERT-based model for automatic behavioral scoring of a specific type of psychotherapy, called Cognitive Behavioral Therapy (CBT), where prior work is limited to frequency-based language features and/or short text excerpts which do not capture the unique elements involved in a spontaneous long conversational interaction. The model focuses on the classification of therapy sessions with respect to the overall score achieved on the widely-used Cognitive Therapy Rating Scale (CTRS), but is trained in a multi-task manner in order to achieve higher interpretability. BERT-based representations are further augmented with available therapy metadata, providing relevant non-linguistic context and leading to consistent performance improvements. We train and evaluate our models on a set of 1,118 real-world therapy sessions, recorded and automatically transcribed. Our best model achieves an F(1) score equal to 72.61% on the binary classification task of low vs. high total CTRS. |
format | Online Article Text |
id | pubmed-8535177 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-85351772021-10-23 Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations Flemotomos, Nikolaos Martinez, Victor R. Chen, Zhuohao Creed, Torrey A. Atkins, David C. Narayanan, Shrikanth PLoS One Research Article During a psychotherapy session, the counselor typically adopts techniques which are codified along specific dimensions (e.g., ‘displays warmth and confidence’, or ‘attempts to set up collaboration’) to facilitate the evaluation of the session. Those constructs, traditionally scored by trained human raters, reflect the complex nature of psychotherapy and highly depend on the context of the interaction. Recent advances in deep contextualized language models offer an avenue for accurate in-domain linguistic representations which can lead to robust recognition and scoring of such psychotherapy-relevant behavioral constructs, and support quality assurance and supervision. In this work, we propose a BERT-based model for automatic behavioral scoring of a specific type of psychotherapy, called Cognitive Behavioral Therapy (CBT), where prior work is limited to frequency-based language features and/or short text excerpts which do not capture the unique elements involved in a spontaneous long conversational interaction. The model focuses on the classification of therapy sessions with respect to the overall score achieved on the widely-used Cognitive Therapy Rating Scale (CTRS), but is trained in a multi-task manner in order to achieve higher interpretability. BERT-based representations are further augmented with available therapy metadata, providing relevant non-linguistic context and leading to consistent performance improvements. We train and evaluate our models on a set of 1,118 real-world therapy sessions, recorded and automatically transcribed. Our best model achieves an F(1) score equal to 72.61% on the binary classification task of low vs. high total CTRS. Public Library of Science 2021-10-22 /pmc/articles/PMC8535177/ /pubmed/34679105 http://dx.doi.org/10.1371/journal.pone.0258639 Text en © 2021 Flemotomos et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Flemotomos, Nikolaos Martinez, Victor R. Chen, Zhuohao Creed, Torrey A. Atkins, David C. Narayanan, Shrikanth Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title | Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title_full | Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title_fullStr | Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title_full_unstemmed | Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title_short | Automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
title_sort | automated quality assessment of cognitive behavioral therapy sessions through highly contextualized language representations |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8535177/ https://www.ncbi.nlm.nih.gov/pubmed/34679105 http://dx.doi.org/10.1371/journal.pone.0258639 |
work_keys_str_mv | AT flemotomosnikolaos automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations AT martinezvictorr automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations AT chenzhuohao automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations AT creedtorreya automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations AT atkinsdavidc automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations AT narayananshrikanth automatedqualityassessmentofcognitivebehavioraltherapysessionsthroughhighlycontextualizedlanguagerepresentations |