Cargando…

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND: Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in beh...

Descripción completa

Detalles Bibliográficos
Autores principales: Choy-Brown, Mimi, Williams, Nathaniel J., Ramirez, Nallely, Esp, Susan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10080877/
https://www.ncbi.nlm.nih.gov/pubmed/37024945
http://dx.doi.org/10.1186/s43058-023-00419-1
_version_ 1785021007124234240
author Choy-Brown, Mimi
Williams, Nathaniel J.
Ramirez, Nallely
Esp, Susan
author_facet Choy-Brown, Mimi
Williams, Nathaniel J.
Ramirez, Nallely
Esp, Susan
author_sort Choy-Brown, Mimi
collection PubMed
description BACKGROUND: Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers’ implementation of evidence-based psychosocial interventions—(1) audit and feedback and (2) active learning. METHODS: Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. RESULTS: In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ(2)=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ(2)=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. CONCLUSIONS: Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT04096274. Registered on 19 September 2019. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-023-00419-1.
format Online
Article
Text
id pubmed-10080877
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-100808772023-04-08 Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy Choy-Brown, Mimi Williams, Nathaniel J. Ramirez, Nallely Esp, Susan Implement Sci Commun Research BACKGROUND: Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers’ implementation of evidence-based psychosocial interventions—(1) audit and feedback and (2) active learning. METHODS: Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. RESULTS: In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ(2)=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ(2)=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. CONCLUSIONS: Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT04096274. Registered on 19 September 2019. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-023-00419-1. BioMed Central 2023-04-06 /pmc/articles/PMC10080877/ /pubmed/37024945 http://dx.doi.org/10.1186/s43058-023-00419-1 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
spellingShingle Research
Choy-Brown, Mimi
Williams, Nathaniel J.
Ramirez, Nallely
Esp, Susan
Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title_full Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title_fullStr Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title_full_unstemmed Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title_short Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
title_sort psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10080877/
https://www.ncbi.nlm.nih.gov/pubmed/37024945
http://dx.doi.org/10.1186/s43058-023-00419-1
work_keys_str_mv AT choybrownmimi psychometricevaluationofapragmaticmeasureofclinicalsupervisionasanimplementationstrategy
AT williamsnathanielj psychometricevaluationofapragmaticmeasureofclinicalsupervisionasanimplementationstrategy
AT ramireznallely psychometricevaluationofapragmaticmeasureofclinicalsupervisionasanimplementationstrategy
AT espsusan psychometricevaluationofapragmaticmeasureofclinicalsupervisionasanimplementationstrategy