Cargando…

Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine

BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and vali...

Descripción completa

Detalles Bibliográficos
Autores principales: Ilic, Dragan, Nordin, Rusli Bin, Glasziou, Paul, Tilson, Julie K, Villanueva, Elmer
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2014
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4062508/
https://www.ncbi.nlm.nih.gov/pubmed/24909434
http://dx.doi.org/10.1186/1472-6920-14-114
_version_ 1782321664537657344
author Ilic, Dragan
Nordin, Rusli Bin
Glasziou, Paul
Tilson, Julie K
Villanueva, Elmer
author_facet Ilic, Dragan
Nordin, Rusli Bin
Glasziou, Paul
Tilson, Julie K
Villanueva, Elmer
author_sort Ilic, Dragan
collection PubMed
description BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis.
format Online
Article
Text
id pubmed-4062508
institution National Center for Biotechnology Information
language English
publishDate 2014
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-40625082014-06-19 Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine Ilic, Dragan Nordin, Rusli Bin Glasziou, Paul Tilson, Julie K Villanueva, Elmer BMC Med Educ Research Article BACKGROUND: While a variety of instruments have been developed to assess knowledge and skills in evidence based medicine (EBM), few assess all aspects of EBM - including knowledge, skills attitudes and behaviour - or have been psychometrically evaluated. The aim of this study was to develop and validate an instrument that evaluates medical trainees’ competency in EBM across knowledge, skills and attitude. METHODS: The ‘Assessing Competency in EBM’ (ACE) tool was developed by the authors, with content and face validity assessed by expert opinion. A cross-sectional sample of 342 medical trainees representing ‘novice’, ‘intermediate’ and ‘advanced’ EBM trainees were recruited to complete the ACE tool. Construct validity, item difficulty, internal reliability and item discrimination were analysed. RESULTS: We recruited 98 EBM-novice, 108 EBM-intermediate and 136 EBM-advanced participants. A statistically significant difference in the total ACE score was observed and corresponded to the level of training: on a 0-15-point test, the mean ACE scores were 8.6 for EBM-novice; 9.5 for EBM-intermediate; and 10.4 for EBM-advanced (p < 0.0001). Individual item discrimination was excellent (Item Discrimination Index ranging from 0.37 to 0.84), with internal reliability consistent across all but three items (Item Total Correlations were all positive ranging from 0.14 to 0.20). CONCLUSION: The 15-item ACE tool is a reliable and valid instrument to assess medical trainees’ competency in EBM. The ACE tool provides a novel assessment that measures user performance across the four main steps of EBM. To provide a complete suite of instruments to assess EBM competency across various patient scenarios, future refinement of the ACE instrument should include further scenarios across harm, diagnosis and prognosis. BioMed Central 2014-06-09 /pmc/articles/PMC4062508/ /pubmed/24909434 http://dx.doi.org/10.1186/1472-6920-14-114 Text en Copyright © 2014 Ilic et al.; licensee BioMed Central Ltd. http://creativecommons.org/licenses/by/2.0 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Ilic, Dragan
Nordin, Rusli Bin
Glasziou, Paul
Tilson, Julie K
Villanueva, Elmer
Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title_full Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title_fullStr Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title_full_unstemmed Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title_short Development and validation of the ACE tool: assessing medical trainees’ competency in evidence based medicine
title_sort development and validation of the ace tool: assessing medical trainees’ competency in evidence based medicine
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4062508/
https://www.ncbi.nlm.nih.gov/pubmed/24909434
http://dx.doi.org/10.1186/1472-6920-14-114
work_keys_str_mv AT ilicdragan developmentandvalidationoftheacetoolassessingmedicaltraineescompetencyinevidencebasedmedicine
AT nordinruslibin developmentandvalidationoftheacetoolassessingmedicaltraineescompetencyinevidencebasedmedicine
AT glaszioupaul developmentandvalidationoftheacetoolassessingmedicaltraineescompetencyinevidencebasedmedicine
AT tilsonjuliek developmentandvalidationoftheacetoolassessingmedicaltraineescompetencyinevidencebasedmedicine
AT villanuevaelmer developmentandvalidationoftheacetoolassessingmedicaltraineescompetencyinevidencebasedmedicine