Cargando…
Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments
INTRODUCTION: There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how...
Autores principales: | , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Association of American Medical Colleges
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6507922/ https://www.ncbi.nlm.nih.gov/pubmed/31139736 http://dx.doi.org/10.15766/mep_2374-8265.10817 |
_version_ | 1783417068484296704 |
---|---|
author | Paul, Caroline R. Ryan, Michael S. Dallaghan, Gary L. Beck Jirasevijinda, Thanakorn Quigley, Patricia D. Hanson, Janice L. Khidir, Amal M. Petershack, Jean Jackson, Joseph Tewksbury, Linda Rocha, Mary Esther M. |
author_facet | Paul, Caroline R. Ryan, Michael S. Dallaghan, Gary L. Beck Jirasevijinda, Thanakorn Quigley, Patricia D. Hanson, Janice L. Khidir, Amal M. Petershack, Jean Jackson, Joseph Tewksbury, Linda Rocha, Mary Esther M. |
author_sort | Paul, Caroline R. |
collection | PubMed |
description | INTRODUCTION: There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. METHODS: The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. RESULTS: The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. DISCUSSION: This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes. |
format | Online Article Text |
id | pubmed-6507922 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | Association of American Medical Colleges |
record_format | MEDLINE/PubMed |
spelling | pubmed-65079222019-05-28 Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments Paul, Caroline R. Ryan, Michael S. Dallaghan, Gary L. Beck Jirasevijinda, Thanakorn Quigley, Patricia D. Hanson, Janice L. Khidir, Amal M. Petershack, Jean Jackson, Joseph Tewksbury, Linda Rocha, Mary Esther M. MedEdPORTAL Original Publication INTRODUCTION: There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data. METHODS: The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work. RESULTS: The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective. DISCUSSION: This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes. Association of American Medical Colleges 2019-04-12 /pmc/articles/PMC6507922/ /pubmed/31139736 http://dx.doi.org/10.15766/mep_2374-8265.10817 Text en Copyright: © 2019 Paul et al. https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode This is an open-access publication distributed under the terms of the Creative Commons Attribution-NonCommercial-Share Alike (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) license. |
spellingShingle | Original Publication Paul, Caroline R. Ryan, Michael S. Dallaghan, Gary L. Beck Jirasevijinda, Thanakorn Quigley, Patricia D. Hanson, Janice L. Khidir, Amal M. Petershack, Jean Jackson, Joseph Tewksbury, Linda Rocha, Mary Esther M. Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title | Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title_full | Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title_fullStr | Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title_full_unstemmed | Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title_short | Collecting Validity Evidence: A Hands-on Workshop for Medical Education Assessment Instruments |
title_sort | collecting validity evidence: a hands-on workshop for medical education assessment instruments |
topic | Original Publication |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6507922/ https://www.ncbi.nlm.nih.gov/pubmed/31139736 http://dx.doi.org/10.15766/mep_2374-8265.10817 |
work_keys_str_mv | AT paulcaroliner collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT ryanmichaels collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT dallaghangarylbeck collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT jirasevijindathanakorn collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT quigleypatriciad collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT hansonjanicel collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT khidiramalm collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT petershackjean collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT jacksonjoseph collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT tewksburylinda collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments AT rochamaryestherm collectingvalidityevidenceahandsonworkshopformedicaleducationassessmentinstruments |