Cargando…

Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies

Here we describe the development and evaluation of a survey instrument to assess the research suitability of veterinary electronic medical records (EMRs) through the conduct of two studies as part of the Dog Aging Project (DAP). In study 1, four reviewers used the instrument to score a total of 218...

Descripción completa

Detalles Bibliográficos
Autores principales: Praczko, Dora, Tinkle, Amanda K., Arkenberg, Crystal R., McClelland, Robyn L., Creevy, Kate E., Tolbert, M. Katherine, Barnett, Brian G., Chou, Lucy, Evans, Jeremy, McNulty, Kellyn E., Levine, Jonathan M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9389294/
https://www.ncbi.nlm.nih.gov/pubmed/35990265
http://dx.doi.org/10.3389/fvets.2022.941036
_version_ 1784770412580700160
author Praczko, Dora
Tinkle, Amanda K.
Arkenberg, Crystal R.
McClelland, Robyn L.
Creevy, Kate E.
Tolbert, M. Katherine
Barnett, Brian G.
Chou, Lucy
Evans, Jeremy
McNulty, Kellyn E.
Levine, Jonathan M.
author_facet Praczko, Dora
Tinkle, Amanda K.
Arkenberg, Crystal R.
McClelland, Robyn L.
Creevy, Kate E.
Tolbert, M. Katherine
Barnett, Brian G.
Chou, Lucy
Evans, Jeremy
McNulty, Kellyn E.
Levine, Jonathan M.
author_sort Praczko, Dora
collection PubMed
description Here we describe the development and evaluation of a survey instrument to assess the research suitability of veterinary electronic medical records (EMRs) through the conduct of two studies as part of the Dog Aging Project (DAP). In study 1, four reviewers used the instrument to score a total of 218 records in an overlapping matrix of pairs to assess inter-rater agreement with respect to appropriate format (qualification), identification match (verification), and record quality. Based upon the moderate inter-rater agreement with respect to verification and the relatively large number of records that were incorrectly rejected the instrument was modified and more specific instructions were provided. In study 2, a modified instrument was again completed by four reviewers to score 100 different EMRs. The survey scores were compared to a gold standard of board-certified specialist review to determine receiver operating curve statistics. The refined survey had substantial inter-rater agreement across most qualification and verification questions. The cut-off value identified had a sensitivity of 95 and 96% (by reviewer 1 and reviewer 2, respectively) and a specificity of 82% and 91% (by reviewer 1 and reviewer 2, respectively) to predict gold standard acceptance or rejection of the record. Using just qualification and verification questions within the instrument (as opposed to full scoring) minimally impacted sensitivity and specificity and resulted in substantial time savings in the review process.
format Online
Article
Text
id pubmed-9389294
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-93892942022-08-20 Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies Praczko, Dora Tinkle, Amanda K. Arkenberg, Crystal R. McClelland, Robyn L. Creevy, Kate E. Tolbert, M. Katherine Barnett, Brian G. Chou, Lucy Evans, Jeremy McNulty, Kellyn E. Levine, Jonathan M. Front Vet Sci Veterinary Science Here we describe the development and evaluation of a survey instrument to assess the research suitability of veterinary electronic medical records (EMRs) through the conduct of two studies as part of the Dog Aging Project (DAP). In study 1, four reviewers used the instrument to score a total of 218 records in an overlapping matrix of pairs to assess inter-rater agreement with respect to appropriate format (qualification), identification match (verification), and record quality. Based upon the moderate inter-rater agreement with respect to verification and the relatively large number of records that were incorrectly rejected the instrument was modified and more specific instructions were provided. In study 2, a modified instrument was again completed by four reviewers to score 100 different EMRs. The survey scores were compared to a gold standard of board-certified specialist review to determine receiver operating curve statistics. The refined survey had substantial inter-rater agreement across most qualification and verification questions. The cut-off value identified had a sensitivity of 95 and 96% (by reviewer 1 and reviewer 2, respectively) and a specificity of 82% and 91% (by reviewer 1 and reviewer 2, respectively) to predict gold standard acceptance or rejection of the record. Using just qualification and verification questions within the instrument (as opposed to full scoring) minimally impacted sensitivity and specificity and resulted in substantial time savings in the review process. Frontiers Media S.A. 2022-08-05 /pmc/articles/PMC9389294/ /pubmed/35990265 http://dx.doi.org/10.3389/fvets.2022.941036 Text en Copyright © 2022 Praczko, Tinkle, Arkenberg, McClelland, Creevy, Tolbert, Barnett, Chou, Evans, McNulty, Dog Aging Project Consortium and Levine. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Veterinary Science
Praczko, Dora
Tinkle, Amanda K.
Arkenberg, Crystal R.
McClelland, Robyn L.
Creevy, Kate E.
Tolbert, M. Katherine
Barnett, Brian G.
Chou, Lucy
Evans, Jeremy
McNulty, Kellyn E.
Levine, Jonathan M.
Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title_full Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title_fullStr Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title_full_unstemmed Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title_short Development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
title_sort development and evaluation of a survey instrument to assess veterinary medical record suitability for multi-center research studies
topic Veterinary Science
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9389294/
https://www.ncbi.nlm.nih.gov/pubmed/35990265
http://dx.doi.org/10.3389/fvets.2022.941036
work_keys_str_mv AT praczkodora developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT tinkleamandak developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT arkenbergcrystalr developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT mcclellandrobynl developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT creevykatee developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT tolbertmkatherine developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT barnettbriang developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT choulucy developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT evansjeremy developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT mcnultykellyne developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies
AT levinejonathanm developmentandevaluationofasurveyinstrumenttoassessveterinarymedicalrecordsuitabilityformulticenterresearchstudies