Cargando…

Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training

BACKGROUND: The 2018 ovarian-adnexal reporting and data system (O-RADS) guidelines are aimed at providing a system for consistent reports and risk stratification for ovarian lesions found on ultrasound. It provides key characteristics and findings for lesions, a lexicon of descriptors to communicate...

Descripción completa

Detalles Bibliográficos
Autores principales: Katlariwala, Prayash, Wilson, Mitchell P, Pi, Yeli, Chahal, Baljot S, Croutze, Roger, Patel, Deelan, Patel, Vimal, Low, Gavin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Baishideng Publishing Group Inc 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9521430/
https://www.ncbi.nlm.nih.gov/pubmed/36186517
http://dx.doi.org/10.4329/wjr.v14.i9.319
_version_ 1784799835524694016
author Katlariwala, Prayash
Wilson, Mitchell P
Pi, Yeli
Chahal, Baljot S
Croutze, Roger
Patel, Deelan
Patel, Vimal
Low, Gavin
author_facet Katlariwala, Prayash
Wilson, Mitchell P
Pi, Yeli
Chahal, Baljot S
Croutze, Roger
Patel, Deelan
Patel, Vimal
Low, Gavin
author_sort Katlariwala, Prayash
collection PubMed
description BACKGROUND: The 2018 ovarian-adnexal reporting and data system (O-RADS) guidelines are aimed at providing a system for consistent reports and risk stratification for ovarian lesions found on ultrasound. It provides key characteristics and findings for lesions, a lexicon of descriptors to communicate findings, and risk characterization and associated follow-up recommendation guidelines. However, the O-RADS guidelines have not been validated in North American institutions or amongst less experienced readers. AIM: To evaluate the diagnostic accuracy and inter-reader reliability of ultrasound O-RADS risk stratification amongst less experienced readers in a North American institution with and without pre-test training. METHODS: A single-center retrospective study was performed using 100 ovarian/adnexal lesions of varying O-RADS scores. Of these cases, 50 were allotted to a training cohort and 50 to a testing cohort via a non-randomized group selection process in order to approximately equal distribution of O-RADS categories both within and between groups. Reference standard O-RADS scores were established through consensus of three fellowship-trained body imaging radiologists. Three PGY-4 residents were independently evaluated for diagnostic accuracy and inter-reader reliability with and without pre-test O-RADS training. Sensitivity, specificity, positive predictive value, negative predictive value (NPV), and area under the curve (AUC) were used to measure accuracy. Fleiss kappa and weighted quadratic (pairwise) kappa values were used to measure inter-reader reliability. Statistical significance was P < 0.05. RESULTS: Mean patient age was 40 ± 16 years with lesions ranging from 1.2 to 22.5 cm. Readers demonstrated excellent specificities (85%-100% pre-training and 91%-100% post-training) and NPVs (89%-100% pre-training and 91-100% post-training) across the O-RADS categories. Sensitivities were variable (55%-100% pre-training and 64%-100% post-training) with malignant O-RADS 4 and 5 Lesions pre-training and post-training AUC values of 0.87-0.95 and 0.94-098, respectively (P < 0.001). Nineteen of 22 (86%) misclassified cases in pre-training were related to mischaracterization of dermoid features or wall/septation morphology. Fifteen of 17 (88%) of post-training misclassified cases were related to one of these two errors. Fleiss kappa inter-reader reliability was ‘good’ and pairwise inter-reader reliability was ‘very good’ with pre-training and post-training assessment (k = 0.76 and 0.77; and k = 0.77-0.87 and 0.85-0.89, respectively). CONCLUSION: Less experienced readers in North America achieved excellent specificities and AUC values with very good pairwise inter-reader reliability. They may be subject to misclassification of potentially malignant lesions, and specific training around dermoid features and smooth vs irregular inner wall/septation morphology may improve sensitivity.
format Online
Article
Text
id pubmed-9521430
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Baishideng Publishing Group Inc
record_format MEDLINE/PubMed
spelling pubmed-95214302022-09-30 Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training Katlariwala, Prayash Wilson, Mitchell P Pi, Yeli Chahal, Baljot S Croutze, Roger Patel, Deelan Patel, Vimal Low, Gavin World J Radiol Retrospective Study BACKGROUND: The 2018 ovarian-adnexal reporting and data system (O-RADS) guidelines are aimed at providing a system for consistent reports and risk stratification for ovarian lesions found on ultrasound. It provides key characteristics and findings for lesions, a lexicon of descriptors to communicate findings, and risk characterization and associated follow-up recommendation guidelines. However, the O-RADS guidelines have not been validated in North American institutions or amongst less experienced readers. AIM: To evaluate the diagnostic accuracy and inter-reader reliability of ultrasound O-RADS risk stratification amongst less experienced readers in a North American institution with and without pre-test training. METHODS: A single-center retrospective study was performed using 100 ovarian/adnexal lesions of varying O-RADS scores. Of these cases, 50 were allotted to a training cohort and 50 to a testing cohort via a non-randomized group selection process in order to approximately equal distribution of O-RADS categories both within and between groups. Reference standard O-RADS scores were established through consensus of three fellowship-trained body imaging radiologists. Three PGY-4 residents were independently evaluated for diagnostic accuracy and inter-reader reliability with and without pre-test O-RADS training. Sensitivity, specificity, positive predictive value, negative predictive value (NPV), and area under the curve (AUC) were used to measure accuracy. Fleiss kappa and weighted quadratic (pairwise) kappa values were used to measure inter-reader reliability. Statistical significance was P < 0.05. RESULTS: Mean patient age was 40 ± 16 years with lesions ranging from 1.2 to 22.5 cm. Readers demonstrated excellent specificities (85%-100% pre-training and 91%-100% post-training) and NPVs (89%-100% pre-training and 91-100% post-training) across the O-RADS categories. Sensitivities were variable (55%-100% pre-training and 64%-100% post-training) with malignant O-RADS 4 and 5 Lesions pre-training and post-training AUC values of 0.87-0.95 and 0.94-098, respectively (P < 0.001). Nineteen of 22 (86%) misclassified cases in pre-training were related to mischaracterization of dermoid features or wall/septation morphology. Fifteen of 17 (88%) of post-training misclassified cases were related to one of these two errors. Fleiss kappa inter-reader reliability was ‘good’ and pairwise inter-reader reliability was ‘very good’ with pre-training and post-training assessment (k = 0.76 and 0.77; and k = 0.77-0.87 and 0.85-0.89, respectively). CONCLUSION: Less experienced readers in North America achieved excellent specificities and AUC values with very good pairwise inter-reader reliability. They may be subject to misclassification of potentially malignant lesions, and specific training around dermoid features and smooth vs irregular inner wall/septation morphology may improve sensitivity. Baishideng Publishing Group Inc 2022-09-28 2022-09-28 /pmc/articles/PMC9521430/ /pubmed/36186517 http://dx.doi.org/10.4329/wjr.v14.i9.319 Text en ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved. https://creativecommons.org/licenses/by-nc/4.0/This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial.
spellingShingle Retrospective Study
Katlariwala, Prayash
Wilson, Mitchell P
Pi, Yeli
Chahal, Baljot S
Croutze, Roger
Patel, Deelan
Patel, Vimal
Low, Gavin
Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title_full Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title_fullStr Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title_full_unstemmed Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title_short Reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
title_sort reliability of ultrasound ovarian-adnexal reporting and data system amongst less experienced readers before and after training
topic Retrospective Study
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9521430/
https://www.ncbi.nlm.nih.gov/pubmed/36186517
http://dx.doi.org/10.4329/wjr.v14.i9.319
work_keys_str_mv AT katlariwalaprayash reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT wilsonmitchellp reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT piyeli reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT chahalbaljots reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT croutzeroger reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT pateldeelan reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT patelvimal reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining
AT lowgavin reliabilityofultrasoundovarianadnexalreportinganddatasystemamongstlessexperiencedreadersbeforeandaftertraining