Cargando…

Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials

BACKGROUND: Radiologic assessments of baseline and post-treatment tumor burden are subject to measurement variability, but the impact of this variability on the objective response rate (ORR) and progression rate in specific trials has been unpredictable on a practical level. In this study, we aimed...

Descripción completa

Detalles Bibliográficos
Autores principales: Yoon, Jeong-Hwa, Yoon, Soon Ho, Hahn, Seokyung
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6498480/
https://www.ncbi.nlm.nih.gov/pubmed/31046712
http://dx.doi.org/10.1186/s12874-019-0727-7
_version_ 1783415617536131072
author Yoon, Jeong-Hwa
Yoon, Soon Ho
Hahn, Seokyung
author_facet Yoon, Jeong-Hwa
Yoon, Soon Ho
Hahn, Seokyung
author_sort Yoon, Jeong-Hwa
collection PubMed
description BACKGROUND: Radiologic assessments of baseline and post-treatment tumor burden are subject to measurement variability, but the impact of this variability on the objective response rate (ORR) and progression rate in specific trials has been unpredictable on a practical level. In this study, we aimed to develop an algorithm for evaluating the quantitative impact of measurement variability on the ORR and progression rate. METHODS: First, we devised a hierarchical model for estimating the distribution of measurement variability using a clinical trial dataset of computed tomography scans. Next, a simulation method was used to calculate the probability representing the effect of measurement errors on categorical diagnoses in various scenarios using the estimated distribution. Based on the probabilities derived from the simulation, we developed an algorithm to evaluate the reliability of an ORR (or progression rate) (i.e., the variation in the assessed rate) by generating a 95% central range of ORR (or progression rate) results if a reassessment was performed. Finally, we performed validation using an external dataset. In the validation of the estimated distribution of measurement variability, the coverage level was calculated as the proportion of the 95% central ranges of hypothetical second readings that covered the actual burden sizes. In the validation of the evaluation algorithm, for 100 resampled datasets, the coverage level was calculated as the proportion of the 95% central ranges of ORR results that covered the ORR from a real second assessment. RESULTS: We built a web tool for implementing the algorithm (publicly available at http://studyanalysis2017.pythonanywhere.com/). In the validation of the estimated distribution and the algorithm, the coverage levels were 93 and 100%, respectively. CONCLUSIONS: The validation exercise using an external dataset demonstrated the adequacy of the statistical model and the utility of the developed algorithm. Quantification of variation in the ORR and progression rate due to potential measurement variability is essential and will help inform decisions made on the basis of trial data. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0727-7) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-6498480
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-64984802019-05-09 Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials Yoon, Jeong-Hwa Yoon, Soon Ho Hahn, Seokyung BMC Med Res Methodol Research Article BACKGROUND: Radiologic assessments of baseline and post-treatment tumor burden are subject to measurement variability, but the impact of this variability on the objective response rate (ORR) and progression rate in specific trials has been unpredictable on a practical level. In this study, we aimed to develop an algorithm for evaluating the quantitative impact of measurement variability on the ORR and progression rate. METHODS: First, we devised a hierarchical model for estimating the distribution of measurement variability using a clinical trial dataset of computed tomography scans. Next, a simulation method was used to calculate the probability representing the effect of measurement errors on categorical diagnoses in various scenarios using the estimated distribution. Based on the probabilities derived from the simulation, we developed an algorithm to evaluate the reliability of an ORR (or progression rate) (i.e., the variation in the assessed rate) by generating a 95% central range of ORR (or progression rate) results if a reassessment was performed. Finally, we performed validation using an external dataset. In the validation of the estimated distribution of measurement variability, the coverage level was calculated as the proportion of the 95% central ranges of hypothetical second readings that covered the actual burden sizes. In the validation of the evaluation algorithm, for 100 resampled datasets, the coverage level was calculated as the proportion of the 95% central ranges of ORR results that covered the ORR from a real second assessment. RESULTS: We built a web tool for implementing the algorithm (publicly available at http://studyanalysis2017.pythonanywhere.com/). In the validation of the estimated distribution and the algorithm, the coverage levels were 93 and 100%, respectively. CONCLUSIONS: The validation exercise using an external dataset demonstrated the adequacy of the statistical model and the utility of the developed algorithm. Quantification of variation in the ORR and progression rate due to potential measurement variability is essential and will help inform decisions made on the basis of trial data. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0727-7) contains supplementary material, which is available to authorized users. BioMed Central 2019-05-02 /pmc/articles/PMC6498480/ /pubmed/31046712 http://dx.doi.org/10.1186/s12874-019-0727-7 Text en © The Author(s). 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Yoon, Jeong-Hwa
Yoon, Soon Ho
Hahn, Seokyung
Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title_full Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title_fullStr Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title_full_unstemmed Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title_short Development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
title_sort development of an algorithm for evaluating the impact of measurement variability on response categorization in oncology trials
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6498480/
https://www.ncbi.nlm.nih.gov/pubmed/31046712
http://dx.doi.org/10.1186/s12874-019-0727-7
work_keys_str_mv AT yoonjeonghwa developmentofanalgorithmforevaluatingtheimpactofmeasurementvariabilityonresponsecategorizationinoncologytrials
AT yoonsoonho developmentofanalgorithmforevaluatingtheimpactofmeasurementvariabilityonresponsecategorizationinoncologytrials
AT hahnseokyung developmentofanalgorithmforevaluatingtheimpactofmeasurementvariabilityonresponsecategorizationinoncologytrials