Cargando…

Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory

BACKGROUND: Meaningfully interpreting patient-reported outcomes (PRO) results from randomized clinical trials requires that the PRO scores obtained in the trial have the same meaning across patients and previous applications of the PRO instrument. Calibration of PRO instruments warrants this propert...

Descripción completa

Detalles Bibliográficos
Autores principales: Loubert, Angély, Regnault, Antoine, Sébille, Véronique, Hardouin, Jean-Benoit
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9375403/
https://www.ncbi.nlm.nih.gov/pubmed/35962310
http://dx.doi.org/10.1186/s12874-022-01680-z
_version_ 1784767957154398208
author Loubert, Angély
Regnault, Antoine
Sébille, Véronique
Hardouin, Jean-Benoit
author_facet Loubert, Angély
Regnault, Antoine
Sébille, Véronique
Hardouin, Jean-Benoit
author_sort Loubert, Angély
collection PubMed
description BACKGROUND: Meaningfully interpreting patient-reported outcomes (PRO) results from randomized clinical trials requires that the PRO scores obtained in the trial have the same meaning across patients and previous applications of the PRO instrument. Calibration of PRO instruments warrants this property. In the Rasch measurement theory (RMT) framework, calibration is performed by fixing the item parameter estimates when measuring the targeted concept for each individual of the trial. The item parameter estimates used for this purpose are typically obtained from a previous “calibration” study. But imposing this constraint on item parameters, instead of freely estimating them directly in the specific sample of the trial, may hamper the ability to detect a treatment effect. The objective of this simulation study was to explore the potential negative impact of calibration of PRO instruments that were developed using RMT on the comparison of results between treatment groups, using different analysis methods. METHODS: PRO results were simulated following a polytomous Rasch model, for a calibration and a trial sample. Scenarios included varying sample sizes, with instrument of varying number of items and modalities, and varying item parameters distributions. Different treatment effect sizes and distributions of the two patient samples were also explored. Cross-sectional comparison of treatment groups was performed using different methods based on a random effect Rasch model. Calibrated and non-calibrated approaches were compared based on type-I error, power, bias, and variance of the estimates for the difference between groups. RESULTS: There was no impact of the calibration approach on type-I error, power, bias, and dispersion of the estimates. Among other findings, mistargeting between the PRO instrument and patients from the trial sample (regarding the level of measured concept) resulted in a lower power and higher position bias than appropriate targeting. CONCLUSIONS: Calibration does not compromise the ability to accurately assess a treatment effect using a PRO instrument developed within the RMT paradigm in randomized clinical trials. Thus, given its essential role in producing interpretable results, calibration should always be performed when using a PRO instrument developed using RMT as an endpoint in a randomized clinical trial. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01680-z.
format Online
Article
Text
id pubmed-9375403
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-93754032022-08-14 Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory Loubert, Angély Regnault, Antoine Sébille, Véronique Hardouin, Jean-Benoit BMC Med Res Methodol Research BACKGROUND: Meaningfully interpreting patient-reported outcomes (PRO) results from randomized clinical trials requires that the PRO scores obtained in the trial have the same meaning across patients and previous applications of the PRO instrument. Calibration of PRO instruments warrants this property. In the Rasch measurement theory (RMT) framework, calibration is performed by fixing the item parameter estimates when measuring the targeted concept for each individual of the trial. The item parameter estimates used for this purpose are typically obtained from a previous “calibration” study. But imposing this constraint on item parameters, instead of freely estimating them directly in the specific sample of the trial, may hamper the ability to detect a treatment effect. The objective of this simulation study was to explore the potential negative impact of calibration of PRO instruments that were developed using RMT on the comparison of results between treatment groups, using different analysis methods. METHODS: PRO results were simulated following a polytomous Rasch model, for a calibration and a trial sample. Scenarios included varying sample sizes, with instrument of varying number of items and modalities, and varying item parameters distributions. Different treatment effect sizes and distributions of the two patient samples were also explored. Cross-sectional comparison of treatment groups was performed using different methods based on a random effect Rasch model. Calibrated and non-calibrated approaches were compared based on type-I error, power, bias, and variance of the estimates for the difference between groups. RESULTS: There was no impact of the calibration approach on type-I error, power, bias, and dispersion of the estimates. Among other findings, mistargeting between the PRO instrument and patients from the trial sample (regarding the level of measured concept) resulted in a lower power and higher position bias than appropriate targeting. CONCLUSIONS: Calibration does not compromise the ability to accurately assess a treatment effect using a PRO instrument developed within the RMT paradigm in randomized clinical trials. Thus, given its essential role in producing interpretable results, calibration should always be performed when using a PRO instrument developed using RMT as an endpoint in a randomized clinical trial. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12874-022-01680-z. BioMed Central 2022-08-12 /pmc/articles/PMC9375403/ /pubmed/35962310 http://dx.doi.org/10.1186/s12874-022-01680-z Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
spellingShingle Research
Loubert, Angély
Regnault, Antoine
Sébille, Véronique
Hardouin, Jean-Benoit
Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title_full Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title_fullStr Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title_full_unstemmed Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title_short Evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on Rasch measurement theory
title_sort evaluating the impact of calibration of patient-reported outcomes measures on results from randomized clinical trials: a simulation study based on rasch measurement theory
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9375403/
https://www.ncbi.nlm.nih.gov/pubmed/35962310
http://dx.doi.org/10.1186/s12874-022-01680-z
work_keys_str_mv AT loubertangely evaluatingtheimpactofcalibrationofpatientreportedoutcomesmeasuresonresultsfromrandomizedclinicaltrialsasimulationstudybasedonraschmeasurementtheory
AT regnaultantoine evaluatingtheimpactofcalibrationofpatientreportedoutcomesmeasuresonresultsfromrandomizedclinicaltrialsasimulationstudybasedonraschmeasurementtheory
AT sebilleveronique evaluatingtheimpactofcalibrationofpatientreportedoutcomesmeasuresonresultsfromrandomizedclinicaltrialsasimulationstudybasedonraschmeasurementtheory
AT hardouinjeanbenoit evaluatingtheimpactofcalibrationofpatientreportedoutcomesmeasuresonresultsfromrandomizedclinicaltrialsasimulationstudybasedonraschmeasurementtheory