Cargando…

Inter-rater reliability of the extended Composite Quality Score (CQS-2)

AIM: To establish the inter-rater reliability of the Composite Quality Score (CQS-2) and to test the null hypothesis that it did not differ significantly from that of the first CQS version (CQS-1). MATERIALS AND METHODS: Four independent raters were selected to rate 45 clinical trial reports using C...

Descripción completa

Detalles Bibliográficos
Autores principales: Mickenautsch, Steffen, Rupf, Stefan, Miletić, Ivana, Strähle, Ulf Tilman, Sturm, Richard, Kimmie-Dhansay, Faheema, Vidosusić, Kata, Yengopal, Veerasamy
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10469905/
https://www.ncbi.nlm.nih.gov/pubmed/37663665
http://dx.doi.org/10.3389/fmed.2023.1201517
_version_ 1785099551765430272
author Mickenautsch, Steffen
Rupf, Stefan
Miletić, Ivana
Strähle, Ulf Tilman
Sturm, Richard
Kimmie-Dhansay, Faheema
Vidosusić, Kata
Yengopal, Veerasamy
author_facet Mickenautsch, Steffen
Rupf, Stefan
Miletić, Ivana
Strähle, Ulf Tilman
Sturm, Richard
Kimmie-Dhansay, Faheema
Vidosusić, Kata
Yengopal, Veerasamy
author_sort Mickenautsch, Steffen
collection PubMed
description AIM: To establish the inter-rater reliability of the Composite Quality Score (CQS-2) and to test the null hypothesis that it did not differ significantly from that of the first CQS version (CQS-1). MATERIALS AND METHODS: Four independent raters were selected to rate 45 clinical trial reports using CQS-1 and CQS-2. The raters remained unaware of each other’s participation in this study until all rating had been completed. Each rater received only one rating template at a time in a random sequence for CQS-1 and CQS-2 rating. Raters completed each template and sent these back to the principal investigator. Each rater received their next template 2 weeks after submission of the completed previous template. The inter-rater reliabilities for the overall appraisal score of the CQS-1 and the CQS-2 were established by using the Brennan-Prediger coefficient (BPC). The coefficients of both CQS versions were compared by using the two-sample z-test. During secondary analysis, the BPCs for every criterion and each corroboration level for both CQS versions were established. RESULTS: The BPC for the CQS-1 was 0.85 (95% CI: 0.64–1.00) and for the CQS-2 it was 1.00 (95% CI: 0.94–1.00), suggesting a very high inter-rater reliability for both. The difference between the two CQS versions was statistically not significant (p = 0.17). The null hypothesis was accepted. CONCLUSION: The CQS-2 is still under development, This study shows that it is associated with a very high inter-rater reliability, which did not statistically significantly differ from that of the CQS-1. The promising results of this study warrant further investigation in the applicability of the CQS-2 as an appraisal tool for prospective controlled clinical therapy trials.
format Online
Article
Text
id pubmed-10469905
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-104699052023-09-01 Inter-rater reliability of the extended Composite Quality Score (CQS-2) Mickenautsch, Steffen Rupf, Stefan Miletić, Ivana Strähle, Ulf Tilman Sturm, Richard Kimmie-Dhansay, Faheema Vidosusić, Kata Yengopal, Veerasamy Front Med (Lausanne) Medicine AIM: To establish the inter-rater reliability of the Composite Quality Score (CQS-2) and to test the null hypothesis that it did not differ significantly from that of the first CQS version (CQS-1). MATERIALS AND METHODS: Four independent raters were selected to rate 45 clinical trial reports using CQS-1 and CQS-2. The raters remained unaware of each other’s participation in this study until all rating had been completed. Each rater received only one rating template at a time in a random sequence for CQS-1 and CQS-2 rating. Raters completed each template and sent these back to the principal investigator. Each rater received their next template 2 weeks after submission of the completed previous template. The inter-rater reliabilities for the overall appraisal score of the CQS-1 and the CQS-2 were established by using the Brennan-Prediger coefficient (BPC). The coefficients of both CQS versions were compared by using the two-sample z-test. During secondary analysis, the BPCs for every criterion and each corroboration level for both CQS versions were established. RESULTS: The BPC for the CQS-1 was 0.85 (95% CI: 0.64–1.00) and for the CQS-2 it was 1.00 (95% CI: 0.94–1.00), suggesting a very high inter-rater reliability for both. The difference between the two CQS versions was statistically not significant (p = 0.17). The null hypothesis was accepted. CONCLUSION: The CQS-2 is still under development, This study shows that it is associated with a very high inter-rater reliability, which did not statistically significantly differ from that of the CQS-1. The promising results of this study warrant further investigation in the applicability of the CQS-2 as an appraisal tool for prospective controlled clinical therapy trials. Frontiers Media S.A. 2023-08-17 /pmc/articles/PMC10469905/ /pubmed/37663665 http://dx.doi.org/10.3389/fmed.2023.1201517 Text en Copyright © 2023 Mickenautsch, Rupf, Miletić, Strähle, Sturm, Kimmie-Dhansay, Vidosusić and Yengopal. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Medicine
Mickenautsch, Steffen
Rupf, Stefan
Miletić, Ivana
Strähle, Ulf Tilman
Sturm, Richard
Kimmie-Dhansay, Faheema
Vidosusić, Kata
Yengopal, Veerasamy
Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title_full Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title_fullStr Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title_full_unstemmed Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title_short Inter-rater reliability of the extended Composite Quality Score (CQS-2)
title_sort inter-rater reliability of the extended composite quality score (cqs-2)
topic Medicine
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10469905/
https://www.ncbi.nlm.nih.gov/pubmed/37663665
http://dx.doi.org/10.3389/fmed.2023.1201517
work_keys_str_mv AT mickenautschsteffen interraterreliabilityoftheextendedcompositequalityscorecqs2
AT rupfstefan interraterreliabilityoftheextendedcompositequalityscorecqs2
AT mileticivana interraterreliabilityoftheextendedcompositequalityscorecqs2
AT strahleulftilman interraterreliabilityoftheextendedcompositequalityscorecqs2
AT sturmrichard interraterreliabilityoftheextendedcompositequalityscorecqs2
AT kimmiedhansayfaheema interraterreliabilityoftheextendedcompositequalityscorecqs2
AT vidosusickata interraterreliabilityoftheextendedcompositequalityscorecqs2
AT yengopalveerasamy interraterreliabilityoftheextendedcompositequalityscorecqs2