Cargando…
Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study
RATIONALE, AIMS AND OBJECTIVES: We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. METHOD: We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BlackWell Publishing Ltd
2014
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4282468/ https://www.ncbi.nlm.nih.gov/pubmed/24819044 http://dx.doi.org/10.1111/jep.12136 |
_version_ | 1782351141041864704 |
---|---|
author | Tuijn, Saskia M van den Bergh, Huub Robben, Paul Janssens, Frans |
author_facet | Tuijn, Saskia M van den Bergh, Huub Robben, Paul Janssens, Frans |
author_sort | Tuijn, Saskia M |
collection | PubMed |
description | RATIONALE, AIMS AND OBJECTIVES: We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. METHOD: We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions. RESULTS: The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients. CONCLUSIONS: Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments. |
format | Online Article Text |
id | pubmed-4282468 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2014 |
publisher | BlackWell Publishing Ltd |
record_format | MEDLINE/PubMed |
spelling | pubmed-42824682015-01-15 Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study Tuijn, Saskia M van den Bergh, Huub Robben, Paul Janssens, Frans J Eval Clin Pract Original Articles RATIONALE, AIMS AND OBJECTIVES: We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus meeting. METHOD: We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions. RESULTS: The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients. CONCLUSIONS: Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments. BlackWell Publishing Ltd 2014-08 2014-05-12 /pmc/articles/PMC4282468/ /pubmed/24819044 http://dx.doi.org/10.1111/jep.12136 Text en © 2014 The Authors. Journal of Evaluation in Clinical Practice published by John Wiley & Sons, Ltd. http://creativecommons.org/licenses/by-nc-nd/3.0/ This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. |
spellingShingle | Original Articles Tuijn, Saskia M van den Bergh, Huub Robben, Paul Janssens, Frans Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title | Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title_full | Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title_fullStr | Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title_full_unstemmed | Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title_short | Experimental studies to improve the reliability and validity of regulatory judgments on health care in the Netherlands: a randomized controlled trial and before and after case study |
title_sort | experimental studies to improve the reliability and validity of regulatory judgments on health care in the netherlands: a randomized controlled trial and before and after case study |
topic | Original Articles |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4282468/ https://www.ncbi.nlm.nih.gov/pubmed/24819044 http://dx.doi.org/10.1111/jep.12136 |
work_keys_str_mv | AT tuijnsaskiam experimentalstudiestoimprovethereliabilityandvalidityofregulatoryjudgmentsonhealthcareinthenetherlandsarandomizedcontrolledtrialandbeforeandaftercasestudy AT vandenberghhuub experimentalstudiestoimprovethereliabilityandvalidityofregulatoryjudgmentsonhealthcareinthenetherlandsarandomizedcontrolledtrialandbeforeandaftercasestudy AT robbenpaul experimentalstudiestoimprovethereliabilityandvalidityofregulatoryjudgmentsonhealthcareinthenetherlandsarandomizedcontrolledtrialandbeforeandaftercasestudy AT janssensfrans experimentalstudiestoimprovethereliabilityandvalidityofregulatoryjudgmentsonhealthcareinthenetherlandsarandomizedcontrolledtrialandbeforeandaftercasestudy |