Cargando…
Comparison of confidence interval methods for an intra-class correlation coefficient (ICC)
BACKGROUND: The intraclass correlation coefficient (ICC) is widely used in biomedical research to assess the reproducibility of measurements between raters, labs, technicians, or devices. For example, in an inter-rater reliability study, a high ICC value means that noise variability (between-raters...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2014
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4258044/ https://www.ncbi.nlm.nih.gov/pubmed/25417040 http://dx.doi.org/10.1186/1471-2288-14-121 |
_version_ | 1782347835550728192 |
---|---|
author | Ionan, Alexei C Polley, Mei-Yin C McShane, Lisa M Dobbin, Kevin K |
author_facet | Ionan, Alexei C Polley, Mei-Yin C McShane, Lisa M Dobbin, Kevin K |
author_sort | Ionan, Alexei C |
collection | PubMed |
description | BACKGROUND: The intraclass correlation coefficient (ICC) is widely used in biomedical research to assess the reproducibility of measurements between raters, labs, technicians, or devices. For example, in an inter-rater reliability study, a high ICC value means that noise variability (between-raters and within-raters) is small relative to variability from patient to patient. A confidence interval or Bayesian credible interval for the ICC is a commonly reported summary. Such intervals can be constructed employing either frequentist or Bayesian methodologies. METHODS: This study examines the performance of three different methods for constructing an interval in a two-way, crossed, random effects model without interaction: the Generalized Confidence Interval method (GCI), the Modified Large Sample method (MLS), and a Bayesian method based on a noninformative prior distribution (NIB). Guidance is provided on interval construction method selection based on study design, sample size, and normality of the data. We compare the coverage probabilities and widths of the different interval methods. RESULTS: We show that, for the two-way, crossed, random effects model without interaction, care is needed in interval method selection because the interval estimates do not always have properties that the user expects. While different methods generally perform well when there are a large number of levels of each factor, large differences between the methods emerge when the number of one or more factors is limited. In addition, all methods are shown to lack robustness to certain hard-to-detect violations of normality when the sample size is limited. CONCLUSIONS: Decision rules and software programs for interval construction are provided for practical implementation in the two-way, crossed, random effects model without interaction. All interval methods perform similarly when the data are normal and there are sufficient numbers of levels of each factor. The MLS and GCI methods outperform the NIB when one of the factors has a limited number of levels and the data are normally distributed or nearly normally distributed. None of the methods work well if the number of levels of a factor are limited and data are markedly non-normal. The software programs are implemented in the popular R language. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/1471-2288-14-121) contains supplementary material, which is available to authorized users. |
format | Online Article Text |
id | pubmed-4258044 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2014 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-42580442014-12-07 Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) Ionan, Alexei C Polley, Mei-Yin C McShane, Lisa M Dobbin, Kevin K BMC Med Res Methodol Research Article BACKGROUND: The intraclass correlation coefficient (ICC) is widely used in biomedical research to assess the reproducibility of measurements between raters, labs, technicians, or devices. For example, in an inter-rater reliability study, a high ICC value means that noise variability (between-raters and within-raters) is small relative to variability from patient to patient. A confidence interval or Bayesian credible interval for the ICC is a commonly reported summary. Such intervals can be constructed employing either frequentist or Bayesian methodologies. METHODS: This study examines the performance of three different methods for constructing an interval in a two-way, crossed, random effects model without interaction: the Generalized Confidence Interval method (GCI), the Modified Large Sample method (MLS), and a Bayesian method based on a noninformative prior distribution (NIB). Guidance is provided on interval construction method selection based on study design, sample size, and normality of the data. We compare the coverage probabilities and widths of the different interval methods. RESULTS: We show that, for the two-way, crossed, random effects model without interaction, care is needed in interval method selection because the interval estimates do not always have properties that the user expects. While different methods generally perform well when there are a large number of levels of each factor, large differences between the methods emerge when the number of one or more factors is limited. In addition, all methods are shown to lack robustness to certain hard-to-detect violations of normality when the sample size is limited. CONCLUSIONS: Decision rules and software programs for interval construction are provided for practical implementation in the two-way, crossed, random effects model without interaction. All interval methods perform similarly when the data are normal and there are sufficient numbers of levels of each factor. The MLS and GCI methods outperform the NIB when one of the factors has a limited number of levels and the data are normally distributed or nearly normally distributed. None of the methods work well if the number of levels of a factor are limited and data are markedly non-normal. The software programs are implemented in the popular R language. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/1471-2288-14-121) contains supplementary material, which is available to authorized users. BioMed Central 2014-11-22 /pmc/articles/PMC4258044/ /pubmed/25417040 http://dx.doi.org/10.1186/1471-2288-14-121 Text en © Ionan et al.; licensee BioMed Central Ltd. 2014 This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Article Ionan, Alexei C Polley, Mei-Yin C McShane, Lisa M Dobbin, Kevin K Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title | Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title_full | Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title_fullStr | Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title_full_unstemmed | Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title_short | Comparison of confidence interval methods for an intra-class correlation coefficient (ICC) |
title_sort | comparison of confidence interval methods for an intra-class correlation coefficient (icc) |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4258044/ https://www.ncbi.nlm.nih.gov/pubmed/25417040 http://dx.doi.org/10.1186/1471-2288-14-121 |
work_keys_str_mv | AT ionanalexeic comparisonofconfidenceintervalmethodsforanintraclasscorrelationcoefficienticc AT polleymeiyinc comparisonofconfidenceintervalmethodsforanintraclasscorrelationcoefficienticc AT mcshanelisam comparisonofconfidenceintervalmethodsforanintraclasscorrelationcoefficienticc AT dobbinkevink comparisonofconfidenceintervalmethodsforanintraclasscorrelationcoefficienticc |