Cargando…

Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions

BACKGROUND: Recent studies of the quality of in-hospital care have used the Quality of Interaction Schedule (QuIS) to rate interactions observed between staff and inpatients in a variety of ward conditions. The QuIS was developed and evaluated in nursing and residential care. We set out to develop m...

Descripción completa

Detalles Bibliográficos
Autores principales: Mesa-Eguiagaray, Ines, Böhning, Dankmar, McLean, Chris, Griffiths, Peter, Bridges, Jackie, Pickering, Ruth M
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2016
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5142422/
https://www.ncbi.nlm.nih.gov/pubmed/27927178
http://dx.doi.org/10.1186/s12874-016-0266-4
_version_ 1782472773364350976
author Mesa-Eguiagaray, Ines
Böhning, Dankmar
McLean, Chris
Griffiths, Peter
Bridges, Jackie
Pickering, Ruth M
author_facet Mesa-Eguiagaray, Ines
Böhning, Dankmar
McLean, Chris
Griffiths, Peter
Bridges, Jackie
Pickering, Ruth M
author_sort Mesa-Eguiagaray, Ines
collection PubMed
description BACKGROUND: Recent studies of the quality of in-hospital care have used the Quality of Interaction Schedule (QuIS) to rate interactions observed between staff and inpatients in a variety of ward conditions. The QuIS was developed and evaluated in nursing and residential care. We set out to develop methodology for summarising information from inter-rater reliability studies of the QuIS in the acute hospital setting. METHODS: Staff-inpatient interactions were rated by trained staff observing care delivered during two-hour observation periods. Anticipating the possibility of the quality of care varying depending on ward conditions, we selected wards and times of day to reflect the variety of daytime care delivered to patients. We estimated inter-rater reliability using weighted kappa, κ(w), combined over observation periods to produce an overall, summary estimate, [Formula: see text] . Weighting schemes putting different emphasis on the severity of misclassification between QuIS categories were compared, as were different methods of combining observation period specific estimates. RESULTS: Estimated [Formula: see text] did not vary greatly depending on the weighting scheme employed, but we found simple averaging of estimates across observation periods to produce a higher value of inter-rater reliability due to over-weighting observation periods with fewest interactions. CONCLUSIONS: We recommend that researchers evaluating the inter-rater reliability of the QuIS by observing staff-inpatient interactions during observation periods representing the variety of ward conditions in which care takes place, should summarise inter-rater reliability by κ(w), weighted according to our scheme A4. Observation period specific estimates should be combined into an overall, single summary statistic [Formula: see text] , using a random effects approach, with [Formula: see text] , to be interpreted as the mean of the distribution of κ(w) across the variety of ward conditions. We draw attention to issues in the analysis and interpretation of inter-rater reliability studies incorporating distinct phases of data collection that may generalise more widely. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12874-016-0266-4) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-5142422
institution National Center for Biotechnology Information
language English
publishDate 2016
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-51424222016-12-15 Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions Mesa-Eguiagaray, Ines Böhning, Dankmar McLean, Chris Griffiths, Peter Bridges, Jackie Pickering, Ruth M BMC Med Res Methodol Research Article BACKGROUND: Recent studies of the quality of in-hospital care have used the Quality of Interaction Schedule (QuIS) to rate interactions observed between staff and inpatients in a variety of ward conditions. The QuIS was developed and evaluated in nursing and residential care. We set out to develop methodology for summarising information from inter-rater reliability studies of the QuIS in the acute hospital setting. METHODS: Staff-inpatient interactions were rated by trained staff observing care delivered during two-hour observation periods. Anticipating the possibility of the quality of care varying depending on ward conditions, we selected wards and times of day to reflect the variety of daytime care delivered to patients. We estimated inter-rater reliability using weighted kappa, κ(w), combined over observation periods to produce an overall, summary estimate, [Formula: see text] . Weighting schemes putting different emphasis on the severity of misclassification between QuIS categories were compared, as were different methods of combining observation period specific estimates. RESULTS: Estimated [Formula: see text] did not vary greatly depending on the weighting scheme employed, but we found simple averaging of estimates across observation periods to produce a higher value of inter-rater reliability due to over-weighting observation periods with fewest interactions. CONCLUSIONS: We recommend that researchers evaluating the inter-rater reliability of the QuIS by observing staff-inpatient interactions during observation periods representing the variety of ward conditions in which care takes place, should summarise inter-rater reliability by κ(w), weighted according to our scheme A4. Observation period specific estimates should be combined into an overall, single summary statistic [Formula: see text] , using a random effects approach, with [Formula: see text] , to be interpreted as the mean of the distribution of κ(w) across the variety of ward conditions. We draw attention to issues in the analysis and interpretation of inter-rater reliability studies incorporating distinct phases of data collection that may generalise more widely. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s12874-016-0266-4) contains supplementary material, which is available to authorized users. BioMed Central 2016-12-07 /pmc/articles/PMC5142422/ /pubmed/27927178 http://dx.doi.org/10.1186/s12874-016-0266-4 Text en © The Author(s). 2016 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Mesa-Eguiagaray, Ines
Böhning, Dankmar
McLean, Chris
Griffiths, Peter
Bridges, Jackie
Pickering, Ruth M
Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title_full Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title_fullStr Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title_full_unstemmed Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title_short Inter-rater reliability of the QuIS as an assessment of the quality of staff-inpatient interactions
title_sort inter-rater reliability of the quis as an assessment of the quality of staff-inpatient interactions
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5142422/
https://www.ncbi.nlm.nih.gov/pubmed/27927178
http://dx.doi.org/10.1186/s12874-016-0266-4
work_keys_str_mv AT mesaeguiagarayines interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions
AT bohningdankmar interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions
AT mcleanchris interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions
AT griffithspeter interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions
AT bridgesjackie interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions
AT pickeringruthm interraterreliabilityofthequisasanassessmentofthequalityofstaffinpatientinteractions