Cargando…

Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project

BACKGROUND: The European Centres of Reference Network for Cystic Fibrosis (ECORN-CF) established an Internet forum which provides the opportunity for CF patients and other interested people to ask experts questions about CF in their mother language. The objectives of this study were to: 1) develop a...

Descripción completa

Detalles Bibliográficos
Autores principales: d'Alquen, Daniela, De Boeck, Kris, Bradley, Judy, Vávrová, Věra, Dembski, Birgit, Wagner, Thomas OF, Pfalz, Annette, Hebestreit, Helge
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2012
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3311602/
https://www.ncbi.nlm.nih.gov/pubmed/22309508
http://dx.doi.org/10.1186/1471-2288-12-11
_version_ 1782227792053665792
author d'Alquen, Daniela
De Boeck, Kris
Bradley, Judy
Vávrová, Věra
Dembski, Birgit
Wagner, Thomas OF
Pfalz, Annette
Hebestreit, Helge
author_facet d'Alquen, Daniela
De Boeck, Kris
Bradley, Judy
Vávrová, Věra
Dembski, Birgit
Wagner, Thomas OF
Pfalz, Annette
Hebestreit, Helge
author_sort d'Alquen, Daniela
collection PubMed
description BACKGROUND: The European Centres of Reference Network for Cystic Fibrosis (ECORN-CF) established an Internet forum which provides the opportunity for CF patients and other interested people to ask experts questions about CF in their mother language. The objectives of this study were to: 1) develop a detailed quality assessment tool to analyze quality of expert answers, 2) evaluate the intra- and inter-rater agreement of this tool, and 3) explore changes in the quality of expert answers over the time frame of the project. METHODS: The quality assessment tool was developed by an expert panel. Five experts within the ECORN-CF project used the quality assessment tool to analyze the quality of 108 expert answers published on ECORN-CF from six language zones. 25 expert answers were scored at two time points, one year apart. Quality of answers was also assessed at an early and later period of the project. Individual rater scores and group mean scores were analyzed for each expert answer. RESULTS: A scoring system and training manual were developed analyzing two quality categories of answers: content and formal quality. For content quality, the grades based on group mean scores for all raters showed substantial agreement between two time points, however this was not the case for the grades based on individual rater scores. For formal quality the grades based on group mean scores showed only slight agreement between two time points and there was also poor agreement between time points for the individual grades. The inter-rater agreement for content quality was fair (mean kappa value 0.232 ± 0.036, p < 0.001) while only slight agreement was observed for the grades of the formal quality (mean kappa value 0.105 ± 0.024, p < 0.001). The quality of expert answers was rated high (four language zones) or satisfactory (two language zones) and did not change over time. CONCLUSIONS: The quality assessment tool described in this study was feasible and reliable when content quality was assessed by a group of raters. Within ECORN-CF, the tool will help ensure that CF patients all over Europe have equal possibility of access to high quality expert advice on their illness.
format Online
Article
Text
id pubmed-3311602
institution National Center for Biotechnology Information
language English
publishDate 2012
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-33116022012-03-24 Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project d'Alquen, Daniela De Boeck, Kris Bradley, Judy Vávrová, Věra Dembski, Birgit Wagner, Thomas OF Pfalz, Annette Hebestreit, Helge BMC Med Res Methodol Research Article BACKGROUND: The European Centres of Reference Network for Cystic Fibrosis (ECORN-CF) established an Internet forum which provides the opportunity for CF patients and other interested people to ask experts questions about CF in their mother language. The objectives of this study were to: 1) develop a detailed quality assessment tool to analyze quality of expert answers, 2) evaluate the intra- and inter-rater agreement of this tool, and 3) explore changes in the quality of expert answers over the time frame of the project. METHODS: The quality assessment tool was developed by an expert panel. Five experts within the ECORN-CF project used the quality assessment tool to analyze the quality of 108 expert answers published on ECORN-CF from six language zones. 25 expert answers were scored at two time points, one year apart. Quality of answers was also assessed at an early and later period of the project. Individual rater scores and group mean scores were analyzed for each expert answer. RESULTS: A scoring system and training manual were developed analyzing two quality categories of answers: content and formal quality. For content quality, the grades based on group mean scores for all raters showed substantial agreement between two time points, however this was not the case for the grades based on individual rater scores. For formal quality the grades based on group mean scores showed only slight agreement between two time points and there was also poor agreement between time points for the individual grades. The inter-rater agreement for content quality was fair (mean kappa value 0.232 ± 0.036, p < 0.001) while only slight agreement was observed for the grades of the formal quality (mean kappa value 0.105 ± 0.024, p < 0.001). The quality of expert answers was rated high (four language zones) or satisfactory (two language zones) and did not change over time. CONCLUSIONS: The quality assessment tool described in this study was feasible and reliable when content quality was assessed by a group of raters. Within ECORN-CF, the tool will help ensure that CF patients all over Europe have equal possibility of access to high quality expert advice on their illness. BioMed Central 2012-02-06 /pmc/articles/PMC3311602/ /pubmed/22309508 http://dx.doi.org/10.1186/1471-2288-12-11 Text en Copyright ©2012 d'Alquen et al; licensee BioMed Central Ltd. http://creativecommons.org/licenses/by/2.0 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Research Article
d'Alquen, Daniela
De Boeck, Kris
Bradley, Judy
Vávrová, Věra
Dembski, Birgit
Wagner, Thomas OF
Pfalz, Annette
Hebestreit, Helge
Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title_full Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title_fullStr Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title_full_unstemmed Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title_short Quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in Europe: the ECORN-CF project
title_sort quality assessment of expert answers to lay questions about cystic fibrosis from various language zones in europe: the ecorn-cf project
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3311602/
https://www.ncbi.nlm.nih.gov/pubmed/22309508
http://dx.doi.org/10.1186/1471-2288-12-11
work_keys_str_mv AT dalquendaniela qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT deboeckkris qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT bradleyjudy qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT vavrovavera qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT dembskibirgit qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT wagnerthomasof qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT pfalzannette qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject
AT hebestreithelge qualityassessmentofexpertanswerstolayquestionsaboutcysticfibrosisfromvariouslanguagezonesineuropetheecorncfproject