Cargando…

Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development

BACKGROUND: Methods for developing a core outcome or information set require involvement of key stakeholders to prioritise many items and achieve agreement as to the core set. The Delphi technique requires participants to rate the importance of items in sequential questionnaires (or rounds) with fee...

Descripción completa

Detalles Bibliográficos
Autores principales: Brookes, Sara T., Macefield, Rhiannon C., Williamson, Paula R., McNair, Angus G., Potter, Shelley, Blencowe, Natalie S., Strong, Sean, Blazeby, Jane M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2016
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4989325/
https://www.ncbi.nlm.nih.gov/pubmed/27534622
http://dx.doi.org/10.1186/s13063-016-1479-x
_version_ 1782448549780258816
author Brookes, Sara T.
Macefield, Rhiannon C.
Williamson, Paula R.
McNair, Angus G.
Potter, Shelley
Blencowe, Natalie S.
Strong, Sean
Blazeby, Jane M.
author_facet Brookes, Sara T.
Macefield, Rhiannon C.
Williamson, Paula R.
McNair, Angus G.
Potter, Shelley
Blencowe, Natalie S.
Strong, Sean
Blazeby, Jane M.
author_sort Brookes, Sara T.
collection PubMed
description BACKGROUND: Methods for developing a core outcome or information set require involvement of key stakeholders to prioritise many items and achieve agreement as to the core set. The Delphi technique requires participants to rate the importance of items in sequential questionnaires (or rounds) with feedback provided in each subsequent round such that participants are able to consider the views of others. This study examines the impact of receiving feedback from different stakeholder groups, on the subsequent rating of items and the level of agreement between stakeholders. METHODS: Randomized controlled trials were nested within the development of three core sets each including a Delphi process with two rounds of questionnaires, completed by patients and health professionals. Participants rated items from 1 (not essential) to 9 (absolutely essential). For round 2, participants were randomized to receive feedback from their peer stakeholder group only (peer) or both stakeholder groups separately (multiple). Decisions as to which items to retain following each round were determined by pre-specified criteria. RESULTS: Whilst type of feedback did not impact on the percentage of items for which a participant subsequently changed their rating, or the magnitude of change, it did impact on items retained at the end of round 2. Each core set contained discordant items retained by one feedback group but not the other (3–22 % discordant items). Consensus between patients and professionals in items to retain was greater amongst those receiving multiple group feedback in each core set (65–82 % agreement for peer-only feedback versus 74–94 % for multiple feedback). In addition, differences in round 2 scores were smaller between stakeholder groups receiving multiple feedback than between those receiving peer group feedback only. Variability in item scores across stakeholders was reduced following any feedback but this reduction was consistently greater amongst the multiple feedback group. CONCLUSIONS: In the development of a core outcome or information set, providing feedback within Delphi questionnaires from all stakeholder groups separately may influence the final core set and improve consensus between the groups. Further work is needed to better understand how participants rate and re-rate items within a Delphi process. TRIAL REGISTRATION: The three randomized controlled trials reported here were each nested within the development of a core information or outcome set to investigate processes in core outcome and information set development. Outcomes were not health-related and therefore trial registration was not applicable. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13063-016-1479-x) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-4989325
institution National Center for Biotechnology Information
language English
publishDate 2016
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-49893252016-08-19 Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development Brookes, Sara T. Macefield, Rhiannon C. Williamson, Paula R. McNair, Angus G. Potter, Shelley Blencowe, Natalie S. Strong, Sean Blazeby, Jane M. Trials Research BACKGROUND: Methods for developing a core outcome or information set require involvement of key stakeholders to prioritise many items and achieve agreement as to the core set. The Delphi technique requires participants to rate the importance of items in sequential questionnaires (or rounds) with feedback provided in each subsequent round such that participants are able to consider the views of others. This study examines the impact of receiving feedback from different stakeholder groups, on the subsequent rating of items and the level of agreement between stakeholders. METHODS: Randomized controlled trials were nested within the development of three core sets each including a Delphi process with two rounds of questionnaires, completed by patients and health professionals. Participants rated items from 1 (not essential) to 9 (absolutely essential). For round 2, participants were randomized to receive feedback from their peer stakeholder group only (peer) or both stakeholder groups separately (multiple). Decisions as to which items to retain following each round were determined by pre-specified criteria. RESULTS: Whilst type of feedback did not impact on the percentage of items for which a participant subsequently changed their rating, or the magnitude of change, it did impact on items retained at the end of round 2. Each core set contained discordant items retained by one feedback group but not the other (3–22 % discordant items). Consensus between patients and professionals in items to retain was greater amongst those receiving multiple group feedback in each core set (65–82 % agreement for peer-only feedback versus 74–94 % for multiple feedback). In addition, differences in round 2 scores were smaller between stakeholder groups receiving multiple feedback than between those receiving peer group feedback only. Variability in item scores across stakeholders was reduced following any feedback but this reduction was consistently greater amongst the multiple feedback group. CONCLUSIONS: In the development of a core outcome or information set, providing feedback within Delphi questionnaires from all stakeholder groups separately may influence the final core set and improve consensus between the groups. Further work is needed to better understand how participants rate and re-rate items within a Delphi process. TRIAL REGISTRATION: The three randomized controlled trials reported here were each nested within the development of a core information or outcome set to investigate processes in core outcome and information set development. Outcomes were not health-related and therefore trial registration was not applicable. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13063-016-1479-x) contains supplementary material, which is available to authorized users. BioMed Central 2016-08-17 /pmc/articles/PMC4989325/ /pubmed/27534622 http://dx.doi.org/10.1186/s13063-016-1479-x Text en © Brookes et al. 2016 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research
Brookes, Sara T.
Macefield, Rhiannon C.
Williamson, Paula R.
McNair, Angus G.
Potter, Shelley
Blencowe, Natalie S.
Strong, Sean
Blazeby, Jane M.
Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title_full Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title_fullStr Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title_full_unstemmed Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title_short Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development
title_sort three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within delphi surveys during core outcome and information set development
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4989325/
https://www.ncbi.nlm.nih.gov/pubmed/27534622
http://dx.doi.org/10.1186/s13063-016-1479-x
work_keys_str_mv AT brookessarat threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT macefieldrhiannonc threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT williamsonpaular threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT mcnairangusg threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT pottershelley threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT blencowenatalies threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT strongsean threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment
AT blazebyjanem threenestedrandomizedcontrolledtrialsofpeeronlyormultiplestakeholdergroupfeedbackwithindelphisurveysduringcoreoutcomeandinformationsetdevelopment