Cargando…

Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool

BACKGROUND: Qualitative evidence synthesis is increasingly used alongside reviews of effectiveness to inform guidelines and other decisions. To support this use, the GRADE-CERQual approach was developed to assess and communicate the confidence we have in findings from reviews of qualitative research...

Descripción completa

Detalles Bibliográficos
Autores principales: Munthe-Kaas, Heather Menzies, Glenton, Claire, Booth, Andrew, Noyes, Jane, Lewin, Simon
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6549363/
https://www.ncbi.nlm.nih.gov/pubmed/31164084
http://dx.doi.org/10.1186/s12874-019-0728-6
_version_ 1783423991150542848
author Munthe-Kaas, Heather Menzies
Glenton, Claire
Booth, Andrew
Noyes, Jane
Lewin, Simon
author_facet Munthe-Kaas, Heather Menzies
Glenton, Claire
Booth, Andrew
Noyes, Jane
Lewin, Simon
author_sort Munthe-Kaas, Heather Menzies
collection PubMed
description BACKGROUND: Qualitative evidence synthesis is increasingly used alongside reviews of effectiveness to inform guidelines and other decisions. To support this use, the GRADE-CERQual approach was developed to assess and communicate the confidence we have in findings from reviews of qualitative research. One component of this approach requires an appraisal of the methodological limitations of studies contributing data to a review finding. Diverse critical appraisal tools for qualitative research are currently being used. However, it is unclear which tool is most appropriate for informing a GRADE-CERQual assessment of confidence. METHODOLOGY: We searched for tools that were explicitly intended for critically appraising the methodological quality of qualitative research. We searched the reference lists of existing methodological reviews for critical appraisal tools, and also conducted a systematic search in June 2016 for tools published in health science and social science databases. Two reviewers screened identified titles and abstracts, and then screened the full text of potentially relevant articles. One reviewer extracted data from each article and a second reviewer checked the extraction. We used a best-fit framework synthesis approach to code checklist criteria from each identified tool and to organise these into themes. RESULTS: We identified 102 critical appraisal tools: 71 tools had previously been included in methodological reviews, and 31 tools were identified from our systematic search. Almost half of the tools were published after 2010. Few authors described how their tool was developed, or why a new tool was needed. After coding all criteria, we developed a framework that included 22 themes. None of the tools included all 22 themes. Some themes were included in up to 95 of the tools. CONCLUSION: It is problematic that researchers continue to develop new tools without adequately examining the many tools that already exist. Furthermore, the plethora of tools, old and new, indicates a lack of consensus regarding the best tool to use, and an absence of empirical evidence about the most important criteria for assessing the methodological limitations of qualitative research, including in the context of use with GRADE-CERQual. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0728-6) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-6549363
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-65493632019-06-06 Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool Munthe-Kaas, Heather Menzies Glenton, Claire Booth, Andrew Noyes, Jane Lewin, Simon BMC Med Res Methodol Research Article BACKGROUND: Qualitative evidence synthesis is increasingly used alongside reviews of effectiveness to inform guidelines and other decisions. To support this use, the GRADE-CERQual approach was developed to assess and communicate the confidence we have in findings from reviews of qualitative research. One component of this approach requires an appraisal of the methodological limitations of studies contributing data to a review finding. Diverse critical appraisal tools for qualitative research are currently being used. However, it is unclear which tool is most appropriate for informing a GRADE-CERQual assessment of confidence. METHODOLOGY: We searched for tools that were explicitly intended for critically appraising the methodological quality of qualitative research. We searched the reference lists of existing methodological reviews for critical appraisal tools, and also conducted a systematic search in June 2016 for tools published in health science and social science databases. Two reviewers screened identified titles and abstracts, and then screened the full text of potentially relevant articles. One reviewer extracted data from each article and a second reviewer checked the extraction. We used a best-fit framework synthesis approach to code checklist criteria from each identified tool and to organise these into themes. RESULTS: We identified 102 critical appraisal tools: 71 tools had previously been included in methodological reviews, and 31 tools were identified from our systematic search. Almost half of the tools were published after 2010. Few authors described how their tool was developed, or why a new tool was needed. After coding all criteria, we developed a framework that included 22 themes. None of the tools included all 22 themes. Some themes were included in up to 95 of the tools. CONCLUSION: It is problematic that researchers continue to develop new tools without adequately examining the many tools that already exist. Furthermore, the plethora of tools, old and new, indicates a lack of consensus regarding the best tool to use, and an absence of empirical evidence about the most important criteria for assessing the methodological limitations of qualitative research, including in the context of use with GRADE-CERQual. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0728-6) contains supplementary material, which is available to authorized users. BioMed Central 2019-06-04 /pmc/articles/PMC6549363/ /pubmed/31164084 http://dx.doi.org/10.1186/s12874-019-0728-6 Text en © The Author(s). 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Munthe-Kaas, Heather Menzies
Glenton, Claire
Booth, Andrew
Noyes, Jane
Lewin, Simon
Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title_full Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title_fullStr Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title_full_unstemmed Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title_short Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool
title_sort systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the camelot tool
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6549363/
https://www.ncbi.nlm.nih.gov/pubmed/31164084
http://dx.doi.org/10.1186/s12874-019-0728-6
work_keys_str_mv AT munthekaasheathermenzies systematicmappingofexistingtoolstoappraisemethodologicalstrengthsandlimitationsofqualitativeresearchfirststageinthedevelopmentofthecamelottool
AT glentonclaire systematicmappingofexistingtoolstoappraisemethodologicalstrengthsandlimitationsofqualitativeresearchfirststageinthedevelopmentofthecamelottool
AT boothandrew systematicmappingofexistingtoolstoappraisemethodologicalstrengthsandlimitationsofqualitativeresearchfirststageinthedevelopmentofthecamelottool
AT noyesjane systematicmappingofexistingtoolstoappraisemethodologicalstrengthsandlimitationsofqualitativeresearchfirststageinthedevelopmentofthecamelottool
AT lewinsimon systematicmappingofexistingtoolstoappraisemethodologicalstrengthsandlimitationsofqualitativeresearchfirststageinthedevelopmentofthecamelottool