Cargando…

Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study

BACKGROUND: Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their o...

Descripción completa

Detalles Bibliográficos
Autores principales: Lai, Nai Ming, Teng, Cheong Lieng, Lee, Ming Lee
Formato: Texto
Lenguaje:English
Publicado: BioMed Central 2011
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3100234/
https://www.ncbi.nlm.nih.gov/pubmed/21450083
http://dx.doi.org/10.1186/1741-7015-9-30
_version_ 1782204165545525248
author Lai, Nai Ming
Teng, Cheong Lieng
Lee, Ming Lee
author_facet Lai, Nai Ming
Teng, Cheong Lieng
Lee, Ming Lee
author_sort Lai, Nai Ming
collection PubMed
description BACKGROUND: Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results. METHODS: We conducted two cross-sectional studies: one with hospital practitioners (n = 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (n = 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors' conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors' conclusions and labelled this as our best conclusion. We compared the participants' choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial ("positive"), and two others did not ("negative"). We also asked the participants their prior beliefs about the intervention. RESULTS: Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (P < 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. "Positive" SRs were more likely than "negative" SRs to change the participants' beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and "convert" those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8). CONCLUSIONS: The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors' conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different. Please see Commentary: http://www.biomedcentral.com/1741-7015/9/31/.
format Text
id pubmed-3100234
institution National Center for Biotechnology Information
language English
publishDate 2011
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-31002342011-05-24 Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study Lai, Nai Ming Teng, Cheong Lieng Lee, Ming Lee BMC Med Research Article BACKGROUND: Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results. METHODS: We conducted two cross-sectional studies: one with hospital practitioners (n = 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (n = 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors' conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors' conclusions and labelled this as our best conclusion. We compared the participants' choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial ("positive"), and two others did not ("negative"). We also asked the participants their prior beliefs about the intervention. RESULTS: Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (P < 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. "Positive" SRs were more likely than "negative" SRs to change the participants' beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and "convert" those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8). CONCLUSIONS: The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors' conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different. Please see Commentary: http://www.biomedcentral.com/1741-7015/9/31/. BioMed Central 2011-03-30 /pmc/articles/PMC3100234/ /pubmed/21450083 http://dx.doi.org/10.1186/1741-7015-9-30 Text en Copyright ©2011 Lai et al; licensee BioMed Central Ltd. http://creativecommons.org/licenses/by/2.0 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Research Article
Lai, Nai Ming
Teng, Cheong Lieng
Lee, Ming Lee
Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title_full Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title_fullStr Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title_full_unstemmed Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title_short Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study
title_sort interpreting systematic reviews: are we ready to make our own conclusions? a cross-sectional study
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3100234/
https://www.ncbi.nlm.nih.gov/pubmed/21450083
http://dx.doi.org/10.1186/1741-7015-9-30
work_keys_str_mv AT lainaiming interpretingsystematicreviewsarewereadytomakeourownconclusionsacrosssectionalstudy
AT tengcheonglieng interpretingsystematicreviewsarewereadytomakeourownconclusionsacrosssectionalstudy
AT leeminglee interpretingsystematicreviewsarewereadytomakeourownconclusionsacrosssectionalstudy