Cargando…

Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches

Background and Objectives: Literature reviews are foundational to understanding medical evidence. With AI tools like ChatGPT, Bing Chat and Bard AI emerging as potential aids in this domain, this study aimed to individually assess their citation accuracy within Nephrology, comparing their performanc...

Descripción completa

Detalles Bibliográficos
Autores principales: Aiumtrakul, Noppawit, Thongprayoon, Charat, Suppadungsuk, Supawadee, Krisanapan, Pajaree, Miao, Jing, Qureshi, Fawad, Cheungpasitporn, Wisit
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10608326/
https://www.ncbi.nlm.nih.gov/pubmed/37888068
http://dx.doi.org/10.3390/jpm13101457
_version_ 1785127753669935104
author Aiumtrakul, Noppawit
Thongprayoon, Charat
Suppadungsuk, Supawadee
Krisanapan, Pajaree
Miao, Jing
Qureshi, Fawad
Cheungpasitporn, Wisit
author_facet Aiumtrakul, Noppawit
Thongprayoon, Charat
Suppadungsuk, Supawadee
Krisanapan, Pajaree
Miao, Jing
Qureshi, Fawad
Cheungpasitporn, Wisit
author_sort Aiumtrakul, Noppawit
collection PubMed
description Background and Objectives: Literature reviews are foundational to understanding medical evidence. With AI tools like ChatGPT, Bing Chat and Bard AI emerging as potential aids in this domain, this study aimed to individually assess their citation accuracy within Nephrology, comparing their performance in providing precise. Materials and Methods: We generated the prompt to solicit 20 references in Vancouver style in each 12 Nephrology topics, using ChatGPT, Bing Chat and Bard. We verified the existence and accuracy of the provided references using PubMed, Google Scholar, and Web of Science. We categorized the validity of the references from the AI chatbot into (1) incomplete, (2) fabricated, (3) inaccurate, and (4) accurate. Results: A total of 199 (83%), 158 (66%) and 112 (47%) unique references were provided from ChatGPT, Bing Chat and Bard, respectively. ChatGPT provided 76 (38%) accurate, 82 (41%) inaccurate, 32 (16%) fabricated and 9 (5%) incomplete references. Bing Chat provided 47 (30%) accurate, 77 (49%) inaccurate, 21 (13%) fabricated and 13 (8%) incomplete references. In contrast, Bard provided 3 (3%) accurate, 26 (23%) inaccurate, 71 (63%) fabricated and 12 (11%) incomplete references. The most common error type across platforms was incorrect DOIs. Conclusions: In the field of medicine, the necessity for faultless adherence to research integrity is highlighted, asserting that even small errors cannot be tolerated. The outcomes of this investigation draw attention to inconsistent citation accuracy across the different AI tools evaluated. Despite some promising results, the discrepancies identified call for a cautious and rigorous vetting of AI-sourced references in medicine. Such chatbots, before becoming standard tools, need substantial refinements to assure unwavering precision in their outputs.
format Online
Article
Text
id pubmed-10608326
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106083262023-10-28 Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches Aiumtrakul, Noppawit Thongprayoon, Charat Suppadungsuk, Supawadee Krisanapan, Pajaree Miao, Jing Qureshi, Fawad Cheungpasitporn, Wisit J Pers Med Article Background and Objectives: Literature reviews are foundational to understanding medical evidence. With AI tools like ChatGPT, Bing Chat and Bard AI emerging as potential aids in this domain, this study aimed to individually assess their citation accuracy within Nephrology, comparing their performance in providing precise. Materials and Methods: We generated the prompt to solicit 20 references in Vancouver style in each 12 Nephrology topics, using ChatGPT, Bing Chat and Bard. We verified the existence and accuracy of the provided references using PubMed, Google Scholar, and Web of Science. We categorized the validity of the references from the AI chatbot into (1) incomplete, (2) fabricated, (3) inaccurate, and (4) accurate. Results: A total of 199 (83%), 158 (66%) and 112 (47%) unique references were provided from ChatGPT, Bing Chat and Bard, respectively. ChatGPT provided 76 (38%) accurate, 82 (41%) inaccurate, 32 (16%) fabricated and 9 (5%) incomplete references. Bing Chat provided 47 (30%) accurate, 77 (49%) inaccurate, 21 (13%) fabricated and 13 (8%) incomplete references. In contrast, Bard provided 3 (3%) accurate, 26 (23%) inaccurate, 71 (63%) fabricated and 12 (11%) incomplete references. The most common error type across platforms was incorrect DOIs. Conclusions: In the field of medicine, the necessity for faultless adherence to research integrity is highlighted, asserting that even small errors cannot be tolerated. The outcomes of this investigation draw attention to inconsistent citation accuracy across the different AI tools evaluated. Despite some promising results, the discrepancies identified call for a cautious and rigorous vetting of AI-sourced references in medicine. Such chatbots, before becoming standard tools, need substantial refinements to assure unwavering precision in their outputs. MDPI 2023-09-30 /pmc/articles/PMC10608326/ /pubmed/37888068 http://dx.doi.org/10.3390/jpm13101457 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Aiumtrakul, Noppawit
Thongprayoon, Charat
Suppadungsuk, Supawadee
Krisanapan, Pajaree
Miao, Jing
Qureshi, Fawad
Cheungpasitporn, Wisit
Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title_full Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title_fullStr Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title_full_unstemmed Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title_short Navigating the Landscape of Personalized Medicine: The Relevance of ChatGPT, BingChat, and Bard AI in Nephrology Literature Searches
title_sort navigating the landscape of personalized medicine: the relevance of chatgpt, bingchat, and bard ai in nephrology literature searches
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10608326/
https://www.ncbi.nlm.nih.gov/pubmed/37888068
http://dx.doi.org/10.3390/jpm13101457
work_keys_str_mv AT aiumtrakulnoppawit navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT thongprayooncharat navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT suppadungsuksupawadee navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT krisanapanpajaree navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT miaojing navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT qureshifawad navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches
AT cheungpasitpornwisit navigatingthelandscapeofpersonalizedmedicinetherelevanceofchatgptbingchatandbardaiinnephrologyliteraturesearches