Cargando…

Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation

Recent immense breakthroughs in generative models such as in GPT4 have precipitated re-imagined ubiquitous usage of these models in all applications. One area that can benefit by improvements in artificial intelligence (AI) is healthcare. The note generation task from doctor-patient encounters, and...

Descripción completa

Detalles Bibliográficos
Autores principales: Yim, Wen-wai, Fu, Yujuan, Ben Abacha, Asma, Snider, Neal, Lin, Thomas, Yetisgen, Meliha
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10482860/
https://www.ncbi.nlm.nih.gov/pubmed/37673893
http://dx.doi.org/10.1038/s41597-023-02487-3
_version_ 1785102264174641152
author Yim, Wen-wai
Fu, Yujuan
Ben Abacha, Asma
Snider, Neal
Lin, Thomas
Yetisgen, Meliha
author_facet Yim, Wen-wai
Fu, Yujuan
Ben Abacha, Asma
Snider, Neal
Lin, Thomas
Yetisgen, Meliha
author_sort Yim, Wen-wai
collection PubMed
description Recent immense breakthroughs in generative models such as in GPT4 have precipitated re-imagined ubiquitous usage of these models in all applications. One area that can benefit by improvements in artificial intelligence (AI) is healthcare. The note generation task from doctor-patient encounters, and its associated electronic medical record documentation, is one of the most arduous time-consuming tasks for physicians. It is also a natural prime potential beneficiary to advances in generative models. However with such advances, benchmarking is more critical than ever. Whether studying model weaknesses or developing new evaluation metrics, shared open datasets are an imperative part of understanding the current state-of-the-art. Unfortunately as clinic encounter conversations are not routinely recorded and are difficult to ethically share due to patient confidentiality, there are no sufficiently large clinic dialogue-note datasets to benchmark this task. Here we present the Ambient Clinical Intelligence Benchmark (aci-bench) corpus, the largest dataset to date tackling the problem of AI-assisted note generation from visit dialogue. We also present the benchmark performances of several common state-of-the-art approaches.
format Online
Article
Text
id pubmed-10482860
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-104828602023-09-08 Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation Yim, Wen-wai Fu, Yujuan Ben Abacha, Asma Snider, Neal Lin, Thomas Yetisgen, Meliha Sci Data Data Descriptor Recent immense breakthroughs in generative models such as in GPT4 have precipitated re-imagined ubiquitous usage of these models in all applications. One area that can benefit by improvements in artificial intelligence (AI) is healthcare. The note generation task from doctor-patient encounters, and its associated electronic medical record documentation, is one of the most arduous time-consuming tasks for physicians. It is also a natural prime potential beneficiary to advances in generative models. However with such advances, benchmarking is more critical than ever. Whether studying model weaknesses or developing new evaluation metrics, shared open datasets are an imperative part of understanding the current state-of-the-art. Unfortunately as clinic encounter conversations are not routinely recorded and are difficult to ethically share due to patient confidentiality, there are no sufficiently large clinic dialogue-note datasets to benchmark this task. Here we present the Ambient Clinical Intelligence Benchmark (aci-bench) corpus, the largest dataset to date tackling the problem of AI-assisted note generation from visit dialogue. We also present the benchmark performances of several common state-of-the-art approaches. Nature Publishing Group UK 2023-09-06 /pmc/articles/PMC10482860/ /pubmed/37673893 http://dx.doi.org/10.1038/s41597-023-02487-3 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Data Descriptor
Yim, Wen-wai
Fu, Yujuan
Ben Abacha, Asma
Snider, Neal
Lin, Thomas
Yetisgen, Meliha
Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title_full Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title_fullStr Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title_full_unstemmed Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title_short Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation
title_sort aci-bench: a novel ambient clinical intelligence dataset for benchmarking automatic visit note generation
topic Data Descriptor
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10482860/
https://www.ncbi.nlm.nih.gov/pubmed/37673893
http://dx.doi.org/10.1038/s41597-023-02487-3
work_keys_str_mv AT yimwenwai acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration
AT fuyujuan acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration
AT benabachaasma acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration
AT sniderneal acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration
AT linthomas acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration
AT yetisgenmeliha acibenchanovelambientclinicalintelligencedatasetforbenchmarkingautomaticvisitnotegeneration