Cargando…

Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning

Background: The primary care service in Catalonia has operated an asynchronous teleconsulting service between GPs and patients since 2015 (eConsulta), which has generated some 500,000 messages. New developments in big data analysis tools, particularly those involving natural language, can be used to...

Descripción completa

Detalles Bibliográficos
Autores principales: López Seguí, Francesc, Ander Egg Aguilar, Ricardo, de Maeztu, Gabriel, García-Altés, Anna, García Cuyàs, Francesc, Walsh, Sandra, Sagarra Castro, Marta, Vidal-Alaball, Josep
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7036927/
https://www.ncbi.nlm.nih.gov/pubmed/32050435
http://dx.doi.org/10.3390/ijerph17031093
_version_ 1783500308022820864
author López Seguí, Francesc
Ander Egg Aguilar, Ricardo
de Maeztu, Gabriel
García-Altés, Anna
García Cuyàs, Francesc
Walsh, Sandra
Sagarra Castro, Marta
Vidal-Alaball, Josep
author_facet López Seguí, Francesc
Ander Egg Aguilar, Ricardo
de Maeztu, Gabriel
García-Altés, Anna
García Cuyàs, Francesc
Walsh, Sandra
Sagarra Castro, Marta
Vidal-Alaball, Josep
author_sort López Seguí, Francesc
collection PubMed
description Background: The primary care service in Catalonia has operated an asynchronous teleconsulting service between GPs and patients since 2015 (eConsulta), which has generated some 500,000 messages. New developments in big data analysis tools, particularly those involving natural language, can be used to accurately and systematically evaluate the impact of the service. Objective: The study was intended to assess the predictive potential of eConsulta messages through different combinations of vector representation of text and machine learning algorithms and to evaluate their performance. Methodology: Twenty machine learning algorithms (based on five types of algorithms and four text representation techniques) were trained using a sample of 3559 messages (169,102 words) corresponding to 2268 teleconsultations (1.57 messages per teleconsultation) in order to predict the three variables of interest (avoiding the need for a face-to-face visit, increased demand and type of use of the teleconsultation). The performance of the various combinations was measured in terms of precision, sensitivity, F-value and the ROC curve. Results: The best-trained algorithms are generally effective, proving themselves to be more robust when approximating the two binary variables “avoiding the need of a face-to-face visit” and “increased demand” (precision = 0.98 and 0.97, respectively) rather than the variable “type of query” (precision = 0.48). Conclusion: To the best of our knowledge, this study is the first to investigate a machine learning strategy for text classification using primary care teleconsultation datasets. The study illustrates the possible capacities of text analysis using artificial intelligence. The development of a robust text classification tool could be feasible by validating it with more data, making it potentially more useful for decision support for health professionals.
format Online
Article
Text
id pubmed-7036927
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-70369272020-03-11 Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning López Seguí, Francesc Ander Egg Aguilar, Ricardo de Maeztu, Gabriel García-Altés, Anna García Cuyàs, Francesc Walsh, Sandra Sagarra Castro, Marta Vidal-Alaball, Josep Int J Environ Res Public Health Article Background: The primary care service in Catalonia has operated an asynchronous teleconsulting service between GPs and patients since 2015 (eConsulta), which has generated some 500,000 messages. New developments in big data analysis tools, particularly those involving natural language, can be used to accurately and systematically evaluate the impact of the service. Objective: The study was intended to assess the predictive potential of eConsulta messages through different combinations of vector representation of text and machine learning algorithms and to evaluate their performance. Methodology: Twenty machine learning algorithms (based on five types of algorithms and four text representation techniques) were trained using a sample of 3559 messages (169,102 words) corresponding to 2268 teleconsultations (1.57 messages per teleconsultation) in order to predict the three variables of interest (avoiding the need for a face-to-face visit, increased demand and type of use of the teleconsultation). The performance of the various combinations was measured in terms of precision, sensitivity, F-value and the ROC curve. Results: The best-trained algorithms are generally effective, proving themselves to be more robust when approximating the two binary variables “avoiding the need of a face-to-face visit” and “increased demand” (precision = 0.98 and 0.97, respectively) rather than the variable “type of query” (precision = 0.48). Conclusion: To the best of our knowledge, this study is the first to investigate a machine learning strategy for text classification using primary care teleconsultation datasets. The study illustrates the possible capacities of text analysis using artificial intelligence. The development of a robust text classification tool could be feasible by validating it with more data, making it potentially more useful for decision support for health professionals. MDPI 2020-02-09 2020-02 /pmc/articles/PMC7036927/ /pubmed/32050435 http://dx.doi.org/10.3390/ijerph17031093 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
López Seguí, Francesc
Ander Egg Aguilar, Ricardo
de Maeztu, Gabriel
García-Altés, Anna
García Cuyàs, Francesc
Walsh, Sandra
Sagarra Castro, Marta
Vidal-Alaball, Josep
Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title_full Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title_fullStr Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title_full_unstemmed Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title_short Teleconsultations between Patients and Healthcare Professionals in Primary Care in Catalonia: The Evaluation of Text Classification Algorithms Using Supervised Machine Learning
title_sort teleconsultations between patients and healthcare professionals in primary care in catalonia: the evaluation of text classification algorithms using supervised machine learning
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7036927/
https://www.ncbi.nlm.nih.gov/pubmed/32050435
http://dx.doi.org/10.3390/ijerph17031093
work_keys_str_mv AT lopezseguifrancesc teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT andereggaguilarricardo teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT demaeztugabriel teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT garciaaltesanna teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT garciacuyasfrancesc teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT walshsandra teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT sagarracastromarta teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning
AT vidalalaballjosep teleconsultationsbetweenpatientsandhealthcareprofessionalsinprimarycareincataloniatheevaluationoftextclassificationalgorithmsusingsupervisedmachinelearning