Cargando…

A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study

BACKGROUND: Numerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals. OBJECTIVE: We aimed to develop machine learning (ML)–based models to predict th...

Descripción completa

Detalles Bibliográficos
Autores principales: Wang, Jun, Chen, Hongmei, Wang, Houwei, Liu, Weichu, Peng, Daomei, Zhao, Qinghua, Xiao, Mingzhao
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10131772/
https://www.ncbi.nlm.nih.gov/pubmed/37023416
http://dx.doi.org/10.2196/43815
_version_ 1785031249954340864
author Wang, Jun
Chen, Hongmei
Wang, Houwei
Liu, Weichu
Peng, Daomei
Zhao, Qinghua
Xiao, Mingzhao
author_facet Wang, Jun
Chen, Hongmei
Wang, Houwei
Liu, Weichu
Peng, Daomei
Zhao, Qinghua
Xiao, Mingzhao
author_sort Wang, Jun
collection PubMed
description BACKGROUND: Numerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals. OBJECTIVE: We aimed to develop machine learning (ML)–based models to predict the risk of PR in older adults. METHODS: This study conducted a cross-sectional secondary data analysis based on 1026 older adults from 6 long-term care facilities in Chongqing, China, from July 2019 to November 2019. The primary outcome was the use of PR (yes or no), identified by 2 collectors’ direct observation. A total of 15 candidate predictors (older adults’ demographic and clinical factors) that could be commonly and easily collected from clinical practice were used to build 9 independent ML models: Gaussian Naïve Bayesian (GNB), k-nearest neighbor (KNN), decision tree (DT), logistic regression (LR), support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and light gradient boosting machine (Lightgbm), as well as stacking ensemble ML. Performance was evaluated using accuracy, precision, recall, an F score, a comprehensive evaluation indicator (CEI) weighed by the above indicators, and the area under the receiver operating characteristic curve (AUC). A net benefit approach using the decision curve analysis (DCA) was performed to evaluate the clinical utility of the best model. Models were tested via 10-fold cross-validation. Feature importance was interpreted using Shapley Additive Explanations (SHAP). RESULTS: A total of 1026 older adults (mean 83.5, SD 7.6 years; n=586, 57.1% male older adults) and 265 restrained older adults were included in the study. All ML models performed well, with an AUC above 0.905 and an F score above 0.900. The 2 best independent models are RF (AUC 0.938, 95% CI 0.914-0.947) and SVM (AUC 0.949, 95% CI 0.911-0.953). The DCA demonstrated that the RF model displayed better clinical utility than other models. The stacking model combined with SVM, RF, and MLP performed best with AUC (0.950) and CEI (0.943) values, as well as the DCA curve indicated the best clinical utility. The SHAP plots demonstrated that the significant contributors to model performance were related to cognitive impairment, care dependency, mobility decline, physical agitation, and an indwelling tube. CONCLUSIONS: The RF and stacking models had high performance and clinical utility. ML prediction models for predicting the probability of PR in older adults could offer clinical screening and decision support, which could help medical staff in the early identification and PR management of older adults.
format Online
Article
Text
id pubmed-10131772
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-101317722023-04-27 A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study Wang, Jun Chen, Hongmei Wang, Houwei Liu, Weichu Peng, Daomei Zhao, Qinghua Xiao, Mingzhao J Med Internet Res Original Paper BACKGROUND: Numerous studies have identified risk factors for physical restraint (PR) use in older adults in long-term care facilities. Nevertheless, there is a lack of predictive tools to identify high-risk individuals. OBJECTIVE: We aimed to develop machine learning (ML)–based models to predict the risk of PR in older adults. METHODS: This study conducted a cross-sectional secondary data analysis based on 1026 older adults from 6 long-term care facilities in Chongqing, China, from July 2019 to November 2019. The primary outcome was the use of PR (yes or no), identified by 2 collectors’ direct observation. A total of 15 candidate predictors (older adults’ demographic and clinical factors) that could be commonly and easily collected from clinical practice were used to build 9 independent ML models: Gaussian Naïve Bayesian (GNB), k-nearest neighbor (KNN), decision tree (DT), logistic regression (LR), support vector machine (SVM), random forest (RF), multilayer perceptron (MLP), extreme gradient boosting (XGBoost), and light gradient boosting machine (Lightgbm), as well as stacking ensemble ML. Performance was evaluated using accuracy, precision, recall, an F score, a comprehensive evaluation indicator (CEI) weighed by the above indicators, and the area under the receiver operating characteristic curve (AUC). A net benefit approach using the decision curve analysis (DCA) was performed to evaluate the clinical utility of the best model. Models were tested via 10-fold cross-validation. Feature importance was interpreted using Shapley Additive Explanations (SHAP). RESULTS: A total of 1026 older adults (mean 83.5, SD 7.6 years; n=586, 57.1% male older adults) and 265 restrained older adults were included in the study. All ML models performed well, with an AUC above 0.905 and an F score above 0.900. The 2 best independent models are RF (AUC 0.938, 95% CI 0.914-0.947) and SVM (AUC 0.949, 95% CI 0.911-0.953). The DCA demonstrated that the RF model displayed better clinical utility than other models. The stacking model combined with SVM, RF, and MLP performed best with AUC (0.950) and CEI (0.943) values, as well as the DCA curve indicated the best clinical utility. The SHAP plots demonstrated that the significant contributors to model performance were related to cognitive impairment, care dependency, mobility decline, physical agitation, and an indwelling tube. CONCLUSIONS: The RF and stacking models had high performance and clinical utility. ML prediction models for predicting the probability of PR in older adults could offer clinical screening and decision support, which could help medical staff in the early identification and PR management of older adults. JMIR Publications 2023-04-06 /pmc/articles/PMC10131772/ /pubmed/37023416 http://dx.doi.org/10.2196/43815 Text en ©Jun Wang, Hongmei Chen, Houwei Wang, Weichu Liu, Daomei Peng, Qinghua Zhao, Mingzhao Xiao. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 06.04.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Wang, Jun
Chen, Hongmei
Wang, Houwei
Liu, Weichu
Peng, Daomei
Zhao, Qinghua
Xiao, Mingzhao
A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title_full A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title_fullStr A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title_full_unstemmed A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title_short A Risk Prediction Model for Physical Restraints Among Older Chinese Adults in Long-term Care Facilities: Machine Learning Study
title_sort risk prediction model for physical restraints among older chinese adults in long-term care facilities: machine learning study
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10131772/
https://www.ncbi.nlm.nih.gov/pubmed/37023416
http://dx.doi.org/10.2196/43815
work_keys_str_mv AT wangjun ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT chenhongmei ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT wanghouwei ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT liuweichu ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT pengdaomei ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT zhaoqinghua ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT xiaomingzhao ariskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT wangjun riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT chenhongmei riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT wanghouwei riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT liuweichu riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT pengdaomei riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT zhaoqinghua riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy
AT xiaomingzhao riskpredictionmodelforphysicalrestraintsamongolderchineseadultsinlongtermcarefacilitiesmachinelearningstudy