Cargando…

Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features

Malnutrition, including both undernutrition and obesity, is a significant problem in low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and communi...

Descripción completa

Detalles Bibliográficos
Autores principales: Chen, Guangzong, Jia, Wenyan, Zhao, Yifan, Mao, Zhi-Hong, Lo, Benny, Anderson, Alex K., Frost, Gary, Jobarteh, Modou L., McCrory, Megan A., Sazonov, Edward, Steiner-Asiedu, Matilda, Ansong, Richard S., Baranowski, Thomas, Burke, Lora, Sun, Mingui
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8047062/
https://www.ncbi.nlm.nih.gov/pubmed/33870184
http://dx.doi.org/10.3389/frai.2021.644712
_version_ 1783678967210835968
author Chen, Guangzong
Jia, Wenyan
Zhao, Yifan
Mao, Zhi-Hong
Lo, Benny
Anderson, Alex K.
Frost, Gary
Jobarteh, Modou L.
McCrory, Megan A.
Sazonov, Edward
Steiner-Asiedu, Matilda
Ansong, Richard S.
Baranowski, Thomas
Burke, Lora
Sun, Mingui
author_facet Chen, Guangzong
Jia, Wenyan
Zhao, Yifan
Mao, Zhi-Hong
Lo, Benny
Anderson, Alex K.
Frost, Gary
Jobarteh, Modou L.
McCrory, Megan A.
Sazonov, Edward
Steiner-Asiedu, Matilda
Ansong, Richard S.
Baranowski, Thomas
Burke, Lora
Sun, Mingui
author_sort Chen, Guangzong
collection PubMed
description Malnutrition, including both undernutrition and obesity, is a significant problem in low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and community levels. In a multinational research project supported by the Bill & Melinda Gates Foundation, we have been using a wearable technology to conduct objective dietary assessment in sub-Saharan Africa. Our assessment includes multiple diet-related activities in urban and rural families, including food sources (e.g., shopping, harvesting, and gathering), preservation/storage, preparation, cooking, and consumption (e.g., portion size and nutrition analysis). Our wearable device (“eButton” worn on the chest) acquires real-life images automatically during wake hours at preset time intervals. The recorded images, in amounts of tens of thousands per day, are post-processed to obtain the information of interest. Although we expect future Artificial Intelligence (AI) technology to extract the information automatically, at present we utilize AI to separate the acquired images into two binary classes: images with (Class 1) and without (Class 0) edible items. As a result, researchers need only to study Class-1 images, reducing their workload significantly. In this paper, we present a composite machine learning method to perform this classification, meeting the specific challenges of high complexity and diversity in the real-world LMIC data. Our method consists of a deep neural network (DNN) and a shallow learning network (SLN) connected by a novel probabilistic network interface layer. After presenting the details of our method, an image dataset acquired from Ghana is utilized to train and evaluate the machine learning system. Our comparative experiment indicates that the new composite method performs better than the conventional deep learning method assessed by integrated measures of sensitivity, specificity, and burden index, as indicated by the Receiver Operating Characteristic (ROC) curve.
format Online
Article
Text
id pubmed-8047062
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-80470622021-04-16 Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features Chen, Guangzong Jia, Wenyan Zhao, Yifan Mao, Zhi-Hong Lo, Benny Anderson, Alex K. Frost, Gary Jobarteh, Modou L. McCrory, Megan A. Sazonov, Edward Steiner-Asiedu, Matilda Ansong, Richard S. Baranowski, Thomas Burke, Lora Sun, Mingui Front Artif Intell Artificial Intelligence Malnutrition, including both undernutrition and obesity, is a significant problem in low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and community levels. In a multinational research project supported by the Bill & Melinda Gates Foundation, we have been using a wearable technology to conduct objective dietary assessment in sub-Saharan Africa. Our assessment includes multiple diet-related activities in urban and rural families, including food sources (e.g., shopping, harvesting, and gathering), preservation/storage, preparation, cooking, and consumption (e.g., portion size and nutrition analysis). Our wearable device (“eButton” worn on the chest) acquires real-life images automatically during wake hours at preset time intervals. The recorded images, in amounts of tens of thousands per day, are post-processed to obtain the information of interest. Although we expect future Artificial Intelligence (AI) technology to extract the information automatically, at present we utilize AI to separate the acquired images into two binary classes: images with (Class 1) and without (Class 0) edible items. As a result, researchers need only to study Class-1 images, reducing their workload significantly. In this paper, we present a composite machine learning method to perform this classification, meeting the specific challenges of high complexity and diversity in the real-world LMIC data. Our method consists of a deep neural network (DNN) and a shallow learning network (SLN) connected by a novel probabilistic network interface layer. After presenting the details of our method, an image dataset acquired from Ghana is utilized to train and evaluate the machine learning system. Our comparative experiment indicates that the new composite method performs better than the conventional deep learning method assessed by integrated measures of sensitivity, specificity, and burden index, as indicated by the Receiver Operating Characteristic (ROC) curve. Frontiers Media S.A. 2021-04-01 /pmc/articles/PMC8047062/ /pubmed/33870184 http://dx.doi.org/10.3389/frai.2021.644712 Text en Copyright © 2021 Chen, Jia, Zhao, Mao, Lo, Anderson, Frost, Jobarteh, McCrory, Sazonov, Steiner-Asiedu, Ansong, Baranowski, Burke and Sun. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Artificial Intelligence
Chen, Guangzong
Jia, Wenyan
Zhao, Yifan
Mao, Zhi-Hong
Lo, Benny
Anderson, Alex K.
Frost, Gary
Jobarteh, Modou L.
McCrory, Megan A.
Sazonov, Edward
Steiner-Asiedu, Matilda
Ansong, Richard S.
Baranowski, Thomas
Burke, Lora
Sun, Mingui
Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title_full Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title_fullStr Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title_full_unstemmed Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title_short Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features
title_sort food/non-food classification of real-life egocentric images in low- and middle-income countries based on image tagging features
topic Artificial Intelligence
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8047062/
https://www.ncbi.nlm.nih.gov/pubmed/33870184
http://dx.doi.org/10.3389/frai.2021.644712
work_keys_str_mv AT chenguangzong foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT jiawenyan foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT zhaoyifan foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT maozhihong foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT lobenny foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT andersonalexk foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT frostgary foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT jobartehmodoul foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT mccrorymegana foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT sazonovedward foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT steinerasiedumatilda foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT ansongrichards foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT baranowskithomas foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT burkelora foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures
AT sunmingui foodnonfoodclassificationofreallifeegocentricimagesinlowandmiddleincomecountriesbasedonimagetaggingfeatures