Cargando…

Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network

The deaf-mutes population always feels helpless when they are not understood by others and vice versa. This is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to re...

Descripción completa

Detalles Bibliográficos
Autores principales: Khan, Rehman Ullah, Khattak, Hizbullah, Wong, Woei Sheng, AlSalman, Hussain, Mosleh, Mogeeb A. A., Mizanur Rahman, Sk. Md.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Hindawi 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8683193/
https://www.ncbi.nlm.nih.gov/pubmed/34925497
http://dx.doi.org/10.1155/2021/9023010
_version_ 1784617360159670272
author Khan, Rehman Ullah
Khattak, Hizbullah
Wong, Woei Sheng
AlSalman, Hussain
Mosleh, Mogeeb A. A.
Mizanur Rahman, Sk. Md.
author_facet Khan, Rehman Ullah
Khattak, Hizbullah
Wong, Woei Sheng
AlSalman, Hussain
Mosleh, Mogeeb A. A.
Mizanur Rahman, Sk. Md.
author_sort Khan, Rehman Ullah
collection PubMed
description The deaf-mutes population always feels helpless when they are not understood by others and vice versa. This is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to recognise Malaysian Sign Language (MSL) from images. Two different experiments were conducted for MSL signs, using CBAM-2DResNet (2-Dimensional Residual Network) implementing “Within Blocks” and “Before Classifier” methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time are recorded to evaluate the models' efficiency. The experimental results showed that CBAM-ResNet models achieved a good performance in MSL signs recognition tasks, with accuracy rates of over 90% through a little of variations. The CBAM-ResNet “Before Classifier” models are more efficient than “Within Blocks” CBAM-ResNet models. Thus, the best trained model of CBAM-2DResNet is chosen to develop a real-time sign recognition system for translating from sign language to text and from text to sign language in an easy way of communication between deaf-mutes and other people. All experiment results indicated that the “Before Classifier” of CBAMResNet models is more efficient in recognising MSL and it is worth for future research.
format Online
Article
Text
id pubmed-8683193
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Hindawi
record_format MEDLINE/PubMed
spelling pubmed-86831932021-12-18 Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network Khan, Rehman Ullah Khattak, Hizbullah Wong, Woei Sheng AlSalman, Hussain Mosleh, Mogeeb A. A. Mizanur Rahman, Sk. Md. Comput Intell Neurosci Research Article The deaf-mutes population always feels helpless when they are not understood by others and vice versa. This is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to recognise Malaysian Sign Language (MSL) from images. Two different experiments were conducted for MSL signs, using CBAM-2DResNet (2-Dimensional Residual Network) implementing “Within Blocks” and “Before Classifier” methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time are recorded to evaluate the models' efficiency. The experimental results showed that CBAM-ResNet models achieved a good performance in MSL signs recognition tasks, with accuracy rates of over 90% through a little of variations. The CBAM-ResNet “Before Classifier” models are more efficient than “Within Blocks” CBAM-ResNet models. Thus, the best trained model of CBAM-2DResNet is chosen to develop a real-time sign recognition system for translating from sign language to text and from text to sign language in an easy way of communication between deaf-mutes and other people. All experiment results indicated that the “Before Classifier” of CBAMResNet models is more efficient in recognising MSL and it is worth for future research. Hindawi 2021-12-10 /pmc/articles/PMC8683193/ /pubmed/34925497 http://dx.doi.org/10.1155/2021/9023010 Text en Copyright © 2021 Rehman Ullah Khan et al. https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
spellingShingle Research Article
Khan, Rehman Ullah
Khattak, Hizbullah
Wong, Woei Sheng
AlSalman, Hussain
Mosleh, Mogeeb A. A.
Mizanur Rahman, Sk. Md.
Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title_full Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title_fullStr Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title_full_unstemmed Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title_short Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network
title_sort intelligent malaysian sign language translation system using convolutional-based attention module with residual network
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8683193/
https://www.ncbi.nlm.nih.gov/pubmed/34925497
http://dx.doi.org/10.1155/2021/9023010
work_keys_str_mv AT khanrehmanullah intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork
AT khattakhizbullah intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork
AT wongwoeisheng intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork
AT alsalmanhussain intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork
AT moslehmogeebaa intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork
AT mizanurrahmanskmd intelligentmalaysiansignlanguagetranslationsystemusingconvolutionalbasedattentionmodulewithresidualnetwork