Static Peruvian Sign Language Classifier Based on Manual Spelling Using a Convolutional Neural Network

Portocarrero-Banda G.; Castro-Gutierrez E.G.; Portocarrero-Banda A.A.; Acra-Despradel C.; Rondon D.; Jimenez-Pacheco H.G.; Ortiz-Esparza M.A. · CEUR Workshop Proceedings (2023)

Resumen

There are great difficulties for people who suffer from mixed language disorders, having only one means for interpretive communication, sign language. A great challenge is to efficiently recognize these static gestures in real environments, therefore, the present research presents a convolutional neural network model that allows recognizing and classifying Peruvian Sign Language (PSL) with a dataset of 3025 frames through 4 stages: a) Generation of a dataset that involves 11 gestural components per frame, which involve invariant characteristics, sign parameters and gestural space, which allows greater generalization of the model compared to samples from other research b) Image preprocessing through the application of techniques and computer vision algorithms, c) Application of a convolutional neural network (CNN) model architecture and d) Execution of the model on a web platform to support model testing. The proposed CNN model obtained an accuracy rate of 99% in training, 88% in validation, and 84% in PSL recognition in the testing stage. The present model is better prepared to recognize static signs of the PSL in real scenarios.

Instituciones: Universidad Nacional de San Agustín de Arequipa; Universidad Continental; Universidad Católica de Santa María