Prototype for Peruvian Sign Language translation based on an artificial neural network approach

Mejia Gamarra J.E.; Alonso Salazar Cubas M.; Sosa Silupu J.D.; Enrique Cordova Chirinos C. · Proceedings of the 2020 IEEE 27th International Conference on Electronics, Electrical Engineering and Computing, INTERCON 2020 (2020) · DOI: 10.1109/intercon50315.2020.9220257

Resumen

This document aims to develop and demonstrate the operation of a prototype capable of recognizing the alphabet of the Peruvian Sign Language and translating it into the Spanish language alphabet, based on an optimized multilayer artificial neural network approach, with the purpose of improving the communication, fluency and reciprocity of a daily conversion with listeners who do not know sign language. Bend sensors were used to record the movement and variation of each letter. Then, through an Arduino-Excel interface, a database of a test user was compiled for training and validation, likewise, the neural network was developed in Matlab software for the recognition of signals (letters). Subsequently, the real-time data acquisition of the test user was performed through a Matlab-Arduino interface that, in conjunction with the online prediction algorithm and the neural network, recognized the desired letter. Finally, an artificial voice algorithm developed in the Matlab software was used to reproduce the recognized lyrics, and another, which allows the corresponding signal to be viewed online. The prototype developed has an accuracy of 94.60% for the training set and 94.32% for the validation set, in turn, the cost was 120 USD. It is concluded that this prototype is capable of translating Peruvian Sign Language with a high percentage of accuracy, also is viable and economical, so it is recommended to test the effectiveness of this prototype in multiple people with speech disabilities.

Instituciones: Universidad de Piura