Smart glove for recognition of sinhala sign language

Abstract

Speech and hearing-impaired people used sign language to communicate with each other. Sign languages are made of gestures. The language consists of different gestures instead of letters or words. The purpose of this research work is to reduce the communication gap between normal people and hearing and speech impaired people. The research incorporates a system comprising of a glove-based mechanism, consisting of sensors to recognize the hand gestures for Sinhala sign language (SSL) alphabet. The solution combines electronics, sensors, embedded systems, machine learning algorithms, and natural language processing. The research based on a data glove with flex sensors that measure finger bending and an Inertial Measurement Unit (IMU) to recognise palm-turning gestures of the alphabet. Further, sample data with eleven independent variables and hundred data samples per gesture was used for the purpose. In the proposed system, data is trained and classified using Random Forest machine learning algorithm. And natural language processing (NLP) is completed using a newly developed Application Programming Interface (API) to make Sinhala words. The results show that the proposed algorithm has a better recognition effect on gestures, and is capable of making words and sentences. The accuracy of the model on the prepared dataset was founded as 99% for the target user with regard to random forest classification. Complete training for all possible combination of letters and preparation of words is necessary to continue NLP. Also, the system can customise as an education platform for sign language learners. Further, the developed smart glove can use separately for any other hand gesture base applications, the developed ML base system can use or customize separately for feature extraction of any smart wearable item, and finally, the newly developed Sinhala API can use separately for any Sinhala sign language base NLP research work.