Hand activity recognition using a wearable smart glove

Abstract

This project is aimed at designing, simulating and constructing a wearable device capable of performing activity recognition to track and monitor activities specific to the manufacturing industry. This was done by designing data capturing glove to capture all necessary signals from the human body and provide necessary filtering to obtain low noise data. This is then passed through suitable pre-processing algorithms to create distinguishing features between activities. The best suited classification and post-processing algorithms were then designed and implemented to classify the captured data in to a specified set of activities. The device was designed with an ESP8266 and a Raspberry Pi coded in C++ and Python respectively. Accelerometer & gyroscope sensors were used to collect data from the human body while a number of classical machine learning algorithms and convolutional neural networks were tested to classify the data. For the activities pointing, wiping, tightening, loosening, picking, holding, pulling, pushing, hammering, walking, holding and walking and turning, the system was capable of classifying the test data with accuracies between 86% - 91%. The null set was classified with an accuracy of 100% with support vector machines with a linear kernel and the post processing algorithm. The same algorithm reached an accuracy of 91.3% for the activity classification while the support vector machine with RBF kernel and post processing algorithm reached an accuracy of 89.7%. The convolutional neural network trained on pre-processed 3D activity images and the post processing algorithm reached an accuracy of 86.2%. The successfully created device will be used to obtain necessary analysis in the manufacturing space to optimize performance of the workers