IMU Data Deep Learning for Stroke Rehabilitation

NSERC Undergraduate Summer Research Awards
Microfluidics and BioMEMS Laboratory

Project Overview

Stroke can cause permanent damage, including partial paralysis and impairment in speech, comprehension and memory. Globally 1 in 4 adults over the age of 25 will have a stroke in their lifetime (American Stroke Association). To make stroke rehabilitation services more accessible I developed an deep learning model to classify movement data collected from IMUs.

My Contributions

I worked with Zhanfeng Zhou and Prof. Liu in the Microfluidics and BioMEMS Lab in UofT in Summer 2023. I helped collect data and fabricate the soft IMUs developed by Zhanfeng. I coded the programs for signal filtering, data segmentation, the LSTM model and trained the model on data we collected. The final accuracy among 8 different upper extremity movements was 92.85%.

Data Collection

Data is collected from 3 IMUs placed on the arm. Each sensor collects gyroscopic and linear acceleration data in three axes and the sampling frequency was 20 Hz. The 8 movements data was collected for were:
‍- extension/ flexion of the wrist;
- supination/pronation of the forearm;
- extension/ flexion of the elbow;
- extension/ flexion of the shoulder;
- ab/adduction of the shoulder;
- circumduction of the shoulder;
- horizontal ab/adduction of the shoulder;
- Lateral rotation of the shoulder
‍

Data Processing

Linear acceleration data was filtered with a Kalman filter and gyro data was filtered with a low-pass filter. Different filters were used because of the different hardware components and nature of the different units.

Data segmentation was implemented by looking for areas with more movement. I coded an algorithm to look for these segments among six different concurrent time series data (3 gyro, 3 linear acceleration). Each segment can be seen in the figure to the left with the dotted red lines. The figure below shows a single segment of the data.

Each segment (containing 6 IMU data for 3 sensors = 18 data series) was then labelled and used as input to train the LSTM model.

Deep Learning Model

I chose to use an LSTM model because of the nature of the time series data. LSTMs are capable of handling variable input sizes and the segments had differing times. In addition LSTMs learn the features of the data contextually because of the "long-term" and "short-term" memory states embedded in the model's architecture.

I tested several different hyperparameters of this model. For 100 hidden units and 3 layers, the model was able to achieve 92.85% while minimizing overfitting when compared to models with fewer layers. The LSTM was able to consistently perform better than a baseline RNN model for most hyperparameter settings.

I used PyTorch to code the LSTM and RNN models. The figure to the left on the top is the training curve for the final LSTM model. The bottom figure shows the architecture for one hidden unit.

I'd like to thank Zhanfeng Zhou and Prof. Liu for giving me the opportunity to work on such an amazing project this summer! I learned a lot and developed a greater passion for AI applications!