Leveraging machine learning techniques to optimally support exoskeleton users

As I am nearing the end of my first year at Panenco, I started to look back on my journey from a mechanical engineer to a software engineer. During my time as a researcher at the robotics research group of KU Leuven, I discovered my interest in software development. In this blogpost, I’ll briefly touch upon some of the work I did during my PhD and highlight some parallels between my work as robotics researcher and software engineer.

Exoskeletons to prevent injuries

Over half of all workers in the EU experience pain in the lower back, shoulder or neck at some point in their career. Besides the discomfort for the workers themselves, this also incurs societal costs due to medical expenses and productivity losses in case workers need to stay absent from work for a while. The root cause of these injuries is repeated lifting of heavy objects. One approach to prevent such injuries would be to simply replace all humans with robots on the workfloor. Although this sounds like a good idea, it turns out that humans are still far superior to robots for tasks that are not strictly repetitive. On the other hand, exoskeletons can leverage the flexibility of humans while still preventing injuries due to lifting.

Exoskeletons are suits that provide support to workers during their day to day working routines, such as the example prototypes shown in the figure below. State-of-the-art exoskeleton prototypes contain motors that are capable of generating the right amount of support at the right time. However, this raises the following questions: When do workers need support? And if they do, how much support do they require? Due to the nature of their work, it is practically infeasible for workers to manually indicate when support is required. Therefore, the exoskeleton needs to automatically infer what a user is doing through the available sensors. 

‍

Recognizing human motion

Exoskeletons contain only the strictly required electronics to remain mobile and lightweight, making it challenging to infer the required support from the limited available sensor data.  We therefore developed motion recognition algorithms that rely on only a handful of sensors. The algorithms are designed such that they rely on simple AI models requiring limited amounts of motion data for the recognition to work. This is an important feature as it is impossible to record hours of motion data at each factory or work station where end-users might want to work with the exoskeletons.

Acting on the output of recognition algorithms

Apart from merely recognizing motions, the exoskeleton also needs to act on the recognition output from the algorithms. This again raises important questions. For example, if the algorithm recognizes a motion with 80% certainty, is this then enough to start providing support? If there is a 20% change that our algorithm is wrong, what could be the consequence if we would generate support anyway? Would it hurt a user? I leveraged ‘decision theory’ to make decisions that would minimise hindrance or discomfort for users while maximising the potential of the exoskeletons to reduce loads.

Evaluating motion recognition algorithms

At Panenco, we tend to use the mantra ‘in production or it didn't happen’, meaning that we need to push features all the way through to avoid that they end up forgotten and abandoned. For research related to exoskeletons we could similarly say ‘evaluated or it didn't happen’. As it is nearly impossible to anticipate how users react to support provided by exoskeletons, it is crucial to evaluate the proposed algorithms with actual exoskeleton prototypes on users.  To this end, the algorithms described above were modified to work on a hip support exoskeleton (shown in the figure above) and a shoulder support exoskeleton (not shown due to IP reasons).

I evaluated the algorithms with several subjects who had no prior experience with any of these exoskeleton prototypes. The motion recognition algorithms achieved up to 90% or more recognition accuracy and the exoskeleton reacted in a timely manner in over 88% of cases. Although there is still room for improvement, this shows the potential of leveraging simple machine learning models.

Looking back 

Evaluating all of these algorithms on actual users requires that the written code is robust and well tested. During my research, I did most of my tests manually. This was feasible because I was the only developer and only user of the prototypes. However, as a software engineer, I learned how to systematically test as well as automatically running tests with every deployment. Setting up such a proper CI pipeline would have saved me a lot of time while figuring out why a particular change broke a different aspect of the algorithms. 

Additionally, I recorded quite some motion data, which were stored and backed up locally. However, there are time series databases, such as InfluxDb, that can be run locally as a docker container. Using such tools could have made data storage, management and processing easier, while at the same time maintaining performance. In particular, this could have helped while fine tuning the algorithms. 

Closing statement

It is during the development and evaluation phase of my research that I discovered my interest in writing robust, clean and well-structured code. In the coming years, I intend to further deepen my knowledge of software architecture and design patterns. With the rise of Artificial Intelligence, I hope to combine my experience in robotics with the expertise available at Panenco to create innovative products that leverage the latest technologies.