Deep-Learning Based Automatic Ergonomic Assessment Using Webcam Data

Abstract

Primarily due to increasing computer use, people are spending more and more time sitting in front of a desk every day. However, prolonged sitting has been associated with tiredness, hypertension, and pain in areas like the lower back or shoulders. These symptoms arise for a variety of reasons, but musculoskeletal disorders in particular are largely associated with poor postures. The adverse results caused by poor postures can be controlled with proper training and monitoring. This study attempts to provide automatic ergonomic assessment using only webcam data. Since laptops, desktops, and phones are now widely available and equipped with built-in cameras, this solution is accessible and convenient for most people. More importantly, automatic posture assessment may help to prevent conditions associated with poor posture by giving reminders whenever improper posture occurs. To create our model, we make use of Mediapipe, which provides a solution to identifying keypoint locations from an image. By training our MLP classifier on this key-point data, we achieved a 96.96% test F1 score, indicating that our system serves as a convenient way to assess posture while maintaining high performance. To illustrate our results, we perform a final video classification by overlaying the model’s pre-dictions on each frame.