Boosting Functional Test Evaluation with Camera-Based System and Machine Learning

By Dr Silpaja Chandrasekar, PhDReviewed by Susha Cheriyedath, M.Sc.Nov 8 2023

In a paper published in the journal PLOS ONE, researchers aimed to evaluate the effectiveness of functional tests using a camera-based system and machine learning techniques. Specifically, their investigation focused on determining whether OpenPose, in combination with a standard camera, could be employed to assess the quality of functional tests such as the Single Leg Squat Test and Step Down Test.

The study involved recording these exercises as performed by 46 healthy subjects, followed by the extraction of motion data. This data was then classified based on assessments provided by three independent physiotherapists, utilizing 15 binary parameters. The results suggested that the AdaBoost classifier effectively used a camera-based approach to evaluate functional test performance.

Background

As the demand for physical therapy grows, there's a need for efficient and cost-effective evaluation methods. Subjectivity often limits traditional visual observation by physiotherapists, while researchers have widely utilized marker-based motion capture systems for precise data collection. Still, recent technologies like OpenPose enable the assessment of functional tests using standard cameras, enhancing efficiency and reducing healthcare resource requirements.

Traditional functional test evaluation often relies on subjective assessments by physiotherapists. Previous research has explored complex motion capture systems, but markerless systems like OpenPose offer cost-effective alternatives. These systems utilize deep learning to detect and evaluate movement patterns, reducing the dependency on specialized equipment.

Validating Camera-Based Functional Test Detection

The methods aimed to validate the hypothesis that a camera-based system could accurately detect the correct execution of functional tests. The participant pool consisted of 46 individuals, representing both men and women, all healthy volunteers. These subjects underwent two functional tests, the single-leg squat Test (SLST) and the Step-Down Test (SDT), which three physiotherapists assessed. The physiotherapists' evaluations provided valuable reference data.

Researchers employed the OpenPose software to capture video recordings with a standard RGB camera, extracting anatomical body landmarks and deriving various features. These features, including joint angles, Keypoint-pair orientation, and relative distances, were calculated from the extracted data.

This approach ensured robust classification, allowing machine learning algorithms to learn from expert assessments and autonomously analyze and measure subjects. This process served as a cost-effective and convenient alternative to traditional evaluation methods, particularly important for simplifying the assessment of functional tests. The results obtained from the expert evaluations provided a crucial reference standard for the classification algorithms. While the focus was not on bringing precise biomechanical descriptions, the goal was to distinguish the presence or absence of evaluated parameters and determine if these parameters contain information regarding the correctness of execution, which would be helpful for machine learning algorithms.

During the classification phase, the researchers addressed six binary research questions related to exercise execution phenomena. They efficiently trained the machine learning model using the PyCaret Python library. AdaBoost, an ensemble learning method, was chosen to improve binary classification. Through iterative processes, the researchers corrected errors in weak classifiers, resulting in a transparent and replicable experimental setup. They assessed the classifier's performance with training, validation, and testing data to ensure reliability. This approach aimed to answer research questions clearly while maintaining transparency and reproducibility.

Quality Assessment and Classification Results

The agreement of expert visual analysis plays a pivotal role in ensuring the quality and reliability of the data for classification. Fleiss kappa, a widely accepted measure of agreement, is employed to evaluate the consensus among three independent PTs. Only categories demonstrating substantial agreement, reflected by Fleis’s kappa coefficient greater than 0.41, are selected for classification. This stringent selection criterion enhances the robustness of the data utilized for machine learning and prevents the inclusion of unreliable or inconsistent results. The research maintains a strong foundation for the subsequent classification process by adhering to established procedures and rigorous quality control measures.

The classification results reveal the method's potential, demonstrating that when at least two raters agree on error presence and ample data is available, the system based on a single 2D camera achieves results comparable to human raters. Furthermore, it highlights the challenges posed by an imbalanced dataset due to the inclusion of healthy subjects only. Despite this imbalance, the focus on sensitivity and specificity underscores the method's effectiveness in detecting errors among subjects with remarkable reliability.

The AdaBoost Classification algorithm contributes to the success of this approach by leveraging the expertise of multiple weak classifiers, refining their performance over time, and ultimately generating a robust and generalizable model suitable for handling novel, unobserved data.

Conclusion

In summary, the cost-effective approach, combining computer vision and machine learning with expert physiotherapist input, offers a precise and efficient method for assessing functional tests and detecting errors during performance. It does not replace physiotherapists but complements their work. With an average accuracy of 0.77, this method can enhance the diagnosis and treatment of movement disorders and increase the efficiency of test measurement and evaluation, expanding the toolkit available to physiotherapists and improving evaluation accuracy and reliability.