Advanced video analytic system for posture and activity recognition: leveraging MediaPipe, CNN-LSTM, and ensemble learning for fall and unstable motion detection

Abstract

Human posture detection and classification are vital in monitoring activities, especially in health and safety contexts, such as fall detection in elderly care. This thesis presents a comparative study of two machine learning approaches for real-time human posture calssification using real time video data, a traditional feature-based approach using a Voting Classifier, and a deep learning appraoch utilizing a Convolutional Neural Network-Long_Short-Term memory (CNN-LSTM) model. The feature-based method incorporates pose estimation using MediaPipe to extract human body landmarks, followed by classification using an ensemble of Rnadom Forest, Support Vector Machine (SVM), and K-Nearest Neighbors (KNN). On the other hand, the CNN-LSTM model captures both spatial and temporal dynamics of video sequences by extracting visual features through Convolutional Neural Network (CNN) and modeling temporal dependencies via LSTM. The models are evaluated on a dataset for four postures-Fall, SIt, Stand and Unstable-with promising result. This work demonstrate the effectiveness of combining pose-based features with voting classifiers and the power of deep learning in sequential data, offering a robust solution for real-time posture classification systems.