HCMMA-Net: A Hybrid Convolutional Multi-Modal Attention Network for Human Activity Recognition in Smart Homes Using Wearable Sensor Data

Ashfaq, Nazish; Aziz, Zeeshan; Hassan Khan, Muhammad; Adeel Nisar, Muhammad; Khalid, Adnan

Abstract

Human activity recognition (HAR) plays a pivotal role in applications such as healthcare monitoring, fitness tracking, and smart homes. Multi-modal sensor data from wearable devices offers diverse perspectives on human motion, enhancing recognition accuracy and robustness. However, integrating these modalities poses challenges due to sensor heterogeneity and variability in placement. This study examines the role of multi-modalities in HAR using a hybrid convolutional multi-modal attention network (HCMMA-Net), designed to exploit spatial and temporal dependencies in sensor data. We evaluate the model on two benchmark datasets Cogage, achieving an accuracy of 93.94%, and WISDM, with an accuracy of 99.29%, demonstrating its strong generalizability across varied sensor configurations. Additionally, we present a newly collected multi-modal dataset, HumcareV1.0, comprising different activities in smart-home-like scenarios. On this real-world dataset, HCMMA-Net attains an accuracy of 97.56%, highlighting its effectiveness in capturing subtle behavioral nuances in practical environments. The model exhibits robust generalization across complex activity patterns and sensor configurations, underscoring the significance of multi-modal integration in advancing HAR systems. These findings highlight the potential of our approach for deployment in real-time, context-aware smart environments.

• Journal Article Type: Article
• Acceptance Date: Jun 4, 2025
• Publication Date: Jun 9, 2025
• Deposit Date: Aug 8, 2025
• Publicly Available Date: Aug 8, 2025
• Journal: IEEE Access
• Electronic ISSN: 2169-3536
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/ACCESS.2025.3577730