Skip to main content

Research Repository

Advanced Search

SFTT: A Spatial-Frequency-Temporal-Based End-to-End Transformer for Heart Rate Estimation

Dey, Rakesh; Palaiahnakote, Shivakumara; Bhattacharya, Saumik; Pal, Umapada; Chanda, Sukalpa

SFTT: A Spatial-Frequency-Temporal-Based End-to-End Transformer for Heart Rate Estimation Thumbnail


Authors

Rakesh Dey

Saumik Bhattacharya

Umapada Pal

Sukalpa Chanda



Abstract

Vision-based Heart Rate (HR) estimation in adverse situations, such as changes in skin tone, arbitrary face movements, and complex backgrounds, etc., is challenging. Unlike state-of-the-art models that use color and spatial-temporal information, the present work exploits a Spatial-Frequency-Temporal Transformer (SFTT) for heart rate estimation. For extracting multi-scale contextual features, we propose an end-to-end transformer that encodes contextual information through a pyramidal structure-based approach. Furthermore, to strengthen the features, the proposed model introduces a new attention approach that performs mutual-sharing operations between spatial-temporal and frequency-temporal domains in an end-to-end fashion. Experimental results on four standard datasets, namely UBFC-rPPG, VIPL-HR, OBF, and MMSE-HR, show that the proposed model is generic and invariant to the aforementioned challenges. Further, a comparative study with the state-of-the-art models demonstrates the effectiveness of the proposed method over the existing methods on all four benchmark datasets. Besides, experiments on cross-dataset validation show that the proposed method is reliable and robust.

Journal Article Type Article
Acceptance Date May 15, 2025
Online Publication Date Jul 3, 2025
Deposit Date Jul 4, 2025
Publicly Available Date Jul 7, 2025
Journal IEEE Transactions on Emerging Topics in Computational Intelligence
Print ISSN 2168-6750
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
DOI https://doi.org/10.1109/TETCI.2025.3582841

Files





You might also like



Downloadable Citations