RGB, a Surrogate of Infrared Facial Videos for Physiological Signs Estimations in Dark

Abstract

Physiological signs are key indicators of cardiovascular health, which can be estimated using remote photoplethysmography. Their estimations in dark environments are particularly important, where infrared based methods were predominantly applied, since they are illumination resistant. However, the extracted signals have poor pulsatile strength with low signal-to-noise ratio, eventually resulting in spurious estimates. Conversely, RGB based methods exhibits stronger pulsatile strength, but hindered by poor illumination. To overcome these limitations, we propose 2E1D-Net, trained using a self-created database acquired in a dark environment with marginal illuminance ≤ 1 lux. It comprises dual encoders that take paired input images captured at different exposure levels, and project them to a latent. The decoder then, elevates the noise (darkness) component from the dark image, followed by multiscale feature fusion, to produce enhanced images. 2E1D-Net was trained using a linear combination of multiscale structured-similarity-index, L1 and L2 losses, respectively. Subsequently, RGB heart rate and oxygen saturation methods cascaded to trained 2E1D-Net, were tested on self-created and public databases. Experimental results proved the superiority of 2E1D-Net, over state-of-the-art, which ensured the extended ability of RGB methods for physiological measurements in dark, thereby proposing RGB as reliable and clinically relevant alternative to infrared methods without performance compromise.