Towards facial expression robustness in multi-scale wild environments

Abstract

Facial expressions are dynamic processes that evolve over temporal segments, including onset, apex, offset, and neutral. However, previous works on automatic facial expression analysis have mainly focused on the recognition of discrete emotions, neglecting the continuous nature of these processes. Additionally, facial images captured from videos in the wild often have varying resolutions due to fixed-lens cameras. To address these problems, our objective is to develop a robust facial expression recognition classifier that provides good performance in such challenging environments. We evaluated several state-of-the-art models on labeled and unlabeled collections and analyzed their performance at different scales. To improve performance, we filtered the probabilities provided by each classifier and demonstrated that this improves decision-making consistency by more than 10%, leading to accuracy improvement. Finally, we combined the models’ backbones into a temporal-sequence classifier, leveraging this consistency-performance trade-off and achieving an additional improvement of 9.6%.