Predicting Soccer Penalty Kick Direction Using Human Action Recognition

Abstract

Action anticipation has become a prominent topic in Human Action Recognition (HAR). However, its application to real-world sports scenarios remains limited by the availability of suitable annotated datasets. This work presents a novel dataset of manually annotated soccer penalty kicks to predict shot direction based on pre-kick player movements. We propose a deep learning classifier to benchmark this dataset that integrates HAR-based feature embeddings with contextual metadata. We evaluate twenty-two backbone models across seven architecture families (MViTv2, MViTv1, SlowFast, Slow, X3D, I3D, C2D), achieving up to 63.9% accuracy in predicting shot direction (left or right)–outperforming the real goalkeepers’ decisions. These results demonstrate the dataset’s value for anticipatory action recognition and validate our model’s potential as a generalizable approach for sports-based predictive tasks.