RoadSense3D: A Framework for Roadside Monocular 3D Object Detection

Abstract

Utilizing monocular cameras for 3D object understanding is widely recognized as a cost-effective approach, spanning applications such as autonomous driving, augmented/virtual reality or roadside monitoring. Despite recent progress, persistent challenges arise in creating generalized models adaptable to unforeseen scenarios and diverse camera configurations. In this work, we focus on the task of monocular 3D object detection within roadside environments. To begin, we introduce a versatile methodology for generating and labeling datasets tailored to roadside scenarios, addressing limitations encountered in real-world settings. Subsequently, we develop an array of deep learning models tailored to this task, refining them to address practical challenges that emerge during real-world application. Lastly, leveraging our framework, we curated a synthetic benchmark dataset comprising 1,415,680 frames and 8,902,636 labeled 3D objects, ultimately assessing the performance of existing models across all datasets.