Vision-based robotics using open FPGAs

Abstract

Robotics increasingly provides practical applications for society, such as manufacturing, autonomous driving, robot vacuum cleaners, robots in logistics, drones for inspection, etc. Typical requirements in this field are fast response time, low power consumption, parallelism, and flexibility. According to these features, FPGAs are a suitable computing substrate for robots. A few vendors have dominated the FPGA market with their proprietary tools and hardware devices, resulting in fragmented ecosystems with few standards and little interoperation. New and complete open toolchains for FPGAs are emerging from the open-source community. This article presents an open-source library of Verilog modules useful for vision-based robots, including reusable image processing blocks for perception and reactive control blocks. This library has been developed using open tools, but its Verilog modules are fully compatible with any proprietary toolchain. In addition, three applications with a real robot and open FPGAs have been developed for experimental validation using this library. In the last application, the mobile robot successfully follows a colored object using two low-cost cameras (to increase the robot's field of view) and includes a third camera on top of a servo-driven turret for tracking a second independent object while following the first one in parallel. Resource consumption of all applications has been measured and compared with state-of-the-art proprietary toolchains, revealing that reconfigurable computing with open FPGAs using open tools is now an attractive alternative to designing and creating intelligent vision-based robotic applications using vendor-dependent proprietary tools and FPGAs.