Verification of a CCSDS121-based Data Compression IP Core through Fuzzing

Abstract

System-level design commonly employs building blocks, also denoted as soft IP cores, to conform complex developments. This is also a trend in the space industry to save costs and development time. Every IP must pass through a verification and validation process before being integrated in a larger design to ensure a proper system behaviour. In order to ensure the functional correctness of the IP core and its compliancy to the requirements, a verification campaign based on a set of defined test cases is performed based on functional simulations and the creation of testbenches. However, as hardware designs grow in complexity, it is more and more difficult to reach all possible corner cases by a test campaign designed to fulfil the defined requirements. Therefore, even if a full code coverage is achieved during the verification process, there is always a chance that the IP will exhibit an unexpected behaviour in certain situations. This fact motivates to look for alternative verification approaches. Among them formal methods are the most effective but its solutions are not scalable to IP core level. In this scenario recently hardware fuzzing appears as an interesting solution to solve the IP verification problem. Fuzzing is a testing technique where inputs are generated randomly and used to identify defects in software. It is commonly utilized in cybersecurity to find vulnerabilities and has also been applied to software testing. The fuzzing process involves two pieces: a fuzzer that generates inputs and manages the execution of the software under test, and a fuzzing harness that connects the fuzzer to the software. The fuzzer is designed to be coverage-guided, meaning it uses information gained from previous inputs to direct future input generation. The fuzzing architecture for hardware testing is more complex than for software and is constituted by three components: IP Core Fuzzer, Orchestrator and Pull of Agents. First, the IP core fuzzer, which encapsulates the fuzzer, the fuzzing harness and a reference software which models the IP core under verification. It generates both the input test vectors and their expected output. These data are sent to the Orchestrator, which manages the execution of the Agents, which simulate the IP core to be validated. Multiple agents can be executed in parallel to increase the number of vectors evaluated per unit time, hence the term Pull of Agents. The Orchestrator selects on every moment which Agent to launch, providing an input test vector and the necessary configuration, retrieves the simulation output and compares it against the expected one. This allows to identify bugs in the IP core and reproduce them later. In this work, we apply the fuzzing methodology to the verification of a universal data compressor compliant with the CCSDS-121.0-B-3 standard. The CCSDS-121 IP verification process includes several verification campaigns (with pre-generated test cases), reaching a 100% code coverage and then a hardware fuzzing verification. The fuzzer architecture has been configured with a pull of 20 agents, which reach throughput of 1.5 tests per second. After testing more than 1.000.000 test vectors, 18 unnoticed bugs have been detected and fixed in this IP core. The bugs detected are related to corner cases with non-common configurations, which were hard to notice in the standard verification campaign. This work demonstrates the strengths of the fuzzing methodology to complement traditional verification campaigns for hardware designs and the benefits of this approach for the verification of new IP cores.