Implementation of Cloud Detection and Processing Algorithms and CCSDS-Compliant Hyperspectral Image Compression for CHIME Mission

Abstract

Hyperspectral sensors are increasing their presence on-board satellites because they provide relevant information for the scientific community in many applications. This is the reason why ESA has included the Copernicus Hyperspectral Imaging Mission for Environment (CHIME) in the future Copernicus 2.0 program. CHIME shall provide contiguous spectral coverage in VNIR and SWIR spectral domain (covering approximately 220 bands between 400nm and 2500nm). Acquisitions shall have typically 106Msps, with a dynamic range of 16 bits per sample and acquired in Band-Interleaved by Line (BIL) order, what leads to input data rates up to almost 2Gbps. However, clouds are estimated to cover more than 54% of the Earth’s land area and 68% of the oceans. Many scientific applications which need to estimate Earth surface properties from satellite images are useless in presence of clouds, making more than half of the acquired scenes unusable. At sensor level, on board detection of cloudy areas and compression combined are then proposed to transmit sensor information to ground within a restricted time and with a limited downlink bandwidth. This work, done in CHIME (phase A/B1) framework, presents a demonstrator including implementation of cloud detection and processing algorithms and hyperspectral image compressor based on CCSDS 123.0-B-2 standard, recommended by the Consultative Committee for Space Data Systems. Cloud algorithms are composed by two stages: detection and processing. The cloud detection is performed by a Support Vector Machine approach (SVM). The SVM algorithm is pixel-based and it is followed by a simple filtering in order to reduce the false positive detection. The output is a spatial mask (with same number of pixels than the image) indicating for each hyperspectral pixel if it is cloud or not. For the pixels detected as cloud, a pre-quantization is done, to improve the posterior compression in these less useful areas. Even for cloudy zones, some bands of the image can have scientific or commercial utility, therefore some interesting selected bands can be excluded from the processing. The CCSDS-123.0-B-2 standard defines a lossless and near-lossless compression solution that specifically targets multispectral and hyperspectral images. The near-lossless compression mode is achieved by introducing a quantization loop in the prediction architecture of its predecessor, the CCSDS-123.0-B-1 lossless standard, and by controlling the compression losses through user defined error values. To adapt the CCSDS-123.0-B-2 standard to CHIME mission requirements a tuning was performed, in order to identify the most suitable values for the different compression parameters proposed in the standard. With a set of images generated from AVIRIS images, to be representative of CHIME scenario, a wide number of simulations were done in order to fine tune parameters such as: Prediction mode, type of Local Sum, Number of Prediction Bands (P), Sample representative resolution (Ɵ), Sample representative damping (ɸz), 12 etc. As entropy coder, the CCSDS 121.0-B-2 block-adaptive coder was selected as a trade-off between compression capability and complexity. Cloud algorithms (detection and processing) modules have been coded by means of RTL VHDL description. In case of the CCSDS-123.0-B-2 standard, it was modeled in C language and implemented by using High Level Synthesis (HLS) techniques, that automatically generate the equivalent RTL description. The system has been successfully validated over the Xilinx KCU105 evaluation board, that mounts a Xilinx KU040 FPGA, new generation device representative of flight hardware. The design consumes low resources leaving growth potential for further evolutions or other additional applications. This paper will present the design implementation, the demonstrator set-up and the validation plan to verify the correct behavior and performances. The test procedure consists in iterations between the simulations of the VHDL in workstation and the demonstrator hardware tests. The compressed images have been validated by comparison with references generated by the external CNES software, compliant with the compression standard. Several test vectors have been defined for the validation with different features, such as image size or percentage of clouds in the scene. Acknowledgments: The research leading to these results has been financed by ESA in the frame of the CHIME phase A/B1 pre-developments. The authors want to thank Mickael Bruno and Mathieu Albinet from CNES for the compression software. Also thanks to Roberto Camarero and Raffaele Vitulli from ESA and Michel-François Foulon and Dimitri Lebedeff from Thales Alenia Space in France for their contributions.