Efficient Data Storage for Real-time High-Speed Lightning Detection from Space-borne Observations

This project develops an FPGA-based algorithm that efficiently stores lightning detection images from the C3IEL satellite by treating them as sparse matrices, storing only the non-zero pixels and their coordinates to dramatically reduce storage and transmission requirements.

Convective clouds play a critical role in shaping the Earth’s climate, yet current observation tools lack the resolution and responsiveness required to fully analyze their development. To address this gap, the C3IEL satellite mission was designed to enable advanced atmospheric research into the development of convective clouds and the formation of lightning.

C3IEL will employ 2 or 3 nanosatellites in a sun-synchronous polar orbit, equipped with advanced imaging instruments that provide high-resolution, high-frame-rate imaging to capture the dynamic nature of convective clouds and lightning activity. However, this also introduces two major engineering constraints: limited onboard storage and restricted downlink bandwidth.

To overcome these limitations, this project developed an FPGA-based algorithm for efficient data storage using sparse matrix techniques, leveraging the fact that lightning events occupy only a small portion of each image frame. The algorithm receives as input a sparse matrix representing a lightning image after pre-processing for detection and enhancement (parallel project) and converts it into a format that stores only the meaningful values, along with their corresponding row and column indices within the matrix. This data is then written directly to onboard DDR memory, significantly reducing both storage usage and the amount of data transmitted to ground stations.

The development was designed to integrate with a parallel project responsible for real-time lightning detection, aiming to establish a complete system for effective data storage.