Designing a data pipeline for total column greenhouse gas measurements with portable EM27/SUN spectrometers
Loading...
URL
Journal Title
Journal ISSN
Volume Title
School of Science |
Master's thesis
Authors
Date
Department
Major/Subject
Mcode
Degree programme
Language
en
Pages
92
Series
Abstract
Understanding the complex systems underlying greenhouse gas dynamics in Earth's climate is crucial in the context of global surface temperature rise. One key method to achieve this understanding is through monitoring greenhouse gases, which is enabled by remote sensing, including space-based and ground-based spectrometry. Two portable ground-based Fourier transform infrared EM27/SUN spectrometers are operated by the Finnish Meteorological Institute, one of which was deployed in an automated enclosure system in spring 2024. The automation facilitates satellite validation campaigns in remote areas and experiments on wetland emission monitoring. However, it has created an influx of data that was difficult to manage with the existing EM27/SUN data processing practices. This thesis addresses the issue by developing a data pipeline for automated processing, aiming to improve usability, scalability, and timeliness. As a corollary, a new data organisation scheme was implemented, increasing the system's capabilities to store and process data from multiple measurement sites. The pipeline includes manual post-processing stages that facilitate comparison of measurements made at a high-latitude mire with those from a nearby reference site to investigate the possibility of measuring local greenhouse gas gradients. The pipeline was deployed in spring 2025, when it was able to process multiple years of data within a few hours. The parallelisation introduced by the pipeline has significantly reduced the actual processing time compared to prior estimates. It continues to run daily, processing new data as it is acquired. To assess the fulfilment of key requirements, a qualitative usability study was conducted with the pipeline's primary operator, in which it scored well in usability and satisfaction metrics. The analysis of the data indicates that some systematic errors may have been induced after the instrument was moved from the reference site to the mire, and this requires further investigation. This data pipeline strengthens the software capabilities in ground-based spectrometry and underscores the role of modern software systems in understanding global carbon dynamics.Description
Supervisor
Jung, AlexThesis advisor
Kutznetsova, OlgaKarppinen, Tomi