METEOR: Deep Meta-Learning for Flexible Earth Observation Problem Solving

By Muhammad OsamaReviewed by Susha Cheriyedath, M.Sc.Jan 18 2024

In an article published in the journal Communications Earth & Environment, researchers from the USA and Germany discussed how deep learning can address various Earth observation problems across different resolutions using satellite data. They developed a deep meta-learning methodology called METEOR that can adapt to new tasks with few labeled examples. Moreover, they demonstrated the performance of their technique on several downstream problems.

Background

Earth observation is the science of monitoring and understanding the Earth’s surface and atmosphere using remote sensing data from satellites, aircraft, or drones. Earth observation data provide valuable information for various applications, such as climate change, disaster management, agriculture, urban planning, and biodiversity conservation. Earth scientists use this data to study a variety of problems, such as land cover classification, urban scene analysis, crop type mapping, deforestation detection, and marine debris segmentation.

However, the analysis of this data is often costly and time-consuming, posing several challenges such as the diversity of data sources, variability in spatial and spectral resolutions, scarcity of labeled data, and heterogeneity of problems. Moreover, these challenges can differ in terms of the spatial, spectral, and temporal resolution of the data, the number of classes, and the geographic regions of interest, making it challenging to transfer knowledge across them.

Previously, researchers developed several machine learning and deep learning methods to automatically extract useful information from Earth observation data. However, most of these methods require large amounts of annotated data for each problem and are not easily transferable to new problems or data sources. Therefore, there is a need for more flexible and adaptable methods that can leverage common knowledge across different problems and data modalities.

Meta-learning is a branch of machine learning where the machine learns how to learn from multiple tasks and quickly adapt to new tasks with few examples or labeled examples. It is a transfer learning approach that aims to learn a generalizable model from a collection of related tasks. Previous meta-learning approaches for remote sensing have focused on homogeneous problems, such as land cover classification or crop mapping, using data from the same sensor and resolution. However, Earth observation problems are heterogeneous and diverse, requiring models that can handle different data modalities and problem scopes.

About the Research

In the present paper, the authors proposed METEOR based on the model-agnostic meta-learning (MAML) algorithm. It facilitates the learning of a generic model, fine-tuned to specific tasks with just a few gradient steps. METEOR was designed with the following three key modifications to enable heterogeneous transfer learning across diverse Earth observation challenges.

• Initially, the study replaced batch normalization layers in the model with instance normalization layers, improving performance on class-imbalanced problems.
• Next, convolutional kernels of the input channels were dynamically changed to adapt to problems with different spectral bands.
• Finally, problems with varying numbers of classes were addressed by pre-training a binary meta-model, fine-tuning it separately to each class, and ensembling a one-vs-all classifier.

The researchers pre-trained the METEOR meta-model on a dataset of land cover classification tasks from different geographic regions, using multi-spectral satellite data from Sentinel-1 and Sentinel-2. They then fine-tuned the meta-model on various downstream tasks, such as land cover classification from different regions, urban scene classification with high-resolution red green blue (RGB) imagery, change detection with multi-temporal imagery, and marine debris segmentation with coastal imagery. Moreover, they compared the performance of METEOR with other state-of-the-art methods, such as self-supervised learning, self-taught learning, and supervised learning.

Research Findings

The outcomes showed that METEOR performed well on both homogeneous and heterogeneous downstream tasks, achieving the best average rank among the compared methods. It can handle different data modalities, such as RGB, multi-spectral, and multi-temporal imagery, and different problem scopes, including classification, segmentation, and change detection.

The study analyzed the behavior of the proposed technique qualitatively on several use cases and provided some insights using interpretability methods. By using a single pre-trained meta-model, domain experts can fine-tune it to their specific problems, thus saving time and effort.  Moreover, the proposed meta-model can capture common knowledge across different problems and data sources, enhancing the generalization and robustness of the model.

Conclusion

In summary, the presented novel method is innovative, efficient, and scalable for Earth observation problems. The study highlighted that METEOR can adapt to various downstream problems with few labeled examples and achieve competitive results. It also indicated the versatility and interpretability of METEOR in several use cases. Moreover, it suggested that meta-learning can be a powerful tool for solving diverse and impactful Earth science problems with remote sensing data.

The authors acknowledged the challenges of using their method. These limitations include the availability of high-quality and diverse Earth observation data, the high cost of pre-training and fine-tuning, and the use of evaluation metrics such as accuracy and F1-score that do not fully capture the performance. They suggested that future research could explore expanding the pre-training scope to other labeled and unlabeled source tasks, training larger and more expressive models, and applying meta-learning to other types of Earth observation data, such as radar or lidar data.