Deep internal learning for inpainting of cloud-affected regions in satellite imagery

Czerkawski, Mikolaj and Upadhyay, Priti and Davison, Christopher and Werkmeister, Astrid and Cardona, Javier and Atkinson, Robert and Michie, Craig and Andonovic, Ivan and Macdonald, Malcolm and Tachtatzis, Christos (2022) Deep internal learning for inpainting of cloud-affected regions in satellite imagery. Remote Sensing, 14 (6). 1342. ISSN 2072-4292 (https://doi.org/10.3390/rs14061342)

[thumbnail of Czerkawaski-etal-RS-2022-Deep-internal-learning-for-inpainting-of-cloud-affected-regions]
Preview
 Text. Filename: Czerkawaski_etal_RS_2022_Deep_internal_learning_for_inpainting_of_cloud_affected_regions.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (61MB)| Preview

Abstract

Cloud cover remains a significant limitation to a broad range of applications relying on optical remote sensing imagery, including crop identification/yield prediction, climate monitoring, and land cover classification. A common approach to cloud removal treats the problem as an inpainting task and imputes optical data in the cloud-affected regions employing either mosaicing historical data or making use of sensing modalities not impacted by cloud obstructions, such as SAR. Recently, deep learning approaches have been explored in these applications; however, the majority of reported solutions rely on external learning practices, i.e., models trained on fixed datasets. Although these models perform well within the context of a particular dataset, a significant risk of spatial and temporal overfitting exists when applied in different locations or at different times. Here, cloud removal was implemented within an internal learning regime through an inpainting technique based on the deep image prior. The approach was evaluated on both a synthetic dataset with an exact ground truth, as well as real samples. The ability to inpaint the cloud-affected regions for varying weather conditions across a whole year with no prior training was demonstrated, and the performance of the approach was characterised.

ORCID iDs

Czerkawski, Mikolaj ORCID logoORCID: https://orcid.org/0000-0002-0927-0416, Upadhyay, Priti ORCID logoORCID: https://orcid.org/0000-0002-6212-5314, Davison, Christopher ORCID logoORCID: https://orcid.org/0000-0002-9450-1791, Werkmeister, Astrid ORCID logoORCID: https://orcid.org/0000-0002-0174-5851, Cardona, Javier ORCID logoORCID: https://orcid.org/0000-0002-9284-1899, Atkinson, Robert ORCID logoORCID: https://orcid.org/0000-0002-6206-2229, Michie, Craig ORCID logoORCID: https://orcid.org/0000-0001-5132-4572, Andonovic, Ivan ORCID logoORCID: https://orcid.org/0000-0001-9093-5245, Macdonald, Malcolm ORCID logoORCID: https://orcid.org/0000-0003-4499-4281 and Tachtatzis, Christos ORCID logoORCID: https://orcid.org/0000-0001-9150-6805;
CORE (COnnecting REpositories)