Exploring the capability of text-to-image diffusion models with structural edge guidance for multi-spectral satellite image inpainting
Czerkawski, Mikolaj and Tachtatzis, Christos (2024) Exploring the capability of text-to-image diffusion models with structural edge guidance for multi-spectral satellite image inpainting. IEEE Geoscience and Remote Sensing Letters, 21. 5001905. ISSN 1545-598X (https://doi.org/10.1109/LGRS.2024.3370212)
Preview |
Text.
Filename: Czerkawski-Tachtatzis-IEEE-GRSL-2024-Exploring-the-capability-of-text-to-image-diffusion.pdf
Accepted Author Manuscript License: Strathprints license 1.0 Download (6MB)| Preview |
Abstract
The letter investigates the utility of text-to-image inpainting models for satellite image data. Two technical challenges of injecting structural guiding signals into the generative process as well as translating the inpainted RGB pixels to a wider set of MSI bands are addressed by introducing a novel inpainting framework based on StableDiffusion and ControlNet as well as a novel method for RGB-to-MSI translation. The results on a wider set of data suggest that the inpainting synthesized via StableDiffusion suffers from undesired artifacts and that a simple alternative of self-supervised internal inpainting achieves a higher quality of synthesis.
ORCID iDs
Czerkawski, Mikolaj ORCID: https://orcid.org/0000-0002-0927-0416 and Tachtatzis, Christos ORCID: https://orcid.org/0000-0001-9150-6805;-
-
Item type: Article ID code: 88256 Dates: DateEvent26 February 2024Published26 February 2024Published Online21 February 2024AcceptedNotes: © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Subjects: Technology > Electrical engineering. Electronics Nuclear engineering > Electrical apparatus and materials Department: Faculty of Engineering > Electronic and Electrical Engineering
Strategic Research Themes > Measurement Science and Enabling TechnologiesDepositing user: Pure Administrator Date deposited: 27 Feb 2024 09:44 Last modified: 01 Nov 2024 01:45 Related URLs: URI: https://strathprints.strath.ac.uk/id/eprint/88256