Multi-scale spatial fusion and regularization induced unsupervised auxiliary task CNN model for deep super-resolution of hyperspectral image

Ha, Viet Khanh and Ren, Jinchang and Wang, Zheng and Sun, Genyun and Zhao, Huimin and Marshall, Stephen (2022) Multi-scale spatial fusion and regularization induced unsupervised auxiliary task CNN model for deep super-resolution of hyperspectral image. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15. pp. 4583-4598. ISSN 1939-1404 (https://doi.org/10.1109/JSTARS.2022.3176969)

[thumbnail of Khanh-Ha-etal-JSTAEORS-2022-Multi-scale-spatial-fusion-and-regularization-induced-unsupervised-auxiliary-task-CNN-model]
Preview
 Text. Filename: Khanh_Ha_etal_JSTAEORS_2022_Multi_scale_spatial_fusion_and_regularization_induced_unsupervised_auxiliary_task_CNN_model.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (15MB)| Preview

Abstract

Hyperspectral images (HSI) feature rich spectral information in many narrow bands but at a cost of a relatively low spatial resolution. As such, various methods have been developed for enhancing the spatial resolution of the low-resolution HSI (Lr-HSI) by fusing it with high-resolution multispectral images (Hr-MSI). The difference in spectrum range and spatial dimensions between the Lr-HSI and Hr-MSI has been fundamental but challenging for multispectral/hyperspectral (MS/HS) fusion. In this article, a multiscale spatial fusion and regularization induced auxiliary task based convolutional neural network model is proposed for deep super-resolution of HSI, where an Lr-HSI is fused with an Hr-MSI to reconstruct a high-resolution HSI (Hr-HSI) counterpart. The multiscale fusion is used to efficiently address the discrepancy in spatial resolutions between the two inputs. Based on the general assumption that the acquired Hr-MSI and the reconstructed Hr-HSI share similar underlying characteristics, the auxiliary task is proposed to learn a representation for improved generality of the model and reduced overfitting. Experimental results on five public datasets have validated the effectiveness of our approach in comparison with several state-of-the-art methods.

ORCID iDs

Ha, Viet Khanh, Ren, Jinchang, Wang, Zheng, Sun, Genyun, Zhao, Huimin and Marshall, Stephen ORCID logoORCID: https://orcid.org/0000-0001-7079-5628;
CORE (COnnecting REpositories)