Operational neural networks for parameter-efficient hyperspectral single-image super-resolution

Ulrichsen, Alexander and Murray, Paul and Marshall, Stephen and Gabbouj, Moncef and Kiranyaz, Serkan and Yamaç, Mehmet and Aburaed, Nour (2023) Operational neural networks for parameter-efficient hyperspectral single-image super-resolution. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17. pp. 1470-1484. ISSN 2151-1535 (https://doi.org/10.1109/JSTARS.2023.3333274)

[thumbnail of Ulrichsen-etal-IEEE-JSTAEORS-Operational-neural-networks-for-parameter-efficient-hyperspectral-single-image-super-resolution]
Preview
 Text. Filename: Ulrichsen-etal-IEEE-JSTAEORS-Operational-neural-networks-for-parameter-efficient-hyperspectral-single-image-super-resolution.pdf
Final Published Version
License: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 logo
Download (5MB)| Preview

Abstract

Hyperspectral imaging is a crucial tool in remote sensing, which captures far more spectral information than standard color images. However, the increase in spectral information comes at the cost of spatial resolution. Super-resolution is a popular technique where the goal is to generate a high-resolution version of a given low-resolution input. The majority of modern super-resolution approaches use convolutional neural networks (CNNs). However, convolution itself is a linear operation and the networks rely on the nonlinear activation functions after each layer to provide the necessary nonlinearity to learn the complex underlying function. This means that CNNs tend to be very deep to achieve the desired results. Recently, self-organized operational neural networks (ONNs) have been proposed that aim to overcome this limitation by replacing the convolutional filters with learnable nonlinear functions through the use of MacLaurin series expansions. This work focuses on extending the convolutional filters of a popular super-resolution model to more powerful operational filters to enhance the model performance on hyperspectral images (HSIs). We also investigate the effects that residual connections and different normalization types have on this type of enhanced network. Despite having fewer parameters than their convolutional network equivalents, our results show that ONNs achieve superior super-resolution performance on small HSI datasets.

CORE (COnnecting REpositories)