Neural knitworks : patched neural implicit representation networks

Czerkawski, Mikolaj and Cardona, Javier and Atkinson, Robert and Michie, Craig and Andonovic, Ivan and Clemente, Carmine and Tachtatzis, Christos (2024) Neural knitworks : patched neural implicit representation networks. Pattern Recognition, 151. 110378. ISSN 0031-3203 (https://doi.org/10.1016/j.patcog.2024.110378)

[thumbnail of Czerkawski-etal-PR-2024-Neural-knitworks-patched-neural-implicit-representation-networks]
Preview
 Text. Filename: Czerkawski-etal-PR-2024-Neural-knitworks-patched-neural-implicit-representation-networks.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (3MB)| Preview

Abstract

Optimizing images as output of a neural network has been shown to introduce a powerful prior for image inverse tasks, capable of producing solutions of reasonable quality in a fully internal learning context, where no external datasets are involved. Two potential technical approaches involve fitting a coordinate-based Multilayer Perceptron (MLP), or a Convolutional Neural Network to produce the result image as output. The aim of this work is to evaluate the two counterparts, as well as a new framework proposed here, named Neural Knitwork, which maps pixel coordinates to local texture patches rather than singular pixel values. The utility of the proposed technique is demonstrated on the tasks of image inpainting, super-resolution, and denoising. It is shown that the Neural Knitwork can outperform the standard coordinate-based MLP baseline for the tasks of inpainting and denoising, and perform comparably for the super-resolution task.

CORE (COnnecting REpositories)