Covariance density neural networks

Tools

Roy, Om and Moshfeghi, Yashar and Smith, Keith (2026) Covariance density neural networks. Transactions on Machine Learning Research, 2026-Marc. pp. 1-46. ISSN 2835-8856

[thumbnail of Roy-Moshfeghi-Smith-2026-Covariance-Density-Neural-Networks]
Preview
 Text. Filename: Roy-Moshfeghi-Smith-2026-Covariance-Density-Neural-Networks.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (6MB)| Preview

Abstract

Graph neural networks have re-defined how we model and predict on network data but there lacks a consensus on choosing the correct underlying graph structure on which to model signals. CoVariance Neural Networks (VNN) address this issue by using the sample covariance matrix as a Graph Shift Operator (GSO). Here, we improve on the performance of VNNs by constructing a Density Matrix where we consider the sample Covariance matrix as a quasi-Hamiltonian of the system in the space of random variables. Crucially, using this density matrix as the GSO allows components of the data to be extracted at different scales, allowing enhanced discriminability and performance. We show that this approach allows explicit control of the stability-discriminability trade-off of the network, provides enhanced robustness to noise compared to VNNs, and outperforms them in useful real-life applications where the underlying covariance matrix is informative. In particular, we show that our model can achieve strong performance in subject-independent Brain Computer Interface EEG motor imagery classification, outperforming EEGnet while being faster. This shows how covariance density neural networks provide a basis for the notoriously difficult task of transferability of BCIs when evaluated on unseen individuals, while providing a principled, tuneable control over the stability–discriminability trade-off via the inverse temperature parameter β.

ORCID iDs

Roy, Om, Moshfeghi, Yashar ORCID logoORCID: https://orcid.org/0000-0003-4186-1088 and Smith, Keith ORCID logoORCID: https://orcid.org/0000-0002-4615-9020;
CORE (COnnecting REpositories)