On the role of activation functions in EEG-to-text decoder

Tools

Lamprou, Zenon and Tenedios, Iakovos and Moshfeghi, Yashar; Nicosia, Giuseppe and Ojha, Varun and Giesselbach, Sven and Pardalos, M. Panos and Umeton, Renato, eds. (2025) On the role of activation functions in EEG-to-text decoder. In: Machine Learning, Optimization, and Data Science. Lecture Notes in Computer Science . Springer, ITA, pp. 46-60. ISBN 9783031824869 (https://doi.org/10.1007/978-3-031-82487-6_4)

[thumbnail of Lamprou-etal-LOD-2024-On-the-role-of-activation-functions-in-EEG-to-text-decoder]
Preview
 Text. Filename: Lamprou-etal-LOD-2024-On-the-role-of-activation-functions-in-EEG-to-text-decoder.pdf
Accepted Author Manuscript
License: Other
Download (1MB)| Preview

Abstract

In recent years, much interdisciplinary research has been conducted exploring potential use cases of neuroscience to advance the field of information retrieval. Initial research concentrated on the use of fMRI data, but fMRI was deemed to be not suitable for real-world applications, and soon, research shifted towards using EEG data. In this paper, we try to improve the original performance of a first attempt at generating text using EEG by focusing on the less explored area of optimising neural network performance. We test a set of different activation functions and compare their performance. Our results show that introducing a higher degree polynomial activation function can enhance model performance without changing the model architecture. We also show that the learnable 3rd-degree activation function performs better on the 1-gram evaluation compared to a 3rd-degree non-learnable function. However, when evaluating the model on 2-grams and above, the polynomial function lacks in performance, whilst the leaky ReLU activation function outperforms the baseline.

ORCID iDs

Lamprou, Zenon, Tenedios, Iakovos and Moshfeghi, Yashar ORCID logoORCID: https://orcid.org/0000-0003-4186-1088; Nicosia, Giuseppe, Ojha, Varun, Giesselbach, Sven, Pardalos, M. Panos and Umeton, Renato
CORE (COnnecting REpositories)