Combining oculo-motor indices to measure cognitive load of synthetic speech in noisy listening conditions

Dubiel, Mateusz and Nakayama, Minoru and Wang, Xin; Spencer, Stephen N., ed. (2021) Combining oculo-motor indices to measure cognitive load of synthetic speech in noisy listening conditions. In: ETRA '21 Short Papers : ACM Symposium on Eye Tracking Research and Applications. Eye Tracking Research and Applications Symposium (ETRA) . ACM, DEU. ISBN 9781450383455 (https://doi.org/10.1145/3448018.3458013)

[thumbnail of Dubiel-etal-ETRA2021-Combining-oculo-motor-indices-to-measure-cognitive-load]
Preview
 Text. Filename: Dubiel_etal_ETRA2021_Combining_oculo_motor_indices_to_measure_cognitive_load.pdf
Accepted Author Manuscript
Download (825kB)| Preview

Abstract

Gaze-based assistive technologies (ATs) that feature speech have the potential to improve the life of people with communication disorders. However, due to a limited understanding of how different speech types affect the cognitive load of users, an evaluation of ATs remains a challenge. Expanding on previous work, we combined temporal changes in pupil size and ocular movements (saccades and fixation differentials) to evaluate cognitive workload of two types of speech (natural and synthetic) mixed with noise, through a listening test. While observed pupil sizes were significantly larger at lower signal-to-noise levels, as participants listened and memorised speech stimuli; saccadic eye-movements were significantly more frequent for synthetic speech. In the synthetic condition, there was a strong negative correlation between pupil dilation and fixation differentials, indicating a higher strain on participants’ cognitive resources. These results suggest that combining oculo-motor indices can aid our understanding of the cognitive implications of different speech types.

CORE (COnnecting REpositories)