Zero-shot recognition of dysarthric speech using commercial automatic speech recognition and multimodal large language models

Tools

Alsayegh, Ali and Masood, Tariq (2025) Zero-shot recognition of dysarthric speech using commercial automatic speech recognition and multimodal large language models. Other. arXiv, Ithaca, NY. (https://doi.org/10.48550/arXiv.2512.17474)

[thumbnail of Alsayegh-Masood-arXiv-2025-Zero-shot-recognition-of-dysarthric-speech]
Preview
 Text. Filename: Alsayegh-Masood-arXiv-2025-Zero-shot-recognition-of-dysarthric-speech.pdf
Final Published Version
License: Strathprints license 1.0
Download (702kB)| Preview

Abstract

Voice-based human-machine interaction is a primary modality for accessing intelligent systems, yet individuals with dysarthria face systematic exclusion due to recognition performance gaps. Whilst automatic speech recognition (ASR) achieves word error rates (WER) below 5% on typical speech, performance degrades dramatically for dysarthric speakers. Multimodal large language models (MLLMs) offer potential for leveraging contextual reasoning to compensate for acoustic degradation, yet their zero-shot capabilities remain uncharacterised. This study evaluates eight commercial speech-to-text services on the TORGO dysarthric speech corpus: four conventional ASR systems (AssemblyAI, Whisper large-v3, Deepgram Nova-3, Nova-3 Medical) and four MLLM-based systems (GPT-4o, GPT-4o Mini, Gemini 2.5 Pro, Gemini 2.5 Flash). Evaluation encompasses lexical accuracy, semantic preservation, and cost-latency trade-offs. Results demonstrate severity-dependent degradation: mild dysarthria achieves 3-5% WER approaching typical-speech benchmarks, whilst severe dysarthria exceeds 49% WER across all systems. A verbatim-transcription prompt yields architecture-specific effects: GPT-4o achieves 7.36 percentage point WER reduction with consistent improvement across all tested speakers, whilst Gemini variants exhibit degradation. Semantic metrics indicate that communicative intent remains partially recoverable despite elevated lexical error rates. These findings establish empirical baselines enabling evidence-based technology selection for assistive voice interface deployment.

ORCID iDs

Alsayegh, Ali ORCID logoORCID: https://orcid.org/0000-0001-7083-3639 and Masood, Tariq ORCID logoORCID: https://orcid.org/0000-0002-9933-6940;
CORE (COnnecting REpositories)