A multimodal sentiment classifier for financial decision making

Todd, Andrew and Bowden, James and Cummins, Mark and Su, Yang (2025) A multimodal sentiment classifier for financial decision making. International Review of Financial Analysis. ISSN 1057-5219 (https://doi.org/10.1016/j.irfa.2025.104322)

Text. Filename: Todd-etal-2025-A-multimodal-sentiment-classifier-for-financial-decision-making.pdf Accepted Author Manuscript Restricted to Repository staff only until 1 January 2099. Download (823kB) | Request a copy

Abstract

This study pioneers a multimodal approach to financial sentiment analysis through the integration of audio and textual data to enhance predictive accuracy. Motivated by the underutilisation of paralinguistic features and deep learning techniques in financial sentiment analysis, we introduce a novel deep learning-enabled multimodal classifier trained on corporate earnings calls using a subset of S&P 100 constituents. Our methodology incorporates FinBERT, a financial variant of Bidirectional Encoder Representation Transformations (BERT), alongside paralinguistic features and a deep learning classifier. Comparative analysis against established sentiment analysis methods, including dictionary approaches and machine learning models, suggests that our multimodal classifier achieves improved out-of-sample accuracy. Specifically, the inclusion of paralinguistic characteristics improves sentiment detection accuracy. Our research provides nuanced insights into sentiment analysis detection of different speakers (managers and analysts) during both the management discussion and Q&A sections of corporate earnings calls. Combined, our results suggest that multimodal sentiment analysis classification possesses the ability to deepen our understanding of the interplay between sentiment and market characteristics.

ORCID iDs

Bowden, James ORCID: https://orcid.org/0000-0002-0419-1882

Cummins, Mark ORCID: https://orcid.org/0000-0002-3539-8843

• Share and Export
  

  EndNoteData Cite XMLMODSRDF+N3OpenURL ContextObjectASCII CitationAtomSimple MetadataMultiline CSVJSONEP3 XMLMPEG-21 DIDLRDF+XMLRDF+N-TriplesMETSBibTeXDublin CoreReferRIOXX2 XMLOpenURL ContextObject in SpanRefWorksHTML CitationRIS (EndNote Online, Zotero, Ref manager, etc)
• Item metadata

  Item type:	Article
  ID code:	92865
  Dates:	Date Event 9 May 2025 Published 9 May 2025 Published Online 6 May 2025 Accepted 20 November 2024 Submitted
  Subjects:	Social Sciences > Finance
  Department:	Strathclyde Business School > Accounting and Finance
  Depositing user:	Pure Administrator
  Date deposited:	16 May 2025 14:19
  Last modified:	16 May 2025 14:19
  URI:	https://strathprints.strath.ac.uk/id/eprint/92865