Exploring AI-Based Emotion Recognition in Swedish: Speech, Text, and Vocal Markers
2025 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE credits
Student thesis
Abstract [en]
This thesis investigates emotion recognition in Swedish speech through a multimodal approach using AI models. Combining speech-based and text-based analysis with self-assessed emotion scores from participants in semi-structured interviews, this study addresses three research questions: (1) How does AI-model for speech emotion recognition compare to research on vocal markers for emotions in Swedish speech?; (2) What similarities and differences emerge between emotions detected from audio features and from the textual transcripts of the same speech data?; (3) How do AI-generated emotion labels (speech & text-based) compare to self-reported emotions? To answer these questions, data was collected in form of spontaneous speech from interviews, resulting in a more naturalistic dataset than acted datasets which are largely used in research. The results from analysing the collected data revealed partial alignments between vocal features and the speech-based AI model, Hume AI, as well as strongly suggesting some emotions are more difficult to detect due. The text-based AI model, NLP Cloud, proved to better align with the self-assessed scores, indicating that the textual context gave important cues more consistently than vocal features alone. The results highlighted the importance of a multimodal approach to capture a wider range of emotional expressions. Contributing to the fields of affective computing and natural language processing particularly by using spontaneous speech over an acted dataset, this study gives a deeper understanding in emotion recognition applied to the Swedish language.
Place, publisher, year, edition, pages
2025. , p. 87
Keywords [en]
Emotion recognition, text-based emotion detection (TBED), speech-based emotion recognition (SER), vocal markers, Swedish speech, self-assessed emotion scores, AI- based emotion detection
National Category
Engineering and Technology Computer Vision and Learning Systems
Identifiers
URN: urn:nbn:se:hj:diva-69116OAI: oai:DiVA.org:hj-69116DiVA, id: diva2:1978866
External cooperation
Knowit
Subject / course
JTH, Computer Engineering
Supervisors
Examiners
2025-07-282025-06-292025-10-13Bibliographically approved