Paper deep dive
ActorMind: Emulating Human Actor Reasoning for Speech Role-Playing
Xi Chen, Wei Xue, Yike Guo
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/14/2026, 2:30:45 AM
Summary
ActorMind is a multi-agent, chain-of-thought reasoning framework designed for speech role-playing. It emulates human theatrical performance through four specialized agents: Eye (context/profile reading), Ear (emotional cue perception), Brain (emotional state reasoning), and Mouth (script delivery via RAG-based speech synthesis). The paper also introduces ActorMindBench, a hierarchical benchmark containing 7,653 utterances across 313 scenes and 6 roles, derived from the television series 'Friends'.
Entities (7)
Relation Signals (3)
ActorMind â comprises â Eye Agent
confidence 100% · ActorMind, shown in Figure 2, conceptualizes four agentsâEye, Ear, Mouth, and Brain.
ActorMindBench â contains â Utterance-Level
confidence 100% · ActorMindBench is hierarchically designed with three levels of data: Utterance-Level, Scene-Level, and Role-Level.
ActorMind â utilizes â ActorMindBench
confidence 90% · Experimental results demonstrate the effectiveness of ActorMind in enhancing speech role-playing on ActorMindBench.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Role-playing has garnered rising attention as it provides a strong foundation for human-machine interaction and facilitates sociological research. However, current work is confined to textual modalities, neglecting speech, which plays a predominant role in daily life, thus limiting genuine role-playing. To bridge this gap, we conceptualize and benchmark speech role-playing through ActorMindBench, and we present a corresponding reasoning framework, called ActorMind. Specifically, (1) Speech Role-Playing enables models to deliver spontaneous responses with personalized verbal traits based on their role, the scene, and spoken dialogue. (2) ActorMindBench is a hierarchical benchmark comprises Utterance-Level content with 7,653 utterances, Scene-Level content with 313 scenes, and Role-Level content with 6 roles. (3) ActorMind is an off-the-shelf, multi-agent, chain-of-though style reasoning framework that emulates how human actors perform in theaters. Concretely, ActorMind first reads its assigned role description via Eye Agent, then comprehends emotional cues within contextual spoken dialogues through Ear Agent. Subsequently, Brain Agent generates a descriptive emotional state, and finally, Mouth Agent delivers the scripts infused with corresponding emotion state. Experimental results demonstrate the effectiveness of ActorMind in enhancing speech role-playing.
Tags
Links
- Source: https://arxiv.org/abs/2604.11103v1
- Canonical: https://arxiv.org/abs/2604.11103v1
Trouble viewing inline? Open PDF directly â
Full Text
57,213 characters extracted from source content.
Expand or collapse full text
ActorMind: Emulating Human Actor Reasoning for Speech Role-Playing Xi Chen, Wei Xue * , Yike Guo The Hong Kong University of Science and Technology chenxi.mail.1005@gmail.com, weixue@ust.hk Abstract Role-playing has garnered rising attention as it provides a strong foundation for human- machine interaction and facilitates sociological research. However, current work is confined to textual modalities, neglecting speech, which plays a predominant role in daily life, thus lim- iting genuine role-playing. To bridge this gap, we conceptualize and benchmark speech role- playing through ActorMindBench, and we present a corresponding reasoning framework, called ActorMind. Specifically, (1) Speech Role-Playing enables models to deliver sponta- neous responses with personalized verbal traits based on their role, the scene, and spoken di- alogue. (2) ActorMindBench is a hierarchi- cal benchmark comprises Utterance-Level con- tent with 7,653 utterances, Scene-Level con- tent with 313 scenes, and Role-Level content with 6 roles. (3) ActorMind is an off-the- shelf, multi-agent chain-of-though style rea- soning framework that emulates how human actors perform in theaters. Concretely, Actor- Mind first reads its assigned role description via Eye Agent, then comprehends emotional cues within contextual spoken dialogues through Ear Agent. Subsequently, Brain Agent gener- ates a descriptive emotional state, and finally, Mouth Agent delivers the scripts infused with corresponding emotion state. Experimental re- sults demonstrate the effectiveness of Actor- Mind in enhancing speech role-playing. The project page is available athttps://github. com/OzymandiasChen/ActorMind. 1 Introduction Role-playing (RP) involves customizing models, particularly Large Language Models (LLMs), to generate spontaneous, human-like responses with personalized traits based on the surrounding con- text Chen et al. (2025). Recently, RP has gar- nered rising attention as it represents genuine ma- chine intelligence and creativity. It enables LLMs * Corresponding author to offer nuanced interaction experiences for users (Moore Wang et al., 2024), provide emotional value (Shao et al., 2023; Johansson, 2025), and support sociological research (Dai et al., 2024; Chan et al., 2024; Zhang et al., 2025a). Numerous benchmarks and methods (Shao et al., 2023; Moore Wang et al., 2024; Zhang et al., 2025b) have been proposed recently. However, they primarily focus on text modality, overlooking that human activities occur across multiple modal- ities, including text, audio, and vision. Among these, audio, especially speech, which is predomi- nant for conveying emotions and attitudes in daily life, reveals persona in a direct and vivid way. Both Large Language-Audio Models (LLAMs) and Text- to-Speech Synthesis (TTS) models are capable of generating speech: LLAMs (Xu et al., 2025; Hurst et al., 2024) exhibit strong capabilities in instruction-following, while recent TTS models en- able fast-speed (Chen et al., 2024a) and zero-shot speech synthesis (Du et al., 2024). Despite these advances, existing models still lack the ability to produce spontaneous, persona-consistent speech re- sponses. Therefore, developing publicly available benchmarks and principled reasoning frameworks for speech role-playing is crucial. To bridge this gap, we (1) conceptualize speech role-playing; (2) propose a publicly available benchmark ActorMindBench, along with corre- sponding tool pipeline; and (3) introduce Actor- Mind, a multi-agent (Mohammadi et al., 2025a) chain-of-thought (CoT) style (Wei et al., 2022) speech role-playing method inspired by emulat- ing the script delivery process of human actors in theaters (Stanislavski and Benedetti, 2009). Speech Role-Playing involves injecting roles into speech generation and interacting via deliv- ering target scripts with personalized verbal at- tributes, such as âWistful flirtation, tinged with a hint of playful vulnerabilityâ, based on the context, including scene descriptions and historical spoken arXiv:2604.11103v1 [cs.SD] 13 Apr 2026 dialogues. ActorMindBench is hierarchically designed with three levels of data.Utterance-Level in- cludes speech segments with text content and speaker labels; Scene-Level includes scene bound- aries and descriptions; Role-Level includes role pro- files. Specifically, ActorMindBench is constructed from well-known television sitcoms, ensuring the authenticity and naturalness of the data. ActorMind is a multi-agent CoT style reason- ing framework that facilitates speech role-playing by emulating how human actors perform in the- aters. Typically, before performing, human actors first read the scripts to understand their roles and gain a rough understanding of how the scene devel- ops. While acting, they carefully listen to the tones and emotions conveyed by other actors. By com- bining their character, scene description, and the emotions of others, they formulate their own emo- tion and tone for delivering the next line. Finally, they deliver the line spontaneously (Stanislavski and Benedetti, 2009). Inspired by this process, Ac- torMind, shown in Figure 2, conceptualizes four agentsâEye, Ear, Mouth, and Brain. The Eye Agent handles character profile and scene script reading; the Ear Agent focuses on listening to the speech tones of others; the Brain Agent is respon- sible for emotion state reasoning; and, aided by Retrieval Argument Generation (RAG) (Fan et al., 2024), the Mouth Agent delivers the script with the desired emotion and voice by referencing emo- tionally similar historical speeches. Experiments on ActorMindBench validate the effectiveness of ActorMind. Notably, ActorMind is an off-the-shelf reasoning framework that can be easily utilized. Briefly, our contributions are threefold: 1. We propose ActorMindBench, a publicly available, hierarchical benchmark for speech role-playing, along with its construction pipeline. It includes Utterance-Level con- tent with 7653 utterances, Scene-Level con- tent with 313 scenes, and Role-Level content with 6 roles. 2.We introduce ActorMind, a multi-agent CoT style, off-the-shelf speech role-playing method inspired by how human actors per- form in theaters. 3.The evaluation results provide compelling evidence of ActorMindâs remarkable perfor- mance. 2 Related Works 2.1 Role-Playing in LLM The advancement of LLMs (Vaswani et al., 2017; Achiam et al., 2023; Dubey et al., 2024) has signif- icantly shaped and catalyzed the development of role-playing. By leveraging supervised fine-tuning (Wei et al., 2021) and in-context learning (Brown et al., 2020; Li et al., 2025), role-playing can be achieved by training (Chen and Zeng, 2025; Chen et al., 2022b) or prompting LLMs with high-quality, character-specific dialogues. The majority of exist- ing work focuses on the text modality. For example, (Chen et al., 2022a) is built upon the well-known Harry Potter universe to establish authentic role- playing, while (Moore Wang et al., 2024) is devel- oped using artificial datasets, enabling role-playing agents across a wide range of environments and cir- cumstances. Furthermore, (Dai et al., 2024) is the first work dedicated to role-playing in the language- vision modality, extending the boundaries of role- playing into the multimodal domain. In this work, we further extend role-playing into the speech domain, as speech is the predominant modality for conveying emotio and information (Chen, 2024). Specifically, ActorMindBench (Sec- tion 3) and ActorMind (Section 4) together provide a comprehensive benchmark and a principled rea- soning framework for speech role-playing. 2.2 Speech Generation Models Both Large LanguageâAudio Models (LLAMs) and Text-to-Speech (TTS) models are capable of generating speech, yet they exhibit complementary strengths and limitations with respect to speech role-playing. (1). LLAMs (Xu et al., 2025; Hurst et al., 2024) are designed to perform complex rea- soning and instruction following, with inputs and outputs spanning both text and audio modalities. Representative models such as Qwen-Omni (Xu et al., 2025) and GPT-4o (Hurst et al., 2024) demon- strate strong capabilities in multimodal understand- ing. However, their supported voice inventories are typically very limited, often ranging from only a few to around ten voices. This constraint fundamen- tally restricts their ability to perform fine-grained role-playing, such as convincingly portraying spe- cific characters (e.g., âHarry Potterâ). (2). TTS models, such as SparkTTS and IndexTTS (Wang et al., 2025; Deng et al., 2025), take text as input and generate corresponding speech. These models exhibit strong in-context learning and zero-shot ca- pabilities for voice cloning and speaking style trans- fer. Nevertheless, they generally lack role-playing abilities: they struggle to adopt role-specific speak- ing styles and to respond spontaneously and coher- ently to dynamic scenes and dialogues. In role-playing with LLMs, generated responses typically exhibit strong textual persona traits, such as characteristic phrasing or catchphrases (Ma et al., 2026). Analogously, speech role-playing requires generated speech to convey spontaneous and au- thentic character traitsâfor example, a speaking style described as âwistful flirtation, tinged with a hint of playful vulnerability.â In this work, Actor- Mind equips speech generation models with such capabilities, serving as a generalizable framework for speech role-playing. 2.3 Chain-of-Thought Style Reasoning Chain-of-thought (CoT) reasoning (Wei et al., 2022; Ling et al., 2026) is a technique in which models are guided to generate explicit intermediate reasoning steps, enabling more effective handling of complex problems that require multi-step infer- ence. It is based on the assumption that generating more tokens for reasoning can lead to improved performance (Muennighoff et al., 2025; Jin et al., 2026). CoT has advanced numerous fields, includ- ing enhancing multilingual factual reasoning (Wei- hua et al., 2026), mitigating hallucinations (Weihua et al., 2025), and extending to multimodal domains such as autonomous driving (Zeng et al., 2025). In this work, we extend CoT to the speech role- playing domain. By emulating human actors per- forming in theater, ActorMind adopts an âeye-ear- brain-mouthâ reasoning process, enabling intuitive and coherent speech-based role-playing. 2.4 LLM Agent An LLM agent (Mohammadi et al., 2025b) is a computational system that leverages a large lan- guage model (LLM) as its core reasoning engine, enabling it to interpret instructions, make deci- sions, and interact with external tools (Yang et al., 2026b,a) or environments (Yao et al., 2022) to ac- complish complex tasks. LLM agents have driven progress across numerous fields, including social network simulation (Zhang et al., 2025c) and logi- cal reasoning (Zhang et al., 2026). In this work, the proposed âeye-ear-brain-mouthâ reasoning process is realized through a set of co- ordinated agents. The Ear Agent is powered by Automatic Speech Recognition (ASR) and Speech Emotion Captioning (SECAP), enabling it to per- ceive both linguistic and emotional signals. The Brain Agent, powered by an LLM, performs high- level emotion state reasoning. Finally, the Mouth Agent leverages RAG to deliver scripts infused with corresponding emotion state. 3 ActorMindBench ActorMindBench is hierarchically designed with three levels of data: Utterance-Level, Scene-Level, and Role-Level. An example from ActorMind- Bench is shown in Figure 4 in Appendix A.1. In this section, we will present our design principle and construction pipeline. 3.1 Design Principle Components. Human beings naturally enjoy role-playing and theatrical performance, which mo- tivates the design of role-oriented benchmarks and modeling methods. Inspired by bau (1965), Ac- torMindBench is structured around three levels of content: âąUtterance-Level: individual lines from theater scripts, including the speaker name, textual con- tent, and corresponding speech data; âą Scene-Level: scene descriptions paired with their associated utterances, where each scene repre- sents a coherent segment reflecting an event or plot development; âą Role-Level: textual profiles. Persona Consistency. Existing role-playing benchmarks often rely on LLM-generated dia- logue (Dai et al., 2024; Moore Wang et al., 2024), which typically requires human verification or con- straints to preserve personality consistency, factual grounding, and other attributes. In contrast, Actor- MindBench is constructed from the widely known Friends Season 1 1 , ensuring naturally consistent personas, stable character knowledge, and high- quality human-written dialogue. 3.2 Construction Pipeline As illustrated in Figure 1, the overall construction pipeline consists of three stages: Utterance-Level. We process original audio episodes through speech denoising, diarization, and recognition to obtain clean speech segments with 1 https://en.wikipedia.org/wiki/Friends_season_ 1 ... Scene: Monica and Phoebe are walking to a newsstand when ...... Scene: The scene takes place at Central Perk, where the gang is hanging out. Rachel ...... Scene-Level Content Raw Audio Original Script with Scene Annotations Web Search ... Rachel: Initially introduced as a spoiled, sheltered rich girl, Rachel evolves over...... Ross: A unique blend of intelligence, insecurity, and lovable quirks,.... Role-Level Content Phoebe: I just wannacheck my horoscope, ...... Rachel: Chandler, I gotta tell you .... ... ... Ross: .... Utterance-Level Content Front-end Speech Processing Speech Denoising,Diarization, and Recognition Semantic-Level Text Processing Scene Cutting and Captioning Figure 1: Overview of ActorMindBench. ActorMindBench comprises three content levels: Utterance-Level includes speech segments with text content and speaker labels; Scene-Level includes scene boundaries and descriptions; Role-Level includes role profiles. speaker labels and text content. (1) Speech Denois- ing removes background noise, music, and environ- mental sounds from speech signal. After denoising, we obtain a clean and high-quality speech signal. We use resemble-enhance 2 . (2) Speech Diariza- tion is the process of partitioning an speech signal containing human speech into segments based on the identity of each speaker. After diarization, the denoised speech signal from an entire episode can be labeled with who spoke at which time, allowing us to obtain speech segments with role labels. We use pyannote-audio 3 . (3) Speech Recognition con- verts speech into text, after which, we can extract the textual content from the speech. We use Whis- per 4 Radford et al. (2023). After these processes, we obtain utterance-level content, including speech segments with corresponding role labels and tex- tual content. Scene-Level. Sceneboundaries,indicating which utterance starts and ends a scene, are obtained by crawling online scripts with scene boundaries and then aligning them with the utterance context.Once the boundaries are identified, we use Llama3 5 Dubey et al. (2024) to generate descriptive scene captions based on the dialogue within the scene. An illustration of the prompt can be seen in Figure 5 of Appendix A.2. 2 https://github.com/resemble-ai/ resemble-enhance 3 https://github.com/pyannote/pyannote-audio 4 https://github.com/openai/whisper 5 https://huggingface.co/meta-llama/ Meta-Llama-3-8B Role-Level. To ensure the high authenticity of ActorMindBench, the roles included are well- known characters: Rachel, Monica, Phoebe, Joey, Chandler, and Ross. Wikipedia 6 already provides a detailed illustration of these roles. As shown in Figure 6 of Appendix A.2, we used Llama3 Dubey et al. (2024) to summarize the Wikipedia pages to obtain the role profiles. 3.3 Statistic ActorMindBench is derived from Season 1 of Friends, which contains 24 episodes. After pro- cessing, we obtain: âąUtterance-Level: 7,653 utterances, correspond- ing to 5 hours and 15 minutes of speech; âąScene-Level: 313 scenes, with an average of 28.7 utterances and 4.23 roles per scene; âąRole-Level: textual profiles for the 6 main char- acters. Comprehensive episode- and role-level statistics are provided in Appendix A.3. 4 ActorMind 4.1 Preliminaries 4.1.1 Notation Utterance-Level. We represent the utterance as U =(U r i , U s i , U t i ) N u i=1 ,(1) whereN u denotes total utterance number, and each utteranceU i is a triplet of the role indicatorU r i , 6 https://en.wikipedia.org/wiki/Main_Page Monica: Phoebe, what are you doing? Emotion: Concerned, slightly exasperated...... Phoebe (Me): Maybe nobody's tried this Monica: I wish we at least knew his name... Emotion: Concerned, slightly exasperated...... Phoebe (Me): Yeah, but did you see the dents in his knuckles? That means he's artistic. Monica: Okay, he's a lawyer, who teaches sculpting on the side. And- he can dance! Emotion: Amused, optimistic, and slightly fanciful, with a hint of romanticism Ear: Who said what, and how they said it I am an actor, destined to portray .... I am performing in the scene .... The dialogue I have heard so far is: Speaker: ..., Line: ..., Emotion: ... My next utterance to deliver is: .... The emotion I should convey in delivering this utterance is: ? Brain: The delivery of my next utterance is shaped by my role, othersâ dialogue and emotions, and the scene context I observe. Powered by retrieval argument generation and text-to-speech synthesis. Mouth: Speak the utterance with the intended emotion. Retrieve: Based on the target emotion state generated by Brain, fetch a previous audio sample expressing a similar feeling. Generation: Prompting LLAMs for speech synthesis via the retrieved audio sample Phoebe: Phoebe Buffay is the quirkiest and most unconventional member of the Friends group. Her personality is a unique blend of optimism, eccentricity, and kindness,... ... Eye: Read role profiles & the dialogue, scene descriptions. Scene:...... Dialogue: Monica: .... Phoebe: ... Monica: ... Ross: ... Scene: The scene takes place in a hospital, where Monica and Phoebe are visiting a guy who is in a coma. Monica is struggling to understand why she reacted so strongly to the guy, even though he's unconscious. Phoebe tries to lift the guy's spirits ... ... Phoebe, what are you doing? Monica Maybe nobody's tried this Phoebe Yeah, but did you see the dents in his knuckles? That means he's artistic. Phoebe Okay, he's a lawyer, who teaches sculpting on the side. And- he can dance! Monica Oh! And, he's the kinda guy who, when you're talking, he's listening, y'know, and not saying 'Yeah, I understand' but really wondering what you look like naked.", Phoebe I wish we at least knew his name... Look at that face. I mean, even sleeping, he looks smart. I bet he's a lawyer. Monica Figure 2: Overview of ActorMind. ActorMind operates in a multi-agent chain-of-thought reasoning style. Specifi- cally: (1) The Eye Agent grasps the scene descriptions and role profiles. (2) The Ear Agent listens to the tones expressed by others. (3) The Brain Agent brainstorms the emotional state for the next line of dialogue based on what has been seen and heard. (4) Powered by RAG, the Mouth Agent retrieves the most similar speech from its won database with a comparable emotional description, mimics it, and spontaneously delivers it. speech signalU s i , and the corresponding textual content U t i . Scene-Level. Scene is represented as S =(S desc j , S bd j ) N s j=1 ,(2) whereN s denotes total scene number, and each sceneS j is a tuple of textual scene description S desc j and scene boundaryS bd j , which indicates the start and end points of utterances. Role-Level. Role information is represented as R =R k N r k=1 ,(3) whereN r is total role number andR k is a textual role profile. 4.1.2 Problem Definition Given scene descriptionS desc j and dialogue se- quence(U p ,· , U qâ1 ), theU r q -playing model should spontaneously perform the next line Ì U t q , cor- responding to the text U t q , in an oral manner. 4.2 Overview ActorMind is a multi-agent CoT reasoning frame- work that facilitates speech role-playing by emulat- ing how human actors perform in theater. Specifi- cally, to conduct the role-playing ofR k in sceneS j : First, the Eye Agent grasps the scene descriptions and role profiles. Then, the Ear Agent listens to the tones and emotion expressed by others. Next, the Brain Agent brainstorms the emotional state for the next line of dialogue based on what has been seen and heard. Finally, powered by RAG, the Mouth Agent retrieves a most similar speech from its own database with a comparable emotional description, mimics it, and spontaneously delivers the target line. 4.3 Eye Agent In the preparatory stage, the Eye Agent reads, con- text textual dialogue(U t p ,· , U t qâ1 ), the prepara- tory descriptive content, including the textual role profileR k and the scene descriptionS desc j , and re- tains them in memory. 4.4 Ear Agent During role-playing, empowered by Speech Emo- tion Captioning tools (SECAP) (Xu et al., 2024), the Ear Agent listens to the dialogue se- quence(U s p ,· , U s qâ1 )and extracts the corre- sponding speech tone and emotional description (E p ,· , E qâ1 ), which will be logged in textual format: (E p ,· ,E qâ1 ) = Ear[(U s p ,· , U s qâ1 )] = SECAP [(U s p ,· , U s qâ1 )]. (4) 4.5 Brain Agent The Brain Agent serves as a central component in speech role-playing, responsible for role injec- tion and deep contextual understanding. Leverag- ing the powerful reasoning capabilities of LLMs (Moore Wang et al., 2024; Dubey et al., 2024), the Brain Agent infers a reasonable emotional state f E q for delivering the next lineU t q based on what ActorMind has just perceived (seen and heard): f E q = Brain[R k , S desc j , (U t p , E p ),· , (U t qâ1 , E qâ1 ), U t q ] = LLM [P rompt ear , R k , S desc j , (U t p , E p ),· , (U t qâ1 , E qâ1 ), U t q ]. (5) 4.6 Mouth Agent Supported by RAG, the Mouth Agent retrieves a previously performed speech segmentU s x from its databaseDatabase U k whose emotional state is most similar to f E q . Leveraging the in-context learning capability of TTS models, the agent is then prompted with the target textU t q together with the retrieved speechU s x , enabling it to render the target utterance with the voice and emotional tone of the retrieved sample: f U s q = M outh( f E q , Database U k , U t q ) = RAG( f E q , Database U k , U t q ). (6) 5 Experiment 5.1 Dataset ActorMindBench is constructed from Friends Sea- son 1 (24 episodes), with details on structure and statistics illustrated in Section 3. Episodes 1â10 and 15â24 are used for training and deployment, while episodes 11â14 are reserved for testing. This split ensures that the training and deployment data encompass a broad range of emotional expressions, from relatively neutral states in earlier episodes to higher-intensity emotions in later episodes, thereby supporting robust role modeling. 5.2 Evaluation Metric We utilize the mean opinion score (MOS) (Chu and Peng, 2006) to measure the perceived quality of the generated speech. To adapt this metric for the role-playing setting, we introduce the RP-MOS. It ranges from 1 to 5, with 1 indicating the lowest quality and 5 the highest. In the speech role-playing context, we identify two pivotal aspects: (1) Exact Delivery and (2) Emotion Expression. Exact Delivery. It refers to the accurate imper- sonation of the intended characterâs voice and the precise articulation of target words. This aspect is fundamental to role-playing; without it, effective speech role-playing may not be feasible. Given this landscape, we consider Exact Delivery a prerequi- site capability. In our RP-MOS, if the generated speech fails to resemble the intended characterâs voice or does not convey the correct content, we assign a lowest score of 1. Emotion Expression.As the adage goes, âthere are a thousand Hamlets in a thousand peopleâs eyesââhuman interpretation of emotion expres- sion can vary greatly.For research purposes, however, a clear and reproducible evaluation cri- terion is crucial. Thus, we consider the emo- tional expression evident in the original speech segmentsâreflected through prosodic cues such as tone, tempo, and intensityâas the ground truth proxy for role-playing emotion expression quality. By evaluating the emotional alignment between the modelâs output and this ground truth, we can capture a measurable dimension of emotion expres- sion: the modelâs ability to reproduce the intended expressive stance of a character within a specific scene. Further details on the RP-MOS instructions and evaluator guidelines are provided in Appendix B.2. 5.3 Baselines We evaluate ActorMind against six baseline meth- ods. These methods are grouped into two cat- egories: (1) LLAM, and (2)â(6) TTS models. Specifically, (1). Qwen_Omni (Xu et al., 2025) is a multimodal foundation model capable of pro- cessing and generating text, images, and audio, supporting real-time, multilingual, and multimodal interactions We report results using the official 7B checkpoint. 7 . The prompt used for Qwen_Omni speech role-playing is shown in Figure 7 (Ap- pendix B.1). (2). CosyVoice (Du et al., 2024) is a semantic-codec-based TTS model. We evalu- ate the official 0.5B checkpoint. 8 (3). SparkTTS (Wang et al., 2025) is an efficient TTS model that combines a single-stream disentangled codec with an LLM backbone. We evaluate the official 0.5B checkpoint. 9 (4). IndexTTS (Deng et al., 2025) is an autoregressive zero-shot TTS model with con- trollable duration and expressive emotion model- ing. We evaluate the officialâŒ0.5B checkpoint. 10 7 https://huggingface.co/Qwen/Qwen2.5-Omni-7B 8 https://w.modelscope.cn/studios/qaz321456/ CosyVoice2-0.5B 9 https://huggingface.co/SparkAudio/ Spark-TTS-0.5B 10 https://huggingface.co/IndexTeam/IndexTTS-2 PhoebeJoeyChandlerRachelRossMonicaAverage YourTTS (Casanova et al., 2022)2.90± 0.892.47± 0.902.30± 1.401.80± 0.842.60± 1.192.30± 0.912.39± 0.93 F5-TTS (Chen et al., 2024b)2.60± 1.082.33± 0.753.60± 0.653.00± 1.582.90± 0.742.80± 0.452.87± 0.77 Cosyvoice (Du et al., 2024)2.30± 0.762.67± 0.712.10± 0.221.40± 0.552.00± 0.941.80± 0.672.04± 0.45 SparkTTS (Wang et al., 2025)3.40± 1.082.53± 0.802.90± 0.422.20± 1.303.20± 0.912.00± 0.942.71± 0.78 IndexTTS (Deng et al., 2025)3.80± 0.672.20± 0.653.30± 0.273.20± 0.842.60± 1.083.20± 0.763.05± 0.56 Qwen_Omni (Xu et al., 2025)1.00± 0.001.00± 0.001.00± 0.001.00± 0.001.00± 0.001.00± 0.001.00± 0.00 ActorMind (Ours)4.00± 0.053.47± 0.613.20± 0.453.40± 0.893.70± 0.573.60± 0.553.56± 0.27 Table 1: Main Results. Subjective evaluation using RP-MOS for ActorMind and baseline models. (5). YourTTS (Casanova et al., 2022) is a flow- matching-based generative model for high-quality speech synthesis. We evaluate the officialâŒ90M checkpoint. 11 (6). F5-TTS (Chen et al., 2024b) is a fully non-autoregressive TTS model based on flow matching with a Diffusion Transformer. We evaluate the official 300M checkpoint. 12 5.4 Implementation Please refer to Appendix B.3. 6 Results and Analysis 6.1 Main Result Subjective evaluation using RP-MOS is presented in Tables 1. âą Overall, on the average score across all roles, Ac- torMind demonstrates superior performance, out- performing all baseline LLAMs and TTS mod- els. This indicates that, in speech role-playing scenarios, ActorMind effectively considers role profiles, scenes, and dialogue to respond spon- taneouslyâcapabilities not present in current models. This positions ActorMind as a pioneer- ing model in the realm of speech role-playing, advancing the field significantly. âąAmong the roles, ActorMind does not achieve optimal performance for Chandler in the subjec- tive evaluation. This may be attributed to their vivid and diverse speaking styles, which demand more advanced reasoning capabilities and metic- ulous design in future models. âąAmong all models evaluated, Qwen_Omni ex- hibited the poorest performance. There are sev- eral contributing factors: (1) The limited set of voices provided by Qwen_Omni does not align with the roles on ActorMindBench; (2) Qwen_Omni is primarily designed for multi- modal understanding, resulting in many gener- ated speech segments that are neutral and lack- 11 https://github.com/Edresson/YourTTS 12 https://huggingface.co/SWivid/F5-TTS/tree/ main/F5TTS_Base ing in expressiveness necessary for role-playing; and (3) During our experiments, when prompts were longâincorporating role profiles and con- textual detailsâQwen_Omni struggled to accu- rately express the intended content, rendering it unsuitable for role-playing applications. 6.2 Ablation Study We conduct ablation studies to evaluate the con- tribution of each agent in ActorMind, as well as the necessity of the role profile, scene description, and context in the speech role-playing setting. Ac- torMind operates as a sequential pipeline. There- fore, removing any component may disrupt this pipeline, leading to interdependent effects in the ablation study. For example, removing the Brain agent effectively disables the RAG mechanism in the Mouth agent. The Eye agent provides all essential textual in- puts for role-playing, including the role profile, scene description, and context textual lines. To as- sess its contribution and the necessity of its inputs, we conduct three ablation settings: (1) w/o Role Profile (w/o Eye), (2). w/o Scene (w/o Eye), (3). w/o Context (w/o Eye, w/o Ear). In (3), remov- ing the textual context eliminates dialogue-based speech emotion processing; therefore, the Ear agent is also removed. The Ear agent processes speech emotion infor- mation. To evaluate its effect, we conduct: (4). w/o Ear, where only textual context is used without speech emotion cues. The Brain agent infers the emotion of the target utterance. Without the Brain agent, RAG in the Mouth agentâ which relies on inferred emotions to retrieve speech prompts for TTSâcannot function. Moreover, information from the Eye and Ear agents becomes ineffective. Therefore: (5). w/o Brain (w/o All), which corresponds to w/o Eye, w/o Ear, w/o Brain, w/o Mouth. We report relative performance with respect to ActorMind across six roles. Subjective evaluation is measured by the average RP-MOS difference. PhoebeJoeyChandlerRachelRossMonicaALL ActorMind + F5-TTS1.00± 0.000.75± 0.290.75± 0.500.50± 0.580.88± 0.250.75± 0.290.77± 0.18 ActorMind + Cosyvoice0.88± 0.250.63± 0.250.75± 0.290.50± 0.58 0.38± 0.480.63± 0.480.63± 0.16 ActorMind + SparkTTS0.50± 0.000.88± 0.251.00± 0.001.00± 0.001.00± 0.001.00± 0.000.90± 0.04 ActorMind + IndexTTS0.88± 0.250.75± 0.5 0.25± 0.500.75± 0.500.88± 0.251.00± 0.000.75± 0.17 ActorMind + YourTTS0.63± 0.480.50± 0.410.88± 0.250.50± 0.581.00± 0.000.50± 0.580.67± 0.17 Table 2: Performance improvement over baseline models after applying ActorMind. RP-MOSâ (1)w/o Role Profile (w/o Eye)-0.37± 0.21 (2)w/o Scene (w/o Eye)-0.30± 0.17 (3)w/o Context (w/o Eye, w/o Ear)-0.22± 0.14 (4)w/o Ear-0.32± 0.23 (5)w/o Brain (w/o All)-0.51± 0.56 Table 3: Ablation study. Relative performance with respect to ActorMind across six roles. As shown in Table 3, the results of (1)-(3) indi- cate that removing any of these components leads to performance degradation, highlighting their im- portance in speech role-playing setting. Among them, removing the role profile results in the largest performance drop, demonstrating that role pro- file information is the most critical component for speech role-playing. This observation aligns with intuitive expectations for role-conditioned genera- tion. Overall, the results from settings (1)â(5) show that each component in ActorMind is necessary, validating the soundness of our method design. 6.3 Generalization of ActorMind ActorMind is a multi-agent CoT reasoning frame- work. It operates in an off-the-shelf manner without requiring additional training, making generaliza- tion a core consideration. To evaluate this prop- erty, we replace the speech generation component with different models, and compare ActorMind + [MODEL] against each corresponding standalone model, thereby assessing ActorMindâs effective- ness as a universal reasoning framework. We inten- tionally omit Qwen_Omni, as it does not support target voice generation and therefore cannot sup- port role-playing. In this experiment, we conduct subjective eval- uations, where evaluators assign a score of 1 to indicate a clear improvement, 0.5 to indicate equiv- alence, and 0 to indicate degradation relative to the baseline model. We recruit six English-speaking evaluators for this study. As shown in Table 2, except for ActorMind + CosyVoice on Ross and ActorMind + IndexTTS (a) CosyVoice(b) SparkTTS(c) IndexTTS (d) F5-TTS(e) YourTTS(f) Qwen-Omni (g) ActorMind(h) Ground Truth Figure 3: Spectrogram Comparison of baselines and ActorMind. All samples are generated for Phoebe per- forming "...So, um, do you think heâs doing any better than he was this morning?" under the same scene and context. on Chandler, all ActorMind + [MODEL] configu- rations achieve scores higher than 0.5, demonstrat- ing consistent performance gains over their corre- sponding baselines. Moreover, five configurations achieve a score of 1, indicating absolute improve- ment and further highlighting the effectiveness and robustness of ActorMind as a general-purpose rea- soning method. 6.4 Qualitative Analysis We visualize spectrograms of the generated speech to qualitatively evaluate speech role-playing ability. In each spectrogram, the x-axis represents time, reflecting the temporal dynamics and tone, while the y-axis represents the energy distribution across frequency bins, reflecting the vocal characteris- tics. Higher similarity to the ground-truth spec- trogram indicates better alignment in both prosody and speaker-specific traits. Figure 3 presents spectrograms of the generated outputs alongside the corresponding ground-truth speech. As shown: âąTTS models (Figure 3 (a)â(e)) use randomly sampled prompts, resulting in arbitrary tone and prosody. Although these models can success- fully generate the target utterance with the target voice, their energy distributions over time and frequency differ substantially from the ground truth. In contrast, as highlighted by the green boxes, ActorMind exhibits significantly higher spectrogram similarity, indicating more accurate role-consistent prosody and expression. âąLLAM, i.e., Qwen_Omni (Figure 3 (f)), fails to reproduce the target voice. This is reflected in its energy distribution across the frequency axis, which deviates significantly from those of the other models and indicates a mismatch in speaker characteristics. 7 Conclusion To establish speech role-playing, we formalize the concept of speech role-playing and introduce Ac- torMindBench, a public benchmark along with corresponding construction pipeline. We also pro- pose ActorMind, a multi-agent CoT style reasoning framework, which is off-the-shelf and can be ap- plied without additional training. Experimental results on ActorMindBench demonstrate the effectiveness of ActorMind com- pared to baseline models. Additional experiments show that ActorMind is effective as a universal framework across different speech generation mod- els, and qualitative analyses on spectrograms fur- ther illustrate its ability to produce spontaneous speech. 8 Limitations ActorMindBench. ActorMindBench is entirely derived from Friends Season 1, covering six roles within the urban comedy domain. As such, it has a limited set of roles and domain coverage. De- spite these limitations, ActorMindBench offers a valuable test bed for current research. Notably, as shown in the main results, current methods perform poorly in speech role-playing setting, indicating that, although ActorMindBench is limited, it is suf- ficient for researchers to explore and develop new approaches. ActorMind. ActorMind is an off-the-shelf method that does not require any training. While it demonstrates strong performance, further improve- ments may be possible through further training, for example, using reinforcement learning to en- hance the RAG mechanism in the Mouth agent or to improve emotion reasoning in the Brain agent. Nevertheless, as the first system of its kind, Ac- torMind represents a meaningful step forward in speech role-playing. 9 Ethical Considerations ActorMindBench benchmark is built upon the well- known TV series Friends, Season 1, and includes annotations at the utterance, scene, and role levels. However, Friends is protected by copyright. Ac- cordingly, we do notâand will notâdistribute any copyrighted audio content from the series. All annotations in ActorMindBench are gener- ated using publicly available tools and consist ex- clusively of structured annotations. We will pub- licly release only the annotation files, enabling researchers to freely access and use our annota- tions and independently obtain the original Friends episodes through legitimate channels for their own research purposes. This design strictly follows es- tablished community practice and ensures that no copyrighted media is redistributed, thereby avoid- ing any copyright or licensing violations. In summary, our work: âąDoes not distribute any copyrighted audio con- tent; âą Releases only copyright-safe annotations; âą Supports reproducibility and extensibility for fu- ture research; We therefore believe that the ethical and legal usage of the source material in our benchmark is appro- priate and rigorously handled. References 1965. On the performing arts: The anatomy of their economic problems. The American economic review, 55(1/2):495â502. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877â1901. Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren Gölge, and Moacir A Ponti. 2022. Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone. In International conference on machine learning, pages 2709â2720. PMLR. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2024. Chateval: Towards better llm-based evaluators through multi-agent debate. In The Twelfth Interna- tional Conference on Learning Representations. Chaoran Chen, Bingsheng Yao, Ruishi Zou, Wenyue Hua, Weimin Lyu, Toby Jia-Jun Li, and Dakuo Wang. 2025. Towards a design guideline for rpa evaluation: A survey of large language model-based role-playing agents. CoRR. Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2022a. Large language models meet harry potter: A bilingual dataset for aligning dialogue agents with characters. arXiv preprint arXiv:2211.06869. Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. 2024a. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesiz- ers. arXiv preprint arXiv:2406.05370. Xi Chen. 2024. Mmrbn: Rule-based network for multi- modal emotion recognition. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8200â8204. Xi Chen, Yongwei Gao, and Wei Li. 2022b. Singing voice detection via similarity-based semi-supervised learning. In Proceedings of the 4th ACM Interna- tional Conference on Multimedia in Asia, MMAsia â22, New York, NY, USA. Association for Computing Machinery. Xi Chen and Min Zeng. 2025. Prototype conditioned generative replay for continual learning in NLP. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies (Volume 1: Long Papers), pages 12754â12770, Albuquerque, New Mexico. Association for Compu- tational Linguistics. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024b. F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching. arXiv preprint arXiv:2410.06885. Min Chu and Hu Peng. 2006. Objective measure for estimating mean opinion score of synthesized speech. US Patent 7,024,362. Yanqi Dai, Huanran Hu, Lei Wang, Shengjie Jin, Xu Chen, and Zhiwu Lu. 2024. Mmrole: A com- prehensive framework for developing and evaluat- ing multimodal role-playing agents. arXiv preprint arXiv:2408.04203. Wei Deng, Siyi Zhou, Jingchen Shu, Jinchao Wang, and Lu Wang. 2025. Indextts: An industrial-level controllable and efficient zero-shot text-to-speech system. arXiv preprint arXiv:2502.05512. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. 2024. Cosyvoice: A scal- able multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv e-prints, pages arXivâ2407. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Pro- ceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 6491â 6501. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. Yizhu Jin, Zhen Ye, Zeyue Tian, Haohe Liu, Qiuqiang Kong, Yike Guo, and Wei Xue. 2026. Inference-time scaling for diffusion-based audio super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 14982â14990. Linus Johansson. 2025. Open weight large language models as a design material in rpgs. Yanshu Li, Yi Cao, Hongyang He, Qisen Cheng, Xiang Fu, Xi Xiao, Tianyang Wang, and Ruixiang Tang. 2025. M 2 iv: Towards efficient and fine-grained mul- timodal in-context learning via representation engi- neering. arXiv preprint arXiv:2504.04633. Guoming Ling, Zhongzhan Huang, Yupei Lin, Junxin Li, Shanshan Zhong, Hefeng Wu, and Liang Lin. 2026. Neural chain-of-thought search: Searching the optimal reasoning path to enhance large language models. arXiv preprint arXiv:2601.11340. Xiaoxu Ma, Xiangbo Zhang, and Zhenyu Weng. 2026. Stable and explainable personality trait evaluation in large language models with internal activations. arXiv preprint arXiv:2601.09833. Mahmoud Mohammadi, Yipeng Li, Jane Lo, and Wendy Yip. 2025a. Evaluation and benchmarking of llm agents: A survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 6129â6139. Mahmoud Mohammadi, Yipeng Li, Jane Lo, and Wendy Yip. 2025b. Evaluation and benchmarking of llm agents: A survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 6129â6139. Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, et al. 2024. Rolellm: benchmarking, eliciting, and enhancing role-playing abilities of large language models. Findings of the Association for Computa- tional Linguistics: ACL 2024, pages 14743â14777. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xi- ang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel CandĂšs, and Tatsunori B Hashimoto. 2025. s1: Simple test-time scaling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20286â20332. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brock- man, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak su- pervision. In International conference on machine learning, pages 28492â28518. PMLR. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-llm: A trainable agent for role- playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153â13187. Konstantin Stanislavski and Jean Benedetti. 2009. An actorâs work on a role. Routledge. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ćukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, et al. 2025. Spark-tts: An efficient llm-based text-to-speech model with single- stream decoupled speech tokens. arXiv preprint arXiv:2503.01710. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc V Le. 2021. Finetuned lan- guage models are zero-shot learners. arXiv preprint arXiv:2109.01652. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824â24837. Zheng Weihua, Xin Huang, Zhengyuan Liu, Tarun Ku- mar Vangani, Bowei Zou, Xiyan Tao, Yuhao Wu, AiTi Aw, Nancy F. Chen, and Roy Ka-Wei Lee. 2026. Adamcot: Rethinking cross-lingual factual reason- ing through adaptive multilingual chain-of-thought. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40):33863â33871. Zheng Weihua, Roy Ka-Wei Lee, Zhengyuan Liu, Wu Kui, AiTi Aw, and Bowei Zou. 2025. CCL- XCoT: An efficient cross-lingual knowledge transfer method for mitigating hallucination generation. In Findings of the Association for Computational Lin- guistics: EMNLP 2025, pages 1768â1788, Suzhou, China. Association for Computational Linguistics. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, et al. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. Yaoxun Xu, Hangting Chen, Jianwei Yu, Qiaochu Huang, Zhiyong Wu, Shi-Xiong Zhang, Guangzhi Li, Yi Luo, and Rongzhi Gu. 2024. Secap: Speech emotion captioning with large language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19323â19331. Shuo Yang, Soyeon Caren Han, Yihao Ding, Shuhe Wang, and Eduard Hoy. 2026a. Tooltree: Efficient llm agent tool planning via dual-feedback monte carlo tree search and bidirectional pruning. arXiv preprint arXiv:2603.12740. Shuo Yang, Soyeon Caren Han, Xueqi Ma, Yan Li, Mohammad Reza Ghasemi Madani, and Eduard Hovy. 2026b. Evotool: Self-evolving tool-use policy optimization in llm agents via blame-aware muta- tion and diversity-aware selection. arXiv preprint arXiv:2603.04900. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations. Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, Xing Wei, and Ning Guo. 2025. Futuresightdrive: Thinking visu- ally with spatio-temporal cot for autonomous driving. arXiv preprint arXiv:2505.17685. Wenyuan Zhang, Tianyun Liu, Mengxiao Song, Xi- aodong Li, and Tingwen Liu. 2025a. SOTOPIA-âŠ: Dynamic strategy injection learning and social in- struction following evaluation for social agents. In Proceedings of the 63rd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 24669â24697, Vienna, Austria. Association for Computational Linguistics. Wenyuan Zhang, Shuaiyi Nie, Jiawei Sheng, Zefeng Zhang, Xinghua Zhang, Yongquan He, and Tingwen Liu. 2025b. Revealing and mitigating the challenge of detecting character knowledge errors in llm role- playing. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33267â33290. Xinglang Zhang, Yunyao Zhang, ZeLiang Chen, Jun- qing Yu, Wei Yang, and Zikai Song. 2026. Logical phase transitions: Understanding collapse in llm log- ical reasoning. arXiv preprint arXiv:2601.02902. Yunyao Zhang, Zikai Song, Hang Zhou, Wenfeng Ren, Yi-Ping Phoebe Chen, Junqing Yu, and Wei Yang. 2025c.gaâ s 3 : Comprehensive social network sim- ulation with group agents. In Findings of the Asso- ciation for Computational Linguistics: ACL 2025, pages 8950â8970, Vienna, Austria. Association for Computational Linguistics. A ActorMindBench Details A.1 ActorMindBench Example Data Example of ActorMindBench data is shown in Fig- ure 4. A.2 Prompts in ActorMindBench Construction Prompts for scene summarization and role sum- marization can be seen in Figure 5 and Figure 6 separately. A.3 ActorMindBench Detail Statistic Utterance-Level statistics regarding the number and duration of utterances for role âxâ in episode âyâ are provided in Table 4. Scnene-Level statistics regarding the average number of utterances and roles performed per scene in each episode are provided in Table 5. B Experiment Details B.1 Qwen_Omni in Speech Role-Playing The prompt used for Qwen_Omni speech role- playing is shown in Figure 7. B.2 RP-MOS This subjective evaluation assesses the modelâs speech role-playing ability by comparing gener- ated speech with reference (ground-truth) record- ings. Participants listen to the generated speech and assign scores based on its similarity to the refer- ence speech. Ten english speakers evaluated twelve rounds (six roles, with two sets per role), using ran- domly selected utterances across all model variants. All evaluators were provided with detailed guide- lines and evaluation criteria prior to the assessment. Each evaluator was compensated at a rate aligned with the average local hourly income, which we consider fair and appropriate given their country of residence and time commitment. Guidelines and Evaluation Criteria Partici- pants should consider the following aspects of emo- tional expression: âąEmotional Consistency: Does the generated speech convey the same emotional tone as the reference speech? âąIntensity Alignment: Is the strength or inten- sity of the emotion comparable between the two speeches? Figure 4: ActorMindBench Example Data EpisodeRachelMonicaPhoebeJoeyChandlerRossOTHERSTOTAL numdurationnumdurationnumdurationnumdurationnumdurationnumdurationnumdurationnumduration SE01_01860:03:15820:03:08220:00:54390:01:42330:01:23630:02:45250:01:073500:14:13 SE01_02560:02:36370:01:15210:00:44110:00:23290:01:101010:04:16970:03:443520:14:09 SE01_03320:01:12760:02:36650:02:20280:01:04720:02:37530:01:54390:01:313650:13:14 SE01_04810:03:16650:02:27470:01:52360:01:15490:01:55570:02:36360:01:233710:14:44 SE01_05480:02:01400:01:44290:00:57580:02:23500:01:51730:03:30480:02:053460:14:31 SE01_06220:00:57540:02:12220:00:48520:02:151140:04:22320:01:28520:02:023480:14:04 SE01_07490:02:00210:00:43440:01:52380:01:26610:02:47710:03:04270:00:523110:12:45 SE01_08230:01:00450:01:34270:00:59150:00:31600:02:09750:03:07940:03:413390:13:02 SE01_09640:02:28740:02:45310:01:13430:01:28540:02:07650:02:43340:01:203650:14:04 SE01_10320:01:28220:00:50580:02:21200:00:46540:01:54530:02:10770:03:203160:12:49 SE01_11280:01:03480:01:46490:01:54440:01:41530:01:46780:02:43660:02:233660:13:15 SE01_12490:02:04380:01:29490:01:51360:01:26390:01:22710:02:45300:01:183120:12:14 SE01_13260:01:15150:00:30270:01:16520:02:12410:01:35140:00:411060:05:012810:12:30 SE01_14190:00:52200:00:50170:00:50320:01:14510:01:55530:02:31830:03:492750:12:02 SE01_15250:00:57440:02:04390:01:34320:01:15700:02:55350:01:32240:01:012690:11:19 SE01_16270:01:08130:00:36410:02:03220:01:01590:02:34360:01:39750:03:172730:12:17 SE01_17540:02:05630:02:40330:01:29300:01:13250:01:04500:02:121020:03:563570:14:38 SE01_18840:03:42380:01:31340:01:25210:00:55330:01:16590:02:1890:00:202780:11:27 SE01_19910:04:04350:01:21180:00:45190:00:51270:01:06880:03:58400:01:383180:13:43 SE01_20850:03:51300:01:13210:00:48340:01:32620:02:32220:01:05690:02:573230:13:58 SE01_21340:01:26640:03:03110:00:34210:00:57250:01:10510:02:35670:03:092730:12:53 SE01_22270:01:05500:02:16530:02:07160:00:38500:02:03410:01:42400:01:532770:11:44 SE01_23240:01:02250:00:55350:01:40340:01:28220:00:57680:02:441020:04:233100:13:09 SE01_24640:02:59350:01:39190:00:54620:02:30280:01:15360:01:34340:01:382780:12:28 ALL11300:47:4710340:41:068120:33:097950:32:0611610:45:4313450:57:3113760:57:507,6535:15:12 Table 4: Utterance-level statistics on number of utterances and speech duration by role and episode. Figure 5: Prompt for scene captioning: black text indi- cates the prompt, while brownish-yellow highlights the utterances within a scene. Figure 6: Prompt for role profile generation: black text indicates the prompt, while brownish-yellow highlights the role content. EpisodeScene NumAvg Utterances per SceneAvg Roles per Scene SE01_011417.293.93 SE01_02829.755.38 SE01_031320.04.85 SE01_041615.754.19 SE01_051614.943.31 SE01_06924.334.78 SE01_072111.142.95 SE01_081016.94.50 SE01_091219.083.92 SE01_10829.06.00 SE01_111223.924.50 SE01_121517.334.33 SE01_131318.694.31 SE01_141711.123.41 SE01_151417.433.43 SE01_161419.55.07 SE01_171420.144.14 SE01_18833.386.25 SE01_19831.385.12 SE01_201220.334.92 SE01_211514.074.00 SE01_221221.424.00 SE01_232112.764.1 SE01_241123.914.00 ALL31318.74.23 Table 5: Scene-level statistics. Figure 7: Qwen_Omni Prompt for Speech Role-Playing. Black text: prompt template; Brownish-yellow: corre- sponding speech and text content. âąNaturalness and Realism: Does the gener- ated audio sound naturally expressive and be- lievable, rather than artificial or flat? âąOverall Impression: Considering all the above factors, how similar is the emotional quality of the generated speech to the real ref- erence? âą Voice and Content Consistency: If the voice is from different people or the text content differs from the reference, directly assign a score of 1. Scoring ScaleUse the following 5-point scale to rate each speech: âą5 â Identical: The generated speech conveys the same emotion as the reference speech, with nearly identical intensity and expression. âą4 â Very Similar: The emotion is highly sim- ilar, with only minor differences in tone or intensity. âą3 â Moderately Similar: The overall emotion is recognizable but with noticeable differences in strength, tone, or expression. âą 2 â Weak Similarity: The emotion type is somewhat related but largely inconsistent with the reference speech. âą1 â No Similarity: The generated audio con- veys a completely different or unrecognizable emotion compared to the reference; If the Figure 8: Prompt for the Brain Agent used for role injection, contextual understanding, and emotion rendering. Here, yellow content represents what the Eye Agent saw, blue content represents what the Ear Agent heard, and purple content represents what the Brain Agent inferred. voice seems to be from a different speaker, or if the text content differs from the reference, directly assign a score of 1. Evaluation Procedure 1. Listen to each speech at least twice before rating. 2.If the voice is from different people or the text content differs from the reference, directly assign a score of 1. 3.Assign a single integer score (1â5) according to the criteria above. 4.If uncertain, choose the score that best repre- sents your overall impression. B.3 ActorMind Implementation Detail Eye Agent reads the preparatory descriptive con- tent and retains it in memory. In practice, a textual memory of a few hundred words is sufficient. Ear Agent Speech Emotion Captioning (SE- CAP) provides textual and intuitive emotional de- scriptions of target speech signals. SECAP (Xu et al., 2024) equips the Ear Agent with listening and emotion-recognition capabilities. Brain Agent is the central component of Ac- torMind. Following previous LLM role-playing works (Dai et al., 2024; Moore Wang et al., 2024), we use LLama3 (Dubey et al., 2024) to perform emotional state reasoning, with the prompts illus- trated in Figure 8. Mouth Agent employs RAG to retrieve rele- vant context from a database for speech genera- tion. In ActorMind, for each roleR k , the database Database k is constructed from that roleâs known speech utterances. Each entry contains the speech signalU s x as content, with indices corresponding to emotional descriptionsE x generated by SE- CAP (Xu et al., 2024). During the retrieval phase, embeddings are computed using OpenAIâs text- embedding-3-large 13 . During the generation phase, we employ IndexTTS 14 (Deng et al., 2025) as speech synthesizer, where the text prompt is the target line and the tone and emotion prompt is the retrieved speech. 13 https://platform.openai.com/docs/models/ embeddings 14 https://github.com/index-tts/index-tts