Paper deep dive
ExRole: From Team Trajectories to Executable Roles in Multi-Agent Language Models
Zhou Liu, Chaoyang Han, Zewei Pan, Zeli Su, Wentao Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Roles provide an interpretable interface for organizing language-model agents, yet most multi-agent systems treat them as hand-written prompt labels disconnected from learned behavior and parameter updates. We argue that a useful role should instead be an executable control variable: it should summarize behavior predictive of future utility, guide subsequent interaction, and identify the trainable capacity responsible for that behavior. We introduce ExRole, a trajectory-to-role framework that learns future-aware role prototypes from prefix-local team traces, resolves them into readable instructions and token-aligned role markers, and optionally routes shared LoRA rank slots with turn-aligned credit. Across MuSiQue and 2WikiMultiHopQA, ExRole improves over single-agent search by 15.0/14.4 and 13.5/16.1 EM/F1 points, respectively. Against the strongest non-ExRole controls, the corresponding gains remain 11.5/11.6 and 7.7/9.7 points. Across both benchmarks, the controlled results consistently favor trajectory-induced role conditioning over role-free, manual, random, and shuffled alternatives. Role-Agent-Turn interventions further show that the induced roles capture transferable behavioral specialization beyond fixed agent identities or turn positions.
Tags
Links
- Source: https://arxiv.org/abs/2608.11949v1
- Canonical: https://arxiv.org/abs/2608.11949v1
Trouble viewing inline? Open PDF directly →
Full Text
90,759 characters extracted from source content.
Expand or collapse full text
★ ExRole: From Team Trajectories to Executable Roles in Multi-Agent Language Models Zhou Liu Chaoyang Han Zewei Pan Zeli Su Wentao Zhang Abstract Roles provide an interpretable interface for organizing language-model agents, yet most multi-agent systems treat them as hand-written prompt labels disconnected from learned behavior and parameter updates. We argue that a useful role should instead be an executable control variable: it should summarize behavior predictive of future utility, guide subsequent interaction, and identify the trainable capacity responsible for that behavior. We introduce ExRole, a trajectory-to-role framework that learns future-aware role prototypes from prefix-local team traces, resolves them into readable instructions and token-aligned role markers, and optionally routes shared LoRA rank slots with turn-aligned credit. Across MuSiQue and 2WikiMultiHopQA, ExRole improves over single-agent search by 15.0/14.4 and 13.5/16.1 EM/F1 points, respectively. Against the strongest non-ExRole controls, the corresponding gains remain 11.5/11.6 and 7.7/9.7 points. Across both benchmarks, the controlled results consistently favor trajectory-induced role conditioning over role-free, manual, random, and shuffled alternatives. Role-Agent-Turn interventions further show that the induced roles capture transferable behavioral specialization beyond fixed agent identities or turn positions. Introduction Large language model (LLM) agents increasingly solve knowledge-intensive tasks by interleaving reasoning with retrieval (Trivedi et al. 2023; Jin et al. 2025); recent multi-agent search work further distributes planning, retrieval, and synthesis across specialized agents (Chen et al. 2026). Roles offer a simple and interpretable abstraction for organizing this collaboration: they expose the intended division of labor without requiring a separate model for every agent. Role-playing designs commonly instantiate these functions as natural-language profiles fixed in prompts before execution, as in CAMEL’s inception prompting (Li et al. 2023). Figure 1 illustrates that changing such labels alone does not reliably improve answer quality or evidence use under a fixed policy. These labels are neither grounded in observed team behavior nor tied to the learning signal for role turns. (a) Answer quality (b) Evidence-use behavior Figure 1: Role-prompt interventions under a fixed MuSiQue policy. Points show paired changes from no role over the same 200 questions; bars denote 95% paired-bootstrap confidence intervals. We ask: how can a system induce roles from prior collaboration and reuse them to coordinate new episodes and assign policy credit? Learning roles from collaboration trajectories is more demanding than clustering surface-level action counts. First, a role representation must be inferred from information available at the current prefix while still capturing how that behavior affects future evidence and return; otherwise it either ignores delayed utility or leaks suffix information. Second, an induced cluster must become executable in a new episode: the same identity should guide the agent’s natural-language instruction and remain recoverable inside the model. Third, shared team rewards create the familiar credit-assignment problem of identifying which action produced the outcome (Foerster et al. 2018); here the credit must further reach the role turn and capacity path that produced a useful textual action. Recent role-decomposed and segment-typed agentic RL methods likewise show why uniform outcome credit can obscure useful intermediate contributions (Park, Cho, and Lee 2026; Xu et al. 2026). Without these links, role induction, role prompting, and policy optimization remain separate mechanisms rather than a learned specialization process. Our key insight is to treat a role as a shared control variable that connects behavioral abstraction with policy execution. We instantiate this idea in ExRole, an executable role-learning framework illustrated in Figure 2. ExRole first encodes prefix-local action, timing, interaction, and evidence statistics, and uses next-action, future-evidence, and final-return targets to organize trajectories by their prospective utility. It then summarizes each cluster as a reusable prototype and deterministically resolves the prototype into a readable instruction and a token-aligned role marker, without requiring an additional role-labeling language model. The resolved identity controls two complementary paths. At the interaction level, the instruction conditions the active agent and the marker remains aligned with the generated role-turn tokens. At the parameter level, ExRole-Routed combines that marker with the current semantic prefix to select a balanced sparse-delta gate over shared LoRA rank slots. GRPO is augmented with turn-aligned credit so that a useful role turn reinforces both its response tokens and, when routing is enabled, its selected capacity. ExRole-Shared retains the same induced roles and turn-aligned policy credit but uses a uniform shared LoRA path, providing a matched control that isolates the contribution of role-conditioned sparse LoRA routing. We evaluate ExRole on MuSiQue and 2WikiMultiHopQA, using single-agent search, no-role and manual-role teams, random role prompts, and shuffled induced roles to separate trajectory-derived specialization from generic prompting effects. On MuSiQue, ExRole-Shared reaches 31.5 EM and 43.2 F1, improving single-agent search by 15.0 and 14.4 points and the strongest non-ExRole control by 11.5 and 11.6 points. On 2WikiMultiHopQA, ExRole-Routed reaches 50.0 EM and 59.7 F1, gains of 13.5 and 16.1 points over single-agent search and 7.7 and 9.7 points over the strongest control. Across both benchmarks, the controlled comparisons favor trajectory-induced roles over role-free, manual, random, and shuffled alternatives. Role-Agent-Turn interventions further show that the induced roles encode transferable action tendencies beyond fixed agent identities and turn positions. To sum up, our contributions are two-fold: • We formulate LLM-agent role learning as trajectory-conditioned role induction, in which prefix-local behavior and future-utility targets define reusable role prototypes. • We make induced roles executable by binding each prototype to a readable instruction, a token-aligned role marker, and a deterministic agent assignment, and connect that identity to role-conditioned sparse LoRA routing with turn-aligned credit. Related Work Role Discovery in Multi-Agent Reinforcement Learning Role-based multi-agent reinforcement learning (MARL) learns reusable abstractions through identifiable embeddings (ROMA), role-specific action spaces (RODE), trajectory-encoded abilities (LDSA), contrastive representations (ACORM), and future behavioral effects (R3DM), while retaining parameter sharing (Wang et al. 2020; Wang et al. 2021; Yang et al. 2022; Hu et al. 2024; Goel et al. 2025). These methods primarily assume compact states and actions; ExRole extends role induction to language-agent trajectories, where a role must organize messages, retrieved evidence, delayed answers, and trainable capacity. Roles and Credit Assignment in LLM Multi-Agent Systems LLM teams commonly begin with prompt-defined roles, as in CAMEL, while MLC, MasRouter, and ReSo learn role differentiation, allocation, or agent selection (Li et al. 2023; Li et al. 2025; Yue et al. 2025; Zhou et al. 2025). MARFT, MAGRPO, MHGPO, and MATPO instead optimize team policies within a specified organization (Liao et al. 2025; Liu et al. 2026; Chen et al. 2026; Mo et al. 2025). Retrieval-oriented reasoning adds delayed credit: IRCoT interleaves retrieval and reasoning, whereas Search-R1 and R1-Searcher optimize single search policies (Trivedi et al. 2023; Jin et al. 2025; Song et al. 2025). COMA addresses team credit in MARL, and DAC and TRIAGE provide cross-agent or segment-level learning signals (Foerster et al. 2018; Park, Cho, and Lee 2026; Xu et al. 2026). ExRole differs by inducing recurring functions from prior trajectories and reusing the same identity across instructions, role-turn credit, and sparse LoRA routing. Parameter-Efficient Specialization and Routing Shared-backbone systems preserve specialization through selective parameter use: Kaleidoscope learns agent-specific masks and ADMN routes agents through shared modules (Li, Pan, and Zhang 2024; Yu et al. 2024). LoraHub, LoraRetriever, X-LoRA, and MoLE compose or route LoRA adapters from task or input signals (Huang et al. 2024; Zhao et al. 2024; Buehler and Buehler 2024; Wu, Huang, and Wei 2024). ExRole instead derives routing identity from collaboration trajectories and modulates rank slots inside a jointly trained shared LoRA parameterization. Problem Formulation Figure 2: Overview of ExRole. Prior team trajectories induce a future-aware role library. The same role identity conditions prompt-level coordination and role-conditioned sparse LoRA routing, while turn-aligned GRPO updates the policy and router. We study role learning for multi-agent language-model teams in evidence-seeking tasks. Each instance is a pair (x,y)(x,y), where x is the input question and y is a gold answer. A team of N agents interacts with a retrieval environment for at most T turns, producing τ=(x,it,ot,mt,at,et,rtt=1Tτ,y^),Tτ≤T.τ= (x,\i_t,o_t,m_t,a_t,e_t,r_t\_t=1^T_τ, y ), T_τ≤ T. (1) Here iti_t is the active agent, oto_t its observation, mtm_t an optional teammate message, ata_t a search, answer, or invalid action, ete_t retrieved evidence, rtr_t the turn reward, and y y the final answer. A team turn is one active-agent response; a role-turn segment is the token span aligned to its role marker. Let ℋtH_t denote the interaction history available before turn t. Agent i is assigned an executable role ziz_i, and the active policy is conditioned on both that role and its adapter-routing state Γt _t: (mt,at)∼πθ,θb(⋅∣x,ℋt,zit,Γt).(m_t,a_t) _θ, _b (· x,H_t,z_i_t, _t ). (2) ExRole induces a source role library =pkk=0K−1P=\p_k\_k=0^K-1 from prior trajectories and resolves it into N executable roles. The policy parameters θ and, when enabled, router parameters θb _b maximize the expected episode reward R(τ,y) R(τ,y) =∑t=1Tτrt, = _t=1^T_τr_t, (3) (θ⋆,θb⋆) (θ , _b ) =argmaxθ,θb(x,y)τ∼πθ,θb(⋅∣x,exec)[R(τ,y)]. = _θ, _b E_(x,y) E_τ _θ, _b(· x,P_exec)[R(τ,y)]. The complete action grammar, reward decomposition, and answer normalizer are provided in the appendix. Method ExRole addresses the gap between discovering recurring team behaviors and turning them into specialization that can be executed and optimized in later episodes. Given prior collaboration trajectories, our goal is to induce roles that capture future-relevant behavior, bind each role to a consistent interaction and model-side identity, and assign learning credit to the role and capacity path that produced an action. Figure 2 summarizes the resulting pipeline: predictive trajectory encoding and clustering produce a role library; deterministic role binding makes each prototype executable; role-conditioned interaction and sparse LoRA routing apply the same identity during generation; and turn-aligned grouped reinforcement learning updates the policy and, when enabled, the router. The following sections describe these four components in order. Future-Aware Role Induction A useful role should summarize not only what an agent has done, but also what its behavior predicts about subsequent collaboration. For each logged agent turn, ExRole constructs a prefix-local behavior vector ϕtobs _t^obs =[ϕtact,ϕtpos,ϕtteam,ϕtevd,ϕtans]∈30, = [ _t^act, _t^pos, _t^team, _t^evd, _t^ans ] ^30, (4) ϕ~t,jobs φ_t,j^obs =ϕt,jobs−μjσj+10−6. = _t,j^obs- _j _j+10^-6. which summarizes action tendencies, turn position, team context, evidence use, and answer behavior observed up to that point; μj _j and σj _j are corpus statistics for coordinate j. Future events are excluded from ϕtobs _t^obs and used only as prediction targets. A role encoder produces an embedding ξt _t and predicts the same agent’s next action, future evidence utility, and final trajectory return using cross-entropy (CE), binary cross-entropy (BCE), and squared-error terms: ξt _t =fθe(ϕ~tobs), =f_ _e( φ_t^obs), (5) (a^t,v^tevd,R^t) ( a_t, v_t^evd, R_t) =hθe(ξt), =h_ _e( _t), ℒrole _role =CE(a^t,ati+)+λevdBCE(v^tevd,vtevd) =CE( a_t,a_t_i^+)+ _evdBCE( v_t^evd,v_t^evd) +λret(R^t−R(τ(t),yτ(t)))2. + _ret ( R_t-R(τ(t),y_τ(t)) )^2. Here ti+t_i^+ is the next turn of the same agent, or a stop target if no such turn exists. This predictive objective shapes the embedding with future utility without placing suffix information in its input. For each candidate role count, we run Euclidean K-means from each configured seed and summarize every sufficiently supported cluster as a source prototype: K(s) _K^(s) =KMeans10(ξtt=1J,K,s), =KMeans_10(\ _t\_t=1^J;K,s), (6) pk p_k =(ξ¯k,ϕ¯k,ηk),k∈K, =( ξ_k, φ_k, _k), k _K, K _K =k:nk≥n0, =\k:n_k≥ n_0\, where the subscript 1010 denotes ten K-means initializations, s is a discovery seed, nkn_k is cluster support, and n0n_0 is the minimum support. The vectors ξ¯k ξ_k and ϕ¯k φ_k are cluster centroids, and ηk _k records the cluster’s dominant action, stage, evidence, and return statistics. When K is not fixed by the execution budget, it is selected by a separation–stability criterion, K⋆=argmaxK∈[Sil¯(K)+λstabStability(K)].K = _K [ Sil(K)+ _stabStability(K) ]. (7) The appendix defines the seed averages and adjusted-Rand stability statistic. Finally, a deterministic joint resolver scores all surviving prototype–template pairs, assigns distinct functional types when possible, and converts the assignments into readable instructions ψk _k and token-aligned role markers χk _k: Sk,υ S_k, =TemplateScoreυ(ϕ¯k,ηk), =TemplateScore_ ( φ_k, _k), (8) υkk∈K \ _k\_k _K =GreedyDistinct(S;≺tie), =GreedyDistinct(S; _tie), (ψk,χk) ( _k, _k) =Templateυk(ϕ¯k,ηk). =Template_ _k( φ_k, _k). Thus, role names are interpretations of learned behavioral clusters rather than manually specified training labels. Feature definitions, future-target construction, role-count settings, and template rules appear in the appendix. Executable Role Binding Induced clusters become useful only when they control subsequent team behavior. ExRole selects at most N source prototypes, remaps them to contiguous runtime role IDs, and pads missing entries with a generic collaborative role: exec=ResolveN(),(zi,ψzi,χzi)=Bindi(exec).P_exec=Resolve_N(P), (z_i, _z_i, _z_i)=Bind_i(P_exec). (9) The resolved role conditions the observation presented to the active agent, ot=(x,it,zit,ψzit,χzit,ℋt,ℬtmsg,et−1),o_t=O (x,i_t,z_i_t, _z_i_t, _z_i_t,H_t,B_t^msg,e_t-1 ), (10) where ℬtmsgB_t^msg is the shared message board. The generated response is projected to a structured team message and environment action. The marker remains in the token sequence and defines role-turn segments. If uq−u_q^- is the start of marker q, then each token u inherits the latest preceding role: seg(u) (u) =maxq:uq−≤u, = \q:u_q^-≤ u\, (11) z(u) z(u) =zseg(u), =z_seg(u), Γ(u) (u) =Γseg(u). = _seg(u). This construction makes the role used in the prompt identical to the role consumed by the model-side router. The exact resolver, marker matching, and cached-decoding rules are deferred to the appendix. Figure 3: Trajectory-to-role binding and execution in ExRole. A deterministic resolver converts trajectory prototypes into executable roles with aligned instructions, markers, and features; fixed role identities then guide round-robin team interaction and model-side sparse routing through shared memory. Role-Conditioned Sparse Routing Prompt conditioning alone does not determine which trainable capacity supports a role. ExRole-Routed therefore modulates the shared LoRA update at the rank-slot level. For adapted module ℓ and role-turn segment q, LoRAℓrole(h)=Bℓ(q,ℓ⊙Aℓh)αℓ,q,ℓ∈d.LoRA_ ^role(h)=B_ ( γ_q, A_ h ) _ , γ_q, ^d. (12) The router combines the executable-role feature zqv_z_q, its discrete identity, and a pooled semantic prefix ζq _q: ζq _q =Norm(1|q|∑u∈qstopgrad(ℰ(idu))), =Norm ( 1|W_q| _u _qstopgrad(E(id_u)) ), (13) hqb h_q^b =Fθb(zq)+Eθb(zq)+λcCθb(ζq), =F_ _b(v_z_q)+E_ _b(z_q)+ _cC_ _b( _q), ωq,ℓs _q, s =Headθbslot(hqb)ℓs+λutanh(Headθbutil(hqb)ℓs), =Head^slot_ _b(h_q^b)_ s+ _u (Head^util_ _b(h_q^b)_ s ), c^q c_q =Headθbcred(hqb). =Head^cred_ _b(h_q^b). Here qW_q is the prefix window before marker q, iduid_u is the token identifier (ID) at position u, and ωq,ℓs _q, s scores rank slot s of module ℓ . The scores define a probability distribution ϖq _q and a hard budget of S directly selected slots: ϖq,ℓs _q, s =softmax(ωq/τb)ℓs, =softmax( _q/ _b)_ s, (14) Mq,ℓs M_q, s =[(ℓ,s)∈TopS(ωq)], =1[( ,s) ( _q)], δq,ℓssel δ^sel_q, s =Mq,ℓs(Ldϖq,ℓs−1), =M_q, s(Ld\, _q, s-1), δ¯q,ℓs δ_q, s =δq,ℓssel−1Ld∑ℓ′,s′δq,ℓ′s′sel, =δ^sel_q, s- 1Ld _ ,s δ^sel_q, s , γq,ℓs _q, s =clip(1+λgδ¯q,ℓs,γmin,γmax). =clip (1+ _g δ_q, s, _ , _ ). The hard mask concentrates role-specific deviations, while centering preserves a balanced shared update before clipping. Both variants retain the same LoRA parameters and role-conditioned interaction; they differ only in the model-side routing state: Γq=,ExRole-Shared,γq,ℓsℓ,s,ExRole-Routed. _q= cases1,&ExRole-Shared,\\ \ _q, s\_ ,s,&ExRole-Routed. cases (15) Turn-Aligned Optimization An episode-level reward alone assigns the same credit to every response in a team trajectory. ExRole retains the group-relative trajectory advantage Adv(g)Adv^(g) from GRPO, while adding discounted local credit to the response tokens produced at turn t: Utturn,(g) U_t^turn,(g) =∑t′=tTτ(g)λdisct′−trt′(g), = _t =t^T_τ^(g) _disc^t -tr_t ^(g), (16) Adv~t,j(g) Adv_t,j^(g) =It,jresp,(g)[Adv(g)+αcU^t(g)], =I_t,j^resp,(g) [Adv^(g)+ _c U_t^(g) ], where U^t(g) U_t^(g) is the normalized turn return and It,jresp,(g)I_t,j^resp,(g) masks non-response tokens. The same return supplies a detached role-turn credit target cqc_q for segment q: cq=stopgrad[clip(ℬR(Ut(q)turn,(g(q))),−2,2)].c_q=stopgrad [clip (N_B_R(U_t(q)^turn,(g(q))),-2,2 ) ]. (17) Here ℬRB_R is the set of role-turn segments in the current normalization batch. The router learns to predict this target and associate positive role-turn credit with its selected capacity. Writing c~q=clip((cq+2)/4,0,1) c_q=clip((c_q+2)/4,0,1) and letting ν¯qutil ν_q^util denote mean predicted utility over selected slots, the two credit-bearing objectives are ℒfuture _future =12Q∑q=1Q[(ν¯qutil−c~q)2+(sigm(c^q)−c~q)2], = 12Q _q=1^Q [( ν_q^util- c_q)^2+(sigm( c_q)- c_q)^2 ], (18) ℒcredit _credit =−1Q∑q=1Qcq∑ℓ,sMq,ℓslogϖq,ℓs∑ℓ,sMq,ℓs. =- 1Q _q=1^Qc_q _ ,sM_q, s _q, s _ ,sM_q, s. Here sigm(⋅)sigm(·) denotes the logistic sigmoid. Load-balance, role-diversity, sparsity, and entropy terms regularize the remaining routing distribution, while Kullback–Leibler (KL) regularization anchors the policy. The two model variants are therefore optimized under the same turn-aligned policy objective and differ only by the router loss: ℒshared _shared =−GRPOturn+βKLℒKL, =-J_GRPO^turn+ _KLL_KL, (19) ℒrouter _router =λfutureℒfuture+λcreditℒcredit+ℒreg, = _futureL_future+ _creditL_credit+L_reg, ℒrouted _routed =ℒshared+ℒrouter. =L_shared+L_router. This matched formulation isolates the contribution of role-conditioned sparse LoRA routing: ExRole-Shared and ExRole-Routed use the same role library, interaction protocol, reward, and turn-aligned policy credit. The appendix gives the full GRPO objective, normalization fallbacks, router regularizers, pseudocode, and hyperparameters. Experiments Experimental Setup Benchmarks. We use MuSiQue and 2WikiMultiHopQA, which require multi-step evidence composition over supporting-document collections. Baselines. The controlled comparison includes single-agent search, multi-agent systems (MAS) without roles or with hand-written roles, and random or shuffled role controls. All controlled systems share the same policy backbone, action interface, and evaluation protocol. ExRole-Shared and ExRole-Routed additionally share the induced role library and training objective, differing only in whether LoRA capacity is uniformly shared or role routed. Table 2 provides RAG, Search-RL, and MAS-RL references; its base icons distinguish SFT and search-specialized backbones. Metrics. We report Exact Match (EM), token-level F1, and strict success (Succ). EM uses benchmark answer normalization, F1 measures token overlap, and Succ requires a normalized exact answer without substring credit. Complete data, retrieval, optimization, and sequence-budget settings are given in Supplementary Appendix B.6; formal metric definitions appear in Supplementary Appendix C.2. Main Results Method #Ag. Role Cap. MuSiQue 2Wiki EM ↑ F1 ↑ Succ. ↑ EM ↑ F1 ↑ Succ. ↑ Trajectory-induced roles ExRole-Shared 3 Induced Shared 31.5 43.2 31.5 49.0 59.1 49.0 ExRole-Routed 3 Induced Routed 30.0 41.5 30.0 50.0 59.7 50.0 Single-agent baseline Single-agent search 1 None Shared 16.5 (-15.0) 28.8 (-14.4) 16.5 (-15.0) 36.5 (-13.5) 43.6 (-16.1) 36.5 (-13.5) Three-agent baselines No-role MAS 3 None Shared 20.0 (-11.5) 31.6 (-11.6) 20.0 (-11.5) 38.0 (-12.0) 44.5 (-15.2) 38.0 (-12.0) Manual-role MAS 3 Human Shared 13.0 (-18.5) 23.4 (-19.8) 12.5 (-19.0) 31.5 (-18.5) 37.8 (-21.9) 31.5 (-18.5) Role-source controls Random role prompt 3 Random Shared 16.5 (-15.0) 28.0 (-15.2) 16.5 (-15.0) 36.0 (-14.0) 43.1 (-16.6) 36.0 (-14.0) Shuffled induced role 3 Mismatch Shared 8.5 (-23.0) 19.1 (-24.1) 8.5 (-23.0) 42.3 (-7.7) 50.0 (-9.7) 42.3 (-7.7) Table 1: Primary ExRole comparison on MuSiQue and 2WikiMultiHopQA. #Ag. is the number of agents, Cap. is the trainable-capacity path, and Succ. is strict success. Parenthesized values for controls are differences from the stronger ExRole variant in each metric. The 2Wiki shuffled-role result is averaged over three role-assignment seeds. (a) Frozen discovery profiles. (b) Action shares by scheduled turn. (c) Standardized router signatures. (d) Frozen trajectory embeddings. Figure 4: Role induction and routed execution on MuSiQue. Panels (b) and (c) summarize 200 ExRole-Routed trajectories across scheduled team turns; panel (d) shows a t-SNE projection of frozen role embeddings. Controlled role comparison. Table 1 compares role sources and trainable capacity under a common evaluation interface. On MuSiQue, ExRole-Shared obtains 31.5 EM and 43.2 F1, compared with 16.5 EM and 28.8 F1 for single-agent search and 20.0 EM and 31.6 F1 for the no-role MAS. On 2WikiMultiHopQA, ExRole-Shared and ExRole-Routed reach 49.0/59.1 and 50.0/59.7 EM/F1, respectively. Both capacity paths preserve the gains from trajectory-induced role conditioning across the two primary benchmarks. Inference behavior. ExRole-Shared and ExRole-Routed use nearly identical numbers of team turns, search calls, retrieved characters, and repeated queries. Both variants retrieve target-bearing evidence for approximately 73% of MuSiQue examples and produce grounded answers for 87%, while strict success remains near 30%. This gap localizes much of the remaining error to answer selection and exact answer formation. Supplementary Appendix D.5 consolidates the matched cost distributions and stage-wise diagnostics. Role mechanism. Figure 4 connects trajectory-level role induction to routed execution on MuSiQue. The induced Researcher, Analyst, and Verifier differ in search, evidence, answer, and timing statistics, and their trajectory embeddings form separated regions. Their scheduled turns also produce distinct action and routing profiles. The figure characterizes role-conditioned execution under the fixed speaker schedule. Method Base MuSiQue 2Wiki EM / F1 EM / F1 Retrieval baselines Direct ★ 3.0 / 11.4 22.5 / 26.2 Naive RAG ★ 12.5 / 22.3 26.5 / 29.6 Search-RL baselines Search-R1 ◆ 33.0 / 44.3 52.0 / 60.3 R1-Searcher ⚫ 47.5 / 59.0 64.5 / 71.2 MAS-RL baselines MAGRPO ★ 12.0 / 23.1 26.5 / 35.2 Dr. MAS ★ 30.5 / 37.8 60.0 / 67.5 MATPO ★ 36.5 / 48.8 45.3 / 54.4 ExRole ExRole-Shared ★ 31.5 / 43.2 49.0 / 59.1 ExRole-Routed ★ 30.0 / 41.5 50.0 / 59.7 Table 2: External EM/F1 references on the primary benchmarks under each method’s stated protocol. Base icons denote ★ shared SFT, ◆ Search-R1/Qwen, and ⚫ R1-Searcher backbones. MuSiQue 2Wiki (a) Router auxiliary loss. (b) Future-utility loss. (c) Absolute turn credit. Figure 5: Routed training signals across MuSiQue and 2WikiMultiHopQA. Pale traces are rank-averaged observations; dark traces are centered rolling means over normalized training progress. Shared Routed (a) MuSiQue policy loss. (b) 2Wiki policy loss. (c) MuSiQue turn credit. (d) 2Wiki turn credit. Figure 6: Matched Shared and Routed optimization dynamics. The upper row reports GRPO policy loss and the lower row reports absolute turn credit; all curves are rank-averaged centered rolling means over the same 128-update horizon. (a) Incremental role information. (b) Cross-agent behavioral consistency. (c) Role-specific action profiles. Figure 7: Role-Agent-Turn disentanglement on MuSiQue using fixed Shared and Routed checkpoints without additional policy training. Role-Agent-Turn Disentanglement Using fixed checkpoints, we orthogonally vary speaker order and agent-role assignment on MuSiQue. Figure 7(a) shows that role identity reduces held-out action log loss by 19.1% for Shared and 18.7% for Routed after accounting for agent identity and turn phase. Between-role Jensen–Shannon distance is 48–60 times the within-role cross-agent distance, while Researcher, Analyst, and Verifier search rates are 96.6%, 85.8%, and 61.3%. These results indicate transferable action tendencies rather than fixed agent or turn labels; Supplementary Appendix D.3 reports the full matrix. External reference. Table 2 reports representative retrieval, Search-RL, MAS-RL, and ExRole results on the primary benchmarks. ExRole-Shared reaches 49.0/59.1 EM/F1 on 2WikiMultiHopQA and 31.5/43.2 on MuSiQue; ExRole-Routed reaches 50.0/59.7 and 30.0/41.5. ExRole is competitive with several MAS-RL systems but remains below the strongest search-specialized policies. Capacity-Path Analysis Paired capacity-path comparison. We evaluate ExRole-Shared and ExRole-Routed on the same 200 examples for each benchmark. Both capacity paths preserve the induced-role gains over the corresponding single-agent and no-role controls, while their relative ordering varies across benchmarks. Supplementary Appendix D.1 reports the complete paired differences, confidence intervals, and per-example comparisons. Router training signals. Figure 7 reports the routing objectives and turn credit across the two primary benchmarks. The auxiliary and future-utility losses remain active throughout training, while turn credit varies with sampled trajectories. Supplementary Appendix D.1 reports the corresponding task-level effects. Matched optimization dynamics. Figure 7 compares Shared and Routed under the same training horizon. Its columns correspond to MuSiQue and 2WikiMultiHopQA, while the two rows show smoothed GRPO policy loss and the magnitude of turn-aligned credit. Both variants receive turn-level credit throughout optimization; only Routed additionally receives the router losses in Figure 7. Conclusion ExRole turns team trajectories into executable roles for interaction and turn-aligned optimization. On MuSiQue and 2WikiMultiHopQA, induced roles yield distinct search, synthesis, and verification behaviors and outperform single-agent and role-free controls. Role-Agent-Turn interventions show that these behaviors generalize beyond fixed agents and turn positions, supporting trajectory-induced role binding as a practical framework for structured language-agent collaboration. This interface makes learned specialization explicit and reusable during policy optimization. References Buehler and Buehler (2024) Buehler, E. L.; and Buehler, M. J. 2024. X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design. arXiv preprint arXiv:2402.07148. Chen et al. (2026) Chen, G.; Yang, S.; Li, C.; Liu, W.; Luan, J.; and Xu, Z. 2026. End-to-End Optimization of LLM-Driven Multi-Agent Search Systems via Heterogeneous-Group-Based Reinforcement Learning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 30319–30338. Foerster et al. (2018) Foerster, J. N.; Farquhar, G.; Afouras, T.; Nardelli, N.; and Whiteson, S. 2018. Counterfactual Multi-Agent Policy Gradients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2974–2982. Goel et al. (2025) Goel, H.; Omama, M.; Chalaki, B.; Tadiparthi, V.; Pari, E. M.; and Chinchali, S. 2025. R3DM: Enabling Role Discovery and Diversity Through Dynamics Models in Multi-Agent Reinforcement Learning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267, 19600–19620. Hu et al. (2024) Hu, Z.; Zhang, Z.; Li, H.; Chen, C.; Ding, H.; and Wang, Z. 2024. Attention-Guided Contrastive Role Representations for Multi-Agent Reinforcement Learning. In International Conference on Learning Representations. Huang et al. (2024) Huang, C.; Liu, Q.; Lin, B. Y.; Pang, T.; Du, C.; and Lin, M. 2024. LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition. In Conference on Language Modeling. Jin et al. (2025) Jin, B.; Zeng, H.; Yue, Z.; Yoon, J.; Arık, S. Ö.; Wang, D.; Zamani, H.; and Han, J. 2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. In Second Conference on Language Modeling. Li et al. (2023) Li, G.; Hammoud, H. A. A. K.; Itani, H.; Khizbullin, D.; and Ghanem, B. 2023. CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society. In Advances in Neural Information Processing Systems, volume 36, 51991–52008. Li et al. (2025) Li, H.; Su, Z.; Xue, Y.; Tian, Z.; Song, Y.; and Huang, M. 2025. Advancing Collaborative Debates with Role Differentiation through Multi-Agent Reinforcement Learning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 22655–22666. Li, Pan, and Zhang (2024) Li, X.; Pan, L.; and Zhang, J. 2024. Kaleidoscope: Learnable Masks for Heterogeneous Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems. Liao et al. (2025) Liao, J.; Wen, M.; Wang, J.; and Zhang, W. 2025. MARFT: Multi-Agent Reinforcement Fine-Tuning. arXiv preprint arXiv:2504.16129. Liu et al. (2026) Liu, S.; Liang, Z.; Lyu, X.; and Amato, C. 2026. LLM Collaboration with Multi-Agent Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 32150–32158. Mo et al. (2025) Mo, Z.; Li, X.; Chen, Y.; and Bing, L. 2025. Multi-Agent Tool-Integrated Policy Optimization. arXiv preprint arXiv:2510.04678. Park, Cho, and Lee (2026) Park, J.; Cho, S.; and Lee, J.-Y. 2026. Divide and Cooperate: Role-Decomposed Multi-Agent LLM Training with Cross-Agent Learning Signals. arXiv preprint arXiv:2606.10684. Song et al. (2025) Song, H.; Jiang, J.; Min, Y.; Chen, J.; Chen, Z.; Zhao, W. X.; Fang, L.; and Wen, J.-R. 2025. R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2503.05592. Trivedi et al. (2023) Trivedi, H.; Balasubramanian, N.; Khot, T.; and Sabharwal, A. 2023. Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 10014–10037. Wang et al. (2020) Wang, T.; Dong, H.; Lesser, V.; and Zhang, C. 2020. ROMA: Multi-Agent Reinforcement Learning with Emergent Roles. In Proceedings of the 37th International Conference on Machine Learning, volume 119, 9876–9886. Wang et al. (2021) Wang, T.; Gupta, T.; Mahajan, A.; Peng, B.; Whiteson, S.; and Zhang, C. 2021. RODE: Learning Roles to Decompose Multi-Agent Tasks. In International Conference on Learning Representations. Wu, Huang, and Wei (2024) Wu, X.; Huang, S.; and Wei, F. 2024. Mixture of LoRA Experts. In International Conference on Learning Representations, volume 2024, 47302–47318. Xu et al. (2026) Xu, Y.; Zhou, Z.; Sang, H.; Li, X.; Zhang, J.; Du, X.; Na, S.; Wang, Z.; and Geramifard, A. 2026. TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning. arXiv preprint arXiv:2606.32017. Yang et al. (2022) Yang, M.; Zhao, J.; Hu, X.; Zhou, W.; Zhu, J.; and Li, H. 2022. LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 35, 1698–1710. Yu et al. (2024) Yu, Y.; Yin, Q.; Zhang, J.; Xu, P.; and Huang, K. 2024. ADMN: Agent-Driven Modular Network for Dynamic Parameter Sharing in Cooperative Multi-Agent Reinforcement Learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 302–310. Yue et al. (2025) Yue, Y.; Zhang, G.; Liu, B.; Wan, G.; Wang, K.; Cheng, D.; and Qi, Y. 2025. MasRouter: Learning to Route LLMs for Multi-Agent Systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 15549–15572. Zhao et al. (2024) Zhao, Z.; Gan, L.; Wang, G.; Zhou, W.; Yang, H.; Kuang, K.; and Wu, F. 2024. LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild. In Findings of the Association for Computational Linguistics: ACL 2024, 4447–4462. Zhou et al. (2025) Zhou, H.; Geng, H.; Xue, X.; Kang, L.; Qin, Y.; Wang, Z.; Yin, Z.; and Bai, L. 2025. ReSo: A Reward-Driven Self-Organizing LLM-Based Multi-Agent System for Reasoning Tasks. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 15979–15998. Appendix A Appendix Guide Index Content B Method and Experimental Details B.1 attr /Border [0 0 0] goto name app.reward-definitionComplete Environment Objective B.2 attr /Border [0 0 0] goto name app.role-induction-detailsRole Features, Targets, and Prototype Construction B.3 attr /Border [0 0 0] goto name app.role-resolution-detailsDeterministic Role Resolution and Marker Alignment B.4 attr /Border [0 0 0] goto name app.router-detailsRouter Objectives and Implementation Details B.5 attr /Border [0 0 0] goto name app.full-turn-grpoFull Turn-Aligned GRPO Objective B.6 attr /Border [0 0 0] goto name app.experimental-configurationExperimental Configuration B.7 attr /Border [0 0 0] goto name app.manual-role-control-promptManual-role Control Prompt B.8 attr /Border [0 0 0] goto name app.role-library-provenanceRole-Library Provenance and Split Isolation C Evaluation, Inference, and Optimization Details C.1 attr /Border [0 0 0] goto name app.limitationsLimitations C.2 attr /Border [0 0 0] goto name app.metric-definitionsEvaluation Metrics and Agreement Analysis C.3 attr /Border [0 0 0] goto name app.inference-procedureInference Procedure C.4 attr /Border [0 0 0] goto name app.sparse-delta-gradientGradient Flow through Balanced Sparse-Delta Routing C.5 attr /Border [0 0 0] goto name app.training-credit-flowTraining-Time Credit Flow C.6 attr /Border [0 0 0] goto name app.training-algorithmTurn-Aligned Routed GRPO Algorithm D Additional Experimental Diagnostics D.1 attr /Border [0 0 0] goto name app.paired-routing-effectsPaired Routing Effects D.2 attr /Border [0 0 0] goto name app.role-routing-diagnosticsRole Induction and Evaluation Diagnostics D.3 attr /Border [0 0 0] goto name app.role-agent-turn-disentanglementRole-Agent-Turn Disentanglement D.4 attr /Border [0 0 0] goto name app.role-credit-complementarityRole Induction and Turn-Aligned Credit D.5 attr /Border [0 0 0] goto name app.inference-diagnosticsInference Cost and Answer Diagnostics D.6 attr /Border [0 0 0] goto name app.hotpot-fullwiki-stressFull-Wikipedia HotpotQA Stress Test D.7 attr /Border [0 0 0] goto name app.external-baseline-curvesExternal Reference Training Curves E Notation E.1 attr /Border [0 0 0] goto name app.symbol-tableSymbol Table Appendix B Method and Experimental Details Complete Environment Objective name app.reward-definition xyz The environment exposes the controlled action space =search(qret),answer(y^),invalid.A=\search(q_ret),answer( y),invalid\. (20) Before turn t, the available team history is ℋt=at′,mt′,et′t′<tH_t=\a_t ,m_t ,e_t \_t <t. Unless the episode terminates early, speakers follow the round-robin schedule it=1+((t−1)modN)i_t=1+((t-1) N). The turn reward used by all ExRole variants is rt r_t =rtbase+rt+−Pent, =r_t^base+r_t^+-Pen_t, (21) rt+ r_t^+ =λansrtans+λsuprtsup = _ansr_t^ans+ _supr_t^sup +λnovrtnov + _novr_t^nov +λverrtver, + _verr_t^ver, Pent _t =λrepPentrep+λearlyPentearly = _repPen_t^rep+ _earlyPen_t^early +λbridgePentbridge + _bridgePen_t^bridge +λinsufPentinsuf + _insufPen_t^insuf +λgwrongPentgwrong + _gwrongPen_t^gwrong +λunsupPentunsup + _unsupPen_t^unsup +λnoansPentnoans. + _noansPen_t^noans. The shared base term covers action format, communication, answer type, and grounding. The positive terms reward a correct answer, supporting evidence, evidence novelty, and a grounded verifier decision. The penalties cover repeated evidence, premature answers, intermediate bridge entities, insufficient evidence, grounded but incorrect answers, unsupported answers, and episodes that terminate without an answer. With (y)Y(y) denoting acceptable aliases, rtans=maxy′∈(y)succ(y^t,y′).r_t^ans= _y (y)1_succ( y_t,y ). (22) The strict indicator compares nsucc(y^t)n_succ( y_t) with nem(y′)n_em(y ), where nsucc=nem∘cansn_succ=n_em c_ans and cansc_ans removes the structured action wrapper and common answer prefixes. Role Features, Targets, and Prototype Construction name app.role-induction-details xyz The prefix feature uses the following fixed 30-coordinate schema: ϕtobs _t^obs =[ti∈14,ti∈4,tteam∈5, = [r_t^i ^14,c_t^i ^4,r_t^team ^5, . (23) pt,bt,onehot5(at)]. .p_t,b_t,onehot_5(a_t) ]. In order, tir_t^i contains the active agent’s search, answer, invalid, and message rates; mean message and retrieval lengths; mean, early, and late turn-position statistics; mean shared-board size; accumulated reward contribution; and evidence-hit, grounded-answer, and repair rates. The count vector tic_t^i contains search, answer, message, and turn counts. The team vector tteamr_t^team contains team search, answer, message, tool-use, and evidence-hit rates; ptp_t and btb_t are progress and remaining-budget ratios. Rates use the number of observed agent or team turns as their denominator, means are zero for empty sets, and missing scalar metadata are set to zero. For coordinate j over the J discovery records, the encoder input is ϕ~t,jobs φ_t,j^obs =ϕt,jobs−μjσj+10−6, = _t,j^obs- _j _j+10^-6, (24) μj _j =1J∑t=1Jϕt,jobs,σj2=1J∑t=1J(ϕt,jobs−μj)2. = 1J _t=1^J _t,j^obs, _j^2= 1J _t=1^J( _t,j^obs- _j)^2. Every component is computed from the logged prefix ending at turn t. Future evidence and final return are used only as targets. For the next-action target, ti+=mint′>t:it′=itt_i^+= \t >t:i_t =i_t\ is the next turn of the same agent; if it does not exist, the target is stop. Let ht′evdh_t ^evd denote the logged evidence-hit or grounding indicator at turn t′t . The three targets are at+ a_t^+ =ati+,ti+ exists,stop,otherwise, = casesa_t_i^+,&t_i^+ exists,\\ stop,&otherwise, cases (25) vtevd v_t^evd =[∃t′>t:it′=it∧ht′evd=1], =1\! [∃ t >t:i_t =i_t h_t ^evd=1 ], Rt R_t =R(τ(t),yτ(t)). =R(τ(t),y_τ(t)). We use λevd=1 _evd=1 and λret=0.5 _ret=0.5 in ℒroleL_role. For a configured discovery-seed set S, K-means uses Euclidean distance, ten initializations per seed, and the following model-selection statistics: K(s) _K^(s) =KMeans10(ξtt=1J,K,s), =KMeans_10(\ _t\_t=1^J;K,s), (26) Sil¯(K) Sil(K) =1||∑s∈Sil(ξt,K(s)), = 1|S| _s Sil(\ _t\,k_K^(s)), Stability(K) (K) =2||(||−1)∑s<s′ARI(K(s),K(s′)), = 2|S|(|S|-1) _s<s ARI(k_K^(s),k_K^(s )), K⋆ K =argmaxK∈[Sil¯(K)+λstabStability(K)]. = _K [ Sil(K)+ _stabStability(K) ]. Ties in the last line are resolved by the larger K, matching the implementation, and the final assignments use the first configured seed s0s_0: kt=kK⋆,t(s0)k_t=k_K ,t^(s_0). Clusters with support below n0n_0 are discarded before prototype construction. For cluster k, let ℐk=t:kt=kI_k=\t:k_t=k\ and nk=|ℐk|n_k=|I_k|. Its centroids and descriptive success lift are ξ¯k ξ_k =1nk∑t∈ℐkξt, = 1n_k _t _k _t, (27) ϕ¯k φ_k =1nk∑t∈ℐkϕtobs, = 1n_k _t _k _t^obs, Liftk _k =1nk∑t∈ℐkYτ(t)succ−1J∑t=1JYτ(t)succ. = 1n_k _t _kY_τ(t)^succ- 1J _t=1^JY_τ(t)^succ. The metadata profile ηk _k records dominant next action, evidence-hit statistics, predicted return, future evidence hit, and final return. Readable instructions are generated without an additional language model. Each cluster is scored against a fixed functional vocabulary, Sk,υ S_k, =TemplateScoreυ(ϕ¯k,ηk), =TemplateScore_ ( φ_k, _k), (28) role _role =Researcher,Coordinator,Verifier,Analyst. =\Researcher,Coordinator,Verifier,Analyst\. High early-search and evidence-hit rates favor a research template, whereas late grounded answers favor a verification template. Let (S)G(S) greedily scan all (k,υ)(k, ) pairs in decreasing (Sk,υ,k,υ)(S_k, ,k, ) order, accepting a pair only when neither its cluster nor functional type has been assigned. The last coordinate follows reverse lexicographic order over the fixed vocabulary, matching the implementation. Any cluster left unmatched after this uniqueness pass takes its own highest-scoring type, with ties resolved by the vocabulary order Researcher, Coordinator, Verifier, Analyst: υk _k =υ,(k,υ)∈(S),argmaxυ∈roleSk,υ,k∉dom(S), = cases ,&(k, ) (S),\\ _ _roleS_k, ,&k (S), cases (29) (ψk,χk) ( _k, _k) =Templateυk(ϕ¯k,ηk). =Template_ _k( φ_k, _k). Thus the role type of one cluster can depend on the scores of other clusters; the resolver is a joint deterministic map, not a pointwise template lookup. We set λstab=0.2 _stab=0.2 in the role-count criterion and discard clusters below the minimum support n0n_0. MuSiQue fixes K=3K=3 to the three-agent execution budget. HotpotQA and 2WikiMultiHopQA search K∈2,3,4K∈\2,3,4\ and select K=4K=4 before resolving three executable roles. The broader sensitivity analysis in the appendix is post-hoc and is not used for model selection. Deterministic Role Resolution and Marker Alignment name app.role-resolution-details xyz The resolver ranks source prototypes using κ(pk) κ(p_k) =(stage(pk),−ηkearly,ηklate, = (stage(p_k),- _k^early, _k^late, . (30) OPENηkpos,−Liftk,−nk,k). . _k^pos,-Lift_k,-n_k,k ). The resolver sorts κ lexicographically in ascending order, so coordinator and planner roles precede researcher and solver roles, followed by analyst and synthesizer roles and then verifier roles; the final source-cluster ID breaks exact ties. Let NselP_N^sel be the first N prototypes under this ordering, Ksel=|Nsel|K_sel=|P_N^sel|, and ιi _i the source ID of its iith entry. Source IDs are preserved when they already form a contiguous runtime range; otherwise they are remapped by execution position: Nsel _N^sel =TakeN(Sortκ()), =Take_N(Sort_κ(P)), (31) zi z_i =ιi,ιjj=1Ksel=0,…,Ksel−1,i−1,otherwise, = cases _i,&\ _j\_j=1^K_sel=\0,…,K_sel-1\,\\ i-1,&otherwise, cases exec _exec =Sortz(PadGeneric((N,isel,zi)i=1Ksel,N)). =Sort_z (PadGeneric(\(P_N,i^sel,z_i)\_i=1^K_sel,N) ). Source IDs remain in metadata for auditability, while the router indexes only contiguous runtime role IDs. If fewer than N roles survive, the resolver appends a generic collaborative fallback with a fresh marker. Equivalent tokenizations of each marker are matched, overlapping occurrences are coalesced, and the resulting spans are ordered as ℳ=(uq−,uq+,zq)q=1Q,u1−<⋯<uQ−.M=\(u_q^-,u_q^+,z_q)\_q=1^Q, u_1^-<·s<u_Q^-. (32) Tokens before the first marker use the first segment, and cached decoding retains the latest segment ID. A malformed sequence without a marker falls back to runtime role 00; formal ExRole prompts always include a marker. Algorithm 1 Deterministic assignment of induced roles 0: Source prototypes P and number of agents N 0: Runtime roles (z1,…,zN)(z_1,…,z_N) and marker-conditioned observations 1: Score prototypes with κ(pk)κ(p_k) and retain the first N 2: while fewer than N roles are available do 3: Append a generic collaborative role with a fresh marker 4: end while 5: Preserve contiguous source IDs; otherwise remap by execution position 6: Sort executable roles by runtime role ID 7: for i=1i=1 to N do 8: Bind role ziz_i and marker χzi _z_i to agent i 9: end for 10: return (z1,…,zN)(z_1,…,z_N) and updated observation templates Router Objectives and Implementation Details name app.router-details xyz The main text writes the normalized routing distribution as ϖq,ℓs _q, s. Equivalently, a total soft budget S is distributed across all adapted-module–rank slots: b~q,ℓs=Sϖq,ℓs,b¯q=1Ld∑ℓ,sb~q,ℓs=SLd. b_q, s=S _q, s, b_q= 1Ld _ ,s b_q, s= SLd. (33) Hence b~q,ℓs/b¯q=Ldϖq,ℓs b_q, s/ b_q=Ld\, _q, s, which yields the compact sparse-delta expression in the main text. The selected-slot utility prediction is ν¯qutil=∑ℓ,sMq,ℓssigm(νq,ℓsutil)∑ℓ,sMq,ℓs. ν_q^util= _ ,sM_q, ssigm( _q, s^util) _ ,sM_q, s. (34) Let ϖ¯ℓs=Q−1∑qϖq,ℓs _ s=Q^-1 _q _q, s. The remaining regularizers are ℒload _load =1Ld∑ℓ,s(ϖ¯ℓs−1Ld)2, = 1Ld _ ,s ( _ s- 1Ld )^2, (35) ℒdiv _div =1N(N−1)∑z≠z′ϖ¯z⊤ϖ¯z′‖ϖ¯z‖2‖ϖ¯z′‖2, = 1N(N-1) _z≠ z _z _z \| _z\|_2\| _z \|_2, ℒl1 _l1 =1QLd∑q,ℓ,sb~q,ℓs(1−Mq,ℓs), = 1QLd _q, ,s b_q, s(1-M_q, s), ℒent _ent =1Q∑q,ℓ,sϖq,ℓslogϖq,ℓs, = 1Q _q, ,s _q, s _q, s, ℒreg _reg =λloadℒload+λdivℒdiv+λl1ℒl1+λentℒent. = _loadL_load+ _divL_div+ _l1L_l1+ _entL_ent. Here ϖ¯z _z is obtained by evaluating executable role z under the minibatch-mean semantic context. This keeps role diversity active even when a microbatch contains only one observed role. Since ℒentL_ent is negative entropy, minimizing it discourages premature slot concentration. Component Setting LoRA structure all-linear, rank d=8d=8, scale 1616 Adapted modules L=196L=196 Direct slot budget S=392S=392 of Ld=1568Ld=1568 slots Router state hidden size 128128, prefix window 256256 Routing coefficients τb=0.8 _b=0.8, λu=0.35 _u=0.35, λc=0.5 _c=0.5 Gate λg=0.5 _g=0.5, range [0.5,1.5][0.5,1.5] Router losses λfuture=0.05 _future=0.05, λcredit=0.10 _credit=0.10 Regularization λload=0.10 _load=0.10, λdiv=0.05 _div=0.05 Small regularizers λl1=λent=10−4 _l1= _ent=10^-4 Optimization βKL=10−3 _KL=10^-3, router gradient scale 10241024 Table 3: Role-routing architecture and optimization settings. Full Turn-Aligned GRPO Objective name app.full-turn-grpo xyz For each task, GRPO samples G trajectories and computes μR _R =1G∑g=1GR(τ(g),y), = 1G _g=1^GR(τ^(g),y), (36) σR _R =1G∑g=1G(R(τ(g),y)−μR)2, = 1G _g=1^G (R(τ^(g),y)- _R )^2, Adv(g) ^(g) =R(τ(g),y)−μRσR+ϵ. = R(τ^(g),y)- _R _R+ε. The actor-side turn return is normalized within each trajectory: U^t(g)=Utturn,(g)−μU(g)σU(g),σU(g)≥10−6,clip(Utturn,(g)−μU(g),−2,2),otherwise. U_t^(g)= cases U_t^turn,(g)- _U^(g) _U^(g),& _U^(g)≥ 10^-6,\\[5.69054pt] clip(U_t^turn,(g)- _U^(g),-2,2),&otherwise. cases (37) Thus, a one-turn or constant-return trajectory contributes zero actor-side local credit. The router normalizer instead retains a raw singleton or zero-variance target before clipping: ℬR(vq)=vq−μℬRσℬR,|ℬR|>1andσℬR≥10−6,vq,otherwise,N_B_R(v_q)= cases v_q- _B_R _B_R,&|B_R|>1\ and\ _B_R≥ 10^-6,\\[5.69054pt] v_q,&otherwise, cases (38) where vq=Ut(q)turn,(g(q))v_q=U_t(q)^turn,(g(q)). For response token j at turn t, define ρt,j(g) _t,j^(g) =πθ,θb(tokt,j(g)∣ot(g),tokt,<j(g))πθold,θb,old(tokt,j(g)∣ot(g),tokt,<j(g)), = _θ, _b(tok_t,j^(g) o_t^(g),tok_t,<j^(g)) _ _old, _b,old(tok_t,j^(g) o_t^(g),tok_t,<j^(g)), (39) ρ¯t,j(g) ρ_t,j^(g) =clip(ρt,j(g),1−ϵclip,1+ϵclip). =clip ( _t,j^(g),1- _clip,1+ _clip ). The turn-aligned policy objective is GRPOturn=x,g,t,j[min(ρt,j(g)Adv~t,j(g),ρ¯t,j(g)Adv~t,j(g))].J_GRPO^turn= E_x,g,t,j [ ( _t,j^(g) Adv_t,j^(g), ρ_t,j^(g) Adv_t,j^(g) ) ]. (40) We use λdisc=0.9 _disc=0.9 and αc=0.35 _c=0.35. Router-only gradients are multiplied by 10241024 after backpropagation and before global clipping; LoRA gradients are not rescaled. This changes the router’s optimization scale but not the forward gate or the objective. Experimental Configuration name app.experimental-configuration xyz The primary MuSiQue and 2WikiMultiHopQA evaluations use the same team-search interface and metric implementation. ExRole-Shared and ExRole-Routed also use the same benchmark-specific warm start, reward, training horizon, and turn-aligned credit; their only architectural difference is the capacity-routing path. HotpotQA is reported separately as a full-Wikipedia retrieval stress test. Component Configuration Primary benchmarks MuSiQue and 2WikiMultiHopQA; 200 held-out examples per benchmark Stress test HotpotQA with 200 held-out examples and a lexical full-Wikipedia index Retrieval Supporting-document collections for the primary benchmarks; top-3 results with 640 characters per retrieved result Team interface Three agents, round-robin execution, at most six team turns, and 640 characters per retrieved result Policy backbone Qwen2.5-7B SFT with all-linear LoRA, rank 8 and α=16α=16 Training data 512 benchmark-specific instances; batch size 4 Policy optimization GRPO group size 8, learning rate 10−610^-6, and 128 policy updates Sequence budgets Data prompt 4096, rollout prompt 6144, response 8192, and total model context 16384 tokens Role induction Three source prototypes for MuSiQue; four for 2WikiMultiHopQA and the HotpotQA stress test; three executable roles after resolution Capacity variants Shared uses a uniform LoRA path; Routed directly selects 392 of 1568 module–rank slots and applies the centered sparse-delta gate Table 4: Training and evaluation configuration for the primary benchmarks and the full-Wikipedia stress test. Router-specific coefficients are reported in Table 3. Manual-role Control Prompt name app.manual-role-control-prompt xyz The manual_role control uses a fixed, ordered library of planner, solver, and verifier instructions. Its training configuration sets role_mode: manual_role and specialization_mode: none. The evaluator resolves the three entries in the displayed order and applies the same per-turn team-search template, state fields, and action parser as ExRole. The implementation renders one prompt string per turn rather than separate API system messages; the box separates its shared context, manual instruction, and action contract for readability. Bracketed fields are filled from the common environment state. Manual-role control prompt System/shared context. The common prompt begins with You are part of a num_agents-agent team solving a structured search task. Question: question Current team turn: turn/max_turns Current speaker: Agent agent_id (role_name) Your functional bias: role_prompt It then inserts the latest retrieved evidence, exact prior search queries, shared message board, and recent team history; it also appends the structured tag [ROLE_ID=k], role-specific operating rules, an optional task answer contract, and the common final-turn rule. Manual instruction and name-dispatched rules. The role IDs follow the listed order. planner “Prioritize task decomposition, decide what evidence is still missing, and send concise coordination messages before issuing searches.” Additional rules: “Route the team away from duplicated work and name the next missing evidence gap concretely.” “Prefer one short message or one focused search over a broad speculative query.” solver “Focus on targeted evidence gathering and drafting candidate answers from retrieved information.” Additional rules: “Synthesize the current evidence into one candidate answer or one narrower follow-up query.” “Avoid broad repeat searches; only search again when you can name the concrete missing fact.” “If the current evidence already points to one short plausible span, prefer <answer> over another broad search.” verifier “Check consistency of the candidate answer against the gathered evidence and only finalize when the evidence is sufficient.” Additional rules: “Audit candidate facts against the retrieved evidence and prefer <answer> over another <search> once one plausible short span appears.” “Only search again when you are resolving one explicit contradiction or one missing entity link.” “Do not leave the episode unfinished if the evidence already supports a grounded exact span.” Action contract shared with ExRole. Each response must contain exactly one final environment action: <search>your query</search> <answer>your final answer</answer> Before that action, the prompt permits only a very short <think>...</think> and one short <message>...</message>. Missing evidence requires <search>, sufficient evidence requires <answer>, and an answer must be only the shortest exact evidence span rather than a sentence or explanation. Unsupported spans require another search, exact prior queries may be repeated only with a new disambiguating term, and malformed output without exactly one final action is penalized. On the final team turn, the shared rule requires <answer> rather than another search; a last-turn search is penalized and ends the episode without a final answer. Thus the manual-role and ExRole conditions share the same team interface, state visibility, role-marker format, and action grammar. They differ in how the role instruction is obtained: manual-role uses the fixed library above, whereas ExRole derives executable role instructions from trajectories; only the routed ExRole variant additionally changes the capacity path. Role-Library Provenance and Split Isolation name app.role-library-provenance xyz For the MuSiQue role library used in the matched MuSiQue experiments, we conducted a provenance and split-isolation audit before policy optimization and evaluation. The frozen library was induced only from 64 train bootstrap trajectories, comprising 323 prefix records; the role encoder consumes prefix-local behavioral statistics rather than question text or retrieved document content. Against the 200-example validation set, we found zero overlap in task identifiers, normalized question strings, answer strings, hop identifiers, or gold supporting-document contents. Because both splits draw contextual passages from Wikipedia, 57 non-gold page titles and one non-gold raw passage recur across the two collections; the repeated passage does not contain the validation answer and is not a supporting document for the corresponding evaluation item. The static role artifact is frozen before policy training, loaded only for role assignment at episode initialization, and is not rebuilt or updated from evaluation rollouts. Future-evidence and return targets are computed solely from the source train trajectories. Thus, the audit establishes isolation at the task, answer, gold-evidence, and role-induction levels while distinguishing this guarantee from the unavoidable reuse of a common background corpus. Appendix C Evaluation, Inference, and Optimization Details Limitations name app.limitations xyz Our primary evaluation focuses on supporting-evidence multi-hop question answering with a fixed team-search interface. Generalization to software engineering, web navigation, longer tool-use episodes, and larger teams remains untested. Appendix attr /Border [0 0 0] goto name app.hotpot-fullwiki-stressD.6 separately reports a full-Wikipedia retrieval stress test in which induced roles do not outperform the strongest controls. ExRole induces its role library offline from prior trajectories and keeps agent-role assignments fixed within an episode. A substantial shift in the task distribution, retrieval system, or tool interface may therefore require the role library to be refreshed. The turn-level reward components are manually weighted, and routed capacity does not improve accuracy on every benchmark. Learned credit models, adaptive role assignment, and larger role libraries are promising directions for more robust specialization. Evaluation Metrics and Agreement Analysis name app.metric-definitions xyz Let y^n y_n be the final answer for evaluation example n∈1,…,Nevaln∈\1,…,N_eval\ and let nY_n be its set of acceptable gold answers. The benchmark normalizer nem(⋅)n_em(·) lowercases text, removes punctuation and English articles, and collapses whitespace. Per-example exact match is EMn=maxy′∈n[nem(y^n)=nem(y′)].EM_n= _y _n1\! [n_em( y_n)=n_em(y ) ]. (41) For token-level F1, let ^n T_n and ny′T_n^y be the token multisets obtained from nem(y^n)n_em( y_n) and nem(y′)n_em(y ). Their multiset intersection counts repeated tokens up to the smaller multiplicity. We compute Ovn(y′) _n(y ) =|^n∩ny′|, =| T_n _n^y |, (42) Precn(y′) _n(y ) =Ovn(y′)max(1,|^n|), = Ov_n(y ) (1,| T_n|), Recn(y′) _n(y ) =Ovn(y′)max(1,|ny′|), = Ov_n(y ) (1,|T_n^y |), F1n 1_n =maxy′∈n2Precn(y′)Recn(y′)Precn(y′)+Recn(y′), = _y _n 2\,Prec_n(y )Rec_n(y )Prec_n(y )+Rec_n(y ), where the last fraction is defined as zero when both precision and recall are zero. Let cans(⋅)c_ans(·) remove the structured action wrapper, common answer prefixes, surrounding whitespace, and terminal punctuation. The environment normalizer is the composition nsucc=nem∘cansn_succ=n_em c_ans. Strict success compares this cleaned prediction with the benchmark-normalized gold answer, so substring matches receive no credit: Succn=maxy′∈nsucc(y^n,y′).Succ_n= _y _n1_succ( y_n,y ). (43) The reported dataset-level scores are percentages, (EM,F1,Succ)=100Neval∑n=1Neval(EMn,F1n,Succn).(EM,F1,Succ)= 100N_eval _n=1^N_eval(EM_n,F1_n,Succ_n). (44) EM and Succ therefore need not be identical because only Succ applies the action-and-prefix cleanup cansc_ans. We quantify their per-example agreement using DSucc,EM=∑n=1Neval[Succn≠EMn].D_Succ,EM= _n=1^N_eval1[Succ_n _n]. (45) For the Shared and Routed evaluations on the two primary 200-example splits, DSucc,EM=0D_Succ,EM=0: the independently computed rules happen to agree on those predictions. The same equality also holds for the two HotpotQA stress-test variants, whereas several HotpotQA control rows have nonzero disagreement because the additional cleanup changes a small number of decisions. We always compute EM, F1, and Succ independently rather than copying one column into another. Inference Procedure name app.inference-procedure xyz At inference time, role induction is performed before evaluation rather than repeated for every sample. ExRole uses a role library induced from prior trajectories, assigns one role prompt to each agent, and then executes a team-search episode. The procedure is: 1. Load source prototypes =pkk=0K−1 P=\p_k\_k=0^K-1 from the induced role library. 2. Select and normalize the executable set exec P_exec, resolve agent-level runtime roles z1,…,zNz_1,…,z_N, and insert marker χzi _z_i into each agent observation. 3. At turn t, choose the active speaker by the team schedule, usually it=1+((t−1)modN)i_t=1+((t-1) N). 4. Build the observation oto_t from the task, active role, latest evidence, shared board, recent search queries, and recent team history. 5. During prefill, coalesce every role marker in the multi-turn sequence and align each token to its role-turn segment; during cached decoding, retain the latest segment ID. 6. Pool the semantic prefix for the active segment and compute slot scores ωq,ℓs _q, s, soft budget b~q,ℓs b_q, s, top-S mask Mq,ℓsM_q, s, and balanced sparse-delta multiplier γq,ℓs _q, s. 7. Generate the assistant response with the role-conditioned LoRA path. 8. Project the response to a structured action: search, answer, or invalid. 9. If the action is search, update evidence and continue. If the action is answer, terminate and score the answer. Gradient Flow through Balanced Sparse-Delta Routing name app.sparse-delta-gradient xyz The forward pass treats the top-S mask as fixed while retaining a differentiable soft budget on the selected slots. Define δq,ℓssel δ^sel_q, s =Mq,ℓs(b~q,ℓsb¯q−1), =M_q, s ( b_q, s b_q-1 ), (46) δ¯q,ℓs δ_q, s =δq,ℓssel−1Ld∑ℓ′,s′δq,ℓ′s′sel, =δ^sel_q, s- 1Ld _ ,s δ^sel_q, s , γq,ℓs _q, s =clip(1+λgδ¯q,ℓs,γmin,γmax). =clip\! (1+ _g δ_q, s, _ , _ ). Away from clipping boundaries and while the discrete mask is unchanged, the softmax couples every routing score to every soft budget. The exact chain rule is ∂γq,ℓs∂ωq,ℓ′s′=λg∑ℓ′=1L∑s′=1d∂δ¯q,ℓs∂b~q,ℓ′s′∂b~q,ℓ′s′∂ωq,ℓ′s′. ∂ _q, s∂ _q, s = _g _ =1^L _s =1^d ∂ δ_q, s∂ b_q, s ∂ b_q, s ∂ _q, s . (47) Selected slots receive direct role-specific deviations, while centering propagates a compensating shift to all slots. The future-utility, role-turn-credit, load-balance, diversity, sparsity, and entropy objectives additionally act directly on the soft routing distribution. Consequently, router learning does not rely on differentiating through the top-S membership itself. Training-Time Credit Flow name app.training-credit-flow xyz For a rollout group, GRPO first converts episode rewards into normalized trajectory advantages. In parallel, ExRole discounts the recorded turn rewards and aligns each return to the role-turn segment that produced it: x x →τ(g)g=1G, →\τ^(g)\_g=1^G, (48) τ(g)g=1G \τ^(g)\_g=1^G →rt(g)g,t→Utturn,(g)g,t, →\r_t^(g)\_g,t→\U_t^turn,(g)\_g,t, (R(τ(g),y),Utturn,(g)) (R(τ^(g),y),U_t^turn,(g) ) →Adv~t,j(g)→GRPOturn, → Adv_t,j^(g) _GRPO^turn, Utturn,(g) U_t^turn,(g) →(ℒfuture,ℒcredit), → ( L_future, L_credit ), The actor keeps the group-relative trajectory advantage and adds a weighted, normalized turn return only to the responsible role-turn segment. The router uses the same detached turn return as its future-utility and role-turn credit target. Thus a high-reward role turn reinforces both its generated action and its selected capacity path, while global role diversity, load balance, entropy, and sparsity regularizers prevent routing collapse. Turn-Aligned Routed GRPO Algorithm name app.training-algorithm xyz Algorithm 2 summarizes the complete update procedure corresponding to the objectives in the main text. Algorithm 2 Turn-aligned routed GRPO update 0: Task x, gold answer y, role library P, policy θ, and router θb _b 0: Updated policy θ and router θb _b 1: ◆ Roll out teams. Sample τ(g)g=1G\τ^(g)\_g=1^G and record turn rewards rt(g)\r_t^(g)\ 2: ◆ Compute group credit. Normalize episode rewards into Adv(g)Adv^(g) 3: for each trajectory τ(g)τ^(g) do 4: ◆ Align role turns. Map every response token to its role-turn segment 5: ◆ Discount local credit. Compute Utturn,(g)U_t^turn,(g) and U^t(g) U_t^(g) 6: ◆ Route capacity. Pool ζq _q and compute balanced gate γq,ℓs _q, s 7: ◆ Align policy credit. Form token advantages Adv~t,j(g) Adv_t,j^(g) 8: end for 9: ◆ Optimize. Compute GRPOturnJ_GRPO^turn and ℒrouterL_router 10: Form ℒrouted=ℒshared+ℒrouterL_routed=L_shared+L_router and backpropagate 11: Multiply router-only gradients by 10241024, apply global gradient clipping, and update (θ,θb)(θ, _b) 12: return ✓ Updated (θ,θb)(θ, _b) Appendix D Additional Experimental Diagnostics Paired Routing Effects name app.paired-routing-effects xyz Table 10 reports Shared and Routed scores together with 95% confidence intervals (CIs), differences in percentage points (p), and per-example win/tie/loss (W/T/L) counts over the same 200 examples. Figure 8 visualizes the corresponding differences. The intervals show a small negative shift on MuSiQue and a small positive shift on 2WikiMultiHopQA; neither is statistically reliable across all metrics. HotpotQA is analyzed separately in Appendix attr /Border [0 0 0] goto name app.hotpot-fullwiki-stressD.6. Figure 8: Paired effect of role-conditioned sparse LoRA routing. Points show Routed minus Shared in percentage points; bars are 95% paired-bootstrap confidence intervals over 200 matched examples. Role Induction and Evaluation Diagnostics name app.role-routing-diagnostics xyz Figure 13 jointly examines role-count sensitivity, cross-seed stability, and paired trajectory changes. Panel (a) shows that K=3K=3 remains competitive in silhouette, within-run stability, and support entropy, whereas K=4K=4 and K=5K=5 achieve slightly stronger clustering geometry. We therefore treat three roles as a compact, team-aligned operating point, not as the uniquely optimal partition. Panel (b) evaluates three K=3K=3 induction runs on the same 323 trajectory prefixes: adjusted Rand index (ARI) ranges from 0.61 to 0.85, normalized mutual information (NMI) from 0.62 to 0.82, and Hungarian-aligned profile cosine from 0.80 to 0.97. Panel (c) provides a qualitative view of the largest positive and negative paired MuSiQue changes. These selected cases do not estimate the population-level routing effect; instead, they show that routing can alter both retrieval decisions and evidence-to-answer conversion. Taken together, the diagnostics support moderately stable role induction while showing that downstream routing effects remain example dependent. Role-Agent-Turn Disentanglement name app.role-agent-turn-disentanglement xyz We test whether an induced role captures a transferable behavior or merely renames a fixed agent index or speaking position. On the MuSiQue validation set, we independently permute three speaker orders and three agent-role assignments for both ExRole-Shared and ExRole-Routed. The resulting 3×3×23× 3× 2 design contains 18 conditions and 3,600 episodes; within each variant, every role is enacted by every agent in early, middle, and late phases. This is an inference-time intervention on fixed checkpoints, not an additional training run. Table 5 reports two complementary tests. First, adding role identity to an action model that already observes agent identity and phase reduces held-out log loss by 19.14% for Shared and 18.68% for Routed, while increasing macro-F1 by approximately 0.236. Second, the action distribution of the same role changes little across agents, whereas the mean distance between different roles is 48–60 times larger. Role identity therefore explains behavior beyond agent index and coarse turn phase. The schedule-level results also preserve the main Routed-versus-Shared finding: routing does not improve the average score under these interventions. Measure Shared Routed Effect Agent + phase log loss 0.4439 0.4408 control + Role log loss 0.3589 0.3585 −19.14%/−18.68%-19.14\%/-18.68\% + Role macro-F1 0.6809 0.6808 ≈+0.236≈+0.236 Same-role cross-agent JS 0.00446 0.00550 low variation Different-role JS 0.2680 0.2652 60.1×/48.2×60.1×/48.2× Mean EM over nine schedules 29.22 28.33 −0.89-0.89 p Mean F1 over nine schedules 40.99 39.97 −1.02-1.02 p Table 5: Role-Agent-Turn disentanglement on MuSiQue. Action prediction uses 8,429 Shared and 8,423 Routed turns; schedule-level scores average nine orthogonal conditions per checkpoint. JS denotes Jensen–Shannon distance. Figure 9 isolates the position of each role in the early, middle, and late phases while retaining all three orthogonal scheduling points. Moving the Analyst–Verifier–Researcher order to these phases lowers both EM and F1, whereas Verifier–Researcher–Analyst is strongest. Thus a role is not reducible to a fixed turn label, but its utility still depends on when it acts in the collaboration sequence. Figure 9: Role-stage effects under orthogonal MuSiQue schedules. Small points are the three speaker-order and agent-role-map conditions associated with each role-stage pattern; diamonds and bars show the mean and one sample standard deviation. R, A, and V denote Researcher, Analyst, and Verifier. Figure 10 gives the complete 3×33× 3 schedule matrix rather than only phase-aggregated values. High- and low-performing cells occur under multiple speaker orders and agent-role mappings, which rules out a single privileged agent assignment. At the same time, the structured variation across cells confirms that coordination order remains part of the task rather than a nuisance variable that can be ignored. Figure 10: Complete Role-Agent-Turn intervention matrix on MuSiQue. Rows vary speaker order and columns vary agent-role assignment; cells report EM or F1 for the same 200 questions. Shared and Routed use identical schedules and evaluation protocol. Complementarity of Role Induction and Turn-Aligned Credit name app.role-credit-complementarity xyz We isolate the two learning components with an independently trained 2×22× 2 design under the shared-capacity setting. Among the four matched runs, jointly enabling trajectory-induced roles and turn-aligned credit achieves the highest observed result, reaching 30.0 EM and 41.5 F1. Relative to this joint configuration, removing role induction reduces EM and F1 by 3.5 and 2.3 points, respectively, while removing turn-aligned credit decreases F1 by 2.0 points. This pattern is consistent with complementary functions: induced role structure organizes collaborative behavior, while turn-aligned credit connects that structure to policy updates. Configuration Role Credit EM ↑ F1 ↑ Full role-credit ✓ ✓ 30.0 41.5 w/o role induction ✗ ✓ 26.5 (-3.5) 39.2 (-2.3) w/o turn-aligned credit ✓ ✗ 29.5 (-0.5) 39.5 (-2.0) w/o both ✗ ✗ 29.0 (-1.0) 40.3 (-1.2) Table 6: Strict 2×22× 2 component ablation on MuSiQue validation-200. The four variants are independently trained with seed 42 and share the same backbone, 128-update horizon, and evaluation protocol. Parenthesized values are changes relative to the complete role-credit configuration. Inference Cost and Answer Diagnostics name app.inference-diagnostics xyz Table 9 separates computational budget from answer quality on the same 200 MuSiQue examples. Shared and Routed have nearly identical turn, search, retrieval, and repetition statistics. Target-bearing evidence is available substantially more often than a strictly correct answer is produced, identifying final answer formation as a major remaining bottleneck. Full-Wikipedia HotpotQA Stress Test name app.hotpot-fullwiki-stress xyz HotpotQA differs from the two primary benchmarks by retrieving from a lexical full-Wikipedia index rather than a bounded supporting-document collection. We therefore report it as a retrieval stress test rather than use it as primary evidence for the role-learning claim. The team interface still uses three agents, six turns, top-3 retrieval, and the same EM/F1/Succ evaluation implementation. Method EM ↑ F1 ↑ Succ. ↑ Single-agent search 27.0 36.0 27.0 No-role MAS 33.5 44.4 33.0 Manual-role MAS 33.0 42.9 32.5 Random role prompt 34.5 43.9 34.0 Shuffled induced role 33.5 42.4 33.5 ExRole-Shared 30.0 38.3 30.0 ExRole-Routed 30.5 37.8 30.5 Table 7: HotpotQA results under the full-Wikipedia FTS5 retrieval stress test. Succ. is strict success, computed independently from EM. Neither ExRole variant surpasses the strongest role-free or prompt-role controls in this setting. Routed changes Shared by +0.5+0.5 EM with a 95% CI of [0.0,+1.5][0.0,+1.5] and by −0.5-0.5 F1 with a CI of [−1.4,+0.3][-1.4,+0.3], indicating negligible routing sensitivity. These results establish a boundary of the current method: role specialization alone does not overcome the retrieval and answer-selection constraints of this full-Wikipedia configuration. They do not, by themselves, isolate retrieval quality as the causal source of the gap. (a) Shared/Routed policy loss. (b) Shared/Routed turn credit. (c) Routed decoding robustness over three seeds. Figure 11: Optimization and decoding diagnostics for the HotpotQA full-Wikipedia stress test. Panels (a) and (b) use the matched 128-update Shared/Routed runs; panel (c) reports three temperature-1 evaluations of a fixed Routed checkpoint on the same 200 questions. Method Base EM ↑ F1 ↑ Retrieval baselines Direct ★ 16.5 24.6 Naive RAG ★ 20.5 26.3 Search-RL baselines Search-R1 ◆ 27.5 35.2 R1-Searcher ⚫ 27.5 35.4 MAS-RL baselines MAGRPO ★ 23.0 29.2 Dr. MAS ★ 27.0 34.4 MATPO ★ 33.5 43.7 ExRole-Shared ★ 30.0 38.3 ExRole-Routed ★ 30.5 37.8 Table 8: HotpotQA external references under each method’s stated protocol. Retrieval, training, and evaluation details differ across rows, so the values provide context rather than a controlled ranking. External Reference Training Curves name app.external-baseline-curves xyz We adapt MAGRPO and Dr. MAS to MuSiQue, HotpotQA, and 2WikiMultiHopQA with the same Qwen2.5-7B SFT policy backbone. Figure 12 combines their learning dynamics: MAGRPO uses a sequential three-agent search interface and grouped updates over 512 training instances, while Dr. MAS uses shared-model grouped rollouts over 128 updates with an 8192-token context budget. Panel (a) reports MAGRPO reward, policy loss, and final 200-example EM/F1; panel (b) reports Dr. MAS reward, valid-action ratio, and native periodic validation. The horizontal axis is normalized within each run, so the curves verify non-degenerate optimization but do not equate method-specific objectives or logging schedules. Diagnostic group Quantity Unit Shared Routed Δ Team turns count 4.48±1.50; 3.0[3.0,6.0]4.48± 1.50;\ 3.0[3.0,6.0] 4.49±1.50; 4.0[3.0,6.0]4.49± 1.50;\ 4.0[3.0,6.0] +0.02+0.02 Search calls count 3.48±1.50; 2.0[2.0,5.0]3.48± 1.50;\ 2.0[2.0,5.0] 3.49±1.50; 3.0[2.0,5.0]3.49± 1.50;\ 3.0[2.0,5.0] +0.02+0.02 Retrieved text kchars 5.35±3.42; 4.50[2.52,7.30]5.35± 3.42;\ 4.50[2.52,7.30] 5.35±3.34; 4.56[2.53,7.41]5.35± 3.34;\ 4.56[2.53,7.41] −0.01-0.01 Resource use Repeated queries count 0.33±0.80; 0.0[0.0,0.0]0.33± 0.80;\ 0.0[0.0,0.0] 0.36±0.83; 0.0[0.0,0.0]0.36± 0.83;\ 0.0[0.0,0.0] +0.03+0.03 Search issued episodes (%) 100.0 100.0 0.00.0 Target retrieved episodes (%) 73.5 73.0 −0.5-0.5 Gold span visible episodes (%) 73.5 73.5 0.00.0 Grounded answer episodes (%) 87.0 87.0 0.00.0 Answer stages Strict success episodes (%) 31.5 30.0 −1.5-1.5 Table 9: Matched MuSiQue inference and answer diagnostics over the same 200 examples. Resource-use cells report mean± deviation (SD) followed by median [interquartile range (IQR)]; stage rows report percentages. Δ is ExRole-Routed minus ExRole-Shared. Stage conditions are evaluated independently and do not form a monotone funnel. (a) MAGRPO training dynamics and final evaluation. (b) Dr. MAS training dynamics and periodic validation. Figure 12: External MAS-RL baseline reproductions on MuSiQue, HotpotQA, and 2WikiMultiHopQA. Both panels use normalized training progress on the horizontal axis. MAGRPO reports reward, policy loss, and final 200-example EM/F1, while Dr. MAS reports reward, valid-action ratio, and its native periodic validation measurements. Benchmark Metric Score Paired Routed −- Shared effect Shared Routed Δ (p) 95% CI W/T/L EM 31.5 30.0 −1.5-1.5 [−4.0,+0.5][-4.0,\,+0.5] 1/195/4 F1 43.2 41.5 −1.8-1.8 [−4.2,+0.3][-4.2,\,+0.3] 2/193/5 MuSiQue Succ. 31.5 30.0 −1.5-1.5 [−4.0,+0.5][-4.0,\,+0.5] 1/195/4 EM 49.0 50.0 +1.0+1.0 [−1.5,+3.5][-1.5,\,+3.5] 4/194/2 F1 59.1 59.7 +0.7+0.7 [−0.8,+2.3][-0.8,\,+2.3] 5/191/4 2Wiki Succ. 49.0 50.0 +1.0+1.0 [−1.5,+3.5][-1.5,\,+3.5] 4/194/2 Table 10: Paired Shared-versus-Routed comparison over 200 examples for each primary benchmark. Δ is measured in percentage points; intervals are 95% paired-bootstrap confidence intervals with 20,000 resamples. W/T/L counts per-example Routed wins, ties, and losses. (a) Role-count sensitivity over three discovery seeds. (b) Cross-seed role agreement for K=3K=3. (c) Extreme paired MuSiQue trajectory changes. Figure 13: Role-induction and evaluation diagnostics. Panel (a) evaluates sensitivity to the number of induced roles; panel (b) measures cross-seed agreement of the induced partition and role profiles; and panel (c) traces paired examples with the largest Routed-versus-Shared F1 changes. Error bars denote one sample standard deviation where applicable. Appendix E Notation Symbol Table name app.symbol-table xyz Symbol Meaning Symbol Meaning x,y,τx,y,τ Task input, gold answer, and team trajectory S,L,dS,L,d Slot budget, adapted linear modules, and LoRA rank N Number of agents in the team s LoRA rank-slot index, 1≤s≤d1≤ s≤ d T,TτT,T_τ Maximum and realized numbers of team turns ωq,ℓs _q, s Score for module ℓ and rank slot s in segment q J,KJ,K Numbers of induction records and source clusters ϖq,ℓs _q, s Normalized routing mass over LoRA rank slots G,QG,Q GRPO group size and number of role-turn segments b~q,ℓs b_q, s Soft slot budget before hard top-S selection iti_t Active speaker at turn t Mq,ℓsM_q, s Hard top-S slot mask oto_t Text observation given to the active agent γq,ℓs _q, s Effective LoRA multiplier after sparse-delta routing at,mta_t,m_t Structured action and optional teammate message cq,c^qc_q, c_q Detached turn-credit target and router prediction ete_t Retrieved evidence after a search action ℬRB_R Role-turn segments in the current normalization batch zi,zq,z(u)z_i,z_q,z(u) Runtime role of an agent, segment, or token θ,θe,θbθ, _e, _b Policy, role-encoder, and router parameters ξt,ϕtobs _t, _t^obs Learned role embedding and prefix-local behavior feature vector τb,λdisc _b, _disc Routing temperature and turn-credit discount pkp_k Induced source prototype for cluster k Adv(g),ρt,j(g)Adv^(g), _t,j^(g) Group advantage and token probability ratio ηk _k Action, stage, evidence, and return statistics for cluster k Utturn,αcU_t^turn, _c Discounted turn return and policy-credit mixing weight ψz,χz _z, _z Readable role instruction and token-aligned role marker R(τ,y)R(τ,y) Episode reward against gold answer y q,seg(u)q,seg(u) Role-turn segment and token-to-segment map nem,nsuccn_em,n_succ Benchmark and strict environment normalizers Γt,Γq,Γ(u) _t, _q, (u) Adapter-routing state at turn, segment, or token level Neval,SuccN_eval,Succ Evaluation-set size and strict success rate zq,ζqv_z_q, _q Executable-role feature and pooled semantic prefix Table 11: Symbols used throughout the method, training objective, and evaluation protocol.