Paper deep dive
RAVEN: Receiver-Conditioned Action-Value Encoding for Finite-Alphabet Multi-Agent Communication
Shuwei Sun, Chenxi Wang, Jian Huang, Weiyun Ru, Hui Cao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 9/30/2026, 6:29:43 AM
Summary
The paper introduces RAVEN (Receiver-conditioned Action-Value ENcoding), a multi-agent communication method for finite-alphabet channels. RAVEN addresses the issue where averaging action values over a receiver's context erases decision-distinctive information. It proposes two estimators: Offline RAVEN, which uses a teacher to build an exact conditional codebook and freezes the sender, and Online RAVEN, which aligns a deployed symbol pathway with a training-only continuous reference within a QMIX learner. RAVEN uses 2-bit messages and significantly outperforms existing methods like NDQ, CACOM, and ExpoComm in navigation and predator-prey tasks.
Entities (11)
Relation Signals (10)
RAVEN → hasvariant → Online RAVEN
confidence 95% · Without a teacher, online RAVEN aligns...
RAVEN → hasvariant → Offline RAVEN
confidence 95% · We give two estimators of this target. With a teacher, offline RAVEN...
RAVEN → achievesbestscoreon → MPE
confidence 90% · on SMAC and MPE it attains the best mean normalized score
RAVEN → achievesbestscoreon → SMAC
confidence 90% · on SMAC and MPE it attains the best mean normalized score
RAVEN → outperforms → CACOM
confidence 90% · Every RAVEN message costs 2 bits, 12-1,024x fewer than those of... CACOM... on navigation
RAVEN → outperforms → NDQ
confidence 90% · Every RAVEN message costs 2 bits, 12-1,024x fewer than those of NDQ... on navigation
RAVEN → outperforms → ExpoComm
confidence 90% · Every RAVEN message costs 2 bits, 12-1,024x fewer than those of... ExpoComm on navigation
Online RAVEN → usesbackbone → QMIX
confidence 90% · online RAVEN aligns, inside a QMIX learner...
Offline RAVEN → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A message drawn from a small alphabet helps a teammate only if it keeps the distinctions that change that teammate's next decision. We show that scoring messages by action values averaged over the receiver's situation can erase exactly these distinctions, and we propose RAVEN (Receiver-conditioned Action-Value ENcoding), which trains a four-symbol, one-step-delayed channel to preserve each receiver's centered action-value profile within the receiver's own context. The sender never needs to know that context: the receiver decodes every symbol with its private information. We give two estimators of this target. With a teacher, offline RAVEN selects the codebook that exactly minimizes an empirical conditional distortion and distills it into a frozen sender; we bound the resulting codebook-selection error and one-step decision loss. Without a teacher, online RAVEN aligns, inside a QMIX learner, the deployed symbol pathway with a training-only continuous reference that shares its routing. Against five recent communication methods on eight navigation settings, offline RAVEN attains the highest return in seven, and removing receiver conditioning forfeits 83% of its communication gain. Online RAVEN raises predator-prey capture success from 53.2% to 96.0% over the same QMIX backbone without communication, and on SMAC and MPE it attains the best mean normalized score of 14 methods, including methods that exchange kilobit messages. Every RAVEN message costs 2 bits, 12-1,024x fewer than those of NDQ, CACOM and ExpoComm on navigation.
Tags
Links
- Source: https://arxiv.org/abs/2609.37566v1
- Canonical: https://arxiv.org/abs/2609.37566v1
Trouble viewing inline? Open PDF directly →
Full Text
106,304 characters extracted from source content.
Expand or collapse full text
RAVEN: Receiver-Conditioned Action-Value Encoding for Finite-Alphabet Multi-Agent Communication Shuwei Sun Affiliation: Xi’an Jiaotong University, Xi’an, China , Chenxi Wang Affiliation: Xi’an Jiaotong University, Xi’an, China , Jian Huang Affiliation: Xi’an Jiaotong University, Xi’an, China , Weiyun Ru Affiliation: Xi’an Jiaotong University, Xi’an, China and Hui Cao Affiliation: Xi’an Jiaotong University, Xi’an, China © none Abstract. A message drawn from a small alphabet helps a teammate only if it keeps the distinctions that change that teammate’s next decision. We show that scoring messages by action values averaged over the receiver’s situation can erase exactly these distinctions, and we propose RAVEN (Receiver-conditioned Action-Value ENcoding), which trains a four-symbol, one-step-delayed channel to preserve each receiver’s centered action-value profile within the receiver’s own context. The sender never needs to know that context: the receiver decodes every symbol with its private information. We give two estimators of this target. With a teacher, offline RAVEN selects the codebook that exactly minimizes an empirical conditional distortion and distills it into a frozen sender; we bound the resulting codebook-selection error and one-step decision loss. Without a teacher, online RAVEN aligns, inside a QMIX learner, the deployed symbol pathway with a training-only continuous reference that shares its routing. Against five recent communication methods on eight navigation settings, offline RAVEN attains the highest return in seven, and removing receiver conditioning forfeits 83% of its communication gain. Online RAVEN raises predator–prey capture success from 53.2% to 96.0% over the same QMIX backbone without communication, and on SMAC and MPE it attains the best mean normalized score of 14 methods, including methods that exchange kilobit messages. Every RAVEN message costs 2 bits, 12–1,024× fewer than those of NDQ, CACOM and ExpoComm on navigation. Keywords: Multi-agent reinforcement learning; discrete communication; limited bandwidth; receiver conditioning; value decomposition 1. Introduction Agents that cooperate under partial observability often share a channel that carries only a few bits per step: a radio link, an acoustic modem, or a protocol budget in a large team. With an alphabet of four symbols, a sender cannot pass on what it sees; it has to decide which distinctions a symbol keeps. The natural answer is to keep the distinctions that change what the receiver should do. We show that this answer depends on where those distinctions are measured, and we build a communication method on the resulting principle. Learned communication has progressed from continuous message passing (Sukhbaatar et al., 2016; Foerster et al., 2016) to attention, gating and scheduling (Jiang and Lu, 2018; Das et al., 2019; Singh et al., 2019; Li et al., 2025; Deng et al., 2026) and to succinct or bottlenecked messages (Wang et al., 2020b; Wang et al., 2020a; Zhang et al., 2020; Tucker et al., 2022; Canesse et al., 2026). Decision-oriented methods go further and compress observations by their effect on value, for example by clustering value representations with a return-gap guarantee (Chen et al., 2024b) or by aggregating observations according to inferred optimal actions (Mostaani et al., 2024). Receiver-aware methods tailor messages to their recipients by modeling teammates (Yuan et al., 2022) or by requesting their context before replying (Li and Zhang, 2024). These lines establish that messages should serve decisions; the question we address is how a sender that sees only its own input, and speaks through four symbols with one step of delay, should decide which of its situations deserve different symbols. Consider two sender types and two receiver contexts (Figure 1). In context C1C_1 the receiver’s centered values over its two actions are (1,−1)(1,-1) under type T1T_1 and (−1,1)(-1,1) under T2T_2; in C2C_2 they are reversed. Averaged over contexts, both types have the profile (0,0)(0,0): an objective that compares value profiles after averaging sees no reason to separate them, and the message becomes useless. Compared within each context, the two types disagree completely, and separating them lets the receiver act optimally in both. Conditioning on everything the receiver observes avoids the cancellation but leaves too few samples per context. RAVEN takes the middle road. It compares centered receiver action values within a small number of receiver conditions, formed from the receiver features to which its action gaps are most sensitive, and it never transmits those conditions: the receiver interprets each symbol with its own information, much as a decoder uses side information in Wyner–Ziv coding (Wyner and Ziv, 1976). One symbol can therefore mean “go left” in one context and “go right” in another at no cost in bandwidth. Two failure modes must be kept apart here. Constructive objectives that score candidate symbols by value profiles averaged over the receiver’s situation—the value-clustering constructions of RGMComm and ABSA (Chen et al., 2024b; Mostaani et al., 2024)—face the cancellation head-on, because the averaging is part of their objective. End-to-end methods such as DIAL and SLIM (Foerster et al., 2016; Canesse et al., 2026) instead backpropagate the receiver’s loss, which is computed from the receiver’s own observation, into the sender; their messages are receiver-conditioned by construction, and what limits them is the biased gradient of the discrete channel, not cancellation. RAVEN addresses each in turn: the conditional target removes the first, and exact construction with a frozen sender avoids the second. Section 4.3 isolates the two effects with separate ablations. Left: a two-by-two grid of bar pairs showing centered values of two receiver actions for two sender types in two contexts; the signs flip between contexts. Top right: after averaging, both types have flat profiles and would share one symbol. Bottom right: RAVEN assigns them different symbols, and the receiver maps symbol 1 to different actions in the two contexts. Figure 1. Why the receiver’s context matters. Two sender types induce opposite action gaps in two receiver contexts. After averaging over contexts both profiles are zero and merging the types costs nothing (Δ=0 =0, Eq. (6)); compared within contexts they must be kept apart (Δ=2 =2), and the receiver decodes the same symbol differently in each context.Left: a two-by-two grid of bar pairs showing centered values of two receiver actions for two sender types in two contexts; the signs flip between contexts. Top right: after averaging, both types have flat profiles and would share one symbol. Bottom right: RAVEN assigns them different symbols, and the receiver maps symbol 1 to different actions in the two contexts. We turn this principle into two estimators that share one execution interface (Figure 2). Offline RAVEN uses a centralized teacher to build a table of receiver-conditioned action values, selects the optimal four-symbol codebook for this table exactly, by enumerating all 611,501 groupings of twelve sender types, distills it into a sender network that is then frozen, and trains receivers on the frozen symbols. Online RAVEN removes the teacher: inside a QMIX learner (Rashid et al., 2018), a training-only reference branch replaces each decoded symbol with the sender’s continuous state under exactly the same routing, and an alignment loss pulls the deployed branch’s action-value profile toward the reference profile. Our contributions are: • A receiver-conditioned target for finite-alphabet messages. We define the target of a symbol as the receiver’s centered action-value profile within its own condition, decompose its reconstruction risk into an abstraction and a compression term, and bound both the codebook-selection error and the one-step decision loss (Section 3.4). • Offline RAVEN, an exact conditional codebook with a frozen sender. Against SLIM, NDQ, CACOM, ExpoComm and MACC on eight navigation settings, it has the highest return in seven, wins 38 of 40 seed-paired comparisons, and needs 82–98% fewer execution FLOPs than NDQ, CACOM and ExpoComm. • Online RAVEN, a teacher-free paired-reference estimator. With the backbone held fixed, its 2-bit channel raises predator–prey capture success by 42.8 percentage points, and on SMAC and MPE it has the best mean normalized score, ahead of methods that exchange 2–4 kbit messages. • Evidence for the mechanism. Conditioning on the receiver accounts for 83% of the communication gain, and the conditional risk that RAVEN minimizes, measured on held-out data, orders the variants by return (within-seed Spearman ρ=0.76ρ=0.76). 2. Related Work Learned communication under bandwidth limits. CommNet and DIAL learn continuous and discretized messages end to end (Sukhbaatar et al., 2016; Foerster et al., 2016); ATOC, TarMAC and IC3Net learn groups, targeted attention and gates (Jiang and Lu, 2018; Das et al., 2019; Singh et al., 2019); ExpoComm and SOPS scale communication with exponential and sparse topologies (Li et al., 2025; Deng et al., 2026). NDQ, IMAC and TMC make messages succinct with information-theoretic regularizers and temporal message control (Wang et al., 2020b; Wang et al., 2020a; Zhang et al., 2020), VQ-VIB trades utility against vocabulary complexity (Tucker et al., 2022), and SLIM decouples policy and message dimensions under bandwidth constraints (Canesse et al., 2026). MASIA, GLC and LMAC ground messages by reconstruction, language-model supervision or state recovery (Guan et al., 2022; Du et al., 2026; Bae et al., 2026). These methods learn what to send by end-to-end gradients or reconstruction; RAVEN instead specifies the target of each symbol, the receiver’s conditional action-value profile, and optimizes it exactly when a teacher is available. Decision-oriented and receiver-aware messages. The RGMComm method bounds the return gap of discrete messages obtained by clustering value representations (Chen et al., 2024b), and ABSA aggregates observations by their inferred optimal actions for a centralized controller (Mostaani et al., 2024). MAIC generates incentive messages from models of teammates (Yuan et al., 2022); CACOM broadcasts a context request and returns personalized, gated replies (Li and Zhang, 2024); SeqComm-DFL scores sequential messages by the improvement of a receiver’s best value (Amoh et al., 2026); MACC assigns credit to delayed discrete messages with a communication critic (Vanneste et al., 2025). RAVEN shares the decision-oriented view but differs in two respects: its distortion is measured within receiver conditions before averaging, which prevents opposite action gaps from canceling, and it needs no request round, since one broadcast symbol is decoded by each receiver with its own information—the side-information principle of Wyner–Ziv coding (Wyner and Ziv, 1976). Task-oriented communication over physical channels. The wireless and semantic-communication communities study the same joint design of learning and communication: Tung et al. (Tung et al., 2021) treat a noisy channel as part of the multi-agent environment and learn coding and control jointly, and task-based information compression (Mostaani et al., 2022) formulates observation compression under rate constraints as a rate–distortion problem whose distortion is measured in task return. RAVEN shares the task-oriented objective but assumes an error-free channel with a four-symbol alphabet; the scarce resource is not the signal-to-noise ratio but the distinctions that four symbols can keep. Value factorization and supervision. We build on centralized training with decentralized execution (Oliehoek and Amato, 2016; Lowe et al., 2017; Foerster et al., 2018; Yu et al., 2022) and on monotonic value factorization (Sunehag et al., 2018; Rashid et al., 2018; Wang et al., 2021b), which RAVEN uses to express each receiver action in team value during training. The offline estimator relates to policy distillation (Rusu et al., 2016) and to vector quantization (Gray and Neuhoff, 1998; van den Oord et al., 2017; Banerjee et al., 2005); unlike policy distillation, RAVEN distills a codebook whose meaning is fixed by construction, and freezes it so that the receivers can rely on it. 3. RAVEN Two lanes. The offline lane goes from a Double-DQN teacher through sender types and receiver conditions to a type-by-condition table, an exact codebook mapping twelve types to four symbols, and a frozen sender with trained receivers. The online lane shows a recurrent sender producing a hard symbol delivered at the next step, a receiver whose shared routing feeds a deployed branch using symbol embeddings and a training-only reference branch using the sender's continuous state, a shared residual head, a QMIX mixer with fixed teammates, centered action profiles of both branches, and the two losses. Figure 2. RAVEN under a four-symbol, one-step-delayed interface. (a) Offline RAVEN builds a table of receiver-conditioned centered values from a teacher, selects the exactly optimal codebook, distills and freezes the sender, and trains the receivers on its symbols (table and grouping shown schematically). (b) Online RAVEN evaluates each receiver twice with one set of weights and one routing decision: once with the decoded symbols (deployed) and once with the senders’ continuous states (reference, training only). The alignment loss ℒCVL_CV between the two centered profiles updates only the encoder and the symbol embedding Φ ; TD losses train all networks.Two lanes. The offline lane goes from a Double-DQN teacher through sender types and receiver conditions to a type-by-condition table, an exact codebook mapping twelve types to four symbols, and a frozen sender with trained receivers. The online lane shows a recurrent sender producing a hard symbol delivered at the next step, a receiver whose shared routing feeds a deployed branch using symbol embeddings and a training-only reference branch using the sender's continuous state, a shared residual head, a QMIX mixer with fixed teammates, centered action profiles of both branches, and the two losses. 3.1. Interface and Target We consider N cooperative agents with shared reward rtr_t and discount γ in a decentralized, partially observable environment (Oliehoek and Amato, 2016). At step t, each active sender j maps its permitted local input XjtX_j^t (its own observation and history; no global state and no other agent’s private state) to one of K=4K=4 symbols, (1) Mjt=sθ,j(Xjt)∈1,…,K,M_j^t=s_θ,j(X_j^t)∈\1,…,K\, which the task’s communication edges deliver at t+1t+1. Receiver i then acts on its local information Zit+1Z_i^t+1 and its mailbox it+1M_i^t+1 of arrived symbols. Each message costs log2K=2 _2K=2 bits, and the symbols are the only quantities exchanged at execution. A receiver’s decision depends on the differences between its action values, not on their level. For a legal action set iA_i, let P=I−⊤/|i|P=I-11 /|A_i| remove the mean of a value vector; then (2) ‖P(u−v)‖22=1|i|∑a<b[(ua−ub)−(va−vb)]2,\|P(u-v)\|_2^2= 1|A_i| _a<b [(u_a-u_b)-(v_a-v_b) ]^2, so the error between centered profiles is exactly the error in all pairwise action gaps. RAVEN asks each symbol to preserve the receiver’s centered action-value profile within the receiver’s condition, a finite summary of the receiver’s context that is used during training only. 3.2. Offline RAVEN: Exact Conditional Codebook Targets. A Double-DQN teacher Q Q (van Hasselt et al., 2016) is trained with centralized inputs. Each construction row n∈n pairs a sender input XnX_n at time t with the receiver’s features Vn⊆ZnV_n Z_n and the teacher context ξn _n at the arrival time t+1t+1, so the target describes the decision the receiver faces when the symbol arrives. With partner actions drawn from a fixed reference ρ (uniform in navigation), the receiver’s centered, scaled profile is (3) una u_na =−i∼ρQ^(ξn,a,−i), =E_a_-i ρ\, Q( _n,a,a_-i), Yna Y_na =1sQ(una−1|n|∑b∈nunb), = 1s_Q (u_na- 1|A_n| _b _nu_nb ), where sQs_Q is the standard deviation of teacher values over D. Sender types and receiver conditions. Sender inputs are clustered into nT=12n_T=12 types T=τ(X)T=τ(X) (speaker–listener tasks use the three target colors). Receiver conditions are obtained by clustering in a metric that stretches the receiver features to which its action gaps are sensitive. With standardized features V¯∈ℝF V ^F, (4) ζℓ _ =1||∑n1|n|∑a∈n(∂Yna∂V¯nℓ)2, = 1|D| _n 1|A_n| _a _n ( ∂ Y_na∂ V_n )^2, C C =kmeans(Ω1/2V¯),Ω=diag(Fζℓ∑kζk)+0.05I, =kmeans ( ^1/2 V ), =diag ( F\, _ _k _k )+0.05I, with nC=12n_C=12 conditions, k-means++ seeding and Lloyd iterations (Arthur and Vassilvitskii, 2007; Lloyd, 1982). Conditional table. For type r, condition c and action a, let NrcaN_rca and SrcaS_rca be the mass and the target sum of the rows with Tn=rT_n=r, Cn=cC_n=c and a legal, each row spreading unit weight over its legal actions. Sparse cells are shrunk toward the type mean Y¯ra Y_ra: (5) μ~rca=Srca+λraY¯raNrca+λra,αrca=Nrca||, μ_rca= S_rca+ _ra Y_raN_rca+ _ra, _rca= N_rca|D|, with pseudo-count λra=32Nra/nr _ra=32N_ra/n_r, where Nra=∑cNrcaN_ra= _cN_rca and nrn_r is the number of rows of type r. Exact codebook. A codebook e maps the types to at most K symbols. Its distortion is the mass-weighted error between each cell and the centroid of its symbol in the same condition, (6) D^(e)=∑r,c,aαrca(μ~rca−μ¯e(r)cae)2,μ¯mcae=∑r:e(r)=mαrcaμ~rca∑r:e(r)=mαrca. D(e)= _r,c,a _rca ( μ_rca- μ^e_e(r)ca )^2, μ^e_mca= _r:e(r)=m _rca μ_rca _r:e(r)=m _rca. Merging two groups with masses v1,v2v_1,v_2 and means μ1,μ2 _1, _2 raises D D by Δ=∑c,av1cav2cav1ca+v2ca(μ1ca−μ2ca)2 = _c,a v_1cav_2cav_1ca+v_2ca( _1ca- _2ca)^2 (Ward, 1963), which vanishes only if the groups agree in every condition where both occur; opposite gaps in different contexts therefore never cancel (Figure 1), whereas groups that never occur in the same condition can share a symbol for free. For twelve types we enumerate all S(12,4)=611,501S(12,4)=611,501 partitions into four groups; because refinement never increases distortion, this returns the exact minimizer e e over all codebooks with at most four symbols. With several recipients, one shared codebook minimizes the sum of their distortions. Sender and receivers. The sender network pθ(m∣X)p_θ(m X) is fitted to the labels e^(τ(Xn)) e(τ(X_n)) by cross-entropy, sθ(X)=argmaxmpθ(m∣X)s_θ(X)= _mp_θ(m X), and then frozen together with every feature that produces symbols, so the meaning of each symbol cannot drift. Receivers keep their full local input ZiZ_i, not the coarse condition, and are trained on the frozen senders’ actual symbols to maximize the teacher’s centered joint value under their product policy, (7) ℒrec=−n[∑∏iπϕ,i(ai∣Zi,i)Q~n()],L_ rec=-E_n [ _a _i _φ,i(a_i Z_i,M_i)\, Q_n(a) ], where Q~n Q_n is the teacher value centered over legal joint actions and scaled by sQs_Q. At execution, senders emit their argmax symbol and receivers act greedily; neither types nor conditions are ever computed. 3.3. Online RAVEN: Paired Reference When no teacher is available or joint actions cannot be enumerated, RAVEN estimates the receiver-conditioned target inside the learner (Figure 2b). The idea is to ask, during training only, what the receiver would do if it could read the sender’s continuous state instead of a 2-bit symbol, and to teach the four symbols to reproduce that decision profile: a training-only reference branch answers the question, and the alignment loss ℒCVL_CV does the teaching. Agents share a recurrent network (a 64-unit layer and a 64-unit GRU (Cho et al., 2014)) that maps observation, previous action and identity to a state hith_i^t and local utilities qiloc,tq_i loc,t; a QMIX mixer fψf_ψ (Rashid et al., 2018) combines utilities during training. Codec. A linear encoder produces four logits ljtl_j^t. The transmitted symbol is the argmax ; the backward pass uses the straight-through estimator (Bengio et al., 2013), (8) m~jt=onehot(argmaxljt)+pjt−sg(pjt),pjt=softmax(ljt), m_j^t=onehot( l_j^t)+p_j^t-sg(p_j^t), p_j^t=softmax(l_j^t), and a shared 4×644× 64 embedding Φ decodes gjt=Φ⊤m~jtg_j^t= m_j^t, the embedding of the received symbol. One routing, two branches. At t+1t+1, receiver i scores each permitted sender by the scaled dot product of its private query WQhit+1W_Qh_i^t+1 and the symbol key WKLN(gjt)W_KLN(g_j^t), keeps the two highest-scoring senders i⋆N_i , and normalizes their scores into weights βij _ij. RAVEN evaluates two branches under this single routing decision: (9) cidep=∑j∈i⋆βijWVLN(gjt),ciref=∑j∈i⋆βijWVLN(hjt).c_i^dep= _j _i _ijW_VLN(g_j^t), c_i^ref= _j _i _ijW_VLN(h_j^t). The deployed branch aggregates what was transmitted; the reference branch aggregates the continuous states that the symbols had to summarize. Because query, keys, selected senders and weights are shared, the two branches differ only in the content of the messages. A residual head RϑR_ with a zero-initialized output layer turns each context into an action correction, (10) Qib(a) Q_i^b(a) =qiloc(a)+i≠∅Δib(a), =q_i loc(a)+1\M_i≠ \\, _i^b(a), Δib _i^b =Rϑ([hi,cib,hi⊙cib]),b∈dep,ref, =R_ ([h_i,c_i^b,h_i c_i^b] ), b∈\dep,ref\, so training starts from the local policy and the channel gains influence only as it becomes useful. Paired profiles. For a probe (t,i)(t,i) sampled from replay, we hold the teammates’ utilities uku_k at their reference values on the replayed actions, vary receiver i’s action over its legal set, and pass both branches through the same mixer and state: (11) Vib(a)=fψ(u1,…,Qib(a),…,uN,s),V¯ib=PVib.V_i^b(a)=f_ψ (u_1,…,Q_i^b(a),…,u_N;s ), V_i^b=PV_i^b. The mixed profiles express every receiver action in team value, and centering removes their level. The alignment loss moves the deployed profile toward the reference profile, (12) ℒCV _CV =1nℬ∑(t,i)∈ℬ∑a∈it[V¯idep(a)−sg(V¯iref(a))]2sg(σi2), = 1n_B _(t,i) \ _a _i^t [ V_i^dep(a)-sg ( V_i^ref(a) ) ]^2sg( _i^2), σi2 _i^2 =max1,1|it|∑aV¯iref(a)2, = \1,\ 1|A_i^t| _a V_i^ref(a)^2 \, where ℬB contains at most eight probe times per update and nℬn_B counts its receiver–action pairs. The gradient of ℒCVL_CV reaches only the encoder and the embedding Φ : states, teammate utilities, reference profiles and scales are detached, and the receiver and mixer weights are held fixed for this term. Both branches are trained by Double-Q TD losses on the team value (van Hasselt et al., 2016; Rashid et al., 2018), (13) ℒ=12(ℒTDdep+ℒTDref)+0.1ℒCV.L= 12 (L_TD^dep+L_TD^ref )+0.1\,L_CV. At execution, only the recurrent network, codec, receiver and local head remain, and agents act greedily on their deployed utilities; the reference branch, the mixer and the global state are used in training only, and no continuous state is ever transmitted. Algorithm 1 RAVEN training Offline RAVEN (teacher Q Q, construction data D, K=4K=4) 1: compute centered targets YnY_n at the arrival time (Eq. 3) 2: cluster sender types T=τ(X)T=τ(X) and receiver conditions C (Eq. 4) 3: build the shrunk table μ~rca μ_rca with masses αrca _rca (Eq. 5) 4: e^←argmineD^(e) e← _e D(e) by enumerating all groupings (Eq. 6) 5: fit the sender sθs_θ to the labels e^(τ(Xn)) e(τ(X_n)), then freeze it 6: train receivers πϕ _φ on the frozen symbols (Eq. 7) Online RAVEN (one update, after each collected episode) 1: act ϵε-greedily with hard symbols; store the episode in replay 2: sample 32 episodes; unroll states h and symbols M 3: for b∈dep,refb∈\dep,ref\ do ⊳ one routing, two branches 4: contexts cbc^b (Eq. 9), utilities QbQ^b (Eq. 10), TD loss ℒTDbL_TD^b 5: for ≤8≤ 8 probe times: paired profiles (Eq. 11) and ℒCVL_CV (Eq. 12) 6: take one gradient step on ℒL (Eq. 13); ∇ℒCV _CV reaches the codec only 3.4. Analysis We state the guarantees of the offline estimator for a fixed teacher, partner reference and input distribution, a fixed legal set with ‖v‖2=||−1∑ava2\|v\|_A^2=|A|^-1 _av_a^2, and the ideal symbol M=e(T)M=e(T). Proofs are in Appendix B. Proposition 3.1 (Decomposition and codebook selection). Let ℛC(e)=∥Y−[Y∣M,C]∥2R_C(e)=E\|Y-E[Y M,C]\|_A^2, ℛZ(e)=∥Y−[Y∣M,Z]∥2R_Z(e)=E\|Y-E[Y M,Z]\|_A^2 and C(e)=∥[Y∣M,Z]−[Y∣M,C]∥2G_C(e)=E\|E[Y M,Z]-E[Y M,C]\|_A^2. Since C is a function of Z, (14) ℛC(e)=ℛZ(e)+C(e)=ℰabs+(e),R_C(e)=R_Z(e)+G_C(e)=E_ abs+D(e), with abstraction error ℰabs=∥Y−[Y∣T,C]∥2E_ abs=E\|Y-E[Y T,C]\|_A^2, independent of e, and compression error (e)=∥[Y∣T,C]−[Y∣M,C]∥2D(e)=E\|E[Y T,C]-E[Y M,C]\|_A^2. If supe|D^(e)−(e)|≤ϵD _e| D(e)-D(e)|≤ _D and e e minimizes D D up to ϵopt _ opt, then (15) ℛZ(e^)−infeℛZ(e)≤osceC(e)+2ϵD+ϵopt.R_Z( e)- _eR_Z(e) _eG_C(e)+2 _D+ _ opt. Proposition 3.2 (One-step decision loss). Let U be the receiver’s true payoffs under the partner reference with |Ua|≤B|U_a|≤ B, ϵQ=‖U−u‖∞ _Q=E\|U-u\|_∞ for the teacher vector u of Eq. (3), q=Pr[sθ(X)≠e^(T)]q= [s_θ(X)≠ e(T)], and let the receiver lose at most ϵrec _ rec in true payoff relative to the decoder argmaxa[ua∣M,Z] _aE[u_a M,Z]. Then its expected payoff JUJ_U satisfies (16) maxaUa−JU≤2ϵQ+sQ||ℛZ(e^)+ϵrec+2Bq,E _aU_a-J_U≤ 2 _Q+s_Q |A|\,R_Z( e)+ _ rec+2Bq, where q≤[−logpθ(e^(T)∣X)]/log2q [- p_θ( e(T) X)]/ 2. Proof idea. Both equalities in Eq. (14) are Pythagorean identities of conditional expectation, because (M,C)(M,C) is coarser than both (M,Z)(M,Z) and (T,C)(T,C); Eq. (15) follows by comparing e e with any codebook through ℛZ=ℰabs+−CR_Z=E_ abs+D-G_C. For Proposition 3.2, the Bayes decoder’s regret with respect to the teacher is at most the expected deviation of the teacher values from their conditional mean at the true best action, which after centering is at most the maximum centered reconstruction error and hence, by the Euclidean and Jensen inequalities, at most sQ||ℛZs_Q |A|R_Z; replacing U by u twice costs 2ϵQ2 _Q, and a wrong symbol costs at most 2B2B. An input whose argmax symbol differs from its label assigns the label a probability of at most one half and so contributes at least log2 2 to the cross-entropy; Markov’s inequality then bounds q. Together, the propositions state what the construction controls: the conditional distortion, which offline RAVEN minimizes exactly, bounds the decision loss caused by compressing the sender’s input into four symbols, up to estimation error and the residual gap of finitely many conditions. Section 4.3 confirms this link empirically. For the online estimator, QMIX’s monotonic mixer preserves the ordering of a receiver’s utilities at fixed state and teammate utilities, and to first order V¯idep−V¯iref=Pdiag(ηia)δQi+O(‖δQi‖2) V_i^dep- V_i^ref=Pdiag( _ia)\,δ Q_i+O(\|δ Q_i\|^2), where ηia _ia are the mixer’s slopes and δQi=Qidep−Qirefδ Q_i=Q_i^dep-Q_i^ref; ℒCVL_CV thus drives the deployed utilities toward the reference ones along the directions that matter for team value, at a cost of 2∑i|i|2 _i|A_i| mixer evaluations per probe instead of an enumeration of joint actions. 4. Experiments We ask four questions. Does RAVEN outperform recent communication methods under the same information boundary (Section 4.2)? Why does it work (Section 4.3)? Does the teacher-free estimator carry the benefit to SMAC and MPE, and is the gain due to communication (Section 4.4)? What does it cost (Section 4.5)? 4.1. Setup Tasks. Navigation: N1-C2 is MPE simple_reference, in which each agent knows only its partner’s goal; N1-C3/C4/C6 extend it to rings of three, four and six agents in which each agent’s goal is known only to its predecessor; N2-L1/L2/L4/L6 are speaker–listener tasks with one, two, four and six listeners (25 steps each). Predator–prey (P1): three agents on a 5×55× 5 grid with vision radius one must reach the prey together within 20 steps. SMAC (Samvelyan et al., 2019): 3m, 8m, M, 3s5z (hard) and M2 (super hard). MPE: Spread, Tag and Crypto; in Crypto the eavesdropper Eve is controlled by the learned policies like the other agents, and her reward is included in the team return. All methods use the same message edges, the same one-step delay and the same execution-time information. Baselines. On navigation we compare with five communication methods, each trained for 1.2M environment steps. SLIM (Canesse et al., 2026), NDQ (Wang et al., 2020b) (three floats per message), CACOM (Li and Zhang, 2024) (a 4-value request and a gated 12-value reply at 2 bits per value) and ExpoComm (Li et al., 2025) (64 floats) are run with their authors’ code, architectures, optimizers and default hyperparameters; only the message edges, the one-step delay and the budget are adapted, and no task-specific tuning is performed. MACC (Vanneste et al., 2025) is implemented from its paper. SLIM and MACC use the same four symbols as RAVEN. MACC diverges under the enforced one-step delay (returns between −38-38 and −166-166): its communication critic credits a message within the step in which it is sent, so a delayed message is credited to the wrong action. We therefore report MACC in Appendix F (Table 6) rather than in Table 1. On SMAC and MPE, online RAVEN is compared at 2.05M steps with 13 methods, including QMIX (Rashid et al., 2018), QPLEX (Wang et al., 2021b) and the QMIX-based communication methods TeamComm (Yang et al., 2024), TGCNet (Zhang et al., 2025), CACOM and a single-neighbor ExpoComm. We run all 13 from the authors’ official open-source implementations and take no numbers from prior publications. Online RAVEN uses one set of hyperparameters for all tasks. Two controls change only the communication pathway: a same-backbone QMIX without codec, receiver and reference, and online RAVEN without the reference-branch TD loss. Protocol. Every RAVEN model, navigation baseline and control is evaluated at its final checkpoint on 1,000 held-out episodes. The training seed is the statistical unit (five seeds, four on SMAC; ten in the ablation): we report means, standard deviations and 95% t-intervals, and paired two-sided t-tests with Holm correction within each family of comparisons. 4.2. Comparison with Communication Methods Table 1. Navigation: team return at the final checkpoint (mean±s.d._ .d. over 5 seeds, 1,000 held-out episodes per model; higher is better). Best per row in bold, second underlined (reference columns excluded). ‡Trained with one listener and deployed unchanged. The last row gives the bits per message. MACC, which diverges under the enforced one-step delay (Section 4.1), is reported in Table 6. The reference columns hold the teacher fixed on N1-C2 (ten-seed controlled study of Section 4.3): the centralized teacher itself (Q Q, evaluated on its selection episodes, hence an optimistic upper bound), receivers supervised by the same teacher without messages (no msg.), and the same supervision with a learned four-symbol channel in place of the constructed codebook (learn. ch.). Setting RAVENoff RAVENon SLIM NDQ CACOM ExpoComm Q Q (ub) no msg. learn. ch. N1-C2 (2 agents, pair) −-9.59±0.60 −-12.35±1.68 −-18.81±0.76 −-21.34±1.87 −-17.25±0.97 −-14.98±1.44 −-8.52±0.16 −-19.27±0.19 −-14.49±0.65 N1-C3 (3 agents, ring) −-11.47±0.39 −-15.21±2.78 −-18.79±0.71 −-19.98±0.95 −-17.84±0.15 −-17.35±0.92 – – – N1-C4 (4 agents, ring) −-13.74±0.64 −-17.86±0.30 −-20.02±0.85 −-20.06±0.67 −-17.89±0.44 −-17.92±0.30 – – – N1-C6 (6 agents, ring) −-18.15±0.12 −-17.96±0.23 −-20.13±0.72 −-24.32±3.55 −-17.78±0.28 −-17.94±0.26 – – – N2-L1 (1 listener) −-7.70±0.17 −-7.86±0.24 −-11.89±3.01 −-14.19±3.02 −-12.64±4.32 −-15.42±0.28 – – – N2-L2 (2 listeners) −-7.86±0.14‡_± 0.14 −-9.13±1.87 −-12.10±2.92‡_± 2.92 −-15.08±2.78 −-9.65±3.58 −-14.16±2.39 – – – N2-L4 (4 listeners) −-7.92±0.21‡_± 0.21 −-9.82±2.31 −-12.19±2.83‡_± 2.83 −-20.52±1.21 −-10.68±3.61 −-13.66±3.09 – – – N2-L6 (6 listeners) −-7.82±0.09‡_± 0.09 −-9.23±0.53 −-12.26±2.99‡_± 2.99 −-24.60±5.51 −-13.81±5.28 −-13.98±2.40 – – – Bits per message 2 2 2 96 24 (+8 request) 2,048 – – 2 Table 1 gives the final returns. Offline RAVEN (RAVENoff) has the highest return in seven of the eight settings, and online RAVEN (RAVENon) is second in the same seven. Seed-paired tests confirm the ranking: of the 40 comparisons between RAVENoff and the five external methods, 38 favor RAVENoff, 28 remain significant after Holm correction, and in 31 every one of the five seeds favors RAVENoff. On N1-C2, RAVENoff beats every external method on every seed. Its margin over the strongest external method is +5.39+5.39, +5.88+5.88 and +4.15+4.15 on the pair and on the three- and four-agent rings, and a confirmation on fresh seeds that took no part in method development reproduces its leads over SLIM on the three rings (+7.32+7.32, +6.28+6.28, +1.97+1.97; 5/5 seeds each, Holm p≤0.0021p≤ 0.0021). The reference columns of Table 1 show where this advantage comes from. Offline RAVEN’s receivers are trained by maximizing the teacher’s joint value (Eq. (7)), so one might suspect that the teacher, rather than the communication principle, produces the gap. Holding the teacher fixed on N1-C2, receivers that are supervised identically but receive no messages reach only −19.27-19.27, behind SLIM, CACOM and ExpoComm, and replacing the constructed codebook by a learned channel of the same alphabet under the same supervision gives −14.49-14.49: teacher supervision alone does not beat reward-trained communication, and it is the exact conditional construction that turns the supervision into a lead. The comparison is also budget-matched. The teacher’s 1.2M training steps are the only environment interaction of offline RAVEN—the construction data are the teacher’s own transitions, and sender distillation and receiver training reuse these rows without new environment steps—so the 1.2M budget of Table 1 already includes the teacher; checkpoint selection and final evaluation use disjoint sets of episodes. Left: in ring navigation every method has a lower return with six agents than with two; offline RAVEN is highest for two to four agents and all methods except NDQ and SLIM meet near minus 18 at six agents. Right: in broadcast, offline RAVEN deployed zero-shot stays near minus 7.8 for all listener counts, above online RAVEN, CACOM, SLIM, ExpoComm and NDQ, the last of which degrades to minus 24.6 at six listeners. Figure 3. Final return against team size (mean and 95% interval over 5 seeds). (a) Rings of two to six agents. (b) Broadcast to one to six listeners: filled markers are models trained with one listener and deployed unchanged (dashed: CACOM and ExpoComm transferred in the same way), hollow markers are retrained at every size.Left: in ring navigation every method has a lower return with six agents than with two; offline RAVEN is highest for two to four agents and all methods except NDQ and SLIM meet near minus 18 at six agents. Right: in broadcast, offline RAVEN deployed zero-shot stays near minus 7.8 for all listener counts, above online RAVEN, CACOM, SLIM, ExpoComm and NDQ, the last of which degrades to minus 24.6 at six listeners. The broadcast family shows a second strength of RAVEN, its reliability: RAVENoff solves the task on every seed, with standard deviations of 0.09–0.21 against 2.8–5.3 for SLIM and CACOM, whose seeds split between runs that solve the task and runs that do not. Because its codebook is shared by all recipients, a model trained with a single listener can be deployed unchanged to two, four or six listeners and loses at most 0.22 return (Figure 3b); CACOM and ExpoComm transferred in the same way stay near −12.95-12.95 and −15.77-15.77, and even when retrained at every size neither reaches the zero-shot RAVENoff, while NDQ degrades from −14.19-14.19 to −24.60-24.60. In the rings (Figure 3a), RAVENoff leads every external method from two to four agents. The teacher-free estimator keeps most of this benefit. RAVENon beats NDQ and MACC in all 16 of its paired comparisons with them and is ahead of CACOM and of ExpoComm in six of the eight settings and within one standard deviation of them in the other two (N1-C4 and N1-C6), while sending 12–1,024× fewer bits; it trails the exact construction by 0.2–4.1 return in seven settings and is slightly ahead of it on the six-agent ring. 4.3. Why RAVEN Works Three panels. Left: a forest plot in which every variant loses return relative to full RAVEN, the largest losses coming from removing messages and from removing the receiver condition. Middle: a scatter plot with a negative trend between reconstruction risk and return, RAVEN points at the lowest risk and highest return. Right: bars showing that RAVEN loses much more return than SLIM when its messages are removed on the three- and four-agent rings. Figure 4. Mechanism of offline RAVEN on navigation. (a) Return lost by each variant relative to full RAVEN on N1-C2 (paired mean and 95% interval over 10 independently trained seeds; filled: 4 symbols, hollow: 2 symbols). (b) Held-out reconstruction risk ℛR against return for RAVEN and three condition variants, both centered within seed (grey lines join the methods of one seed). (c) Return lost when incoming messages are removed at test time (fresh seeds; dots are seeds); SLIM uses the same 2-bit budget.Three panels. Left: a forest plot in which every variant loses return relative to full RAVEN, the largest losses coming from removing messages and from removing the receiver condition. Middle: a scatter plot with a negative trend between reconstruction risk and return, RAVEN points at the lowest risk and highest return. Right: bars showing that RAVEN loses much more return than SLIM when its messages are removed on the three- and four-agent rings. Table 2. SMAC (win rate, %) and MPE (team return) at 2.05M environment steps. RAVENon: mean±s.d._ .d. with 1,000 evaluation episodes per model (SMAC: 4 seeds; MPE: 5 seeds). Reference methods: our runs of the authors’ official open-source code under the same budget; mean over runs. “Norm.” and “rank” average the per-task min–max normalized score and the rank over all 14 methods on the eight tasks (the remaining seven methods are listed in Table 15). †Single-neighbor variant. Best per column in bold, second underlined. Method 3m 8m M M2 3s5z Spread Tag Crypto Bits/msg Norm. Rank RAVENon 99.72±0.22 99.95±0.06 100.0±0.0 87.35±8.30 97.80±1.12 −-29.00±0.59 277.3±3.0 48.03±0.04 2 0.98 1.75 QMIX (Rashid et al., 2018) 99.80 98.80 98.60 57.80 86.40 −-43.42 23.39 0.38 0 0.74 7.3 QPLEX (Wang et al., 2021b) 99.75 99.50 30.75 62.00 96.50 −-31.57 235.1 45.44 0 0.80 5.0 TeamComm (Yang et al., 2024) 99.25 99.00 98.00 64.25 91.00 −-43.73 68.56 48.00 ≥ 2,048 0.84 5.6 TGCNet (Zhang et al., 2025) 99.00 99.25 74.00 66.50 96.75 −-44.95 55.46 48.00 ≥ 2,048 0.80 5.9 CACOM (Li and Zhang, 2024) 96.25 43.50 98.75 34.25 97.25 −-48.47 251.1 3.18 ≥ 16 0.67 8.1 ExpoComm† (Li et al., 2025) 99.00 99.50 100.0 94.50 96.25 −-26.80 276.9 −-31.20 4,096 0.87 4.0 Conditioning on the receiver is the key design choice. The ablation in Figure 4a trains ten independent seeds per variant on N1-C2; each seed has its own teacher, and all variants of a seed share it, the network capacity and the total number of updates, so each variant changes one factor. Communication is worth +9.82+9.82 return. Compressing the marginal value instead of the conditional one loses 8.188.18 of it (83%) on all ten seeds; this variant keeps the exact construction but scores symbols by the marginal profile, the quantity that value-clustering constructions optimize (Chen et al., 2024b; Mostaani et al., 2024), and its collapse is the cancellation illustrated in Figure 1. The way conditions are formed also matters: conditions built from the receiver’s full observation (−7.54-7.54), from the sensitivity of the joint value instead of the receiver’s own action gaps (−5.76-5.76), or from unweighted clusters (−4.09-4.09) are all significantly worse. A condition that distinguishes everything the receiver sees spends the four symbols on distinctions that do not change its decision. Construction and freezing also matter, and they address the second failure mode: a learnable four-symbol channel of the same capacity loses 5.045.04, and continuing to update the distilled sender together with the receivers loses 5.125.12. Both variants keep the receiver-conditioned supervision but train through the discrete channel, as end-to-end methods do, and both forfeit about half of the communication gain, consistent with biased gradients through the discrete channel eroding the code. All seven differences are positive on 10/10 seeds with Holm-adjusted p<3×10−4p<3× 10^-4; with two symbols the ordering is largely preserved at about half the scale. The construction’s objective predicts return. For each seed we computed, on held-out samples, the conditional reconstruction risk that the codebook minimizes (Proposition 3.1). A lower risk agrees with a higher return in 83% of within-seed method pairs (Spearman ρ=0.76ρ=0.76; Figure 4b), and RAVEN has the lowest risk on every seed. Restricted to the three condition variants, the compression term alone orders their returns with 87% concordance (ρ=0.80ρ=0.80): among these variants, return tracks how well the four symbols preserve the conditional values. The receivers rely on the symbols. Removing incoming messages at test time costs RAVENoff 6.44 and 4.64 return on the three- and four-agent rings, against 1.17 and 0.49 for SLIM at the same budget (Figure 4c), and the empirical symbol entropy is 1.94 of 2 bits, so all four symbols are used. Because the symbols are discrete and their meaning is fixed, the frozen models also degrade gracefully under channel constraints: sending a fresh symbol only every fourth step changes the return by −0.009-0.009 (95% interval [−0.022,0.004][-0.022,0.004]) while cutting the serialized payload by 72%, and 10% or 30% random message loss moves the return only from −9.78-9.78 to −9.80-9.80 and −9.88-9.88. 4.4. Online RAVEN on SMAC and MPE Left: capture success of online RAVEN rises to about 85 percent within 0.4 million steps and above 90 percent by 0.5 million steps, while the same-backbone QMIX falls to about 12 percent and recovers only to about 50 percent. Right: on Spread, online RAVEN is above the variant without the reference TD loss, which is above the same-backbone QMIX, during most of training. Figure 5. Online RAVEN against controls that change only the communication pathway (mean ± s.d.; 100 monitoring episodes per point). (a) Predator–prey capture success, 5 seeds. (b) MPE Spread team return, 5 seeds (variant without the reference TD loss: 3 matched seeds).Left: capture success of online RAVEN rises to about 85 percent within 0.4 million steps and above 90 percent by 0.5 million steps, while the same-backbone QMIX falls to about 12 percent and recovers only to about 50 percent. Right: on Spread, online RAVEN is above the variant without the reference TD loss, which is above the same-backbone QMIX, during most of training. Table 3. Resources saved by RAVEN, 100(1−RAVEN/other)100(1-RAVEN/other) in %, as the range over settings (navigation) or tasks (SMAC and MPE). Parameters and execution FLOPs per team step are counted on the networks used in the runs; training time is compared on the same machine. ⋆Lower bound (only part of the message counted). vs. Bits/msg FLOPs Params Train time RAVENoff, navigation SLIM same – 95.6–98.0 39.6–55.4 NDQ 97.9 82.2–96.8 −-1.4 – 85.1 – CACOM 91.7 85.8–97.7 −-2.1 – 85.0 – ExpoComm 99.9 91.5–98.3 48.4–92.3 – RAVENon, navigation CACOM 91.7 −-18.6 – 1.6 −-48.3 – −-38.2 70.6–74.3 RAVENon, SMAC and MPE TeamComm 99.9⋆ 4.7–21.0 23.7–29.6 – TGCNet 99.9⋆ −-2.9 – 13.7 33.6–40.2 – ExpoComm† 99.95 78.5–82.1 78.8–82.1 – The gain comes from communication. Holding the recurrent backbone, optimizer, replay, exploration, budget and initialization fixed and removing only the channel isolates its contribution. In predator–prey, where three agents with vision radius one must agree on a common target, the 2-bit channel raises capture success from 53.20%53.20\% to 96.04%96.04\%, +42.84+42.84 percentage points (95% interval [15.5,70.2][15.5,70.2], 5/5 seeds, Holm-adjusted p=0.024p=0.024). RAVEN reaches about 85% success after 0.4M steps and ends between 94.5% and 97.0% on every seed, whereas the same-backbone QMIX collapses to 10–14% before recovering slowly (Figure 5a). On Spread, where every agent already sees all landmarks, the channel still adds +2.87+2.87 return ([1.77,3.97][1.77,3.97], 5/5 seeds, p=0.002p=0.002). Removing only the reference-branch TD loss costs 0.690.69 (3/3 seeds, p=0.043p=0.043) and lowers the return by about three units between 0.7M and 1.1M steps (Figure 5b): training the reference branch speeds up learning of the deployed channel. The teacher-free estimator is competitive on SMAC and MPE. On SMAC (Table 2), RAVENon wins 99.7–100% of the games on 3m, 8m and M and has the highest win rate on the hard 3s5z (97.80%). On the super-hard M2, value factorization alone reaches 57.8% (QMIX) and 62.0% (QPLEX) and the QMIX-based communication methods 64.3% (TeamComm) and 66.5% (TGCNet); RAVENon reaches 87.35%, at least 20.8 points higher, with 2-bit messages; only the single-neighbor ExpoComm is higher. It also learns M fastest of all methods, with a time-averaged win rate of 92.8% against 90.8% for the next best. On MPE it is best on Tag and second on Spread and Crypto, behind the single-neighbor ExpoComm and an attention-based QMIX variant, respectively (Table 15 lists all 14 methods). Over the eight tasks it has the highest mean normalized score (0.98) and the best mean rank (1.75) of the 14 methods, is in the top three on seven tasks, and never falls below a normalized score of 0.92, whereas every other method falls below 0.4 on at least one task. The single-neighbor ExpoComm, which sends 4,096 bits per message, is the closest competitor (0.87). 4.5. Cost Every RAVEN message is 2 bits (Table 3). Over complete training runs on navigation, RAVENon delivers 97.9% fewer bits than NDQ, 79.3–90.0% fewer than CACOM, whose gate suppresses some replies, and 99.8% fewer than ExpoComm. The offline implementation is also lightweight: its execution FLOPs are 82–98% below those of NDQ, CACOM and ExpoComm, its broadcast model has 5,257 parameters, 98% fewer than SLIM’s, and, including its teacher, it trains in 40–55% less time than SLIM on the same server in the ring settings. The online implementation needs 71–74% less training time than CACOM on the same machine and, on SMAC and MPE, needs 24–40% fewer parameters than TeamComm and TGCNet and about 80% fewer parameters and FLOPs than the single-neighbor ExpoComm. 5. Discussion and Limitations RAVEN’s advantage is largest when a few symbols must carry decision-relevant information that the receiver cannot observe—pairs, small rings and broadcast—and where the receiver’s context changes the meaning of that information. Several limitations remain. First, in the six-agent ring CACOM and ExpoComm are ahead of RAVENoff by 0.37 and 0.21 return; removing all messages costs RAVENoff only 0.41 there, so a one-hop ring carries little decision value at this size, and multi-hop relaying is a natural extension. Second, the online implementation learns more slowly early in training on the hardest maps: on 3s5z it reaches 80% win rate after 1.2M steps, later than QPLEX, and on M2 the single-neighbor ExpoComm, with 2,048 times more bits per message, is higher (94.5%). Third, the online implementation has 24–54% more parameters than CACOM on SMAC and MPE (Table 17), so its savings there are in bandwidth rather than network size. Finally, the offline implementation requires a teacher and an enumerable joint action space for the receiver objective; the online implementation removes both requirements. 6. Conclusion RAVEN rests on one principle: a finite-alphabet message should preserve the receiver’s action-value differences in the receiver’s own situation. The offline estimator realizes it exactly and gives the strongest results where a teacher is available; the online estimator carries it, without a teacher, to ten-agent StarCraft teams. In both, two bits per message, interpreted by each receiver with its private information, are enough to outperform or match methods that exchange tens to thousands of bits. Code is available at https://github.com/sswun/RAVEN. References Amoh et al. (2026) Benjamin Amoh, Geoffrey Parker, and Wesley Marrero. 2026. Multi-Agent Decision-Focused Learning via Value-Aware Sequential Communication. arXiv preprint. Version 2. arXiv:2604.08944 https://arxiv.org/abs/2604.08944v2 Arthur and Vassilvitskii (2007) David Arthur and Sergei Vassilvitskii. 2007. k-means++: The Advantages of Careful Seeding. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms. 1027–1035. https://research.google/pubs/k-means-the-advantages-of-careful-seeding/ Bae et al. (2026) Sangjun Bae, Yisak Park, Sanghyeon Lee, and Seungyul Han. 2026. LLM-Guided Communication for Cooperative Multi-Agent Reinforcement Learning. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 306). https://icml.c/virtual/2026/poster/61721 Banerjee et al. (2005) Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. 2005. Clustering with Bregman Divergences. Journal of Machine Learning Research 6, 58 (2005), 1705–1749. https://jmlr.org/papers/v6/banerjee05b.html Bengio et al. (2013) Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint. arXiv:1308.3432 https://arxiv.org/abs/1308.3432 Canesse et al. (2026) Alexi Canesse, Benoît Goupil, Jesse Read, and Sonia Vanier. 2026. Decoupling Communication from Policy: Robust MARL under Bandwidth Constraints. arXiv preprint. Version 1. arXiv:2605.21085 https://arxiv.org/abs/2605.21085v1 Chen et al. (2024b) Jingdi Chen, Tian Lan, and Carlee Joe-Wong. 2024b. RGMComm: Return Gap Minimization via Discrete Communications in Multi-Agent Reinforcement Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17327–17336. doi:10.1609/aaai.v38i16.29680 Chen et al. (2024a) Wentse Chen, Shiyu Huang, and Jeff Schneider. 2024a. Soft-QMIX: Integrating Maximum Entropy For Monotonic Value Function Factorization. arXiv preprint. arXiv:2406.13930 https://arxiv.org/abs/2406.13930 Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1724–1734. doi:10.3115/v1/D14-1179 Das et al. (2019) Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Mike Rabbat, and Joelle Pineau. 2019. TarMAC: Targeted Multi-Agent Communication. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97). 1538–1546. https://proceedings.mlr.press/v97/das19a.html Deng et al. (2026) Zhibo Deng, Feng Liang, Yong Zhang, Xiaoxi Zhang, and Xiping Hu. 2026. Sparse Topology-Aware Pairwise Scoring for Large-Scale Multi-Agent Reinforcement Learning. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 306). https://icml.c/virtual/2026/poster/65907 Du et al. (2026) Wei Du, Benyu Wu, Yuqing Sun, Wei Guo, Yuntao Du, Zhongmin Yan, Guoxian Yu, and Lizhen Cui. 2026. Learning Efficient and Interpretable Multi-Agent Communication. In International Conference on Learning Representations. https://openreview.net/forum?id=a3CUE06G5Y Foerster et al. (2016) Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. 2016. Learning to Communicate with Deep Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, Vol. 29. https://proceedings.neurips.c/paper/2016/hash/c7635bfd99248a2cdef8249ef7bfbef4-Abstract.html Foerster et al. (2018) Jakob N. Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2018. Counterfactual Multi-Agent Policy Gradients. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32. doi:10.1609/aaai.v32i1.11794 Gray and Neuhoff (1998) Robert M. Gray and David L. Neuhoff. 1998. Quantization. IEEE Transactions on Information Theory 44, 6 (1998), 2325–2383. doi:10.1109/18.720541 Guan et al. (2022) Cong Guan, Feng Chen, Lei Yuan, Chenghe Wang, Hao Yin, Zongzhang Zhang, and Yang Yu. 2022. Efficient Multi-agent Communication via Self-supervised Information Aggregation. In Advances in Neural Information Processing Systems, Vol. 35. 1020–1033. doi:10.52202/068431-0075 Hu et al. (2021) Jian Hu, Siyang Jiang, Seth Austin Harding, Haibin Wu, and Shih wei Liao. 2021. Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning. arXiv preprint. arXiv:2102.03479 https://arxiv.org/abs/2102.03479 Jiang and Lu (2018) Jiechuan Jiang and Zongqing Lu. 2018. Learning Attentional Communication for Multi-Agent Cooperation. In Advances in Neural Information Processing Systems, Vol. 31. https://proceedings.neurips.c/paper/2018/hash/6a8018b3a00b69c008601b8becae392b-Abstract.html Kingma and Ba (2015) Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations. https://arxiv.org/abs/1412.6980 Li et al. (2025) Xinran Li, Xiaolu Wang, Chenjia Bai, and Jun Zhang. 2025. Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning. In International Conference on Learning Representations. https://arxiv.org/abs/2502.19717 Li and Zhang (2024) Xinran Li and Jun Zhang. 2024. Context-aware Communication for Multi-agent Reinforcement Learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems. https://arxiv.org/abs/2312.15600 Lloyd (1982) Stuart P. Lloyd. 1982. Least Squares Quantization in PCM. IEEE Transactions on Information Theory 28, 2 (1982), 129–137. doi:10.1109/TIT.1982.1056489 Lowe et al. (2017) Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. In Advances in Neural Information Processing Systems, Vol. 30. https://proceedings.neurips.c/paper/2017/hash/68a9750337a418a86fe06c1991a1d64c-Abstract.html Mostaani et al. (2022) Arsham Mostaani, Thang X. Vu, Symeon Chatzinotas, and Björn Ottersten. 2022. Task-Oriented Data Compression for Multi-Agent Communications Over Bit-Budgeted Channels. IEEE Open Journal of the Communications Society 3 (2022), 1867–1886. doi:10.1109/OJCOMS.2022.3213213 Mostaani et al. (2024) Arsham Mostaani, Thang X. Vu, Symeon Chatzinotas, and Björn Ottersten. 2024. Task-Effective Compression of Observations for the Centralized Control of a Multiagent System Over Bit-Budgeted Channels. IEEE Internet of Things Journal 11, 4 (2024), 6131–6143. doi:10.1109/JIOT.2023.3312553 Oliehoek and Amato (2016) Frans A. Oliehoek and Christopher Amato. 2016. A Concise Introduction to Decentralized POMDPs. Springer International Publishing. doi:10.1007/978-3-319-28929-8 Rashid et al. (2020) Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. 2020. Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, Vol. 33. https://arxiv.org/abs/2006.10800 Rashid et al. (2018) Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. 2018. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). 4295–4304. https://proceedings.mlr.press/v80/rashid18a.html Rusu et al. (2016) Andrei A. Rusu, Sergio Gomez Colmenarejo, Çağlar Gülçehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell. 2016. Policy Distillation. In International Conference on Learning Representations. https://arxiv.org/abs/1511.06295 Samvelyan et al. (2019) Mikayel Samvelyan, Tabish Rashid, Christian Schroeder de Witt, Gregory Farquhar, Nantas Nardelli, Tim G. J. Rudner, Chia-Man Hung, Philip H. S. Torr, Jakob Foerster, and Shimon Whiteson. 2019. The StarCraft Multi-Agent Challenge. In Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems. Extended abstract. https://w.ifaamas.org/Proceedings/aamas2019/pdfs/p2186.pdf Singh et al. (2019) Amanpreet Singh, Tushar Jain, and Sainbayar Sukhbaatar. 2019. Learning When to Communicate at Scale in Multiagent Cooperative and Competitive Tasks. In International Conference on Learning Representations. https://openreview.net/forum?id=rye7knCqK7 Sukhbaatar et al. (2016) Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. 2016. Learning Multiagent Communication with Backpropagation. In Advances in Neural Information Processing Systems, Vol. 29. https://proceedings.neurips.c/paper/2016/hash/55b1927fdafef39c48e5b73b5d61ea60-Abstract.html Sunehag et al. (2018) Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel. 2018. Value-Decomposition Networks for Cooperative Multi-Agent Learning Based on Team Reward. In Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 2085–2087. Extended abstract. https://w.ifaamas.org/Proceedings/aamas2018/pdfs/p2085.pdf Tucker et al. (2022) Mycal Tucker, Roger P. Levy, Julie A. Shah, and Noga Zaslavsky. 2022. Trading off Utility, Informativeness, and Complexity in Emergent Communication. In Advances in Neural Information Processing Systems, Vol. 35. 22214–22228. doi:10.52202/068431-1614 Tung et al. (2021) Tze-Yang Tung, Szymon Kobus, Joan Pujol Roig, and Deniz Gündüz. 2021. Effective Communications: A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning Over Noisy Channels. IEEE Journal on Selected Areas in Communications 39, 8 (2021), 2590–2603. doi:10.1109/JSAC.2021.3087248 van den Oord et al. (2017) Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. In Advances in Neural Information Processing Systems, Vol. 30. https://proceedings.neurips.c/paper/2017/hash/7a98af17e63a0ac09ce2e96d03992fbc-Abstract.html van Hasselt et al. (2016) Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep Reinforcement Learning with Double Q-Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30. doi:10.1609/aaai.v30i1.10295 Vanneste et al. (2025) Simon Vanneste, Astrid Vanneste, Kevin Mets, Tom De Schepper, Ali Anwar, Siegfried Mercelis, and Peter Hellinckx. 2025. Learning to communicate using a communication critic and counterfactual reasoning. Neural Computing and Applications 37 (2025), 25645–25662. doi:10.1007/s00521-024-10598-0 Wang et al. (2021b) Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. 2021b. QPLEX: Duplex Dueling Multi-Agent Q-Learning. In International Conference on Learning Representations. https://arxiv.org/abs/2008.01062 Wang et al. (2020a) Rundong Wang, Xu He, Runsheng Yu, Wei Qiu, Bo An, and Zinovi Rabinovich. 2020a. Learning Efficient Multi-agent Communication: An Information Bottleneck Approach. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). 9908–9918. https://proceedings.mlr.press/v119/wang20i.html Wang et al. (2021a) Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson, and Chongjie Zhang. 2021a. RODE: Learning Roles to Decompose Multi-Agent Tasks. In International Conference on Learning Representations. https://arxiv.org/abs/2010.01523 Wang et al. (2020b) Tonghan Wang, Jianhao Wang, Chongyi Zheng, and Chongjie Zhang. 2020b. Learning Nearly Decomposable Value Functions Via Communication Minimization. In International Conference on Learning Representations. https://arxiv.org/abs/1910.05366 Ward (1963) Joe H. Ward, Jr. 1963. Hierarchical Grouping to Optimize an Objective Function. J. Amer. Statist. Assoc. 58, 301 (1963), 236–244. doi:10.1080/01621459.1963.10500845 Wyner and Ziv (1976) Aaron D. Wyner and Jacob Ziv. 1976. The Rate-Distortion Function for Source Coding with Side Information at the Decoder. IEEE Transactions on Information Theory 22, 1 (1976), 1–10. doi:10.1109/TIT.1976.1055508 Xie et al. (2026) Zaipeng Xie, Sitong Shen, Yaowu Wang, Chentai Qiao, Bin Tang, and Wenzhan Song. 2026. ROCO: Role-Oriented Communication for Efficient Multi-Agent Reinforcement Learning. Expert Systems with Applications 297 (2026), 129421. doi:10.1016/j.eswa.2025.129421 Yang et al. (2024) Ming Yang, Kaiyan Zhao, Yiming Wang, Renzhi Dong, Yali Du, Furui Liu, Mingliang Zhou, and Leong Hou U. 2024. Team-wise Effective Communication in Multi-Agent Reinforcement Learning. Autonomous Agents and Multi-Agent Systems 38, 2 (2024), 36. doi:10.1007/s10458-024-09665-6 Yu et al. (2022) Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games. In Advances in Neural Information Processing Systems, Vol. 35. 24611–24624. doi:10.52202/068431-1787 Yuan et al. (2022) Lei Yuan, Jianhao Wang, Fuxiang Zhang, Chenghe Wang, Zongzhang Zhang, Yang Yu, and Chongjie Zhang. 2022. Multi-Agent Incentive Communication via Decentralized Teammate Modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 9466–9474. doi:10.1609/aaai.v36i9.21179 Zhang et al. (2020) Sai Qian Zhang, Qi Zhang, and Jieyu Lin. 2020. Succinct and Robust Multi-Agent Communication With Temporal Message Control. In Advances in Neural Information Processing Systems, Vol. 33. 17271–17282. https://papers.nips.c/paper/2020/hash/c82b013313066e0702d58dc70db033ca-Abstract.html Zhang et al. (2025) Zhuohui Zhang, Bin He, Bin Cheng, and Gang Li. 2025. Bridging Training and Execution via Dynamic Directed Graph-Based Communication in Cooperative Multi-Agent Systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 23395–23403. doi:10.1609/aaai.v39i22.34507 Appendix overview The appendices list the notation (Appendix A), prove the statements of Section 3.4 (Appendix B), give the implementation of both estimators (Appendices C and D), describe tasks, baselines and the statistical protocol (Appendix E), and report the complete results behind every number in the paper (Appendices F–I). All results are computed from the final checkpoints of the runs described above. Code is available at https://github.com/sswun/RAVEN; the result files and the scripts that produce every table and figure from them will be released there as well. Appendix A Notation Table 4 lists the symbols used throughout the paper. Table 4. Notation. Symbol Meaning N, K=4K=4 number of agents; alphabet size (2 bits per message) XjtX_j^t, MjtM_j^t permitted local input and symbol of sender j at step t ZitZ_i^t, itM_i^t local information and mailbox of receiver i iA_i, P legal actions of receiver i; centering matrix I−⊤/|i|I-11 /|A_i| ‖v‖2\|v\|_A^2 ||−1∑ava2|A|^-1 _av_a^2 Q Q, ξn _n, ρ teacher, its input at the arrival time, partner-action reference unau_na, YnaY_na, sQs_Q receiver action value, centered and scaled target, global scale D construction data (rows n) T=τ(X)T=τ(X), nTn_T sender type and number of types (12) V, V¯∈ℝF V ^F, C, nCn_C receiver features, standardized features, condition, number of conditions (12) ζℓ _ , Ω action-gap sensitivity of feature ℓ ; condition metric NrcaN_rca, SrcaS_rca, μ~rca μ_rca, αrca _rca cell mass, target sum, shrunk mean, empirical mass e, e e, D^(e) D(e) codebook (types → symbols), selected codebook, empirical distortion sθs_θ, pθp_θ, πϕ _φ frozen sender, its symbol distribution, offline receiver policy hith_i^t, qiloc,tq_i loc,t recurrent state and local utilities (online) ljtl_j^t, m~jt m_j^t, Φ , gjtg_j^t encoder logits, straight-through one-hot, 4×644× 64 symbol embedding, decoded symbol i⋆N_i , βij _ij two selected senders and their attention weights cibc_i^b, Δib _i^b, QibQ_i^b context, correction and utilities of branch b∈dep,refb∈\dep,ref\ fψf_ψ, VibV_i^b, V¯ib V_i^b QMIX mixer, mixed receiver profile, centered profile ℬB, σi2 _i^2 probe set, scale of ℒCVL_CV ℛC,ℛZ,CR_C,R_Z,G_C reconstruction risks and the gap between them ℰabsE_ abs, (e)D(e) abstraction and compression errors Appendix B Proofs Throughout, the legal set is fixed with d=||d=|A| actions, ‖v‖2=d−1∑ava2\|v\|_A^2=d^-1 _av_a^2, all random quantities are square integrable, and M=e(T)M=e(T) with T=τ(X)T=τ(X). B.1. Centering identity (Eq. (2)) Let w=u−vw=u-v and w¯=d−1∑awa w=d^-1 _aw_a. Then ‖Pw‖22=∑a(wa−w¯)2=12d∑a,b(wa−wb)2=1d∑a<b(wa−wb)2\|Pw\|_2^2= _a(w_a- w)^2= 12d _a,b(w_a-w_b)^2= 1d _a<b(w_a-w_b)^2, and wa−wb=(ua−ub)−(va−vb)w_a-w_b=(u_a-u_b)-(v_a-v_b). B.2. Merge cost and exactness of the enumeration Fix a cell (c,a)(c,a) and two groups of types with masses v1,v2v_1,v_2 and weighted means μ1,μ2 _1, _2. The weighted sum of squared deviations of the union about its mean equals the sum of the two within-group sums plus v1v2v1+v2(μ1−μ2)2 v_1v_2v_1+v_2( _1- _2)^2 (the Ward identity). Summing over cells gives the merge cost Δ of Section 3.2. Consequently (i) Δ≥0 ≥ 0, so splitting a group never increases D D, and (i) Δ=0 =0 if and only if the two groups have equal means in every cell where both have mass; in particular, groups that never have mass in the same condition merge for free. Let ℰ≤KE_≤ K be the codebooks with at most K nonempty groups over nT≥Kn_T≥ K types and ℰ=KE_=K those with exactly K. By (i), every e∈ℰ≤Ke _≤ K can be refined to some e′∈ℰ=Ke _=K with D^(e′)≤D^(e) D(e )≤ D(e), so minℰ=KD^=minℰ≤KD _E_=K D= _E_≤ K D. Symbols are exchangeable, so it suffices to enumerate unlabeled partitions; their number is the Stirling number (17) S(12,4)=412−4⋅312+6⋅212−424=611,501.S(12,4)= 4^12-4· 3^12+6· 2^12-424=611,501. Ties are broken by a fixed order of the partitions. For several recipients, D D is the equally weighted sum of the recipients’ tables under one shared map, and the same argument applies. B.3. Proof of Proposition 3.1 First equality. Because C=c(V)C=c(V) and V⊆ZV Z, the σ-algebra of (M,C)(M,C) is contained in that of (M,Z)(M,Z). Write Y−[Y∣M,C]=A+BY-E[Y M,C]=A+B with A=Y−[Y∣M,Z]A=Y-E[Y M,Z] and B=[Y∣M,Z]−[Y∣M,C]B=E[Y M,Z]-E[Y M,C]. B is (M,Z)(M,Z)-measurable and [A∣M,Z]=0E[A M,Z]=0, so ⟨A,B⟩=0E A,B _A=0 and ℛC=ℛZ+CR_C=R_Z+G_C. Second equality. Because M=e(T)M=e(T), the σ-algebra of (M,C)(M,C) is contained in that of (T,C)(T,C). With μ=[Y∣T,C]μ=E[Y T,C], write Y−[Y∣M,C]=(Y−μ)+(μ−[Y∣M,C])Y-E[Y M,C]=(Y-μ)+(μ-E[Y M,C]); the second term is (T,C)(T,C)-measurable and [Y−μ∣T,C]=0E[Y-μ T,C]=0, so ℛC=ℰabs+(e)R_C=E_ abs+D(e). ℰabsE_ abs does not depend on e. Selection bound. Combining both equalities, ℛZ(e)=ℰabs+(e)−C(e)R_Z(e)=E_ abs+D(e)-G_C(e) for every e. For any codebook e′e , ℛZ(e^)−ℛZ(e′) _Z( e)-R_Z(e ) =(e^)−(e′)+C(e′)−C(e^) =D( e)-D(e )+G_C(e )-G_C( e) ≤[D^(e^)+ϵD]−[D^(e′)−ϵD]+osceC(e) ≤ [ D( e)+ _D ]- [ D(e )- _D ]+osc_eG_C(e) ≤ϵopt+2ϵD+osceC(e), ≤ _ opt+2 _D+osc_eG_C(e), using D^(e^)≤infeD^(e)+ϵopt≤D^(e′)+ϵopt D( e)≤ _e D(e)+ _ opt≤ D(e )+ _ opt. Taking the infimum over e′e gives Eq. (15). The empirical distortion D D weights each row’s legal actions by 1/|n|1/|A_n|, which is the norm ∥⋅∥\|·\|_A; ϵD _D therefore covers finite-sample and shrinkage effects only. □ B.4. Proof of Proposition 3.2 Let u^=[u∣M,Z] u=E[u M,Z] (a vector over actions), let the Bayes decoder choose a⋆=argmaxau^a = _a u_a, and let a∘=argmaxauaa = _au_a. Then (18) [ua∘−ua⋆]= [u_a -u_a ]= [ua∘−u^a∘]+[u^a∘−u^a⋆] [u_a - u_a ]+E[ u_a - u_a ] +[u^a⋆−ua⋆]. +E[ u_a -u_a ]. The middle term is non-positive by the definition of a⋆a , and the last is zero because a⋆a is (M,Z)(M,Z)-measurable. With δ=u−u^δ=u- u and δ¯=d−1∑aδa δ=d^-1 _a _a, δa∘=(Pδ)a∘+δ¯ _a =(Pδ)_a + δ and δ¯=[δ¯∣M,Z]=0E δ=E\,E[ δ M,Z]=0. Hence (19) [ua∘−ua⋆] [u_a -u_a ] ≤maxa|(Pδ)a|≤‖Pδ‖2 _a|(Pδ)_a| \|Pδ\|_2 =d‖Pδ‖≤d‖Pδ‖2. = d\,E\|Pδ\|_A≤ d\,E\|Pδ\|_A^2. For a fixed legal set, centering commutes with conditional expectation, Pu=sQYPu=s_QY and Pu^=sQ[Y∣M,Z]P u=s_QE[Y M,Z], so ‖Pδ‖2=sQ2ℛZ(e^)E\|Pδ\|_A^2=s_Q^2\,R_Z( e) and the teacher regret of the Bayes decoder is at most sQdℛZ(e^)s_Q d\,R_Z( e). For the true payoffs, maxaUa−maxaua≤ϵQE _aU_a-E _au_a≤ _Q and ua⋆−Ua⋆≤ϵQEu_a -EU_a ≤ _Q, so the Bayes decoder loses at most 2ϵQ+sQdℛZ(e^)2 _Q+s_Q d\,R_Z( e) in true payoff. The receiver loses at most ϵrec _ rec more under the ideal symbol. The deployed symbol sθ(X)s_θ(X) differs from e^(T) e(T) with probability q, and on that event the payoff changes by at most 2B2B; this gives Eq. (16). Finally, if argmaxmpθ(m∣X)≠e^(T) _mp_θ(m X)≠ e(T) then pθ(e^(T)∣X)≤12p_θ( e(T) X)≤ 12, i.e. −logpθ(e^(T)∣X)≥log2- p_θ( e(T) X)≥ 2, and Markov’s inequality gives q≤[−logpθ(e^(T)∣X)]/log2q [- p_θ( e(T) X)]/ 2. □ B.5. Online estimator For fixed state s and teammate utilities, the QMIX mixer is non-decreasing in each agent’s utility, so Qib(a)≥Qib(a′)Q_i^b(a)≥ Q_i^b(a ) implies Vib(a)≥Vib(a′)V_i^b(a)≥ V_i^b(a ): mixing preserves the ranking of a receiver’s actions. If the mixer has a locally Lipschitz gradient, a first-order expansion around the reference utility of each action gives Videp(a)−Viref(a)=ηiaδQi(a)+O(δQi(a)2)V_i^dep(a)-V_i^ref(a)= _ia\,δ Q_i(a)+O(δ Q_i(a)^2) with slopes ηia≥0 _ia≥ 0, and centering yields the relation stated in Section 3.4. Each probe evaluates the mixer once per legal action and branch, i.e. 2∑i|i|2 _i|A_i| times, instead of enumerating joint actions. Appendix C Offline RAVEN: Implementation Teachers. Pair navigation (N1-C2, MPE simple_reference; 11 local observation features, five movement actions, 25-step episodes) uses a Double-DQN teacher with ReLU layers 22→128→128→2522→ 128→ 128→ 25 over joint actions. Ring navigation uses 11N→128→128→5N11N→ 128→ 128→ 5N with centralized-input additive heads Q^(x,)=N−1∑kuk(x,ak) Q(x,a)=N^-1 _ku_k(x,a_k) (Sunehag et al., 2018); the speaker–listener teacher is 11→128→128→511→ 128→ 128→ 5. Teachers are trained for 1.2M environment steps with Adam (Kingma and Ba, 2015) at 3×10−43× 10^-4, discount 0.95, a replay capacity of 200,000 and batches of 512; after 5,000 collection steps they update every 16 steps and synchronize targets every 1,000 updates, and exploration decays from 1 to 0.05 over 600,000 steps. Pair-navigation and speaker–listener teachers are selected on independent 200-episode sets among the checkpoints at 0, 0.1M, 0.3M, 0.6M and 1.2M steps; ring teachers use the final checkpoint. Construction. The construction data consist of the first 10,000 transitions and 10,000 transitions sampled from the final replay, without terminal transitions; normalization, types, conditions and codebooks are all computed on these rows. The partner reference is uniform over the partner’s actions in navigation; in speaker–listener tasks the speaker’s action is held fixed. The scale sQs_Q is the global standard deviation of teacher values (floored at 10−610^-6); rows with a single legal action have a zero target. Types come from k-means on standardized sender inputs (speaker–listener tasks use the three target colors). Conditions use nC=12n_C=12, k-means++ seeding (Arthur and Vassilvitskii, 2007), 40 Lloyd iterations (Lloyd, 1982) and the metric of Eq. (4) with derivatives taken with the partner reference and all other teacher inputs held fixed; the construction is rejected if the sensitivity trace ∑ℓζℓ _ _ is below 10−1610^-16. The pseudo-count is λra=32Nra/nr _ra=32N_ra/n_r; unsupported type–action pairs are omitted and zero-mass cells contribute nothing. The codebook is the exact minimizer of Eq. (6) (Appendix B); in broadcast tasks one map serves all listeners, and each ring sender has one recipient. Sender and receivers. Pair-navigation senders and receivers are Tanh networks 11→64→64→411→ 64→ 64→ 4 and 15→32→515→ 32→ 5 (the receiver input is its observation and the one-hot symbol). The sender is fitted by 2,048 cross-entropy updates and then frozen (parameters, normalization statistics and all features that produce symbols); receivers are trained for 8,192 updates on Eq. (7) with batches of 512, Adam at 0.003 and gradient-norm clipping at 5. At execution the sender emits the argmax symbol, which the receiver uses at the next step, and the receiver acts greedily. In the ablation, the learned four-symbol channel and the no-message policy receive 10,240 policy updates, the same total number of updates as RAVEN; all variants of a seed share its teacher and construction data. Appendix D Online RAVEN: Implementation Networks. Each agent’s observation, previous action (one-hot) and identity pass through a 64-unit layer and a 64-unit GRU (Cho et al., 2014), followed by a linear local utility head; parameters are shared across agents. The encoder is linear (64→464→ 4); transmission uses the hard argmax and the straight-through estimator of Eq. (8) with temperature one and no Gumbel noise; Φ is a shared 4×644× 64 embedding. Queries WQhiW_Qh_i and keys WKLN(gj)W_KLN(g_j) have 32 dimensions and are compared by the dot product divided by 32 32; the self edge is excluded, the two highest-scoring senders are kept (min2,N−1 \2,N-1\), and the softmax runs over the kept senders only. Values have 64 dimensions. Senders that sent the same symbol have the same key and the same deployed value, so the deployed utilities do not depend on how equal scores are ordered. The residual head maps [hi,cib,hi⊙cib][h_i,c_i^b,h_i c_i^b] through 192→64→||max192→ 64→|A|_ with ReLU; its last layer is zero-initialized, illegal actions are masked after the head, and an empty mailbox gives a zero correction. The mixer is a state-conditioned QMIX network with embedding width 32 and hypernetwork width 64 (Rashid et al., 2018). Training. After each collected episode, one update is made on 32 episodes sampled from a 5,000-episode replay buffer, starting as soon as 32 episodes are stored; the TD and alignment losses are active from the first update. Complete episodes are unrolled to recompute recurrent states. Rewards are standardized with running statistics of the valid steps of the sampled batches (updated before the targets are computed). TD targets use Double-Q learning (van Hasselt et al., 2016): online local utilities select greedy legal next actions and target networks evaluate them; targets are synchronized every 200 updates. Exploration decays from 1 to 0.05 over 50,000 environment steps, the discount is 0.99, Adam learning rates are 5×10−45× 10^-4 for the recurrent network, action head and mixer and 3×10−43× 10^-4 for codec and receiver, and gradients are clipped at norm 10. Each update draws at most eight probe times from the batch, excluding padding and receivers with at most one legal action; teammate utilities are fixed at uk=Qkref(akreplay)u_k=Q_k^ref(a_k replay). The same hyperparameters are used on every task. Budgets are 2.05M environment steps (1.2M on navigation), with 100 monitoring episodes every 50,000 steps and 1,000 final evaluation episodes. The same-backbone QMIX control uses the identical backbone, reward standardization, optimizer, replay, exploration schedule, budget and per-seed initialization, without codec, receiver, reference branch and ℒCVL_CV. Appendix E Tasks, Baselines and Protocol Table 5. Tasks. “Edges” are the directed message edges that every method may use; all methods use the same one-step delay and the same execution-time information. Task Agents Description Edges Metric RAVEN N1-C2 reference navigation 2 MPE simple_reference: each agent knows only its partner’s goal color; 25 steps pair, both ways team return offline, online N1-C3/C4/C6 ring navigation 3/4/6 each agent’s goal is known only to its predecessor predecessor → successor team return offline, online N2-L1/L2/L4/L6 broadcast 1+L1+L MPE simple_speaker_listener: an immobile speaker sees the target; listeners see landmarks speaker → listeners listener return offline, online Predator–prey P1 3 5×55× 5 grid, vision radius 1; all agents must reach the same target within 20 steps all-to-all capture success online SMAC 3m, 8m, M, M2, 3s5z 3/8/10/10/8 StarCraft I micromanagement (Samvelyan et al., 2019); M2 super hard, 3s5z hard all-to-all, receiver keeps 2 win rate online MPE Spread, Tag, Crypto 3/4/3 coverage, pursuit and encrypted signaling; in Crypto the eavesdropper Eve is controlled like the other agents and her reward is included in the summed team return; 25 steps all-to-all team return online Navigation baselines. SLIM (Canesse et al., 2026), NDQ (Wang et al., 2020b), CACOM (Li and Zhang, 2024) and ExpoComm (Li et al., 2025) are run from their authors’ official code with their own architectures, losses, optimizers and default hyperparameters—no task-specific tuning is performed on our tasks; only the message edges, the delay and the budget are adapted to the common protocol. MACC (Vanneste et al., 2025) is implemented from its paper. Every model is trained for 1.2M environment steps. SLIM and MACC use the same four-symbol channel as RAVEN; NDQ sends three float32 values (96 bits), and ExpoComm 64 float32 values (2,048 bits). CACOM’s two-round exchange is computed inside step t, as in the authors’ implementation—a 4-value request (8 bits) and a gated 12-value reply at 2 bits per value (24 bits)—and the reply is delivered at t+1t+1, so every message respects the same one-step delay. In the broadcast settings, RAVEN (offline) and SLIM deploy the one-listener model unchanged; all other methods are retrained at every size, and CACOM and ExpoComm are additionally transferred zero-shot (Table 10). SMAC and MPE reference methods. The 13 reference methods are QMIX (Rashid et al., 2018), QMIX with larger networks and QMIX with attention (Hu et al., 2021), QPLEX (Wang et al., 2021b), OW-QMIX and CW-QMIX (Rashid et al., 2020), Soft-QMIX (Chen et al., 2024a), RODE (Wang et al., 2021a), ROCO (Xie et al., 2026), and the communication methods TeamComm (Yang et al., 2024), TGCNet (Zhang et al., 2025), CACOM (Li and Zhang, 2024) and a single-neighbor ExpoComm (Li et al., 2025) on a QMIX backbone. All of them are run by us from the authors’ official open-source implementations under the same 2.05M-step budget; no numbers are taken from prior publications. Their final value is the last evaluation before 2.05M steps, averaged over their runs. Statistics. The training seed is the statistical unit. Navigation methods and controls use five seeds, the mechanism ablation ten, and the rate study eight; on SMAC, online RAVEN uses four seeds and on MPE five. Every final model is evaluated on 1,000 held-out episodes whose environment instances are disjoint from training and teacher selection, always at the final checkpoint. We report means with 95% t-intervals over seeds and paired two-sided t-tests on seed-level means, with Holm correction inside each family of comparisons (for example, offline RAVEN against NDQ, CACOM and ExpoComm over the ring settings, or the online implementation against one external method over the eight settings). The evaluation episodes reduce the noise of each model and are never treated as independent samples. The ring comparison with SLIM is repeated on fresh seeds that took no part in method development. Appendix F Navigation: Complete Results Table 6 extends Table 1 with MACC, and Figure 6 shows every seed. Table 7 lists all paired comparisons, including online against offline RAVEN. Figure 7 and Table 8 show the learning curves of the methods that record intermediate evaluations: RAVENon has the highest time-averaged return in four of the six settings, and RAVENoff, which is not trained by trial and error, lies above every curve except on N1-C6. Table 9 compares the development and fresh seeds of the ring study, Table 10 and Figure 8 the zero-shot broadcast transfer, and Figure 9 the margin over the strongest external method. Table 6. Navigation: team return at the fixed final checkpoint (mean±s.d._ .d. over 5 training seeds, 1,000 held-out episodes per model; higher is better). Best per row in bold, second best underlined. ‡ RAVEN (offline) and SLIM deploy the L1 model zero-shot on L2/L4/L6; all other methods are retrained at every size. MACC is implemented from its paper; under the enforced one-step delay its communication critic assigns credit to a message within the step it is sent, this attribution no longer matches the action the message influenced, and training diverges, which is why Table 1 omits it. Setting RAVEN (offline) RAVEN (online) SLIM NDQ CACOM ExpoComm MACC N1-C2 −-9.59±0.60 −-12.35±1.68 −-18.81±0.76 −-21.34±1.87 −-17.25±0.97 −-14.98±1.44 −-48.12±7.75 N1-C3 −-11.47±0.39 −-15.21±2.78 −-18.79±0.71 −-19.98±0.95 −-17.84±0.15 −-17.35±0.92 −-43.44±15.20 N1-C4 −-13.74±0.64 −-17.86±0.30 −-20.02±0.85 −-20.06±0.67 −-17.89±0.44 −-17.92±0.30 −-49.47±6.68 N1-C6 −-18.15±0.12 −-17.96±0.23 −-20.13±0.72 −-24.32±3.55 −-17.78±0.28 −-17.94±0.26 −-37.97±6.34 N2-L1 −-7.70±0.17 −-7.86±0.24 −-11.89±3.01 −-14.19±3.02 −-12.64±4.32 −-15.42±0.28 −-129.72±58.17 N2-L2 −-7.86±0.14‡ −-9.13±1.87 −-12.10±2.92‡ −-15.08±2.78 −-9.65±3.58 −-14.16±2.39 −-115.26±71.17 N2-L4 −-7.92±0.21‡ −-9.82±2.31 −-12.19±2.83‡ −-20.52±1.21 −-10.68±3.61 −-13.66±3.09 −-138.76±61.12 N2-L6 −-7.82±0.09‡ −-9.23±0.53 −-12.26±2.99‡ −-24.60±5.51 −-13.81±5.28 −-13.98±2.40 −-165.82±6.16 Table 7. Paired seed-level comparisons on navigation: mean difference in team return, Holm-adjusted significance within each family of comparisons (∗p<0.001^***p<0.001, p∗∗<0.01^**p<0.01, ∗p<0.05^*p<0.05, †p<0.1 p<0.1) and the number of seeds with positive/negative difference. C3/C4/C6 comparisons with SLIM use fresh seeds that never took part in method development; L2/L4/L6 comparisons with SLIM compare two zero-shot transfers. RAVEN (offline) minus … Setting SLIM NDQ CACOM ExpoComm MACC N1-C2 ++9.22∗ (5/0) ++11.75∗ (5/0) ++7.66∗ (5/0) ++5.39∗ (5/0) ++38.54∗ (5/0) N1-C3 ++7.32∗ (5/0) ++8.51∗ (5/0) ++6.37∗ (5/0) ++5.88∗ (5/0) ++31.97∗ (5/0) N1-C4 ++6.28∗ (5/0) ++6.32∗ (5/0) ++4.15∗ (5/0) ++4.18∗ (5/0) ++35.73∗ (5/0) N1-C6 ++1.97∗ (5/0) ++6.17† (5/0) −-0.38† (1/4) −-0.21 (1/4) ++19.82∗ (5/0) N2-L1 ++4.19† (4/1) ++6.49∗ (5/0) ++4.94† (4/1) ++7.72∗ (5/0) ++122.02∗ (5/0) N2-L2 ++4.24† (4/1) ++7.22∗ (5/0) ++1.79 (2/3) ++6.31∗ (5/0) ++107.40∗ (5/0) N2-L4 ++4.27† (4/1) ++12.60∗ (5/0) ++2.77 (5/0) ++5.74† (5/0) ++130.84∗ (5/0) N2-L6 ++4.44† (4/1) ++16.78∗ (5/0) ++5.99 (4/1) ++6.16∗ (5/0) ++158.00∗ (5/0) RAVEN (online) minus … Setting SLIM NDQ CACOM ExpoComm MACC RAVEN (offline) N1-C2 – ++8.99∗ (5/0) ++4.90∗ (5/0) ++2.63 (4/1) ++35.77∗ (5/0) −-2.76 (0/5) N1-C3 – ++4.77∗ (5/0) ++2.63 (4/1) ++2.14 (4/1) ++28.23∗ (5/0) −-3.74 (1/4) N1-C4 – ++2.20∗ (5/0) ++0.02 (2/3) ++0.05 (4/1) ++31.61∗ (5/0) −-4.12∗ (0/5) N1-C6 – ++6.36∗ (5/0) −-0.19 (2/3) −-0.02 (1/4) ++20.01∗ (5/0) ++0.19 (5/0) N2-L1 – ++6.32∗ (5/0) ++4.78 (5/0) ++7.56∗ (5/0) ++121.85∗ (5/0) −-0.16 (2/3) N2-L2 – ++5.95∗ (5/0) ++0.52 (2/3) ++5.03 (5/0) ++106.13∗ (5/0) −-1.27 (0/5) N2-L4 – ++10.70∗ (5/0) ++0.86 (3/2) ++3.84 (4/1) ++128.94∗ (5/0) −-1.90 (0/5) N2-L6 – ++15.37∗ (5/0) ++4.57 (3/2) ++4.75† (5/0) ++156.59∗ (5/0) −-1.41∗ (0/5) Figure 6. Final team return of every training seed (small dots), the seed mean (large marker) and its 95% interval (bar) in the eight navigation settings. MACC is omitted for scale. Figure 7. Navigation learning curves (mean ± s.d. over 5 seeds). Horizontal lines are final values of methods without recorded curves; the y-axis is clipped at −30-30. Table 8. Sample efficiency on navigation: time-averaged team return over 0.05–1.2M training steps (area under the mean learning curve divided by its length; higher is better). Only methods that record intermediate evaluations are listed. Setting RAVEN (online) CACOM ExpoComm MACC N1-C3 −-17.03 −-18.19 −-18.10 −-46.92 N1-C4 −-18.12 −-18.47 −-18.29 −-47.44 N1-C6 −-18.32 −-18.87 −-18.26 −-38.22 N2-L2 −-12.54 −-11.95 −-15.35 −-115.92 N2-L4 −-12.48 −-13.01 −-15.68 −-135.90 N2-L6 −-13.69 −-15.88 −-16.18 −-148.49 Table 9. Ring navigation, RAVEN (offline) minus SLIM on the development seeds and on fresh confirmation seeds, and the return each method loses when its incoming messages are removed at test time (message dependence, fresh seeds). development seeds fresh seeds gain from messages Setting Δ seeds >0>0 Δ seeds >0>0 RAVEN SLIM N1-C3 ++7.04 5/5 ++7.32 5/5 ++6.44 ++1.17 N1-C4 ++5.52 5/5 ++6.28 5/5 ++4.64 ++0.49 N1-C6 ++2.49 5/5 ++1.97 5/5 ++0.41 ++0.28 Table 10. Zero-shot scale transfer on the broadcast family: a model trained with one listener is deployed unchanged to 2/4/6 listeners (“zero-shot”) and compared with the same method retrained at each size. RAVEN (offline) SLIM CACOM ExpoComm Setting zero-shot zero-shot zero-shot retrained zero-shot retrained N2-L2 −-7.86 −-12.10 −-12.96 −-9.65 −-15.78 −-14.16 N2-L4 −-7.92 −-12.19 −-12.94 −-10.68 −-15.75 −-13.66 N2-L6 −-7.82 −-12.26 −-12.95 −-13.81 −-15.78 −-13.98 Figure 8. Zero-shot transfer from one listener (filled markers) against retraining at every size (hollow markers); mean and 95% interval over 5 seeds. Figure 9. Margin of each RAVEN implementation over the strongest external method of each setting (named under the axis). Appendix G Mechanism: Complete Results Table 11 gives all ablation variants with K=4K=4 and K=2K=2. Table 12 reports the agreement between the diagnostics of Proposition 3.1 and return for all four methods and for the three condition variants alone. Table 13 and Figure 10 report the frozen offline models under reduced sending rates and message loss: at one fresh symbol every four steps, RAVEN still beats the every-step learned channel by +5.26+5.26 and the unweighted condition at the same rate by +4.45+4.45 (8/8 seeds each, Holm-significant). Figure 11 shows the message dependence and the symbol entropy (1.94 bits on average of 2 with K=4K=4; 0.93 of 1 with K=2K=2). Table 11. Mechanism ablation of RAVEN (offline) on N1-C2 (10 independently trained seeds, each with its own teacher; all variants share the teacher and construction data of their seed and match capacity and number of updates). Δ : full method minus variant (paired, 95% CI). p: Holm-adjusted within the pre-registered K=4 family (the sender-freezing comparison belongs to the candidate-internal family). K=2 comparisons are descriptive (∘: computed from the per-seed records, not part of a registered family). K = 4 (2 bits) K = 2 (1 bit) Variant changes return Δ seeds ++/−- p (Holm) return Δ RAVEN (offline), full – −-9.45±0.46 – – – −-14.78 – local-residual target + geometric condition target and condition −-10.75±1.72 ++1.30 [0.10, 2.50] 7/2 0.146 −-15.31 ++0.53 geometric receiver condition condition −-13.54±1.79 ++4.09 [2.85, 5.32] 10/0 2.6×10−42.6× 10^-4 −-16.12 ++1.33 plain learnable 4-symbol channel (no codebook) codebook construction −-14.49±0.65 ++5.04 [4.62, 5.46] 10/0 9.7×10−99.7× 10^-9 −-16.80 ++2.02 sender unfrozen after distillation sender freezing −-14.57±0.99 ++5.12 [4.53, 5.72] 10/0 3.5×10−83.5× 10^-8 −-17.37 ++2.59∘ joint-Q sensitivity condition condition −-15.21±1.46 ++5.76 [4.76, 6.77] 10/0 4.8×10−64.8× 10^-6 −-17.79 ++3.01 full local observation condition condition −-16.99±0.39 ++7.54 [7.18, 7.91] 10/0 8.1×10−118.1× 10^-11 −-18.69 ++3.91 no receiver condition (marginal average) condition −-17.63±0.41 ++8.18 [7.76, 8.60] 10/0 1.3×10−101.3× 10^-10 −-18.64 ++3.86 no messages (retrained) communication −-19.27±0.19 ++9.82 [9.44, 10.21] 10/0 1.4×10−111.4× 10^-11 −-19.27 ++4.49 Table 12. Does the finite objective predict task return? For each seed, diagnostics are computed on held-out samples for RAVEN (offline) and three condition variants (“all four”) or for the three variants only; entries are the mean over 10 seeds of pairwise direction concordance / Spearman ρ between lower diagnostic and higher return. all four methods controls only Diagnostic K=4 K=2 K=4 K=2 total risk R 0.833 / 0.76 0.800 / 0.72 0.667 / 0.40 0.633 / 0.35 abstraction term A 0.850 / 0.78 0.817 / 0.72 0.700 / 0.45 0.667 / 0.35 compression term D 0.800 / 0.72 0.617 / 0.26 0.867 / 0.80 0.833 / 0.70 teacher-action regret 0.783 / 0.72 0.767 / 0.66 0.767 / 0.65 0.767 / 0.60 Table 13. Communication rate and loss on N1-C2 (8 training seeds, frozen models). The receiver keeps the last symbol that arrived; “every k” sends a fresh symbol every k steps, “once” only at the first step, and “loss” drops each message independently. Entries are mean team returns. Method every step every 2 every 4 once 10% loss 30% loss bytes / episode 50 26 14 2 50 50 RAVEN (offline), full −-9.78 −-9.78 −-9.79 −-9.84 −-9.80 −-9.88 local-residual target + geometric condition −-11.60 −-11.60 −-11.61 −-11.71 −-11.62 −-11.68 geometric receiver condition −-14.20 −-14.22 −-14.24 −-14.37 −-14.22 −-14.25 plain learnable 4-symbol channel (no codebook) −-15.05 −-15.10 −-15.15 −-15.73 −-15.07 −-15.10 Figure 10. Return under reduced sending rates and message loss (N1-C2, 8 seeds). Numbers under the axis give the serialized payload per episode. Figure 11. Left: return lost when incoming messages are removed at test time (ring navigation, fresh seeds). Right: empirical symbol entropy of offline RAVEN on N1-C2 for each of 10 seeds; dashed lines mark the maximum. Appendix H Online RAVEN: Complete Results Table 14 lists the controlled comparisons, and Figures 12 and 13 show them per seed. Figure 14 shows the TD losses of both branches and ℒCVL_CV: the two TD losses track each other, as intended by the shared-weight design, and ℒCVL_CV stays small but non-zero, so the codec keeps adapting to the receivers’ action values throughout training. Table 15 lists all 14 methods on SMAC and MPE, Table 16 the learning speed, and Figures 15–18 the curves and the normalized scores. Table 14. Online implementation against controls that differ only in the communication pathway. The same-backbone QMIX shares the GRU backbone, optimizer, replay, exploration schedule, budget and per-seed initialization and removes codec, receiver, reference branch and ℒCVL_CV; the ablation removes only the reference-branch TD loss. p: paired two-sided t-test (P1: Holm-adjusted). Task Metric Control RAVEN control Δ [95% CI] seeds ++/−- p Predator–prey P1 capture success (%) same-backbone QMIX 96.04 53.20 ++42.84 [15.5, 70.2] 5/0 0.024 MPE Spread team return same-backbone QMIX −-29.00 −-31.87 ++2.87 [1.77, 3.97] 5/0 0.002 MPE Spread team return w/o reference TD loss −-28.83 −-29.52 ++0.69 [0.06, 1.32] 3/0 0.043 Figure 12. Predator–prey. Left: capture success during training (mean ± s.d., 5 seeds). Right: final success of each seed; lines connect runs with the same seed and initialization. Figure 13. Spread: online RAVEN against the variant without the reference-branch TD loss (3 matched seeds). Left: learning curves; right: final return per seed. Figure 14. Training losses (means over seeds, 50k-step bins). Top: TD loss of the deployed branch, of the reference branch and of the same-backbone QMIX. Bottom: the alignment loss ℒCVL_CV. Table 15. SMAC (win rate, %) and MPE (team return) at 2.05M environment steps. RAVEN (online): mean±s.d._ .d. (SMAC: 4 seeds; MPE: 5 seeds), 1,000 final evaluation episodes per model. Reference methods: mean over our runs (4–9) of the authors’ official open-source code, at the last evaluation before 2.05M steps (about 100 test episodes). Best per column in bold, second underlined (ties marked alike). The last two columns give the mean per-task min–max normalized score and the mean rank over the eight tasks. § single-neighbor ExpoComm variant. Method 3m 8m M M2 3s5z Spread Tag Crypto norm. rank RAVEN (online) 99.72±0.22 99.95±0.06 100.00±0.00 87.35±8.30 97.80±1.12 −-29.00±0.59 277.3±3.0 48.03±0.04 0.98 1.75 QMIX 99.80 98.80 98.60 57.80 86.40 −-43.42 23.39 0.38 0.74 7.3 QMIX (large) 97.57 99.31 96.88 26.91 94.91 −-44.17 −-46.04 21.69 0.70 8.6 QPLEX 99.75 99.50 30.75 62.00 96.50 −-31.57 235.1 45.44 0.80 5.0 OW-QMIX 100.00 96.20 98.00 38.75 89.60 −-51.50 −-51.09 8.70 0.69 8.6 CW-QMIX 98.00 96.50 96.00 2.25 80.50 −-49.39 −-22.87 47.21 0.69 9.9 RODE 96.75 98.50 98.00 14.00 63.00 −-60.18 106.9 7.25 0.66 9.9 ROCO 74.50 69.25 97.25 30.25 94.00 −-77.83 95.39 21.89 0.54 9.9 Soft-QMIX 99.00 88.75 28.50 0.00 0.00 −-140.0 −-58.21 36.72 0.32 11.9 QMIX-att 97.71 98.96 99.79 42.92 91.67 −-124.4 −-42.83 50.00 0.68 7.5 QMIX+TeamComm 99.25 99.00 98.00 64.25 91.00 −-43.73 68.56 48.00 0.84 5.6 QMIX+TGCNet 99.00 99.25 74.00 66.50 96.75 −-44.95 55.46 48.00 0.80 5.9 CACOM 96.25 43.50 98.75 34.25 97.25 −-48.47 251.1 3.18 0.67 8.1 ExpoComm (1-nbr.)§ 99.00 99.50 100.00 94.50 96.25 −-26.80 276.9 −-31.20 0.87 4.0 Same-backbone QMIX – – – – – −-31.87 – – – – Table 16. SMAC learning speed: time-averaged win rate over 0.05–2.05M steps (%) / first training step (M) at which the mean curve reaches 80% (“–”: never within budget). Reference curves are binned to 50k steps. Method 3m 8m M M2 3s5z RAVEN (online) 92.5 / 0.25 93.2 / 0.25 92.8 / 0.30 44.8 / 1.50 61.4 / 1.20 QMIX 90.8 / 0.40 93.1 / 0.15 86.3 / 0.45 29.3 / – 59.1 / 1.15 QPLEX 97.8 / 0.15 97.2 / 0.10 86.9 / 0.30 25.9 / – 90.0 / 0.30 QMIX+TeamComm 82.1 / 0.60 91.9 / 0.20 88.7 / 0.35 34.8 / – 64.8 / 1.00 QMIX+TGCNet 78.6 / 0.60 93.1 / 0.30 84.0 / 0.50 42.5 / – 67.3 / 1.00 CACOM 92.2 / 0.20 38.9 / – 82.9 / 0.45 12.4 / – 64.7 / 0.90 ExpoComm (1-nbr.)§ 92.1 / 0.30 91.9 / 0.35 87.2 / 0.40 58.1 / 1.10 74.6 / 0.70 Figure 15. SMAC learning curves of online RAVEN (mean ± s.d.) and six reference methods (mean over runs, binned to 50k steps). †Single-neighbor variant. Figure 16. MPE learning curves. Online RAVEN and the same-backbone QMIX: mean ± s.d. over 5 seeds; reference methods: mean over runs, binned to 50k steps. Figure 17. Final SMAC win rate of each online RAVEN seed (squares) against the mean of every reference method (ticks; the best one in black). Figure 18. Final score of each method on each task, colored by the per-task min–max normalized score (cell values are win rates or returns; the mean normalized score is in brackets). Appendix I Cost: Complete Results Parameters and execution FLOPs (matrix multiplications of one team decision step) are counted on networks rebuilt from the code and configurations of the runs; for both RAVEN implementations, NDQ and MACC the rebuilt counts equal the parameter counts recorded by the runs. Bits are logical payload sizes. Measured traffic is the number of bits delivered during a complete training run. Training time is compared only between runs on the same machine: offline RAVEN (teacher included) and SLIM on one GPU server, and online RAVEN, CACOM and MACC on one Apple M4 Pro CPU. Table 17 gives all savings, Table 18 representative absolute values, and Figures 19 and 20 the full picture. Figure 19. Training time per model, compared within one machine: (a) offline RAVEN (teacher included) and SLIM; (b) online RAVEN, CACOM and MACC (log scale). Table 17. Resource savings of RAVEN against every external communication-learning method, 100(1−RAVEN/other)100(1-RAVEN/other) in %; “lo / hi” is the range over settings or maps; negative values mean RAVEN uses more. Parameters and FLOPs (one team decision step) are counted on networks rebuilt from the code and configurations of the runs. Measured traffic is the number of bits delivered during the whole training run. Training time is compared only between runs on the same machine (offline vs. SLIM on one GPU server; online vs. CACOM and MACC on one Mac). ⋆ lower bound: only the request round of CACOM and only the message vector of TeamComm/TGCNet are counted. RAVEN vs. params exec. FLOPs bits / message measured traffic training time RAVEN (offline) SLIM 95.6 / 98.0 – 0.00 – 39.6 / 55.4 RAVEN (offline) NDQ −-1.4 / 85.1 82.2 / 96.8 97.92 – – RAVEN (offline) CACOM −-2.1 / 85.0 85.8 / 97.7 91.67 – – RAVEN (offline) ExpoComm 48.4 / 92.3 91.5 / 98.3 99.90 – – RAVEN (offline) MACC −-229.0 / 50.9 45.4 / 89.3 0.00 – – RAVEN (online) SLIM 81.7 / 81.8 – 0.00 – – RAVEN (online) NDQ −-47.1 / −-37.2 −-48.3 / −-39.6 97.92 97.9 – RAVEN (online) CACOM −-48.3 / −-38.2 −-18.6 / 1.6 91.67 79.3 / 90.0 70.6 / 74.3 RAVEN (online) ExpoComm 29.2 / 29.4 28.2 / 29.0 99.90 99.8 – RAVEN (online) MACC −-351.8 / −-348.9 −-363.5 / −-355.8 0.00 – −-1525.1 / 93.1 RAVEN (online), SMAC CACOM −-42.3 / −-24.2 −-4.5 / 41.2 87.50⋆ – – RAVEN (online), SMAC QMIX+TeamComm 23.7 / 28.2 4.7 / 19.8 99.90⋆ – – RAVEN (online), SMAC QMIX+TGCNet 33.6 / 38.8 −-2.9 / 12.7 99.90⋆ – – RAVEN (online), SMAC ExpoComm (1-nbr.)§ 78.8 / 81.4 78.5 / 81.4 99.95 – – RAVEN (online), MPE CACOM −-54.4 / −-51.8 −-35.6 / −-28.7 87.50⋆ – – RAVEN (online), MPE QMIX+TeamComm 29.2 / 29.6 16.3 / 21.0 99.90⋆ – – RAVEN (online), MPE QMIX+TGCNet 39.8 / 40.2 8.4 / 13.7 99.90⋆ – – RAVEN (online), MPE ExpoComm (1-nbr.)§ 81.9 / 82.1 81.8 / 82.1 99.95 – – Table 18. Absolute cost. Top: navigation, deployed parameters / execution FLOPs per team decision step (SLIM: parameters recorded by its runs, no FLOP count). Bottom: SMAC and MPE, parameters / FLOPs / bits per agent message (lower bounds for CACOM, TeamComm and TGCNet). Setting RAVEN (offline) RAVEN (online) SLIM NDQ CACOM ExpoComm MACC N1-C2 11.7k / 23k 48.0k / 190k 264.0k / – 32.7k / 128k 32.4k / 160k 68.0k / 268k 10.7k / 42k N1-C6 35.2k / 68k 48.3k / 578k – / – 34.7k / 409k 34.5k / 567k 68.2k / 807k 10.7k / 125k N2-L1 5.3k / 10k 48.0k / 190k 264.0k / – 32.7k / 128k 32.4k / 160k 68.0k / 268k 10.7k / 42k N2-L6 5.3k / 16k 48.3k / 677k 264.0k / – 35.2k / 485k 35.0k / 688k 68.3k / 943k 10.7k / 146k Setting RAVEN (online) CACOM QMIX+TeamComm QMIX+TGCNet ExpoComm (1-nbr.)§ 3m 50.1k / 0.30M / 2 35.2k / 0.29M / 16 69.8k / 0.37M / 2048 81.9k / 0.34M / 2048 269.4k / 1.61M / 4096 M2 61.6k / 1.24M / 2 48.8k / 2.10M / 16 80.7k / 1.30M / 2048 92.8k / 1.20M / 2048 290.2k / 5.77M / 4096 Spread 48.5k / 0.29M / 2 31.4k / 0.21M / 16 68.5k / 0.36M / 2048 80.6k / 0.33M / 2048 267.4k / 1.59M / 4096 Figure 20. (a) Bits per message. (b) Deployed parameters against execution FLOPs per team decision step on SMAC and MPE. (c) The same for the eight navigation settings; SLIM’s recorded parameter count is marked by the dashed line.