Paper deep dive
Bounded Coupled AI Learning Dynamics in Tri-Hierarchical Drone Swarms
Oleksii Bychkov
Intelligence
Status: succeeded | Model: anthropic/claude-sonnet-4.6 | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/24/2026, 2:34:02 AM
Summary
This paper studies a tri-hierarchical drone swarm learning system combining Hebbian online learning (fast timescale), multi-agent reinforcement learning (MARL) for tactical coordination (medium timescale), and meta-learning (MAML) for strategic adaptation (slow timescale). Four main theorems are established: (1) Bounded Total Error Theorem showing component-wise upper bounds on total suboptimality under contractual constraints; (2) Bounded Representation Drift Theorem bounding the impact of Hebbian updates on MARL-level embeddings; (3) Meta-Level Compatibility Theorem providing sufficient conditions for strategic adaptation to preserve lower-level invariants; and (4) Non-Accumulation Theorem proving error does not grow unboundedly over time. Numerical illustrations with 10-100 UAV swarms confirm the realism of estimates.
Entities (30)
Relation Signals (26)
Oleksii Bychkov โ affiliatedwith โ Taras Shevchenko National University of Kyiv
confidence 99% ยท Oleksii Bychkov Taras Shevchenko National University of Kyiv, oleksiibychkov@knu.ua
Oleksii Bychkov โ authored โ Bounded Coupled AI Learning Dynamics in Tri-Hierarchical Drone Swarms
confidence 99% ยท Bounded Coupled AI Learning Dynamics in Tri-Hierarchical Drone Swarms Oleksii Bychkov
Tri-Hierarchical Drone Swarm Learning System โ incorporates โ Meta-Learning (MAML)
confidence 99% ยท tri-hierarchical swarm learning system where three mechanisms act simultaneously: (3) meta-learning (MAML) for strategic adaptation
Tri-Hierarchical Drone Swarm Learning System โ incorporates โ Multi-Agent Reinforcement Learning (MARL)
confidence 99% ยท tri-hierarchical swarm learning system where three mechanisms act simultaneously: (2) multi-agent reinforcement learning (MARL) for tactical group coordination
Tri-Hierarchical Drone Swarm Learning System โ incorporates โ Hebbian Online Learning
confidence 99% ยท tri-hierarchical swarm learning system where three mechanisms act simultaneously: (1) local Hebbian online learning at the level of individual agents (fast timescale)
Multi-Agent Reinforcement Learning (MARL) โ operatesattimescale โ 1-10 s
confidence 99% ยท multi-agent reinforcement learning (MARL) for tactical group coordination (medium timescale, ฯ2 โผ1โ10 s)
Meta-Learning (MAML) โ operatesattimescale โ
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern autonomous multi-agent systems combine heterogeneous learning mechanisms operating at different timescales. An open question remains: can one formally guarantee that coupled dynamics of such mechanisms stay within the admissible operational regime? This paper studies a tri-hierarchical swarm learning system where three mechanisms act simultaneously: (1) local Hebbian online learning at individual agent level (fast timescale, 10-100 ms); (2) multi-agent reinforcement learning (MARL) for tactical group coordination (medium timescale, 1-10 s); (3) meta-learning (MAML) for strategic adaptation (slow timescale, 10-100 s). Four results are established. The Bounded Total Error Theorem shows that under contractual constraints on learning rates, Lipschitz continuity of inter-level mappings, and weight stabilization, total suboptimality admits a component-wise upper bound uniform in time. The Bounded Representation Drift Theorem gives a worst-case estimate of how Hebbian updates affect coordination-level embeddings during one MARL cycle. The Meta-Level Compatibility Theorem provides sufficient conditions under which strategic adaptation preserves lower-level invariants. The Non-Accumulation Theorem proves that error does not grow unboundedly over time.
Tags
Links
- Source: https://arxiv.org/abs/2603.20333v1
- Canonical: https://arxiv.org/abs/2603.20333v1
Trouble viewing inline? Open PDF directly โ
Full Text
58,782 characters extracted from source content.
Expand or collapse full text
Bounded Coupled AI Learning Dynamics in Tri-Hierarchical Drone Swarms Oleksii Bychkov Taras Shevchenko National University of Kyiv, oleksiibychkov@knu.ua; 0000-0002-9378-9535 Abstract Modern autonomous multi-agent systems increasingly combine heterogeneous learning mechanisms that operates at different timescales. However, an open question remain: can one formally guarantee that coupled dynamics of such mechanisms do not leave the admissible operational regime? In this paper we study a tri-hierarchical swarm learning system where three mechanisms act simultaneously: (1) local Hebbian online learning at the level of individual agents (fast timescale, ํ 1 โผ10โ100 ms); (2) multi-agent reinforcement learning (MARL) for tactical group coordination (medium timescale, ํ 2 โผ1โ10 s); and (3) meta-learning (MAML) for strategic adaptation (slow timescale, ํ 3 โผ10โ100 s). We establish four results. The Bounded Total Error Theorem shows that under contractual constraints on learning rates, Lipschitz continuity of inter-level mappings, and weight stabilization, the total suboptimality of the system admits a component-wise conservative upper bound that is uniformly bounded in time. The Bounded Representation Drift Theorem provides a conservative worst-case estimate of how Hebbian updates at the lowest level affect the embeddings used by the coordination level during one MARL update cycle. The Meta-Level Compatibility Theorem establishes sufficient conditions under which strategic adaptation from above preserves the invariants of lower levels. The Non-Accumulation Theorem proves that the error does not accumulate unboundedly over time. Numerical illustrations with parameters typical for swarms of 10โ100 UAVs confirms the realism of obtained estimates and demonstrate that violation of even a single contractual condition can leads to unbounded error growth. Keywords: coupled learning dynamics, bounded representation drift, multi-agent systems, Hebbian plasticity, MARL, meta-learning, contract compatibility, autonomous drone swarms, tri-hierarchical architecture 1. Introduction 1.1. Context and Motivation Autonomous multi-agent systems (MAS) have reached a level of complexity where a single learning mechanism can no longer provide all required cognitive capabilities. A UAV swarm executing a complex mission simultaneously needs: reactive adaptation to local conditions (obstacle avoidance, wind compensation), tactical coordination between agents (target allocation, formation control), and strategic adaptation to changes in the global situation (threat type changes, priority redistribution). Each of these types of intelligence is realized by a qualitatively different learning mechanism. Reactive adaptation is naturally described by local neuroplasticity following the Hebbian rule [1], where the weights of a neurocontroller are continuously modified based on sensory experience without external error signal. Tactical coordination is implemented through multi-agent reinforcement learning (MARL) [8, 9], where agents optimize coordination policy based on a global reward signal. Strategic adaptation requires meta-learning [2], which allow the system to quickly adapt to fundamentally new situations. Combining these three mechanisms into a single system โ a tri-hierarchical architecture โ creates a complex cognitive apparatus, but gives rise to a specific technical question: do the coupled dynamics of three qualitatively different learning types remain within admissible operational regime? This paper provides a constructive answer: under formalized contractual constraints, the drift of each level is bounded, the levels are compatible with each other, and the total error does not accumulate over time. 1.2. The Problem of Inter-Level Non-Stationarity To precisely formulate what needs to be proven, let us consider the mechanism by which inter-level non-stationarity arises. Level 1 Hebbian plasticity continuously changes the weights of each agentโs neurocontroller. These weights determine the embeddings (internal representations) that are fed to the graph neural network (GNN) of the coordination Level 2. From the perspective of the MARL coordinator, the input data is therefore non-stationary โ it changes not because of environment changes but because the agents themselves are learning. This non-stationarity propagates in a cascading manner. Changes in embeddings affects the quality of GNN aggregation, which in turn affect the MARL policy, which affects the modulation signal that returns to Level 1 and modulates the Hebbian learning rate. A closed feedback loop arises: L1: ฮํคโL2: ฮํโฮํโฮํโL1: ฮํ eff โฮํคโโฏ An analogous problem arises in the interaction between Level 2 and Level 3. Meta-learning adapts system hyperparameters (including the parameters of Hebbian rules), which changes the dynamics of lower levels. Without formal guarantees, a situation is possible where strategic adaptation from above โbreaksโ the coordination that was tuned to previous parameters. The central question of this paper is whether one can quantitatively bound the influence of each level on the others and prove that the total error does not exceed controlled limits. 1.3. Existing Results and Their Limitations To position our results, we review three most relevant directions in existing research and identify what exactly is missing for our problem. The theory of two-timescale stochastic approximation [3] proves convergence of systems where two iterative procedures operate at different speeds: the slower one โseesโ the faster one as quasi-static. This result is used for analysis of actor-critic algorithms [4], where the actor (policy) updates slower than the critic (value function). However, existing results have substantial limitations in the context of our problem: (a) They considers two timescales, while our system has three qualitatively different ones. (b) They assume both processes optimize the same type of objective function (e.g., both minimize a loss), whereas our three levels uses qualitatively different learning mechanisms: Hebbian correlation (L1), gradient-based reward optimization (L2), and meta-gradient (L3). (c) They do not account for the cascading influence of lower levels on upper ones through changes in input representations. In MARL theory, error boundedness and convergence have been studied under the assumption of a stationary environment [6, 7, 5]. Works on Hebbian plasticity in robotics [10, 11, 12] study adaptability of individual agents but do not formalize the impact of plasticity on coordination in MAS. Meta-learning for multi-agent systems [5] is considered without a Hebbian level. Gap that we address: quantitative estimates of boundedness of coupled learning dynamics and their mutual compatibility are absent for systems where Hebbian plasticity, MARL coordination, and meta-learning act simultaneously. 1.4. Contributions We do not claim to build a complete stability theory for tri-hierarchical systems โ such a theory would require significantly stronger assumptions and different mathematical tools. Instead, we establish four concrete results, each proven with full details, which together give a practically useful picture of bounded coupled dynamics. 1. Theorem 1 (Bounded Total Error): under five contractual conditions (S1)โ(S5), the total suboptimality admits a component-wise conservative upper bound, uniform in time. 2. Theorem 2 (Bounded Representation Drift): an upper bound on the impact of Level 1 Hebbian updates on Level 2 embeddings during one MARL update cycle. Corollary: a constructive constraint on the Hebbian learning rate. 3. Theorem 3 (Meta-Level Compatibility): sufficient conditions under which Level 3 strategic adaptation does not violate lower-level contracts. Central tool โ contract robustness margin. 4. Theorem 4 (Non-Accumulation in the Sense of Uniform Boundedness): proof that the total error is uniformly bounded in time โ i.e., it does not accumulate unboundedly. 5. Numerical illustrations (Section 5) for three swarm configurations (ํ=10,30,100), demonstrating realism of estimates and sensitivity to condition violations. These results are not a self-contained theory โ they are a set of technical tools that allows an engineer to verify whether a given configuration of a tri-hierarchical system will remain within admissible bounds. 2. Related Work The problem of boundedness of coupled learning dynamics intersects several established research directions. In this section we systematically analyze five most relevant areas and show that none of them individually covers our problem. Each subsection concludes with a formulation of how our approach differs. The positioning is summarized in Table 1. 2.1. Multi-Timescale Stochastic Approximation The closest mathematical apparatus for our problem is the theory of multi-timescale stochastic approximation, which investigates convergence of systems with iterative procedures running at different speeds. The classical two-timescale stochastic approximation theory [3] considers a pair of iterative processes: ํฅ ํ+1 =ํฅ ํ +ํ(ํ)ํ(ํฅ ํ ,ํฆ ํ )+ํ ํ+1 (1) ํฆ ํ+1 =ํฆ ํ +ํ(ํ)ํ(ํฅ ํ ,ํฆ ํ )+ํ ํ+1 (2) where ํ(ํ)/ํ(ํ)โ0 โ meaning ํฅ updates โinfinitely slowerโ than ํฆ. Under technical conditions (Lipschitz continuity, noise assumptions) convergence (ํฅ ํ ,ํฆ ํ )โ(ํฅ โ ,ํฆ โ ) is proven. Konda and Tsitsiklis [4] applied this approach to actor-critic algorithms. Our problem differs substantially. First, we have three timescales: ํ 1 โชํ 2 โชํ 3 . Pairwise application of two-timescale theory does not account for the end-to-end cascade L1โL2โL3โL1. Second, the update mechanisms at different levels are qualitatively different: Hebbian correlation is not gradient descent, and MAML update is not a standard SA iteration. Third, all three levels operate simultaneously with fixed (though different) rates, whereas classical theory requires ํ(ํ)/ํ(ํ)โ0 asymptotically. 2.2. Error Boundedness in MARL The second relevant direction concerns quality estimates for multi-agent reinforcement learning. This area is directly relevant since MARL coordination constitute the middle level of tri-hierarchical architecture. For cooperative games, Zhang et al. [6] provide an overview of convergence results for various algorithm classes. Rashid et al. [7] propose QMIX with convergence proof under stationary assumptions. Yu et al. [8] prove MAPPO convergence for cooperative tasks. The principal limitation: all these results assumes a stationary environment. In our system, Level 1 Hebbian plasticity makes the environment non-stationary from the viewpoint of the MARL coordinator: agent embeddings continuously change. Hernandez-Leal et al. [9] study MARL in non- stationary environments but consider external non-stationarity (environment changes), not internal (agent changes through parallel learning). Our approach is the first to formalize precisely internal non-stationarity generated by Hebbian plasticity. 2.3. Hebbian Plasticity in Multi-Agent Systems The third relevant direction concerns the lower-level mechanism โ Hebbian neuroplasticity. Miconi et al. [10] proposed differentiable plasticity โ the ability to learn plasticity rules through meta- gradients. Najarro and Risi [11] developed this approach for evolutionary robotics, showing that Hebbian controllers adapt to new environments without retraining. Pedersen and Risi [12] demonstrated self-modifying policies for continual learning tasks. However, none of these works investigates the impact of Hebbian updates on coordination in a multi-agent system. Plasticity is treated as a property of an individual agent, without quantitative assessment of how weight changes in one agent affects the joint policy of the group. Our Theorem 2 fills this gap by providing an upper bound on representation drift โ the effect of L1 Hebbian updates on embeddings fed to L2 coordinator. 2.4. Meta-Learning for Multi-Agent Adaptation The upper level of the architecture implements meta-learning. MAML [2] and its variants enable fast adaptation to new tasks. Al-Shedivat et al. [5] applied meta-learning to multi-agent adaptation. MAML convergence was formally studied by Fallah et al. [13] and Ji et al. [14]. These results prove outer loop convergence under technical conditions on the inner loop, which is implemented by several gradient descent steps. In our system the inner loop is implemented by Hebbian plasticity (which is not gradient descent). Our Theorem 3 establishes compatibility conditions between MAML outer loop and Hebbian inner loop โ that is, conditions under which meta-adaptation does not violate lower-level contracts. 2.5. Contract-Based Design for Autonomous Systems Finally, the fifth direction forms the methodological basis of our approach. The Design by Contract approach [15] is widely used in software engineering and cyber-physical systems [16]. A contract ํถ=โจPre,Post,Invโฉ specifies preconditions, postconditions, and invariants of a component. Sangiovanni-Vincentelli et al. [16] proposed contract theory for CPS, where overall correctness follows from composition of contracts. We adapt the contract approach for systems where component parameters change through learning. This adds a requirement: the contract must hold in a static configuration and throughout the entire learning process. It is precisely this requirement that gives rise to the notion of contract robustness margin (Definition 9), which is central to Theorem 3. 3. Formal Framework In this section we introduce the formal apparatus needed for formulating and proving the main results. The construction proceeds in five steps: first the structure of the tri-hierarchical system is defined (Subsection 3.1), then timescales and dynamics equations for each level are specified (Subsection 3.2), then quantitative drift measures are introduced (Subsection 3.3), after which the contract system is formalized (Subsection 3.4), and finally โ technical assumptions (Subsection 3.5). 3.1. Tri-Hierarchical Learning System Model The central object of our analysis is the tri-hierarchical learning system โ an architecture that combines three learning levels with qualitatively different mechanisms and timescales. Definition 1 (Tri-Hierarchical Learning System). The system is defined as a quadruple ํฎ= โจโ 1 ,โ 2 ,โ 3 ,ํโฉ, where: โข โ 1 =(ํค ํ (ํก),ํป ํ ) ํ=1 ํ โ Level 1 (neuroplasticity): ํ agents, each with weights ํค ํ (ํก)โ โ ํ and a fixed Hebbian rule ํป ํ ; โข โ 2 =(ํ coord ,ํ,ํ) โ Level 2 (MARL coordination): coordination policy ํ coord , embedding map ํ:โ ํ โโ ํ , and modulation signal ํ:โ ํ โโ; โข โ 3 =(ํ meta ,โณ) โ Level 3 (meta-learning): meta-parameters ํ meta and meta-knowledge library โณ; โข ํ=ํถ ํ ํ=1 ํพ โ set of contracts. 3.2. Timescales and Level Dynamics A defining feature of the tri-hierarchical system is the presence of three clearly separated timescales. Each level updates at its own frequency, and these frequencies differs by orders of magnitude. The timescale separation is simultaneously the source of both the opportunity to obtain bounds (the upper level โseesโ the lower one as quasi-static) and the potential problem (drift of lower levels during one upper-level update cycle can be substantial). Definition 2 (Timescales). The system is characterized by three timescales: ํ 1 โชํ 2 โชํ 3 where ํ ํ is the update period of level ํ. Specifically: โข ํ 1 โผ10โ100 ms: Hebbian update is performed on each processing cycle (tick); โข ํ 2 โผ1โ10 s: MARL update is performed after collecting a batch of experience; โข ํ 3 โผ10โ100 s: meta-update is performed after episode completion or at SMC trigger. Corresponding learning rates: ํ 1 โซํ 2 โซํ 3 Level 1 Dynamics (Hebbian Update). At each step ํก with period ํ 1 : ํค ํ (ํก+ํ 1 )=ํค ํ (ํก)+ํ 1 โ โ ํ (ํฅ ํ (ํก),ํค ํ (ํก),ํ ํ (ํก)) where ํฅ ํ (ํก)โโ ํ is the sensory observation of agent ํ, ํ ํ (ํก)โโ is the modulation signal from Level 2, and โ ํ is the Hebbian function. For the four-component Hebbian rule [10]: โ ํ (ํฅ,ํค,ํ)=ํ(ํ)โ (ํผโ (ํฅ pre โํฅ post )+ํฝโ ํฅ pre +ํพโ ํฅ post +ํฟโ ํค) where (ํผ,ํฝ,ํพ,ํฟ) are plasticity parameters (determined by ํป ํ ), โ denotes element-wise (Hadamard) product, and ํ(โ ) is a sigmoid function that limits modulation. Vectors ํฅ pre ,ํฅ post โโ ํ are pre- and post-synaptic activations. Level 2 Dynamics (MARL Update). The coordination policy is parameterized by vector ํโโ ํ ํ : ํ coord (โ |ํ )=ํ ํ (โ |ํ ). At each step with period ํ 2 : ํ(ํก+ํ 2 )=Proj ฮ (ํ(ํก)+ํ 2 โ โ ํ ํฝ coord ( ํ,ํ(ํค(ํก)) ) ) where ฮ is the admissible parameter set (trust region / clipping in PPO [17]), Proj ฮ is a projection onto ฮ that guarantee the update result remains a valid policy parameterization, ํ(ํค(ํก))= (ํ(ํค 1 (ํก)),...,ํ(ํค ํ (ํก))) is the embedding vector of all agents, and ํฝ coord is the total coordination reward function. For brevity, we write ํ coord (ํก) instead of ํ ํ(ํก) . Relation to base architecture. In the base tri-hierarchical architecture, the tactical level can be implemented as either a full online MARL update or an offline-trained policy with small incremental corrections. Equation (3) models both cases as bounded incremental adaptation: contract (C3) and the projection Proj ฮ bound the update step regardless of whether the gradient update is truly online or merely fine-tuning of a pre-trained policy. Level 3 Dynamics (Meta-Update). At each step with period ํ 3 : ํ meta (ํก+ํ 3 )=ํ meta (ํก)โํ 3 โ โ ํ โ meta ( ํ,ํ coord ,ํค(ํก) ) where โ meta is the meta-loss function (MAML outer loss). 3.3. Drift Measures To transition from qualitative understanding of non-stationarity to quantitative estimates, we need to introduce formal drift measures. Each measure corresponds to one level of the architecture and quantitatively describes how much the corresponding component has changed over a given time interval. The sequence of definitions reflects the cascading influence structure: weight drift (Level 1) generates embedding drift (L1โL2 interface), which generates coordination policy drift (Level 2), while meta-updates (Level 3) are described by meta-parameter drift. Definition 3 (Weight Drift). Maximum drift of plastic controller weights over interval [ํก 1 ,ํก 2 ]: ํท ํค (ํก 1 ,ํก 2 )=max ํโ1,...,ํ โฅํค ํ (ํก 2 )โํค ํ (ํก 1 )โฅ Definition 4 (Embedding Drift). Maximum drift of coordination-level embeddings: ํท ํ (ํก 1 ,ํก 2 )=max ํโ1,...,ํ โฅํ(ํค ํ (ํก 2 ))โํ(ํค ํ (ํก 1 ))โฅ Definition 5 (Policy Drift). Maximum drift of the coordination policy: ํท ํ (ํก 1 ,ํก 2 )=max ํ โํฎ ํท ํํ ( ํ coord (ํก 2 )(โ |ํ ), ํ coord (ํก 1 )(โ |ํ ) ) where ํท ํํ is total variation distance. Definition 6 (Meta-Parameter Drift). ํท ํ (ํก 1 ,ํก 2 )=โฅํ meta (ํก 2 )โํ meta (ํก 1 )โฅ 3.4. Contract System Contracts formalize the constraints under which three learning levels can function without mutual destruction. Each contract specifies a verifiable invariant for one component. From the full system of 22 contracts described in the context of tri-hierarchical architecture, we select six that directly affect boundedness of inter-level interactions. Definition 7 (Contract). A contract ํถ ํ =โจPre ํ ,Post ํ ,Inv ํ โฉ specifies a precondition (Pre), postcondition (Post), and invariant (Inv) that must hold during system operation. For purposes of our analysis we select six critical contracts: Contract NP-C1 (Bounded Plasticity). Inv: โฅํค ํ (ํก+ํ 1 )โํค ํ (ํก)โฅโคฮ NP โํ,โํก Contract NP-C2 (Safety-Compatible Plasticity). Inv: ํฆ safety (ํฅ,ํค ํ (ํก))=ํฆ safety (ํฅ,ํค ํ (0)) โํ,โํก,โํฅโํณ danger Contract MARL-C1 (Bounded Policy Update). Inv: ํท ํํ ( ํ(ํก+ํ 2 )(โ |ํ ),ํ(ํก)(โ |ํ ) ) โคฮ ํ โํ Contract GNN-C1 (Bounded GNN Approximation Error). Notation remark. In this contract we distinguish two roles: ํ is the learned encoder (GNN or other network) that actually computes the embeddings; ํ โ is the ideal (target) mapping of weights to embeddings in the absence of approximation error. The contract bounds the distance between them: Inv: โฅํ(ํค)โํ โ (ํค)โฅโคํ GNN โํค:โฅํคโฅโคํ max In Assumption A1 and in drift formulas (11)โ(13), ํ denotes the real encoder. Thus ํ GNN enters bounds as an additive approximation error term, separate from the multiplicative drift ํฟ ํ โ ํท ํค . Contract ML-C1 (Bounded Adaptation Time). Post: ํ adapt โคํ critical Contract ML-C2 (Monotone Meta-Improvement). Inv: ํผ[ํพ inner (ํ+1) ]โคํผ[ํพ inner (ํ) ] 3.5. Assumptions To conclude the formal framework, we fix the technical assumptions used in the proofs of Section 4. These assumptions concern smoothness of inter-level mappings, boundedness of activations and signals, and properties of the meta-loss function. In Section 6 we discuss the realism of each assumption. Assumption A1 (Lipschitz Embeddings). The mapping ํ:โ ํ โโ ํ is ํฟ ํ -Lipschitz: โฅํ(ํค)โํ(ํค โฒ )โฅโคํฟ ํ โ โฅํคโํค โฒ โฅ โํค,ํค โฒ Assumption A2 (Lipschitz Coordination Policy). The coordination policy ํ coord is ํฟ ํ -Lipschitz in embeddings: ํท ํํ ( ํ(โ |ํ ,ํ),ํ(โ |ํ ,ํ โฒ ) ) โคํฟ ํ โ โฅํโํ โฒ โฅ โํ Assumption A3 (Weight Stabilization). The autoregulation parameter in the Hebbian rule is negative: ํฟ<0. Assumption A4 (Bounded Activations). Neural activations are normalized: โฅํฅ pre โฅ,โฅํฅ post โฅโค1. Assumption A5 (Bounded Modulation). The modulation signal is bounded: |ํ ํ (ํก)|โคํ max for all ํ,ํก. Assumption A6 (Smooth Meta-Function). The meta-loss function โ meta is ํฟ โ -smooth in ํ: โฅโ ํ โ meta (ํ)โโ ํ โ meta (ํ โฒ )โฅโคํฟ โ โ โฅํโํ โฒ โฅ Assumption A7 (Sufficiently Small Hebbian Step). The Hebbian learning rate satisfies ํ 1 <ํ ฬ 1 , where ํ ฬ 1 is defined in (9a). This is a conservative sufficient condition that guarantees dominance of the stabilizing contribution of ํฟ<0 over the quadratic term of the discrete update (see Lemma 1). Assumption A8 (Reference-MDP Bounding). There exists a reference MDP โณ ref = (ํฎ,ํ,ํ ref ,ํ ref ,ํพ) with fixed transitions and reward, such that the coordination quality loss caused by representation drift โฅฮํโฅโคํ is upper-bounded by the effective policy change in โณ ref : |ํฝ(ํ ํ (โ |ํ))โํฝ(ํ ํ (โ |ํ โฒ ))|โค|ํฝ ref (ํ ํ (โ |ํ))โํฝ ref (ํ ํ (โ |ํ โฒ ))| Remark. A8 is the central bounding assumption of this work: it allows estimating one-step coordination loss through a fixed-latent reference MDP without claiming stationarity of the real system. In the real system the MDP is non-stationary due to embedding drift, whereas โณ ref is fixed. A8 only asserts that the fixed-MDP bound is a correct upper estimate for the non-stationary case. This is justified because embedding changes are already accounted for in ํฟ ํ โ ฮ NP through Lipschitz continuity, and we estimate one-step policy change rather than accumulated trajectory divergence. A8 allows applying the classical Performance Difference Lemma [18] in our setting (see proof of Theorem 1). 4. Main Results This section contains the main results of the paper. The structure follows a principle of progressive complexity: first we establish five auxiliary lemmas, each describing a local property of one component or one interaction (Subsection 4.1). Then these lemmas are building blocks for four theorems: Theorem 1 (Subsection 4.2) gives an overall boundedness estimate; Theorem 2 (Subsection 4.3) details the impact of the fastest level on the middle one; Theorem 3 (Subsection 4.4) establishes compatibility of the slowest level with lower ones; and Theorem 4 (Subsection 4.5) closes the analysis by proving non-accumulation of error over time. 4.1. Auxiliary Lemmas Before proceeding to the main theorems, we establish five auxiliary results. Lemmas 1โ2 describe the behavior of an individual Hebbian controller: weight boundedness and single-step update boundedness. Lemmas 3โ4 describe the cascading impact through Lipschitz continuity: how weight changes translate into embedding changes, and then into policy changes. Lemma 5 combines previous results into a single L1โL2 chain. Lemma 1 (Boundedness of Hebbian Weights). Let Assumptions A3โA5, contract (C1), and ํ 1 <ํ ฬ 1 โ 2|ํฟ| (|ํผ|+|ํฝ|+|ํพ|+|ํฟ|ํ 0 ) 2 โ ํ(ํ max ) 2 hold, where ํ 0 โ(|ํผ|+|ํฝ|+|ํพ|)/|ํฟ|. Then for all ํ and ํกโฅ0: โฅํค ํ (ํก)โฅโคํ max โํ 0 +1= |ํผ|+|ํฝ|+|ํพ| |ํฟ| +1 Notation remark. In the Hebbian rule (2), the product ํฅ pre โํฅ post is the element-wise (Hadamard) product of vectors ํฅ pre ,ํฅ post โโ ํ , yielding a vector in โ ํ . All estimates below use the Euclidean norm โฅโ โฅ=โฅโ โฅ 2 . By Assumption A4, โฅํฅ pre โฅ 2 ,โฅํฅ post โฅ 2 โค1 and โฅํฅ pre โฅ โ โค1, whence โฅํฅ pre โ ํฅ post โฅ 2 โคโฅํฅ pre โฅ โ โ โฅํฅ post โฅ 2 โค1. Proof. Consider the auxiliary quadratic function ํ ํ (ํก)=โฅํค ํ (ํก)โฅ 2 . By (1) and (2): ํ ํ (ํก+ํ 1 )โํ ํ (ํก)=2ํ 1 โจํค ํ (ํก),โ ํ (โ )โฉ+ํ 1 2 โฅโ ํ (โ )โฅ 2 We estimate both terms. First term. By CauchyโSchwarz and the remark above: โจํค ํ ,โ ํ โฉ=ํ(ํ ํ )โ (ํผโจํค ํ ,ํฅ pre โํฅ post โฉ+ํฝโจํค ํ ,ํฅ pre โฉ+ํพโจํค ํ ,ํฅ post โฉ+ํฟโฅํค ํ โฅ 2 ) โคํ(ํ max )โ ( (|ํผ|+|ํฝ|+|ํพ|)โฅํค ํ โฅ+ํฟโฅํค ํ โฅ 2 ) Denote ํดโ|ํผ|+|ํฝ|+|ํพ|. When โฅํค ํ โฅ>ํด/|ํฟ|=ํ 0 we have โจํค ํ ,โ ํ โฉโคํ(ํ max )โ ํฟโฅํค ํ โฅ(โฅ ํค ํ โฅโํ 0 )<0. Second term. By triangle inequality and activation boundedness: โฅโ ํ โฅโคํ(ํ max )โ (ํด+|ํฟ|โฅํค ํ โฅ) Hence ํ 1 2 โฅโ ํ โฅ 2 โคํ 1 2 ํ(ํ max ) 2 (ํด+|ํฟ|โฅํค ํ โฅ) 2 . Combining. Denote ํ=โฅํค ํ โฅ. For any ํ we have: ฮํ ํ โคโ2ํ 1 ํ(ํ max )|ํฟ|โ ํโ (ํโํ 0 ) โ stabilizing (linear in ํ 1 ) +ํ 1 2 ํ(ํ max ) 2 (ํด+|ํฟ|ํ) 2 โ perturbing (quadratic in ํ 1 ) Since the quadratic term grows as ํ 2 for large ํ, while the stabilizing term also grows as ํ 2 (because ํ(ํโํ 0 )โผํ 2 ) but with different coefficients (ํ 1 vs ํ 1 2 ), condition (9a) on the smallness of ํ 1 ensures ฮํ ํ <0 only in a bounded outer region around ํ 0 . For very large ํ purely discrete dynamics with finite ํ 1 may not have ํ ํ decreasing. Therefore weight boundedness in this paper is ensured by the combination of two mechanisms: (i) Locally stabilizing tendency. Under ํฟ<0 and ํ 1 <ํ ฬ 1 (A7), the function ํ ํ decreases in a neighborhood ํ:ํ 0 <ํโคํ 0 +ํ for some ํ>0 depending on ํ 1 . This provides a locally stabilizing tendency near the boundary ํ 0 , which together with the step constraint (C1) is sufficient for the boundedness estimates in this paper. (i) Step constraint. Contract (C1) through clamping bounds each step: โฅํค ํ (ํก+ํ 1 )โํค ํ (ํก)โฅโคฮ NP (formula (10a)). This guarantees that even if ํ ํ does not decrease monotonically, the weights cannot โjumpโ beyond the controlled region. Together: under (A3)โ(A7) and (C1), the weight trajectory remains in some bounded set. For subsequent estimates we fix a constructive practical bound: ํ max โํ 0 +1 which is used as a conservative margin in all subsequent bounds. The choice of ํ 0 +1 is not a rigorously derived optimal bound โ it is a convenient constructive constant that accounts for update discreteness. Remark on condition (9a). Condition ํ 1 <ํ ฬ 1 is a conservative sufficient condition for the boundedness estimates of this paper. We do not claim it is a sharp threshold for discrete Hebbian dynamics. For simplicity of subsequent estimates we take: ํ ฬ 1 โ 2|ํฟ| (ํด+|ํฟ|ํ 0 ) 2 โ ํ(ํ max ) 2 which is not tight but sufficient for our boundedness estimates. For typical parameters (Section 5) ํ ฬ 1 โ4.5ร10 โ3 , while ํ 1 =10 โ3 , and the condition is satisfied with margin. โซ Lemma 2 (Single-Step Weight Drift). Under conditions of Lemma 1, the weight change per one step ํ 1 is bounded by the intrinsic bound: โฅํค ํ (ํก+ํ 1 )โํค ํ (ํก)โฅโคํ 1 โ ํ(ํ max )โ (|ํผ|+|ํฝ|+|ํพ|+|ํฟ|โ ํ max )โฮ ฬ 1 int Remark (Intrinsic vs. enforced bound). Formula (10) is a consequence of the dynamics: it follows from update equation (1) and weight boundedness (Lemma 1). Contract (C1) adds an external constraint: clamping min(ฮ ฬ 1 int ,ฮ NP ), which can be much tighter than the intrinsic bound. Throughout the paper we denote the effective update step: ฮ ฬ 1 โmin(ฮ ฬ 1 int ,ฮ NP ) Thus contract (C1) both verifies the bound and strengthens it through clamping โ this is the price that pays off through tighter estimates in Theorems 1โ2. Proof. Direct application of triangle inequality to (1) with estimates from Lemma 1. โซ Lemma 3 (Lipschitz Embedding Drift). Under Assumption A1: ํท ํ (ํก 1 ,ํก 2 )โคํฟ ํ โ ํท ํค (ํก 1 ,ํก 2 ) Proof. Direct consequence of Lipschitz continuity of ํ: โฅํ(ํค ํ (ํก 2 ))โํ(ํค ํ (ํก 1 ))โฅโคํฟ ํ โฅํค ํ (ํก 2 )โํค ํ (ํก 1 )โฅโคํฟ ํ โ ํท ํค (ํก 1 ,ํก 2 ) โซ Lemma 4 (Coordination Policy Sensitivity). Under Assumption A2, an embedding change of ํ causes a policy change: ํท ํ โคํฟ ํ โ ํ Proof. Direct consequence of Lipschitz continuity of ํ coord . โซ Lemma 5 (Cascading Bound L1 โ L2). Under (C1), (A1), (A2), the coordination policy drift per one Hebbian update step is bounded: ํท ํ (ํก,ํก+ํ 1 )โคํฟ ํ โ ํฟ ํ โ ฮ NP Proof. Combining Lemmas 2, 3, 4: ํท ํ (ํก,ํก+ํ 1 )โคํฟ ํ โ ํท ํ (ํก,ํก+ํ 1 )โคํฟ ํ โ ํฟ ํ โ ํท ํค (ํก,ํก+ํ 1 )โคํฟ ํ โ ํฟ ํ โ ฮ NP โซ 4.2. Theorem 1: Bounded Total Error The first result combines auxiliary lemmas with contractual conditions and establishes an upper bound on total suboptimality. The central idea is additive decomposition of error into three components, each of which is bounded. Remark on the nature of additive decomposition. The estimate ํ total โคํ Hebb +ํ coord +ํ meta is a structural upper bound, not a rigorous decomposition. It relies on triangle inequality and independent worst-case estimation of each component, without accounting for possible compensations between levels. In particular, cross-terms (e.g., simultaneous impact of meta-update and Hebbian drift) are upper-bounded by the sum of individual effects. This makes the bound conservative but guarantees correctness. Definition 8a (Total Suboptimality). We define total suboptimality as the difference between optimal and actual coordination quality: ํ total (ํก)โํฝ โ (ํ coord โ ,ํ โ ,ํ โ )โํฝ(ํ coord (ํก),ํ(ํค(ํก)),ํ meta (ํก)) where ํฝ โ is the quality under optimal parameters of all three levels, and ํฝ(โ ) is the actual quality under current parameters. Both quantities are measured through coordination reward ํฝ coord . Theorem 1 (Bounded Total Error). Let Assumptions A1โA8 and contracts (C1)โ(C6) hold. Introduce five contractual boundedness conditions: โข (S1) ํฟ<0 and |ํ 1 |โคํ ฬ 1 (weight stabilization); โข (S2) ํ 1 /ํ 2 โคํ 12 and ํ 2 /ํ 3 โคํ 23 (timescale separation); โข (S3) ํฟ ํ โ ํฟ ํ โ ฮ NP โคํ coord โ (bounding L1 impact on L2); โข (S4) ํ 3 โ ํฟ โ โคํ meta โ (bounding meta-learning step); โข (S5) contracts (C1)โ(C6) hold at time ํก 0 . Then for all ํกโฅํก 0 the total suboptimality is bounded: ํ total (ํก)โคํ ฬ =ํ Hebb +ํ coord +ํ meta where: ํ Hebb = 2ํฟ ํ ํฟ ํ ฮ NP โฅโํโฅ โ (1โํพ) ํ coord =2ํป eff โ ํฟ ํ โ (ํฟ GNN โ ํโ ํฟ ํ โ ํ 12 โ ฮ ฬ 1 +ํ GNN )โ ํ max ํ meta =2ํป eff โ ํฟ ํ โ ํฟ ํ โ ํฟ ํปโฮํค โ ํฟ ํโํป โ ํ 3 ํบ max โ ํ 3 ํ 1 โ ํ max where ํ 12 =ํ 2 /ํ 1 is the number of Hebbian steps per one MARL cycle (for simplicity we assume ํ 2 /ํ 1 โโ; generalization to non-integer case via โโ โ is straightforward), ํป eff is the effective horizon (Definition 8), ํ max =max ํ ,ํ |ํ (ํ ,ํ)|, ํบ max =sup ํ โฅโ ํ โ meta โฅ, and ํฟ ํโํป , ํฟ ํปโฮํค are Lipschitz constants of the meta-parameter cascade. Character of estimates (15)โ(17). These formulas constitute a conservative component-wise upper bound obtained through separate estimation of three coupled mechanisms. Additivity is a consequence of triangle inequality, not of exact decomposition; cross-terms are upper-bounded by the sum rather than computed exactly. Actual suboptimality may be significantly smaller than ํ ฬ . Moreover, the system preserves: โข (G1) Safety: ํฆ safety (ํฅ,ํค ํ (ํก))=ํฆ safety (ํฅ,ํค ํ (0)) for all ํ,ํก and ํฅโํณ danger ; โข (G3) Bounded sub-optimality: ํฝ โ โํฝ(ํก)โคํ ฬ for all ํก. Remark (Architectural Liveness Guarantee, G2). Unlike G1 and G3, which are consequences of bounded-learning analysis, liveness is an orthogonal architectural-temporal guarantee: each task completes in finite time โคํ live,max , which follows from contractually bounded timeouts of the BDI/HTN/Auction/BT pipeline and does not depend on learning dynamics. Proof. Step 1: Safety (G1). By contract (C2), safety synapses are frozen: ํค ํ frozen (ํก)=ํค ํ frozen (0). Since safety output depends exclusively on frozen synapses (architectural isolation, safety mask), we have ํฆ safety (ํฅ,ํค ํ (ํก))=ํ safety (ํฅ,ํค ํ frozen )=ํ safety (ํฅ,ํค ํ frozen (0)) for all ํก. This result does not depend on plastic weight dynamics and holds unconditionally. โซ G1 Step 2: Bounded sub-optimality (G3). Total suboptimality is upper-bounded by the sum of three components. Component 1 (ํ Hebb ): direct plasticity impact on policy. By Lemma 5, each Hebbian update step perturbs the coordination policy by โคํฟ ํ ํฟ ํ ฮ NP in total variation distance. By Assumption A8 (Reference-MDP Bounding, Section 3.5) and PDL [18]: |ํฝ(ํ)โํฝ(ํ โฒ )|โค 2โฅํโํ โฒ โฅ ํํ โ โฅโํโฅ โ 1โํพ Substituting โฅํโํ โฒ โฅ ํํ โคํฟ ํ ํฟ ํ ฮ NP , we obtain (15). Component 2 (ํ coord ): accumulated impact over MARL update cycle. During one MARL update cycle (ํ 2 seconds) there occur ํ 12 Hebbian update steps. By Lemma 3: ํท ํ (ํก,ํก+ํ 2 )โคํฟ ํ โ โโฅ ํ 12 โ1 ํ=0 ํค ํ (ํก+ํํ 1 +ํ 1 )โํค ํ (ํก+ํํ 1 )โฅโคํฟ ํ โ ํ 12 โ ฮ ฬ 1 GNN aggregation collects messages from |ํ(ํ)|โคํ neighbors. By Lipschitz continuity of GNN with constant ํฟ GNN : โฅํง ํ (ํก+ํ 2 )โํง ํ โ (ํก+ํ 2 )โฅโคํฟ GNN โ โ ํโ ํฟ ํ โ ํ 12 โ ฮ ฬ 1 +ํ GNN For obtaining a less conservative estimate we use the effective horizon. Definition 8 (Effective Horizon). The effective horizon ํป eff is defined as the minimum of three quantities: ํป eff =min ( 1 1โํพ , ํ 3 ํ 2 , ํป mission ) where (1โํพ) โ1 is the standard discounted horizon, ํ 3 /ํ 2 is the number of MARL cycles between meta-updates, and ํป mission is the maximum duration of a mission phase. Status of ํป eff . The effective horizon is a practical truncation heuristic (practical finite-horizon truncation parameter), not a rigorously derived or asymptotically exact object. Further, ํป eff is used only as a technical truncation parameter for constructing a finite-horizon upper-bound estimate and does not claim the role of exact asymptotic horizon of the process. It rests on three observations: (a) discounting naturally truncates the influence of distant steps; (b) meta-updates every ํ 3 seconds redirect adaptation; (c) mission has finite duration. For rigorous justification of point (b) one would need to prove that each meta-cycle brings the system closer to the admissible set โ this remains an open question (see Section 6.3). We use ํป eff as a conservative (but not proven optimal) choice of truncation horizon that gives useful practical bounds. Lemma 6 (Finite-Horizon Policy Difference Bound). For any two policies ํ,ํ โฒ with โฅํโ ํ โฒ โฅ ํํ โคํ and truncation horizon ํป eff : |ํฝ ํป eff (ํ)โํฝ ํป eff (ํ โฒ )|โค2ํป eff โ ํโ ํ max Proof of Lemma 6. |ํฝ ํป (ํ)โํฝ ํป (ํ โฒ )|โค โ ํพ ํก ํปโ1 ํก=0 โ 2ํโ ํ max โค2ํปํํ max . โซ Applying Lemma 6 with ํ=ํฟ ํ (ํฟ GNN โ ํํฟ ํ ํ 12 ฮ ฬ 1 +ํ GNN ), we obtain (16). Component 3 (ํ meta ): meta-update impact. By (S4) and (A6), the meta-update step is bounded: โฅ ฮํโฅโคํ 3 ํบ max . Meta-parameter changes affect coordination quality through a cascade: ฮํโฮํปโ ฮ(plasticity dynamics)โฮํ. Remark (nature of estimate). Formula (17) below is a compositional cascade estimate: each cascade step is upper-bounded via the corresponding Lipschitz constant, and results are multiplied. This gives a correct upper bound but not a sharp one โ actual meta-update impact may be considerably smaller due to partial compensation at different cascade levels. Let ํฟ ํโํป be the Lipschitz constant of the mapping from meta-parameters to Hebbian rule parameters, and ํฟ ํปโฮํค be the Lipschitz constant of the impact of rule changes on weight dynamics over one ํ 3 cycle. Then the weight change caused by meta-update: โฅฮํค ํ meta โฅโคํฟ ํปโฮํค โ ํฟ ํโํป โ ํ 3 ํบ max โ ํ 3 ํ 1 Cascading through embeddings and policy, we obtain (17). Summing three components: ํ total โคํ Hebb +ํ coord +ํ meta =ํ ฬ . โซ G3 โซ 4.3. Theorem 2: Bounded Representation Drift Theorem 1 established a general estimate but did not answer the principal design question: how much exactly do lower-level Hebbian updates โcorruptโ the embeddings fed to the coordinator? Theorem 2 provides an upper bound on this drift over one MARL update cycle. This result have practical consequences: it allows computing the maximum allowable Hebbian learning rate. Important caveat. The estimate of Theorem 2 is a conservative worst-case drift bound: it is constructed via triangle inequality for the sum of steps, without accounting for correlations between successive updates. In practice successive Hebbian steps are partially correlated (and can partially cancel each other), so actual drift is typically much smaller than the bound. Tighter bounds can be obtained with additional assumptions on the distribution of sensory experience (e.g., mixing conditions). Theorem 2 (Bounded Representation Drift โ conservative worst-case estimate). Under conditions (S1)โ(S3) of Theorem 1, for any interval [ํก,ํก+ํ 2 ] (one MARL update cycle): ํท ํ (ํก,ํก+ํ 2 )โคํฟ ํ โ ํ 2 ํ 1 โ ฮ ฬ 1 โฮฆ max where ฮ ฬ 1 is defined in (10). Corollary 2.1. For preserving near-optimal coordination, a necessary condition is: ํฟ ํ โ ํ 2 ํ 1 โ ฮ ฬ 1 <ํ ํ โ where ํ ํ โ is the admissible embedding drift threshold for ํ-convergence of MARL. Corollary 2.2 (Practical Recommendation). From (19) we obtain a constraint on the Hebbian learning rate: ํ 1 < ํ ํ โ โ ํ 1 ํฟ ํ โ ํ 2 โ ํ(ํ max )โ (|ํผ|+|ํฝ|+|ํพ|+|ํฟ|ํ max ) Proof of Theorem 2. During the interval [ํก,ํก+ํ 2 ] there occur ํ 12 Hebbian update steps. Weight drift of agent ํ: โฅํค ํ (ํก+ํ 2 )โํค ํ (ํก)โฅ=โฅโํ 1 ํ 12 โ1 ํ=0 โ ํ (ํฅ ํ (ํก+ํํ 1 ),ํค ํ (ํก+ํํ 1 ),ํ ํ (ํก+ํํ 1 ))โฅ By triangle inequality and Lemma 2: โฅํค ํ (ํก+ํ 2 )โํค ํ (ํก)โฅโคโฮ ฬ 1 ํ 12 โ1 ํ=0 =ํ 12 โ ฮ ฬ 1 Applying Lemma 3: ํท ํ (ํก,ํก+ํ 2 )โคํฟ ํ โ ํท ํค (ํก,ํก+ํ 2 )โคํฟ ํ โ ํ 2 ํ 1 โ ฮ ฬ 1 . โซ 4.4. Theorem 3: Meta-Level Compatibility with Contracts Theorems 1 and 2 describe boundedness of the Hebbian and coordination levels. However, a question remains: can meta-adaptation violate the very contracts under which these bounds hold? Theorem 3 formalizes sufficient conditions under which meta-updates preserve lower-level invariants. The central tool is the contract robustness margin โ the gap between the current system state and the boundary of contract violation. Definition 9 (Contract Robustness Margin). For contract ํถ ํ with invariant Inv ํ , define the closed failure set โฑ ํ =ํโฮ:Inv ํ (ํ)=FALSEโฮ (closedness follows from continuity of invariants). The robustness margin: ํ ํ (ํ)=dist(ํ,โฑ ํ )=inf ํ โฒ โโฑ ํ โฅํโํ โฒ โฅ i.e., the distance from current meta-parameters to the failure set of invariant ํ. As a distance-to- closed-set function, ํ ํ is 1-Lipschitz: |ํ ํ (ํ)โํ ํ (ํ โฒ )|โคโฅํโํ โฒ โฅ for all ํ,ํ โฒ . Definition 10 (Cascading Sensitivity of Meta-Update). A change ฮํ in meta-parameters affects lower levels through the chain: ฮํโ ํฟ ํโํป ฮํปโ ํฟ ํปโํค ฮ(plasticity dynamics)โ ํฟ ํ ฮํโ ํฟ ํ ฮํ Total cascading sensitivity: ํฆ=ํฟ ํ โ ํฟ ํ โ ํฟ ํปโํค โ ํฟ ํโํป Theorem 3 (Meta-Level Compatibility). Let the following conditions hold: (M1) Each contract ํถ ํ has positive margin: ํ ํ (ํ meta (ํก))>0 for all ํ. (M2) Meta-update step is bounded: โฅํ meta (ํก+ํ 3 )โํ meta (ํก)โฅโคฮ ํ , where ฮ ํ =ํ 3 ํบ max . (M3) The step does not exceed minimum margin: ฮ ํ <min ํ ํ ํ (ํ meta (ํก)). Then: (i) No invariant is violated: Inv ํ (ํ meta (ํก+ํ 3 ))=TRUE for all ํ. (i) The margin remains positive: ํ ํ (ํ meta (ํก+ํ 3 ))โฅํ ํ (ํ meta (ํก))โฮ ํ >0 for all ํ. (i) Cascading impact on coordination is bounded: ํท ํ meta (ํก,ํก+ํ 3 )โคํฆโ ฮ ํ . Proof. Part (i). By the definition of margin (26), invariant Inv ํ is violated only when perturbation โฅฮํโฅโฅ ํ ํ . By (M2), perturbation โคฮ ํ . By (M3), ฮ ํ <ํ ํ for all ํ. Hence invariant is preserved. โซ (ํ) Part (i). By the 1-Lipschitz property of distance function ํ ํ =dist(โ ,โฑ ํ ): ํ ํ (ํ โฒ )โฅํ ํ (ํ)โโฅ ํ โฒ โํโฅ. Substituting: ํ ํ (ํ(ํก+ํ 3 ))โฅํ ํ (ํ(ํก))โฮ ํ >0 by (M3). โซ (ํ) Part (i). Meta-update changes Hebbian rules: ฮํป=ํฟ ํโํป โ ฮํ. This changes the weight dynamics: the difference in weights under old and new rules over time ํ 3 : โฅฮํค meta โฅโคํฟ ํปโํค โ โฅฮํปโฅ=ํฟ ํปโํค โ ํฟ ํโํป โ ฮ ํ Through embeddings and policy: ํท ํ meta โคํฟ ํ โ ํฟ ํ โ ํฟ ํปโํค โ ํฟ ํโํป โ ฮ ํ =ํฆโ ฮ ํ โซ (ํ) โซ Corollary 3.1 (Constructive Constraint on ํ 3 ). From (M3) we obtain: ํ 3 < min ํ ํ ํ (ํ meta (ํก)) ํบ max This gives an adaptive rule: the meta-learning rate should decrease as the system approaches the boundary of any contract (margin ํ ํ decreases). Corollary 3.2 (Increasing Robustness). If the meta-update improves margins (i.e., ํ ํ (ํ(ํก+ ํ 3 ))>ํ ํ (ํ(ํก)) for some ํ), the system becomes more robust over time. Remark 1 (Practical Implementation). The margin ํ ํ (ํ) can be computed as a runtime monitor: for linear invariants โ analytically, for nonlinear ones โ through numerical approximation. Margin dropping below threshold ํ alarm is an early warning. 4.5. Theorem 4: Non-Accumulation of Error Theorems 1โ3 established that the system error is bounded at each step. But a question remains: does this error gradually grow over time, even while remaining bounded at each individual step? Such a phenomenon โ unbounded error accumulation โ is a characteristic problem of non- stationary systems: slow monotonic degradation that does not violate any local constraint but over long time leads to degradation. Theorem 4 proves that in our system under contractual conditions, unbounded accumulation is impossible. Theorem 4 (Non-Accumulation in the Sense of Uniform Boundedness). Under conditions of Theorems 1โ3, the total error ํ total (ํก) is uniformly bounded over time: limsup ํกโโ ํ total (ํก)โคํ ฬ i.e., error does not accumulate unboundedly over time (no unbounded error accumulation). Remark on strength of result. Theorem 4 guarantees the absence of unbounded growth of error. It does not exclude: (a) bounded oscillations of ํ total (ํก) within interval [0,ํ ฬ ]; (b) slow regime wandering within the admissible set; (c) monotonic growth that stops upon reaching ํ ฬ . Stronger results (e.g., convergence of ํ total (ํก) to a stationary value) would require additional assumptions about ergodicity or contractivity of the dynamics. Proof. A potential source of error accumulation is growth of โฅํค ํ (ํก)โฅ over time, which would increase ฮ ฬ 1 . However by Lemma 1, โฅํค ํ (ํก)โฅโคํ max for all ํก โ weights are bounded thanks to the stabilizing term ํฟ<0. We show that each error component is bounded independently of ํก: Component 1 (Hebbian): ํ Hebb depends on ฮ NP , ํฟ ํ , ํฟ ํ โ all constants that do not change with time. Hence ํ Hebb =const for all ํก. Component 2 (Coordination): ํ coord depends on ฮ ฬ 1 , which in turn depends on ํ max (bounded by Lemma 1) and fixed parameters. Hence ํ coord is bounded for all ํก. Component 3 (Meta): ํ meta depends on ํ 3 ํบ max , where ํบ max =sup ํ โฅโ ํ โ meta โฅ โ bounded by (A6) and boundedness of meta-parameter domain (which in turn is ensured by boundedness of lower-level components and MAML convergence [13]). Hence ํ total (ํก)โคํ Hebb +ํ coord +ํ meta =ํ ฬ for all ํก, where ํ ฬ does not depend on ํก. This precludes unbounded error accumulation. โซ 5. Numerical Illustrations The estimates obtained in Section 4 are formally correct, but their practical value depend on how realistic they are for concrete systems. In this section we substitute specific numerical parameters typical for autonomous UAV swarms and compute all bounds explicitly. This allows verifying that estimates carry information (are not trivially large), identifying the most sensitive parameters, and demonstrating necessity of each contractual condition through counterexamples. 5.1. Parameters and Three Swarm Configurations We consider three swarm configurations: small (ํ=10), tactical (ํ=30), and large (ํ=100). Common parameters (same for all three configurations): Parameter Notation Value Justification Hebbian rate ํ 1 10 โ3 Typical for diff. plasticity [10] MARL rate ํ 2 10 โ4 Standard for MAPPO [8] Meta rate ํ 3 10 โ5 Standard for MAML [2] L1 period ํ 1 20 ms 50 Hz tick rate L2 period ํ 2 2 s Every 100 ticks L3 period ํ 3 20 s Every 1000 ticks Embedding Lipschitz ํฟ ํ 5.0 Typical for GNN [19] Policy Lipschitz ํฟ ํ 3.0 Estimate for MAPPO [8] Stabilization ํฟ โ0.01 Moderate weight decay Hebbian params |ํผ|,|ํฝ|,|ํพ| 0.5, 0.1, 0.1 Typical [10] Modulation ํ(ํ max ) 1.5 Moderate amplification Discount ํพ 0.99 Standard for RL ํ max โ 1.0 Normalized reward ํฟ GNN โ 4.0 3-layer GNN ํ GNN โ 0.05 Contract GNN-C1 ฮ NP (clamping) โ 10 โ4 Contract NP-C1 ํฟ ํโํป โ 2.0 Lipschitz metaโHebb ํฟ ํปโํค โ 1.0 Lipschitz Hebbโweights ํบ max โ 1.0 Bounded meta-gradient Configuration-dependent parameters: Parameter ํ=10 ํ=30 ํ=100 โ ํ 3.16 5.48 10.0 Parameter ํ=10 ํ=30 ํ=100 ํป eff =min(100,ํ 3 /ํ 2 ,ํป mission ) 10 10 10 Typical communication graph diameter 3 5 8 5.2. Base Quantities We determine quantities common to all three configurations โ they describe behavior of an individual agent and do not depend on swarm size. Weight boundedness (Lemma 1): ํ 0 = 0.5+0.1+0.1 0.01 =70.0, ํ max =ํ 0 +1=71.0 Single-step weight drift โ intrinsic bound (Lemma 2): ฮ ฬ 1 int =10 โ3 โ 1.5โ (0.7+0.01โ 71)=10 โ3 โ 1.5โ 1.41=2.115ร10 โ3 Effective step with clamping (10a): ฮ ฬ 1 =min(ฮ ฬ 1 int ,ฮ NP )=min(2.1ร10 โ3 ,10 โ4 )=10 โ4 Clamping reduces the step by a factor of 21 โ contract (C1) substantially strengthens the intrinsic bound, and it is the effective step ฮ ฬ 1 that enters all subsequent estimates. Number of Hebbian steps per MARL cycle: ํ 12 =ํ 2 /ํ 1 =2 s/0.02 s=100 Embedding drift per MARL cycle (Theorem 2): ฮฆ max =5.0โ 100โ 10 โ4 =0.05 Suboptimality from plasticity (15): ํ Hebb = 2โ 3.0โ 5.0โ 10 โ4 โ 1.0 0.01 =0.30 5.3. Comparison of Three Configurations Having base quantities, we compute full bounds for each configuration. The main question: how does suboptimality scale with swarm size ํ? Coordination suboptimality (16): For ํ=10: ํ coord (10) =2โ 10โ 3.0โ (4.0โ 3.16โ 0.05+0.05)โ 1.0=60โ 0.682=40.9 For ํ=30: ํ coord (30) =2โ 10โ 3.0โ (4.0โ 5.48โ 0.05+0.05)โ 1.0=60โ 1.146=68.8 For ํ=100: ํ coord (100) =2โ 10โ 3.0โ (4.0โ 10.0โ 0.05+0.05)โ 1.0=60โ 2.05=123.0 Meta-learning suboptimality (17): ํ meta =2โ 10โ 3.0โ 5.0โ 1.0โ 2.0โ 10 โ5 โ 1.0โ 20 0.02 โ 1.0=6.0 This component does not depend on ํ (meta-learning is global). Summary bounds table: Component ํ=10 ํ=30 ํ=100 Dependence on ํ ํ Hebb 0.30 0.30 0.30 ํ(1) ํ coord 40.9 68.8 123.0 ํ( โ ํ) ํ meta 6.0 6.0 6.0 ํ(1) ํ ฬ total 47.2 75.1 129.3 ํ( โ ํ) Share of ํ coord in ํ ฬ 86.7% 91.6% 95.1% Increasing Analysis. The coordination component ํ coord dominates and grows as ํ( โ ํ) โ a consequence of GNN aggregation where each agent collects messages from ํ(ํ) neighbors. For large swarms (ํโฅ100) this motivates: limiting the number of GNN neighbors, hierarchical decomposition of coordination, and reducing ฮฆ max through smaller ฮ NP . Normalized interpretation. If ํฝ โ โํป eff โ ํโ ํ max , then relative suboptimality: ํ ํ ฬ ํฝ โ ํ ฬ /ํฝ โ 10 47.2 100 47.2% 30 75.1 300 25.0% 100 129.3 1000 12.9% Relative suboptimality decreases with growing ํ: a larger swarm compensates drift through statistical averaging. 5.4. Sensitivity Analysis For system design it is necessary to understand which parameters have the largest impact on bounds. We vary each influential parameter by a factor of two (up and down) and compute elasticity โ relative change of ํ ฬ per relative change of parameter. Parameter Base ํ ฬ ร2 ร0.5 Elasticity ฮ NP (clamping) 75.1 143.9 41.3 ~0.92 ํป eff (horizon) 75.1 143.9 41.3 ~0.92 ํฟ ํ (Lip. 75.1 143.9 41.3 ~0.92 Parameter Base ํ ฬ ร2 ร0.5 Elasticity embed.) ํฟ ํ (Lip. policy) 75.1 150.2 37.6 ~1.0 ํ (swarm size) 75.1 97.2 55.6 ~0.46 ํ 1 (with ฮ NP fixed) 75.1 75.1 75.1 0 (clamping) ํ 3 (meta rate) 75.1 81.1 72.1 ~0.08 The system is most sensitive to ฮ NP , ํป eff , ํฟ ํ , and ํฟ ํ (elasticity โ1). Meta rate ํ 3 has minimal impact. With fixed clamping ฮ NP , changing ํ 1 does not affect the bound. Design recommendations: 1. Keep ฮ NP as small as possible (dominant parameter). 2. Reduce effective horizon through frequent meta-updates (ํ 3 โ). 3. Apply spectral normalization for controlling ํฟ ํ and ํฟ ํ . 4. For large swarms โ limit the number of GNN neighbors. 5.5. Illustration of Unbounded Growth Under Condition Violation The theorems of Section 4 establish sufficient boundedness conditions. Are these conditions truly necessary? To answer, we construct three counterexamples โ one for each critical condition โ and show that violating any of them leads to qualitative deterioration of bounds. Counterexample 1: ํฟ=0 (violation of S1). Without stabilization, weights grow unboundedly: โฅํค ํ (ํก)โฅโโฅํค ํ (0)โฅ+ํ 1 (|ํผ|+|ํฝ|+|ํพ|)โ ํก/ํ 1 At 100 s: โฅํค ํ โฅโ3.5; at 1000 s: โฅํค ํ โฅโ35; at 10,000 s: โฅํค ํ โฅโ350. Correspondingly ฮ ฬ 1 โโ, ฮฆ max โโ, ํ ฬ โโ. ํก (s) โฅํค ํ (ํก)โฅ with ํฟ=โ0.01 โฅํค ํ (ํก)โฅ with ํฟ=0 0 0 0 100 โค70.0 (bounded) 3.5 1000 โค70.0 35.0 10000 โค70.0 350.0 Counterexample 2: No clamping (violation of C1). With ฮ ฬ 1 =ฮ ฬ 1 int =2.1ร10 โ3 instead of 10 โ4 : ฮฆ max no clamp =5.0โ 100โ 2.1ร10 โ3 =1.05 (vs 0.05 with clamping) ํ ฬ no clamp โ21โ ํ ฬ clamped โ1577 (for ํ=30) The system formally has bounded weights (ํฟ<0), but suboptimality is unacceptably large. Counterexample 3: ํ 2 /ํ 1 =1000 instead of 100 (violation of S2). If MARL updates infrequently (ํ 2 =20 s instead of 2 s): ฮฆ max slow MARL =5.0โ 1000โ 10 โ4 =0.5 (vs 0.05) Suboptimality grows proportionally: ํ ฬ โ10รํ ฬ base . Counterexamples summary. Each of conditions (S1), (C1), (S2) is necessary for achieving acceptable bounds. Violation of any leads to qualitative deterioration: from moderate suboptimality to unbounded error growth. 6. Discussion In this section we discuss four aspects of the obtained results: relation to existing results (Subsection 6.1), practical significance for design (Subsection 6.2), limitations of our approach (Subsection 6.3), and the place of this work in a research series (Subsection 6.4). 6.1. Relation to Existing Results Our results uses ideas from timescale separation [3], but with fundamentally different assumptions: three timescales instead of two, heterogeneous learning mechanisms, and cascading influence through representation changes that has no analogue in classical stochastic approximation. Theorem 2 (Bounded Representation Drift) is, to the best of our knowledge, the first quantitative estimate of how Hebbian updates impact the coordination level in a MAS. Previous work on non- stationary MARL [9] considered external non-stationarity, whereas our result describes internal non-stationarity generated by the agentsโ own learning. Theorem 3 (Meta-Level Compatibility) complements MAML convergence results [13, 14] with a new perspective: instead of outer loop convergence we ask whether lower-level contracts are preserved under meta-adaptation. This is a different question requiring different tools โ specifically, the contract robustness margin. It is important to emphasize that we do not claim a complete stability theory for tri-hierarchical systems in the sense of dynamical systems theory. Our results are a set of quantitative bounds on drift, compatibility, and error accumulation. A complete stability theory would require, among other things, analysis of coupled system attractors, characterization of basins of attraction, and investigation of response to large perturbations โ all of which remains subject of future research. 6.2. Practical Significance Beyond the technical contribution, the results have direct significance for engineers designing multi- agent systems with multiple learning mechanisms. 1. Learning rate selection: inequality (20) gives an explicit upper bound for ํ 1 as a function of architectural parameters. Corollary 3.1 gives an analogous constraint for ํ 3 . 2. Contracts as runtime monitors: conditions (S1)โ(S5) can be verified in real time. Violation of any condition is a signal about potential departure from admissible bounds. 3. Contract-preserving adaptation: Theorem 3 gives a constructive criterion: a meta-update is admissible if its step is smaller than the minimum contract robustness margin. 6.3. Limitations Any set of quantitative estimates rests on assumptions, and honest analysis of their limitations is necessary. 1. Lipschitz continuity. Assumptions A1โA2 require Lipschitz continuity of ํ and ํ. For GNNs with bounded number of layers and smooth activation functions this holds. For deep networks with ReLU โ requires spectral normalization. Relaxing this assumption is a direction for future work. 2. Conservativeness of bounds. Our estimates are worst-case. In practice Hebbian updates are correlated (not adversarial), and actual drift is significantly smaller than the theoretical bound. Tighter bounds can be obtained with additional assumptions on sensory experience distribution. 3. Effective horizon as heuristic. Definition ํป eff (16a) is a practical truncation heuristic, not a rigorously derived object. In particular, the claim that meta-update โresetsโ the accumulated effect requires separate proof that each meta-cycle brings the system closer to the admissible set. This remains an open question. 4. Reference-MDP bounding (Assumption A8). The estimate of representation drift impact on coordination quality relies on Assumption A8, which postulates the existence of a reference MDP for upper-bounding. A more rigorous analysis would require fixed latent MDP formalization or explicit non-stationary PDL extension. 5. Communication delays. The framework does not account for message transmission delays between agents, which can increase effective embedding drift. 6. Adversarial perturbations. We assume non-stationarity is generated exclusively by learning, not by adversarial actions (EW, spoofing). 6.4. Relation to Research Series This paper is the second in a series of works forming a technical toolkit for analysis of adaptive swarm systems. Paper 1 (Hebbian-driven specialization) studies the Level 1 mechanism in isolation โ how local plasticity gives rise to functional roles. Our paper establishes that the coupled dynamics of three levels remain bounded and compatible. Paper 3 (Measuring swarm meta- cognition) will propose metrics for assessing the quality of meta-cognitive cycle โ using, in particular, our bounds as baselines. Paper 4 (Contract-based degradation and recovery) will extend the contract approach to off-nominal regimes where contracts may be violated โ and where the robustness margin (Definition 9) will become a diagnostic tool. 7. Conclusions In this paper we established four quantitative results concerning coupled learning dynamics in a tri- hierarchical swarm system. 1. Theorem 1 (Bounded Total Error) proves that under five contractual conditions, the total suboptimality admits a component-wise conservative upper bound, uniform in time. These conditions are formalized as verifiable contracts. 2. Theorem 2 (Bounded Representation Drift) provides a conservative worst-case upper bound on the impact of Level 1 Hebbian updates on Level 2 embeddings, from which a constraint on the admissible Hebbian learning rate follows. 3. Theorem 3 (Meta-Level Compatibility) establishes that meta-adaptation preserves lower- level contracts provided the meta-update step is smaller than the minimum contract robustness margin. 4. Theorem 4 (Non-Accumulation in the Sense of Uniform Boundedness) guarantees that error does not accumulate unboundedly over time, although bounded oscillations and regime wandering are not excluded. Numerical illustrations for three swarm configurations (ํ=10,30,100) confirm the realism of bounds and demonstrates necessity of each condition: violation of even one of them (e.g., ํฟ=0 or absence of clamping) leads to unbounded error growth. These results are technical tools, not a complete theory. They allows verifying whether a given configuration of a tri-hierarchical system will remain within admissible bounds and provide constructive design recommendations. A complete stability theory โ with analysis of attractors, basins of attraction, and response to large perturbations โ remains an open problem. Directions for future research include: relaxation of Lipschitz condition, accounting for communication delays, extension to adversarial perturbations, and experimental validation in simulation environment. The English text was corrected and edited using AI in accordance with the KNU Regulation "On the Use of Artificial Intelligence" References [1] D. O. Hebb, The Organization of Behavior: A Neuropsychological Theory. New York: Wiley, 1949. [2] C. Finn, P. Abbeel, and S. Levine, โModel-agnostic meta-learning for fast adaptation of deep networks,โ in Proc. ICML, 2017, p. 1126โ1135. [3] V. S. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, 2008. [4] V. R. Konda and J. N. Tsitsiklis, โOn actor-critic algorithms,โ SIAM J. Control Optim., vol. 42, no. 4, p. 1143โ1166, 2003. [5] M. Al-Shedivat, T. Bansal, Y. Burda, I. Sutskever, I. Mordatch, and P. Abbeel, โContinuous adaptation via meta-learning in nonstationary and competitive environments,โ in Proc. ICLR, 2018. [6] K. Zhang, Z. Yang, and T. Baลar, โMulti-agent reinforcement learning: A selective overview of theories and algorithms,โ in Handbook of Reinforcement Learning and Control, Springer, 2021, p. 321โ384. [7] T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson, โMonotonic value function factorisation for deep multi-agent reinforcement learning,โ J. Mach. Learn. Res., vol. 21, no. 178, p. 1โ51, 2020. [8] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, โThe surprising effectiveness of PPO in cooperative multi-agent games,โ in Proc. NeurIPS, 2022, p. 24611โ24624. [9] P. Hernandez-Leal, M. Kaisers, T. Baarslag, and E. M. de Cote, โA survey of learning in multiagent environments: Dealing with non-stationarity,โ arXiv:1707.09183, 2017. [10] T. Miconi, J. Clune, and K. O. Stanley, โDifferentiable plasticity: Training plastic neural networks with backpropagation,โ in Proc. ICML, 2018, p. 3559โ3568. [11] E. Najarro and S. Risi, โMeta-learning through Hebbian plasticity in random networks,โ in Proc. NeurIPS, 2020, p. 20719โ20731. [12] J. K. Pedersen and S. Risi, โEvolving and merging Hebbian learning rules: Increasing generalization by decreasing the number of rules,โ in Proc. GECCO, 2021, p. 892โ900. [13] A. Fallah, A. Mokhtari, and A. Ozdaglar, โOn the convergence theory of gradient-based model- agnostic meta-learning algorithms,โ in Proc. AISTATS, 2020, p. 1082โ1092. [14] K. Ji, J. Yang, and Y. Liang, โMulti-step model-agnostic meta-learning: Convergence and improved algorithms,โ arXiv:2002.07836, 2020. [15] B. Meyer, โApplying Design by Contract,โ IEEE Computer, vol. 25, no. 10, p. 40โ51, 1992. [16] A. Sangiovanni-Vincentelli, W. Damm, and R. Passerone, โTaming Dr. Frankenstein: Contract- based design for cyber-physical systems,โ Eur. J. Control, vol. 18, no. 3, p. 217โ238, 2012. [17] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, โProximal policy optimization algorithms,โ arXiv:1707.06347, 2017. [18] S. Kakade and J. Langford, โApproximately optimal approximate reinforcement learning,โ in Proc. ICML, 2002, p. 267โ274. [19] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, โHow powerful are graph neural networks?,โ in Proc. ICLR, 2019.