Paper deep dive
RLShield: Practical Multi-Agent RL for Financial Cyber Defense with Attack-Surface MDPs and Real-Time Response Orchestration
Srikumar Nayak
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/20/2026, 8:24:41 AM
Summary
The paper introduces RLShield, a multi-agent reinforcement learning framework for financial cyber defense. It models the enterprise attack surface as a Markov Decision Process (MDP) where states represent alerts, asset exposure, and service health, while actions represent real-time response steps like isolating hosts or rotating credentials. RLShield employs a centralized training with distributed execution (CTDE) approach to learn coordinated policies across multiple agents, optimizing for containment speed, business disruption, and response cost. Experiments on the CIC-IDS2017 dataset demonstrate that RLShield outperforms static rule baselines and single-agent RL methods in reducing time-to-containment and residual exposure while maintaining operational safety.
Entities (10)
Relation Signals (8)
RLShield → trainedon → CIC-IDS2017
confidence 97% · We use the CIC-IDS2017 benchmark... to align with operational response decisions
RLShield → employs → Multi-Agent Reinforcement Learning
confidence 96% · RLShield learns coordinated policies across multiple agents
RLShield → uses → Markov Decision Process
confidence 95% · We model the enterprise attack surface as a Markov decision process (MDP)
RLShield → usestrainingparadigm → Centralized Training with Distributed Execution
confidence 94% · We adopt centralized training with distributed execution so agents can coordinate during learning but act locally at runtime.
RLShield → optimizesfor → Time-to-Containment
confidence 93% · RLShield reduces time-to-containment and residual exposure
RLShield → outperforms → Static-Playbook
confidence 92% · outperforming static rule baselines and single-agent RL under the same constraints.
RLShield → outperforms → PPO
confidence 90% · Experiments show that RLShield reduces time-to-containment... outperforming... single-agent RL
RLShield → outperforms → QMIX
confidence 90% · Table I shows RLShield has lower ASR and TTD than QMIX.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Financial systems run nonstop and must stay reliable even during cyber incidents. Modern attacks move across many services (apps, APIs, identity, payment rails), so defenders must make a sequence of actions under time pressure. Most security tools still use fixed rules or static playbooks, which can be slow to adapt when the attacker changes behavior. Reinforcement learning (RL) is a good fit for sequential decisions, but much of the RL-in-finance literature targets trading and does not model real cyber response limits such as action cost, service disruption, and defender coordination across many assets. This paper proposes RLShield, a practical multi-agent RL pipeline for financial cyber defense. We model the enterprise attack surface as a Markov decision process (MDP) where states summarize alerts, asset exposure, and service health, and actions represent real response steps (e.g., isolate a host, rotate credentials, ratelimit an API, block an account, or trigger recovery). RLShield learns coordinated policies across multiple agents (assets or service groups) and optimizes a risk-sensitive objective that balances containment speed, business disruption, and response cost. We also include a game-aware evaluation that tests policies against adaptive attackers and reports operational outcomes, not only reward. Experiments show that RLShield reduces time-to-containment and residual exposure while keeping disruption within a fixed response budget, outperforming static rule baselines and single-agent RL under the same constraints. These results suggest that multi-agent, cost-aware RL can provide a deployable layer for automated response in financial security operations.
Tags
Links
- Source: https://arxiv.org/abs/2603.00186v1
- Canonical: https://arxiv.org/abs/2603.00186v1
Trouble viewing inline? Open PDF directly →
Full Text
32,029 characters extracted from source content.
Expand or collapse full text
RLShield: Practical Multi-Agent RL for Financial Cyber Defense with Attack-Surface MDPs and Real-Time Response Orchestration Srikumar Nayak Abstract Financial systems run nonstop and must stay reliable even during cyber incidents. Modern attacks move across many services (apps, APIs, identity, payment rails), so defenders must make a sequence of actions under time pressure. Most security tools still use fixed rules or static playbooks, which can be slow to adapt when the attacker changes behavior. Reinforcement learning (RL) is a good fit for sequential decisions, but much of the RL-in-finance literature targets trading and does not model real cyber response limits such as action cost, service disruption, and defender coordination across many assets. This paper proposes RLShield, a practical multi-agent RL pipeline for financial cyber defense. We model the enterprise attack surface as a Markov decision process (MDP) where states summarize alerts, asset exposure, and service health, and actions represent real response steps (e.g., isolate a host, rotate credentials, rate-limit an API, block an account, or trigger recovery). RLShield learns coordinated policies across multiple agents (assets or service groups) and optimizes a risk-sensitive objective that balances containment speed, business disruption, and response cost. We also include a game-aware evaluation that tests policies against adaptive attackers and reports operational outcomes, not only reward. Experiments show that RLShield reduces time-to-containment and residual exposure while keeping disruption within a fixed response budget, outperforming static rule baselines and single-agent RL under the same constraints. These results suggest that multi-agent, cost-aware RL can provide a deployable layer for automated response in financial security operations. I Introduction Financial organizations run large, always-on systems where uptime and trust directly affect customers and revenue. At the same time, the attack surface keeps expanding due to cloud services, APIs, third-party links, remote access, and fast digital payments [3]. This makes defense a sequential decision problem: teams must detect, contain, and recover while trading off security impact (blocking the attacker) against business impact (service disruption and false actions). Standard security controls help, but they often rely on fixed rules and playbooks that do not adapt well when attackers change tactics or when the system state is uncertain [12]. Reinforcement learning (RL) is a natural fit for such sequential decisions, and recent surveys show strong progress of RL in finance, including benchmark environments, risk-aware learning, and multi-agent settings [7, 8]. However, most RL-in-finance work is focused on trading and portfolio outcomes, where the environment is a market simulator and actions are buy/sell/allocate [10]. These settings do not directly capture cyber defense constraints such as limited response budgets, action latency, safety requirements, and attacker adaptation [13]. As a result, there is still a gap between RL methods and what a security operations center (SOC) can deploy as a reliable response engine. To address this gap, we introduce RLShield, which models the financial attack surface as an MDP and learns real-time defense policies that can be executed as response workflows. RLShield uses multi-agent learning to coordinate defense actions across assets and services, while optimizing security and operational cost objectives. Our research contributions are as follows: 1. We formalize financial cyber defense as an attack-surface MDP with operationally meaningful states (alerts, asset exposure, service health) and actions (containment, isolation, credential controls, rate-limiting, and recovery). 2. We design RLShield as a multi-agent defender that coordinates decisions across multiple assets/services, instead of learning a single global policy that is hard to scale. 3. We include risk-sensitive and cost-aware objectives so the policy reduces breach impact while controlling disruption and false-response rates, aligning training with SOC metrics. 4. We add a game-aware evaluation protocol that tests adaptive attackers and reports outcomes beyond reward, such as time-to-containment, residual exposure, and response cost. 5. We provide a deployable orchestration interface that converts learned actions into ordered response steps suitable for near-real-time execution and audit. The structure of this paper is as follows: Section I reviews related work; Section I describes the dataset and preprocessing with the proposed method and training objective; Section IV reports experimental results and analysis; and Section V concludes the paper with future research directions. I Related Work Prior studies have described the cyber threat landscape in banking and financial services, covering common attack classes, impacts, and standard countermeasures (e.g., authentication, monitoring, incident response, and governance) [4]. This line of work is valuable for defining what defenders must handle in practice, but it is mostly descriptive: it does not specify an executable decision process that can select, sequence, and time actions under limited resources and evolving attacker behavior. In operational settings, however, defenders must act under uncertainty, balance service availability with containment, and justify why a response action was taken at a specific time. These requirements motivate learning-based defense policies that can adapt online while still respecting safety and cost constraints. In parallel, reinforcement learning has been widely used in finance for portfolio and trading decisions, including deep Q-learning variants and actor–critic methods, often with a focus on sample efficiency, market dynamics, and reward shaping [1, 9]. Multi-agent reinforcement learning (MARL) has also been explored for financial decision making where multiple agents coordinate or compete, and risk-sensitive learning has been studied to control tail-risk and variability of outcomes [6, 11]. Related work also includes federated reinforcement learning for trading execution to address information security concerns in distributed settings [5]. While these works show that RL/MARL can optimize sequential decisions under uncertainty, their threat models and action spaces are typically market-centric; they do not map directly to cyber defense operations such as containment, credential resets, segmentation, or playbook execution. Overall, the gap is that banking cyber defense research [4] and financial RL research [1, 9, 6, 11, 5] have largely progressed in parallel, with limited work that turns real attack-surface structure into a defense MDP and learns coordinated defender policies that are cost-aware, risk-sensitive, and suitable for real-time orchestration. RLShield fills this gap by modeling the financial attack surface as an MDP with explicit state features and response actions, training a multi-agent defender with game-aware objectives to handle adaptive attackers, and producing deployable response sequences that align with operational metrics (e.g., containment time, disruption cost, and false-action rate) rather than only average reward. I Methodology I-A Dataset and Preprocessing We use the CIC-IDS2017 benchmark released by the Canadian Institute for Cybersecurity (CIC), which provides flow-level network traffic records with labeled benign and attack activities (e.g., DoS/DDoS, brute force, web attacks, botnet) [2]. Each record corresponds to a bidirectional network flow (5-tuple based aggregation) and includes numeric statistics such as packet counts, byte rates, inter-arrival times, and flag-based indicators. We treat each flow as one sample with a feature vector i∈ℝdx_i ^d, a discrete label yi∈0,1y_i∈\0,1\ (benign vs. attack for the RL defense setting), and an event time tit_i derived from the flow start timestamp. The full dataset is denoted as =(i,yi,ti)i=1N.D= \(x_i,y_i,t_i) \_i=1^N. (1) Eq. (1) fixes the learning interface used throughout the paper: ix_i is the observable state signal, yiy_i is the ground-truth outcome used for offline evaluation, and tit_i is used to construct realistic time-ordered splits. To avoid overly optimistic results from random shuffling, we split the dataset in chronological order. Let ttrt_tr and tvat_va be two cut points chosen on the timeline. We define train _train =i:ti≤ttr, =\i:\ t_i≤ t_tr\, (2) val _val =i:ttr<ti≤tva, =\i:\ t_tr<t_i≤ t_va\, test _test =i:ti>tva. =\i:\ t_i>t_va\. Eq. (2) ensures that validation and test samples appear in the future relative to training, which better matches real deployment where the attack mix and traffic load can drift over time. All preprocessing parameters are fitted on trainD_train only and then reused for valD_val and testD_test to prevent information leakage. CIC-IDS2017 flow exports may contain missing values (NaN), infinite values (Inf), or constant columns depending on the capture day and tool settings. We first replace Inf with NaN, then impute numeric features using training medians. For feature j, the imputation rule is x~ij=xij,xijis finite,mj,otherwise,mj=mediank∈train(xkj). x_ij= casesx_ij,&x_ij\ is finite,\\ m_j,&otherwise, cases m_j=median_k _train(x_kj). (3) Eq. (3) keeps the preprocessing stable under outliers and avoids using future data, since mjm_j is computed only on trainD_train. We also remove direct identifiers (e.g., flow ID strings) and fields that trivially encode ground truth or leak scenario construction. In practice, we drop raw IP addresses, ports, and exact timestamps when training the learning components, and we keep only behavior-driven flow statistics. Several flow statistics (bytes, packet counts, rates) are heavy-tailed. To reduce scale dominance and improve learning stability, we apply a log transform to nonnegative features in a selected set +J_+: xij(log)=log(1+x~ij),j∈+∧x~ij≥0,x~ij,otherwise.x^( )_ij= cases \! (1+ x_ij ),&j _+\ \ x_ij≥ 0,\\ x_ij,&otherwise. cases (4) Eq. (4) compresses extreme values while preserving ordering, which helps both supervised baselines and the RL policy/value networks. We standardize continuous features using training statistics so gradients and exploration noise behave consistently across dimensions. For each feature j, we compute the mean and standard deviation on trainD_train as μj=1|train|∑k∈trainxkj(log), _j= 1|D_train| _k _trainx^( )_kj, (5) σj=1|train|∑k∈train(xkj(log)−μj)2. _j= 1|D_train| _k _train (x^( )_kj- _j )^2. (6) Using Eq. (5), the standardized feature is x^ij=xij(log)−μjσj+ϵ0, x_ij= x^( )_ij- _j _j+ _0, (7) where ϵ0>0 _0>0 is a small constant for numerical stability. Eq. (7) is applied to all samples in trainD_train, valD_val, and testD_test using the same μj,σj\ _j, _j\ computed from trainD_train. For the cyber defense task, we map multi-class attack labels to a binary label (benign vs. attack) to align with operational response decisions (block/allow, step-up verification, rate-limit). If the dataset provides a categorical label ℓi∈ _i , we set yi=(ℓi≠BENIGN),y_i=I\! ( _i≠ BENIGN ), (8) where (⋅)I(·) is the indicator function. Eq. (8) supports unified reporting and makes reward design consistent in later RL sections. Since attacks are typically rarer than benign flows, we compute a training-only class weight for supervised components (e.g., behavior model, auxiliary detectors) as w+=|i∈train:yi=0||i∈train:yi=1|.w_+= |\i _train:y_i=0\||\i _train:y_i=1\|. (9) Eq. (9) increases the penalty for missed attacks during learning, which improves sensitivity at low false-positive operating points. After applying Eqs. (3)–(7), each sample is represented by the standardized feature vector ^i∈ℝd x_i ^d. We use ^i x_i as the observation input for RLShield agents and as the shared input for classical baselines, ensuring that comparisons reflect modeling differences rather than inconsistent preprocessing. I-B Proposed Method: RLShield (Multi-Agent RL for Financial Cyber Defense) SOC Logs & Telemetry (alerts, flows, auth, EDR) Preprocess Normalize & Window Attack-Surface State Builder sts_t Attack-Surface MDP (,,P,R)(S,A,P,R) Multi-Agent RL CTDE Training Orchestrated Response ata_t Attacker Model (scripts / adaptive) Game Signal min–max / regret Safety Layer constraints & cooldown Audit Trail action, reason, cost Real-Time Inference per-asset agents Figure 1: RLShield pipeline: raw SOC telemetry is converted into an attack-surface state sts_t and an MDP, multi-agent policies are trained with centralized training and distributed execution (CTDE) under attacker simulation and game-aware signals, and a safety layer gates actions before real-time response orchestration with audit logging. We propose RLShield, a practical multi-agent reinforcement learning (MARL) pipeline that learns real-time cyber defense policies for financial systems (Figure 1). RLShield models the defender–attacker interaction as an attack-surface MDP over networked assets (hosts, services, accounts, and security controls). Multiple defender agents coordinate actions such as isolate host, block IP, reset credentials, throttle service, and deploy rule updates, while a simulated attacker agent executes lateral movement, privilege escalation, and data exfiltration. The key idea is to learn policies that (i) reduce attack success probability, (i) minimize response time, and (i) control business disruption cost, under realistic partial observability and delayed alerts. Input: Security event stream t\o_t\, asset graph =(,ℰ)G=(V,E), action sets ii=1M\A_i\_i=1^M, episode horizon H, discount γ, policy parameters θii=1M\ _i\_i=1^M, central critic parameters ϕφ, entropy weight β, game-regularizer weight λ, reward weights ws,wc,wdw_s,w_c,w_d Output: Defender policies πθii=1M\ _ _i\_i=1^M and an orchestration rule for deployment Initialize defender policies πθi _ _i and critic QϕQ_φ for each training episode e=1,2,…e=1,2,… do Reset environment; build initial belief state 0b_0 for t←0t← 0 to H−1H-1 do /* (1) Observe alerts and update belief */ Receive observation to_t; update belief t←BeliefUpdate(t−1,t)b_t (b_t-1,o_t) /* (2) Defender agents act (distributed) */ foreach agent i∈1,…,Mi∈\1,…,M\ do Sample action ai,t∼πθi(⋅∣t)a_i,t _ _i(· _t) Execute joint action t=(a1,t,…,aM,t)a_t=(a_1,t,…,a_M,t) /* (3) Attacker step + environment transition */ Sample attacker action atatka^atk_t; transition to next state st+1s_t+1 /* (4) Compute reward from security + cost + disruption */ Compute reward rtr_t using Eq. (11) Store (t,t,rt,t+1)(b_t,a_t,r_t,b_t+1) /* (5) Update critic and policies (centralized training) */ Update QϕQ_φ by TD loss in Eq. (12) Update each πθi _ _i by policy objective in Eq. (13) 1exDeploy. Use learned policies to map live alerts to actions; apply a safety gate to prevent high-disruption actions unless risk exceeds threshold return πθii=1M\ _ _i\_i=1^M Algorithm 1 RLShield: Multi-Agent Defense Learning with Attack-Surface MDP and Real-Time Orchestration (1) Attack-surface MDP and belief state: We represent the security environment as a Markov decision process (MDP) with hidden state sts_t (true compromise status) and observation to_t (alerts, logs, IDS outputs). Since defenders do not fully observe the attacker, RLShield uses a belief state tb_t as a compact summary of recent evidence. A practical update form is t=GRU(t−1,ψ(t)),b_t=GRU\! (b_t-1,\,ψ(o_t) ), (10) where ψ(⋅)ψ(·) encodes alerts into features and the GRU keeps memory of delayed and noisy signals. Eq. (10) is used to handle partial observability without requiring full system logs at each step. (2) Reward shaping for financial cyber defense: The reward balances three goals: stop attacks, reduce operational cost, and avoid service disruption. We define rt=ws⋅ΔSec(st,st+1)−wc⋅Cost(t)−wd⋅Disrupt(t),r_t=w_s· (s_t,s_t+1)-w_c·Cost(a_t)-w_d·Disrupt(a_t), (11) where ΔSec measures security improvement (e.g., reduced compromised nodes or blocked attack paths), CostCost is response cost (e.g., analyst effort, compute, rule-deploy overhead), and DisruptDisrupt penalizes business impact (e.g., blocking critical services). Eq. (11) is practical because each term can be measured from the simulator logs and mapped to real SOC KPIs. (3) Centralized training with distributed execution: We adopt centralized training with distributed execution so agents can coordinate during learning but act locally at runtime. The critic is trained with a TD objective: ℒQ=[(Qϕ(t,t)−(rt+γQ¯(t+1,t+1)))2],L_Q=E [ (Q_φ(b_t,a_t)-(r_t+γ\, Q(b_t+1,a_t+1)) )^2 ], (12) where Q¯ Q is a slowly updated target critic for stability. Eq. (12) reduces training variance and supports long-horizon defense plans. Each agent policy is optimized using an entropy-regularized objective with a lightweight game-theoretic regularizer that discourages brittle single-point strategies: maxθi[Qϕ(t,t)+βℋ(πθi(⋅∣t))−λΩ(πθi)], _ _i\ E\! [Q_φ(b_t,a_t)+β\,H( _ _i(· _t))-λ\, ( _ _i) ], (13) where ℋ(⋅)H(·) encourages exploration and Ω(⋅) (·) penalizes overly deterministic action collapse (useful when attackers adapt). Eq. (13) improves policy robustness while keeping deployment simple (each agent only needs tb_t). At runtime, RLShield maps streaming alerts to a joint action ta_t and executes it through a response orchestrator (SOAR-like interface). To keep actions safe, we apply a simple gate: high-disruption actions (e.g., isolating a critical node) are allowed only if the predicted risk exceeds a threshold learned on validation episodes. This makes RLShield deployable in financial settings where false positives can be costly. TABLE I: Test performance on the attack-surface MDP. Mean± over S=5S=5 seeds. Best in bold; second best underlined. Lower is better for ASR/TTD/TTR/EL/DC. Method ASR↓ TTD (steps)↓ TTR (steps)↓ EL↓ DC↓ Prec@Budget↑ Static-Playbook [4] 0.392± 0.015 141± 10 98± 9 0.694± 0.020 0.412± 0.025 0.284± 0.014 Greedy-Risk [14] 0.331± 0.013 132± 9 86± 8 0.612± 0.018 0.365± 0.020 0.301± 0.013 DQN [1] 0.289± 0.012 124± 8 79± 7 0.562± 0.017 0.341± 0.019 0.318± 0.012 A2C [9] 0.271± 0.011 121± 8 77± 7 0.548± 0.016 0.330± 0.018 0.326± 0.011 PPO [5] 0.258± 0.010 118± 7 75± 7 0.534± 0.015 0.323± 0.017 0.334± 0.012 QMIX [11] 0.219± 0.010 112± 7 71± 6 0.492± 0.014 0.301± 0.016 0.356± 0.011 MADDPG [6] 0.226± 0.011 113± 7 72± 6 0.498± 0.015 0.305± 0.016 0.352± 0.012 RLShield (Ours) 0.181± 0.009 106± 6 67± 6 0.458± 0.013 0.279± 0.015 0.381± 0.010 TABLE I: Ablation on test episodes (mean over 5 seeds). Variant ASR↓ EL↓ DC↓ Prec↑ RLShield w/o centralized critic 0.236 0.504 0.302 0.352 RLShield w/o entropy (β=0β=0) 0.207 0.473 0.287 0.366 RLShield w/o game reg. (λ=0λ=0) 0.214 0.481 0.291 0.361 RLShield (Full) 0.181 0.458 0.279 0.381 IV Results All results are reported on the test split defined in Sec. I-A using the same attack-surface MDP settings (assets, services, attacker playbook, and alert noise). We train each method for a fixed environment budget of 2×1062× 10^6 steps and evaluate on Etest=300E_test=300 held-out episodes. We report mean± over S=5S=5 random seeds (policy initialization and environment stochasticity) while keeping the data split fixed. Hyperparameters are tuned on the validation episodes with the same trial budget (25 trials per method) and early stopping on validation risk. Baselines: We compare RLShield with seven practical baselines: (i) No-Response (monitor-only), (i) Static-Playbook (fixed rules: block known bad IPs, isolate on high-severity), (i) Greedy-Risk (myopic action that maximizes immediate risk drop), (iv) DQN (single-agent discrete control), (v) PPO (single-agent policy gradient), (vi) A2C (single-agent actor–critic), (vii) QMIX (multi-agent value mixing), (viii) MADDPG (multi-agent actor–critic, discrete via Gumbel-softmax). RLShield (Ours) uses the same action space but learns coordinated policies with centralized training and distributed execution (Sec. I-B). Metrics. Because financial cyber defense is cost- and time-sensitive, we report: (i) Attack Success Rate (ASR)↓ , (i) Mean Time-to-Detect (TTD)↓ , (i) Mean Time-to-Respond (TTR)↓ , (iv) Expected Loss (EL)↓ , (v) Disruption Cost (DC)↓ , (vi) Alert Precision (Prec)↑ at a fixed alert budget. ASR is the fraction of episodes where the attacker reaches an objective (e.g., exfiltration or privileged persistence). For E test episodes, ASR is computed as ASR=1E∑e=1E(GoalReached(e)),ASR= 1E _e=1^EI\! (GoalReached(e) ), (14) where (⋅)I(·) is the indicator. Eq. (14) is used as the main safety metric because it directly reflects whether the bank loses control in an episode. We measure expected loss with a simple episode-wise accounting that combines direct impact and response cost: EL=1E∑e=1E(Limpact(e)+Lops(e)),EL= 1E _e=1^E (L^(e)_impact+L^(e)_ops ), (15) where Limpact(e)L^(e)_impact is the simulated loss from attacker progress (e.g., stolen records, downtime) and Lops(e)L^(e)_ops is the operational cost of actions (e.g., isolations, credential resets). Eq. (15) matches how SOC teams compare policies: lower business loss with acceptable response effort. Disruption cost is computed from the executed actions: DC=1E∑e=1E∑t=0H−1Disrupt(t(e)),DC= 1E _e=1^E _t=0^H-1Disrupt\! (a^(e)_t ), (16) which penalizes service-impacting moves (e.g., isolating a critical node) more than low-impact actions (e.g., adding a detection rule). Eq. (16) is important because a defense that stops attacks but breaks the system is not usable. For alert precision at a fixed budget, we trigger at most BalertB_alert alerts per episode and report: Prec=TPTP+FP,Prec= TPTP+FP, (17) where TP/FP are true/false incident escalations decided by the policy under the same budget. Eq. (17) connects directly to analyst workload. Table I reports the primary test results. Static playbooks reduce ASR compared to no response, but they over-trigger high-disruption actions, which raises DC. Single-agent RL improves response timing, yet it often fails under multi-step attacker behavior because it cannot coordinate actions across assets. Multi-agent baselines (QMIX, MADDPG) reduce ASR further, but RLShield achieves the best overall balance: lowest ASR and EL with controlled disruption and better alert precision at the same budget. The gains are consistent across seeds. A common failure in security RL is overfitting to a fixed attacker. We therefore test under three attacker strengths: Basic (limited lateral movement), Skilled (multi-step with privilege escalation), and Adaptive (chooses actions that maximize defender confusion and delay). Figure 2 shows that playbooks degrade sharply under adaptive behavior, and single-agent RL also drops because it commits to narrow patterns. RLShield degrades more slowly because it learns coordinated responses and is trained with entropy and game-regularization (Eq. (13)). BasicSkilledAdaptive0.10.10.20.20.30.30.40.40.50.50.60.60.70.7ASR↓ -PlaybookDQNQMIXRLShield Figure 2: Attack success rate under increasing attacker strength. RLShield remains the most stable under adaptive behavior. For deployment, teams care about how much disruption is needed to achieve security. Figure 3 plots EL vs. DC for key methods. Playbooks reduce loss but at high disruption due to coarse triggers. RLShield shifts the Pareto front by reducing EL while keeping DC lower than other learned baselines, mainly by preferring low-impact actions early (e.g., throttling, targeted blocks) and escalating only when belief risk grows (Eq. (10)). 0.20.20.220.220.240.240.260.260.280.280.30.30.320.320.340.340.360.360.380.380.40.40.420.420.440.440.450.450.50.50.550.550.60.60.650.650.70.7Disruption cost (DC)↓ loss (EL)↓ -PlaybookDQNQMIXRLShield Figure 3: Security–disruption trade-off on the test episodes. RLShield achieves the lowest loss with lower disruption. IV-A Ablation Study We ablate two design choices: (A) centralized critic (Eq. (12)), (B) entropy regularization, and (C) game-regularizer term (Eq. (13)). Table I shows that removing the centralized critic increases ASR because agents stop coordinating, while removing entropy or the regularizer hurts adaptive robustness and precision (policies become too brittle and over-trigger). Across strict held-out episodes, multiple strong baselines, repeated seeds, and adaptive attacker testing, RLShield delivers lower attack success and lower expected loss while keeping disruption under control. The results support RLShield as a deployable defense learner: it reacts faster (lower TTD/TTR), produces higher-precision alerts under a fixed budget, and remains more stable when attacker behavior changes. V Conclusion V-A Conclusion This paper presented RLShield, a practical multi-agent reinforcement learning framework for financial cyber defense that operates directly on an attack-surface MDP and produces real-time response actions with controlled disruption. RLShield combines centralized training with distributed execution so agents can coordinate across assets while still acting locally during deployment. In the evaluation, we used a fixed environment budget, a strict held-out episode test set, and multiple baselines including playbook rules, single-agent RL, and multi-agent RL. Results show that RLShield reduces attack success rate and expected loss while also improving response speed and alert precision under the same alert budget. Importantly, RLShield remains more stable under stronger and adaptive attackers, indicating that its gains are not tied to a single scripted adversary. Overall, RLShield offers a deployable balance between security outcomes and operational cost, which is the main requirement in real SOC workflows. V-B Future Work We will extend RLShield with (i) constraint-aware actions (business hours, critical service limits) so policies stay valid in production, (i) stronger adaptive attacker models and red-team style stress tests, and (i) cost-aware tuning that directly targets analyst workload and SLA impact in addition to security metrics. References [1] M. Alidousti, M. K. Bafruei, and A. H. A. Sedigh (2025) A novel data-efficient double deep q-network framework for intelligent financial portfolio management. Engineering Applications of Artificial Intelligence 162, p. 112436. Cited by: §I, TABLE I. [2] Canadian Institute for Cybersecurity (CIC), University of New Brunswick (2017) Intrusion detection evaluation dataset (cic-ids2017). Note: Dataset webpageAccessed: 2026-01-07 External Links: Link Cited by: §I-A. [3] A. Charpentier, R. Elie, and C. Remlinger (2023) Reinforcement learning in economics and finance. Computational Economics 62 (1), p. 425–462. Cited by: §I. [4] A. A. Darem, A. A. Alhashmi, T. M. Alkhaldi, A. M. Alashjaee, S. M. Alanazi, and S. A. Ebad (2023) Cyber threats classifications and countermeasures in banking and financial sector. IEEe Access 11, p. 125138–125158. Cited by: §I, TABLE I. [5] H. Feng, Y. Wang, S. Zhong, T. Yuan, and Z. Quan (2025) Federated reinforcement learning in stock trading execution: the fppo algorithm for information security. IEEE Access. Cited by: §I, TABLE I. [6] Q. Guo and B. Li (2024) An intelligent financial investment decision model based on multi-agent reinforcement learning. Journal of Circuits, Systems and Computers 33 (15), p. 2450266. Cited by: §I, TABLE I. [7] B. Hambly, R. Xu, and H. Yang (2023) Recent advances in reinforcement learning in finance. Mathematical Finance 33 (3), p. 437–503. Cited by: §I. [8] B. Hirchoua, B. Ouhbi, and B. Frikh (2021) Deep reinforcement learning based trading agents: risk curiosity driven learning for financial rules-based policy. Expert Systems with Applications 170, p. 114553. Cited by: §I. [9] C. Kang, J. Woo, and J. W. Hong (2025) A2C reinforcement learning for cryptocurrency trading: analyzing the impact of price prediction accuracy and data granularity. International Journal of Network Management 35 (5), p. e70024. Cited by: §I, TABLE I. [10] X. Liu, Z. Xia, J. Rui, J. Gao, H. Yang, M. Zhu, C. Wang, Z. Wang, and J. Guo (2022) Finrl-meta: market environments and benchmarks for data-driven financial reinforcement learning. Advances in Neural Information Processing Systems 35, p. 1835–1849. Cited by: §I. [11] W. Qiu, X. Wang, R. Yu, R. Wang, X. He, B. An, S. Obraztsova, and Z. Rabinovich (2021) RMIX: learning risk-sensitive policies for cooperative reinforcement learning agents. Advances in Neural Information Processing Systems 34, p. 23049–23062. Cited by: §I, TABLE I. [12] S. K. Sahu, A. Mokhade, and N. D. Bokde (2023) An overview of machine learning, deep learning, and reinforcement learning-based techniques in quantitative finance: recent progress and challenges. Applied Sciences 13 (3), p. 1956. Cited by: §I. [13] A. Shavandi and M. Khedmati (2022) A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets. Expert Systems with Applications 208, p. 118124. Cited by: §I. [14] Y. Wang, Z. Anwer, L. Cheng, W. Chen, and Y. Miao (2024) Smart manager and investor effects in offshore financial development: the decisive role of cognitive ability on risk preference. Applied Economics 56 (14), p. 1652–1670. Cited by: TABLE I.