Paper deep dive
Learning Intrusion Response Strategies for OT Systems
Duc Huy Le, Rolf Stadler
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/12/2026, 2:16:04 AM
Summary
This paper presents a formal model for automated intrusion response in Operational Technology (OT) systems using a Partially Observable Markov Decision Process (POMDP). The authors propose two learning-based solution methods, kk-Obs-PPO and BF-PPO, derived from Proximal Policy Optimization (PPO), to handle the large state space and partial observability based on network traffic measurements. The approach is evaluated on an emulated OT system, demonstrating effectiveness against various MITRE ATT&CK attacks.
Entities (8)
Relation Signals (6)
POMDP → models → OT Systems
confidence 98% · We present a formal model of an OT intrusion response use case using the POMDP framework.
kk-Obs-PPO → uses → PPO
confidence 95% · The first method, kk-Obs-PPO, simplifies the structure of the input for the defender strategy, while the second method, BF-PPO, uses particle filter... based on Proximal Policy Optimization (PPO)
BF-PPO → uses → Particle Filter
confidence 95% · the second method, BF-PPO, uses particle filter [26] to estimate the system state.
BF-PPO → uses → PPO
confidence 95% · we propose two learning methods, which are based on Proximal Policy Optimization (PPO)... BF-PPO
MITRE ATT&CK → categorizes → OT Attacks
confidence 90% · We consider attacks from the MITRE ATT&CK framework [36]
kk-Obs-PPO → simplifies → Input Structure
confidence 90% · kk-Obs-PPO, simplifies the structure of the input for the defender strategy
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Cyberattacks against Operational Technology (OT) systems, which monitor and control industrial processes, pose an increasing threat to essential societal services. For this reason, developing automated intrusion response strategies is highly important. In this paper, we present a formal model of an OT intrusion response use case using the POMDP framework. It includes a realistic model of partial observability that is based on traffic measurements. This approach allows us to develop tractable, learning-based solution methods for automated intrusion response, which are based on PPO. We evaluate the obtained response strategies on an emulated OT system and find that they are effective against several types of MITRE attacks for the studied use case.
Tags
Links
- Source: https://arxiv.org/abs/2609.10298v1
- Canonical: https://arxiv.org/abs/2609.10298v1
Trouble viewing inline? Open PDF directly →
Full Text
55,652 characters extracted from source content.
Expand or collapse full text
Learning Intrusion Response Strategies for OT Systems Duc Huy Le, Rolf Stadler Affiliation: Dept. of Network and Systems Engineering KTH Royal Institute of Technology, Stockholm, Sweden Email: dhle, stadler@kth.se Affiliation: Abstract Cyberattacks against Operational Technology (OT) systems, which monitor and control industrial processes, pose an increasing threat to essential societal services. For this reason, developing automated intrusion response strategies is highly important. In this paper, we present a formal model of an OT intrusion response use case using the POMDP framework. It includes a realistic model of partial observability that is based on traffic measurements. This approach allows us to develop tractable, learning-based solution methods for automated intrusion response, which are based on PPO. We evaluate the obtained response strategies on an emulated OT system and find that they are effective against several types of MITRE attacks for the studied use case. Index Terms: Operational Technology (OT), security management, automated security, defender strategy, reinforcement learning, Partially Observable Markov Decision Process (POMDP) I Introduction Operational Technology (OT) systems, which monitor and control industrial processes, are increasingly exposed to cyber threats that target critical infrastructure and industrial production. The number of OT cyberattacks grew by more than 90%90\% annually between 2019 and 2023 [1]. Ransomware attacks against industrial organizations increased by 87%87\% in 2024 [2]. Although the number of OT attacks is currently lower than that of Information Technology (IT) attacks, their consequences can be more severe, by disrupting essential services and affecting communities beyond the targeted organizations [1]. Notable examples of such attacks include TRITON/TRISIS [3], Industroyer2 [4], and FrostyGoop [2]. A factor contributing to the rise of OT attacks is the IT/OT convergence, where IT systems are integrated with physical OT networks to facilitate monitoring, data collection, and efficient operation. This integration expands the attack surface and creates paths from enterprise networks into OT environments, which adversaries can exploit [5, 6]. Traditionally, OT infrastructures have a lower level of cybersecurity than IT infrastructures [7]. OT cybersecurity guidelines, e.g. [8, 9, 10], emphasize proactive measures, such as network segmentation and vulnerability analysis, as well as monitoring-based control, including resetting hosts and isolating network segments. They leave intrusion response actions largely to human operators and do not stress automated responses. Early research on automated intrusion response for IT and OT systems has focused on rule-based policies, which configure firewall rules [11] or SDN flows [12, 13], for instance. These rules are defined and maintained by human experts. Recently, methods for OT intrusion response have been developed that do not rely on predefined rules, but are based on learning from system measurements. These methods use Reinforcement Learning (RL) as the dominant concept. Some RL approaches learn heuristic policies without a formal system model [14, 15, 16], which precludes the understanding of an achievable optimal strategy. Other works apply formal models, such as Markov Decision Process (MDP) [17, 18, 19], Partially Observable Markov Decision Process (POMDP) [20, 21], and stochastic game [22, 23]. Most of these lines of research assume full observability of the system state or attacker actions, which is unrealistic and limits the practical relevance of the results. Alternatively, some studies assume specific observation models, without detailing how such observations can be obtained in real systems [20, 21]. Fig. 1: The OT infrastructure of the intrusion response use case In this paper, we investigate an intrusion response use case for an OT infrastructure (see Fig. 1). The infrastructure follows the Purdue model architecture with network segmentation [24]. The objective of the attacker is to disrupt the industrial processes that control the tanks in the OT infrastructure. The attacker creates a path through the infrastructure by executing a sequence of actions, including reconnaissance, vulnerability scanning, exploitation, and host inspection. The defender’s objective is to maintain normal process operation and mitigate the attacker actions. The defender monitors the infrastructure and executes a sequence of defensive actions based on continuous observations of the network traffic. We model this scenario as a discrete-time dynamical system and formalize the defender’s problem as a Partially Observable Markov Decision Process (POMDP) [25]. The model allows us to capture attacker and defender actions and define an optimal defender strategy for the use case. To learn effective defender strategies, we propose two learning methods, which are based on Proximal Policy Optimization (PPO), a state-of-the-art RL algorithm. Applying PPO to our problem formulation is not computationally feasible, since the state space (and thus the belief space) is too large. The first method, k-Obs-PPO, simplifies the structure of the input for the defender strategy, while the second method, BF-PPO, uses particle filter [26] to estimate the system state. We have developed an emulation system, which serves as a replica of the target infrastructure shown in Fig. 1. On this system, we perform the attacks for this scenario. While the attacks are occurring, we collect traffic measurements. These measurements are used to estimate the observation function of the POMDP model, which allows us to simulate this model and to compute effective defender strategies. The obtained defender strategies are evaluated on the emulation system. The key results of this investigation are as follows. We can compute effective defender strategies for the selected scenario using our methods. They converge in reasonable time on a simulator (about 11 hour on an Apple M3 Pro processor). BF-PPO produces a defender strategy that performs on the emulation system close to a strategy that assumes full observability, for three different types of attacks. We make four contributions with this paper: • We present a formal model of the OT intrusion response use case using the POMDP framework and formally define an optimal defender strategy. • We present two solution methods, k-Obs-PPO and BF-PPO, for automated response. The methods are learning based, not rule-based, and not set up by experts. They efficiently learn effective defender strategies. • Our POMDP model includes a realistic model of partial observability, which is based on online traffic measurements. • We evaluate the solution methods in a realistic emulation environment of the OT infrastructure, demonstrating that the learned defender strategies are effective. In particular, BF-PPO achieves performance close to a defender strategy that has been computed under the assumption of full observability. We see the main limitations of this work as follows. First, like all formal models, our model relies on simplifying assumptions. For instance, we assume that attacker and defender actions are taken simultaneously at discrete times and have immediate effects. In reality, such actions occur asynchronously in continuous time, have different durations, and affect the system with delays. Capturing such dynamics would make the model more realistic, at the expense of increasing the computational complexity of the learning defender strategies. Also, we could have modeled the state space with higher granularity, with the same drawback. Second, our POMDP model is valid for the specific use case, system configuration and attack types. In this work, we did not study the generalization property of our model with respect to those aspects. Third, this work currently does not consider operational safety, which constrains defender strategy. We will do so in the future. I Related Work Many studies have investigated the intrusion response problem in IT systems. They use a variety of modeling frameworks, including POMDP [27, 28], game theory [29, 30], or causal models [31]. They apply numerous solution techniques, such as RL [27, 28], attack-graph-based planning [32], and large language models [33]. Automated intrusion response for OT systems has been studied using rule-based and learning-based approaches. Rule-based methods define and enforce predefined response actions, for example through automated firewall configuration [11] or software-defined networking [12, 13]. These methods automatically detect possible incidents and their response strategies are defined by human experts. Recent advances in RL have initiated research on learning-based intrusion response solutions for OT systems, which allows defender strategies to be learned from measurements. Some studies propose heuristic solution methods that do not require a formal system model of the use case [14, 15, 16, 34]. However, without a formal model, it cannot be assessed to which extent the learned defender strategies are optimal. Other works formulate intrusion response problems using decision-theoretic frameworks, including MDP [17, 18, 19], POMDP [20, 21], and stochastic games [22, 23]. Most of them investigate scenarios in which the defender can observe the attacker’s actions and progress. This assumption of full observability is unrealistic. Some works consider partial observability using abstract observation models, but do not discuss how such models are obtained for real systems [16, 20, 21]. A methodology for developing a defender strategy, which first computes the strategy based on a formal model and subsequently evaluates it on an emulation platform, is presented in [35]. The work in this paper is unique in that it considers the following aspects in combination. It formulates a formal model of an OT intrusion response use case, presents learning-based solution methods for automated intrusion response, includes a realistic model of partial observability and evaluation results from an emulated OT system. I The OT Intrusion Response Use Case We study an intrusion response use case for the OT infrastructure shown in Fig. 1. The infrastructure follows the Purdue model [24] and is partitioned into enterprise, supervisory, control, and physical subnets. The enterprise subnet contains client systems that provide access to the industrial environment. The supervisory subnet contains servers with Human–Machine Interfaces (HMIs) and Engineering Workstations (EWSs) for process monitoring, control, and engineering operations. The control subnet contains Programmable Logic Controllers (PLCs), which manage two water tanks in the physical subnet. We assume the attacker has a foothold in the enterprise subnet, either through a compromised enterprise host or through insider access to the IT network. The attacker’s objective is to disrupt the industrial processes by tampering with the tanks. We consider attacks from the MITRE ATT&CK framework [36], where an attacker performs reconnaissance, exploitation, and inspection on supervisory hosts, and then uses a compromised host to interact with the PLCs controlling the tanks. We consider three attacker strategies for this use case, which differ in the way they perform reconnaissance, exploitation, etc. The defender cannot directly observe the attacker’s progression through the infrastructure. Instead, it receives network measurements from the switch through the IDS in Fig. 1. The defender’s objective is to maintain normal operation of the industrial processes and mitigate the attacker actions. The defender can perform three types of defensive actions: reset a supervisory host, reset an industrial process, and reset all hosts in the supervisory subnet and the control subnet. A reset action on a component reboots the machine, renews its credentials, changes its IP address, etc. Such an action can cause a temporary disruption of industrial operations. Note that the traffic measurements not only relate to the attacker actions but also to normal process operations, which makes their interpretation for the defender difficult. IV Formalizing the Use Case Using a POMDP Model We study the intrusion response use case introduced in §I from the defender’s perspective. We assume that the system evolves in discrete time steps with a finite horizon. At each time, both the defender and the attacker take an action. The defender cannot observe the attacker’s progression and obtains knowledge about the attack through network measurements. We formalize the problem of intrusion response as a sequential decision-making problem under partial observability. We choose POMDP as the modeling framework. IV-A Partially Observable Markov Decision Process A Partially Observable Markov Decision Process (POMDP) models the decision process of a discrete-time Markovian system with partial observability [25]. It is defined by a 10-tuple =(,,,,,,γ,ρ1,T,)P=(S,D,T,O,Z,C,γ, _1,T,B). S denotes the state space and D denotes the action space. The probability measure :×→[0,1]T:S×S×D→[0,1] denotes the system dynamics, where (St+1|St,Dt)T(S_t+1|S_t,D_t) is the transition probability from state StS_t to St+1S_t+1 by taking action DtD_t. The state transition is partially observable through variable Ot∈O_t , where O is the observation space. The conditional observation distribution is denoted as Z. Taking an action DtD_t in state StS_t induces a cost Ct=(St,Dt)∈ℝC_t=C(S_t,D_t)∈ R. The objective in a POMDP is to find the sequence of T actions, D1,…,DTD_1,…,D_T, to minimize the expected cumulative cost [J] E[J], with discount factor γ∈(0,1]γ∈(0,1]: J=∑t=1Tγt−1CtJ= _t=1^Tγ^t-1C_t (1) A belief state bt=⟨bt(st)⟩st∈b_t= b_t(s_t) _s_t is associated with time t, where bt(s)=ℙ[St=s|ht]b_t(s)= P[S_t=s|h_t] with ht=(ρ1,o1,d1,o2,…,dt−1,ot)∈ℋh_t=( _1,o_1,d_1,o_2,…,d_t-1,o_t) . The belief state is a distribution over the state space S. At every time t, the belief is recursively computed: bt+1(s)=η−1(ot+1|s,dt)∑st∈(s|st,dt)bt(st)b_t+1(s)=η^-1Z(o_t+1|s,d_t) _s_t T(s|s_t,d_t)b_t(s_t) (2) where η=∑s∈(ot+1|s,dt)∑st∈(s|st,dt)bt(st)η= _s Z(o_t+1|s,d_t) _s_t T(s|s_t,d_t)b_t(s_t) is the normalization factor. At the beginning of an episode, the initial state S1S_1 is sampled from the initial state distribution ρ1:→[0,1] _1:S→[0,1], which also defines the initial belief state. IV-B A POMDP Model of the Intrusion Response Use Case This subsection defines the POMDP model of the defender’s problem. The system state captures attacker progression and process integrity, while the observation represents IDS-derived network measurements. The transition function models the interaction between attacker and defender actions, and the cost function encodes the defender’s objective. The resulting POMDP defines the optimization problem for computing an optimal defender strategy. IV-B1 Scenario configuration To simplify notation, we denote the supervisory hosts and the processes in Fig. 1 by h0=HMI0h_0=HMI_0, h1=HMI1h_1=HMI_1, and h2=EWS0h_2=EWS_0, p0=Tank0p_0=Tank_0 and p1=Tank1p_1=Tank_1. Let =h0,h1,h2H=\h_0,h_1,h_2\ be the set of hosts and =p0,p1P=\p_0,p_1\ be the set of tanks. In this topology, each supervisory host h∈h can monitor and control both p0p_0 and p1p_1 through their corresponding control units, PLC0PLC_0 and PLC1PLC_1. During an episode, the attacker follows a fixed strategy πA∈ΠA _A∈ _A, where ΠA _A is the attacker strategy space. IV-B2 System state space S and initial state distribution ρ1 _1 The system state StS_t at time t represents the states of the supervisory hosts, the states of the industrial processes, and the attacker action state from the previous time step. Formally, StS_t is defined as the tuple St=(Sth0,Sth1,Sth2,Stp0,Stp1,At−1)S_t=(S^h_0_t,S^h_1_t,S^h_2_t,S^p_0_t,S^p_1_t,A_t-1). For each host h∈h , its state Sth∈=U,D,S,E,IS^h_t ^h=\ U, D, S, E, I\ represents the attacker’s progression on host h at time t. The values denote whether h is undiscovered by the attacker (U), discovered by the attacker (D), scanned by the attacker (S), exploited and accessed by the attacker (E), or inspected for PLC control by the attacker (I). These states are motivated by MITRE ATT&CK framework [36] and reflect the main attack stages of an OT cyberattack, where an attacker first discovers assets, then gathers information, gains access, and learns control patterns. For each process p∈p , its state Stp∈=W,CS^p_t ^p=\ W, C\ denotes the operation condition of the water tanks. The values denote whether p is in its normal operation state (W) or has been corrupted by the attacker (C). The component At−1A_t-1 denotes the attacker action at time t−1t-1 and is defined as At−1=(At−1h,At−1p)A_t-1=(A^h_t-1,A^p_t-1). At−1hA^h_t-1 represents the targeted host at time t−1t-1 and takes three types of values: ∅ , if no host is targeted; h∈h , if host h is targeted; and H, if the attacker scans the supervisory subnet. At−1pA^p_t-1 represents the targeted process at time t−1t-1 and can be either ∅ or a process p∈p . We require At−1p≠∅A^p_t-1≠ only when At−1h∈A^h_t-1 , meaning that process tampering is performed through a targeted host. The attacker action space is =∪∅,×∪∅A=\H∪\ ,H\\×\P∪\ \\. The state space is defined as =3×2×S=S^h^3×S^p^2×A. At time t=1t=1, no intrusion has occurred, hence, the initial state distribution ρ1 _1 is a degenerate distribution with ρ1(s)=1 _1(s)=1, where s=(U,U,U,W,W,(∅,∅))s=( U, U, U, W, W,( , )). IV-B3 Defender action space D Fig. 2: The transitions of host states SthS^h_t and the process states StpS^p_t caused by attacker action AtA_t (red arrows) and defender action DtD_t (blue arrows). The dashed arrow illustrates that host h must be in state I for the attacker to tamper with process p using host h (OPENAth=h,Atp=p)A^h_t=h,A^p_t=p). At time t, the defender takes an action Dt=(Dth,Dtp)D_t=(D^h_t,D^p_t). DthD^h_t is the target host(s) and DtpD^p_t is the target process(es) at time t. DtD_t can take one of the following values: (Dth=∅,Dtp=∅)(D^h_t= ,D^p_t= ) if the defender passively monitors the system; (Dth=h∈,D∅p)(D^h_t=h ,D^p_ ) if the defender resets host h; (Dth=∅,Dtp=p∈)(D^h_t= ,D^p_t=p ) if the defender resets process p; and (Dth=,Dtp=)(D^h_t=H,D^p_t=P) if the defender resets the whole system. The defender action space is thereby defined as =(∅,∅)∪(h,∅):h∈∪(∅,p):p∈∪(,).D=\( , )\∪\(h, ):h \∪\( ,p):p \∪\(H,P)\. IV-B4 Transition function T The state transition (St+1|St,Dt)T(S_t+1|S_t,D_t) from time t to t+1t+1 can be described through the transitions of each component of the system state: (a) At−1→AtA_t-1→ A_t; (b) Sth→St+1hS^h_t→ S^h_t+1; and (c) Stp→St+1pS^p_t→ S^p_t+1. The transitions are presented as follows. Attacker action At−1→AtA_t-1→ A_t The attacker action AtA_t at time t is defined by the system state StS_t and its strategy πA _A, i.e., At=πA(St)A_t= _A(S_t). Host state Sth→St+1hS^h_t→ S^h_t+1 The state St+1hS^h_t+1 of a host h∈h is dependent on its previous state SthS^h_t, the attacker action AthA^h_t and the defender action DthD^h_t. Sth=U S^h_t= U →St+1h=D if Ath=,h∉Dth → S^h_t+1= D if A^h_t=H,h∉ D^h_t (3a) Sth=D S^h_t= D →St+1h=S if Ath=,h∉Dth → S^h_t+1= S if A^h_t=h,h∉ D^h_t (3b) Sth=S S^h_t= S →St+1h=E if Ath=,Ne=1,h∉Dth → S^h_t+1= E if A^h_t=h,N_e=1,h∉ D^h_t (3c) Sth=S S^h_t= S →St+1h=S if Ath=,Ne=0,h∉Dth → S^h_t+1= S if A^h_t=h,N_e=0,h∉ D^h_t (3d) Sth=E S^h_t= E →St+1h=I if Ath=,h∉Dth → S^h_t+1= I if A^h_t=h,h∉ D^h_t (3e) Sth S^h_t →St+1h=U if h∈Dth → S^h_t+1= U if h∈ D^h_t (3f) Sth S^h_t →St+1h=Sth otherwise → S^h_t+1=S^h_t otherwise (3g) where NeN_e is a binary random variable that determines the success of the attacker’s Exploit action on host h. (3a-3g) describe the transition of the host state Sth→St+1hS^h_t→ S^h_t+1, ∀h∈∀ h . (3a) describes the attacker’s network reconnaissance action on the supervisory subnet, which exposes the hosts H to the attacker and causes the transition U→D U→ D. (3b) describes the transition D→S D→ S, where the attacker performs a host-level discovery on host h. (3c) and (3d) describe the exploitation attempt of the attacker on host h, which has a success probability defined by a binary random variable NeN_e. After gaining access to host h, the attacker inspects it to learn control patterns to the underlying physical processes, which is presented by the transition E→I E→ I. All transitions in (3a-3e) occur under the condition that the defender does not reset host h, which brings St+1hS^h_t+1 to the secure state U (3f). The transitions are illustrated in Fig. 2, which describes the attacker actions on a host toward state I. The defender can reset the host and bring it to the secure state U. Process state Stp→St+1pS^p_t→ S^p_t+1 The state St+1pS^p_t+1 of a process p∈p is dependent on its state StS_t, the attacker action AtpA^p_t and the defender action DtpD^p_t. The transition is illustrated in Fig. 2. Stp→ S^p_t→ St+1p=C if Atp=p,Ath=h,Sth=I,p∉Dtp S^p_t+1= C if A^p_t=p,A^h_t=h,S^h_t= I,p∉ D^p_t (4a) Stp→ S^p_t→ St+1p=W if p∈Dtp S^p_t+1= W if p∈ D^p_t (4b) Stp→ S^p_t→ St+1p=Stp otherwise S^p_t+1=S^p_t otherwise (4c) (4a-4c) describe the transition of the process state Stp→St+1pS^p_t→ S^p_t+1, ∀p∈∀ p . (4a) describes the process tampering action, which requires an inspected host (∃h∈:Sth=I∃\;h :S^h_t= I), and the process is not reset by the defender at the same time (p∉Dtpp∉ D^p_t). (4b) describes the defender resetting process p to its normal working state. IV-B5 Defender observation space O The defender cannot observe the system state StS_t. Instead, the defender observes Ot=(Gt,Oth0,Oth1,Oth2,Otp0,Otp1)O_t=(G_t,O^h_0_t,O^h_1_t,O^h_2_t,O^p_0_t,O^p_1_t) (5) where GtG_t represents the number of IP packets between the subnets, OthiO^h_i_t represents the number of IP packets between supervisory host i and the enterprise subnet, and OtpjO^p_j_t represents the number of IP packets between PLCjPLC_j and the supervisory subnet, during time step t. IV-B6 Defender observation function Z The observation function is the conditional probability mass function of the discrete observation vector Ot+1O_t+1: (o,s,d)=ℙ[Ot+1=o∣St+1=s,Dt=d]Z(o,s,d)= P[O_t+1=o S_t+1=s,D_t=d] (6) where s∈s , d∈d , and o∈o . IV-B7 Cost function C and defender objective J At each time t, the defender pays a cost CtC_t: Ct=∑h∈σh(Sth)+∑p∈σp(Stp)+σd(Dt)C_t= _h _h(S^h_t)+ _p _p(S^p_t)+ _d(D_t) (7) where σh,σp _h, _p, and σd _d are parameter functions, which reflect the defender’s objective by denoting costs for the host states, the process states and the defensive actions. The functions are specified in Tab. I. The high cost for process corruption, σp(C)=10 _p( C)=10, prioritizes the safety of the physical processes. The increasing host state costs σh _h penalize the defender for attacker progression and encourage preventive responses. The action cost σd _d penalizes resets, preventing unnecessarily disruptive responses. TABLE I: Parameters of the cost function C Cost function Value σh _h σh(U)=σh(D)=0,σh(S)=0.2,σh(E)=1.5,σh(I)=3.5 _h( U)= _h( D)=0, _h( S)=0.2, _h( E)=1.5, _h( I)=3.5 σp _p σp(W)=0,σp(C)=10 _p( W)=0, _p( C)=10 σd _d σd(Dth,Dtp)=2×|Dth|+3×|Dtp| _d(D^h_t,D^p_t)=2× D^h_t +3× D^p_t Hence, the objective of the defender is formulated as minimizing the expected cumulative cost CtC_t over the time horizon T with discount factor γ=1γ=1: J=∑t=1T[Ct]J= _t=1^T E[C_t] (8) IV-B8 The defender’s optimization problem We define πD:ℋ→ _D:H or πD:→ _D:B as the defender strategy, where ℋH is the history space and B is the belief space. The defender problem is defined as finding the optimal strategy πD∗ _D^* that minimizes the expected cumulative cost J. Problem 1. Find the optimal defender strategy : πD∗ _D^*\;\;\;\;\; =argminπDπD[J] =\; _ _D E_ _D[J] (9a) subject to Dt=πD(bt)∀t D_t= _D(b_t)∀ t (9b) πA∼P(ΠA) _A P( _A) (9c) Because the POMDP has finite state, action, and observation spaces over a finite horizon, an optimal strategy πD∗π^*_D exists [25, Thm. 7.4.1]. V Learning Defender Strategies with Reinforcement Learning Problem 1 can be solved with Dynamic Programming methods [37]. Due to the large state space, such methods are computationally intractable for our use case [38]. We therefore parameterize the defender strategy πD _D with parameter θ. We apply PPO [39], a state-of-the-art RL algorithm, to learn the (almost) optimal defender strategy. PPO is an actor–critic policy-gradient method that uses a clipped surrogate objective to improve training stability by limiting policy changes between updates. A direct application of PPO to Problem 1 is not feasible due to the large state space and belief space. We therefore propose two PPO-based solution methods that use different techniques to address the issue. V-A PPO with k latest observations: k-Obs-PPO The first method, k-Obs-PPO, uses the k most recent IDS observations, (Ot,…,Ot−k+1)(O_t,…,O_t-k+1) as input for πDθπ^θ_D. It allows training the defender strategy directly from observation histories, without computing the belief state. The parameter k controls the trade-off between the amount of information used to learn the strategy on the one hand and the computational complexity and sampling efficiency on the other hand. A larger k provides more temporal information to the defender strategy but increases the input dimension and the learning complexity. A smaller k keeps the input small and simplifies defender strategy learning. V-B Belief Filter Proximal Policy Optimization: BF-PPO The second method, BF-PPO, uses an approximate and simplified belief state to learn the defender strategy. In a POMDP, belief updates are computed using Bayes filter (Eq. (2)). Such an update has a quadratic time complexity with respect to the size of the state space. Therefore, the Bayes Filter is computationally infeasible in our case. We address the issue by approximating the belief using particle filter [26]. At time t, the particle filter represents the belief by M sampled states, t=st(1),…,st(M)P_t=\s_t^(1),…,s_t^(M)\. The belief state is then approximated by the relative frequency of each system state in tP_t, i.e., b^t(s)=1M∑i=1Mst(i)=s b_t(s)= 1M _i=1^M 1\s_t^(i)=s\. Due to the large state space, the belief vector ⟨b^t(s)⟩ b_t(s) is high dimensional, which increases the learning complexity and makes it inefficient as input to the defender strategy. We therefore exploit the factorized structure of the system state with components Sth0,Sth1,Sth2,Stp0,Stp1,At−1S^h_0_t,S^h_1_t,S^h_2_t,S^p_0_t,S^p_1_t,A_t-1. For each such component XtX_t of the state and component state x, we compute the marginal belief b^tX(x)=1M∑i=1MXt(i)=x. b_t^X(x)= 1M _i=1^M 1\X_t^(i)=x\. (10) The concatenation of these marginal beliefs is used as input to the defender strategy πDθπ^θ_D, providing a compact input while preserving the information about each component of the system state. VI Constructing an Emulation Environment and a Simulation Environment to Train and Evaluate Defender Strategies Learning defender strategies using our solution methods and evaluating them requires repeated POMDP episodes. Such episodes cannot be performed directly in an operating OT infrastructure, which is both inefficient due to the large number of episodes required for training and impractical because it requires performing attacker actions in the system. For these reasons, we develop an emulation environment and a simulation environment. The emulation environment is designed to closely match the operational conditions of the target infrastructure. It serves two purposes: collecting IDS measurements to identify the observation model of the POMDP, and evaluating defender strategies under realistic conditions. The simulation environment generates POMDP episodes to efficiently train the defender strategies. VI-A Constructing the Emulation Environment The infrastructure in Fig. 1 is emulated with Docker [40] containers and orchestrated by ContainerLab [41]. Each functional component is instantiated as a separate container, including the enterprise clients, supervisory hosts, PLCs, water tanks, router, attacker, defender, and IDS. The PLCs are implemented using OpenPLCv3 [42]. The main communication protocol between OT components (HMI, PLC, EWS, and water tanks) is implemented using ModbusTCP [43]. The configuration of the components is listed in Appendix A. The network is implemented using Linux virtual Ethernet interfaces. Logical network segmentation is enforced through VLAN configurations using Open vSwitch [44]. Inter-subnet routing and firewall rules are implemented by the router container. The IDS is connected to a mirrored Open vSwitch port, which allows it to observe and collect network measurements. The defender’s reset action on a host or a process changes the IP address and login credentials of the target component, thereby removing potential attacker access. The updated configuration is then propagated to the components that require it. For example, when a PLC is reset, its new IP address is distributed to the HMIs and the EWS so that supervisory communication can be re-established. The attacker actions, including reconnaissance and exploitation, are implemented by a sequence of commands executed from the attacker container. To capture a broad range of traffic patterns for the same action, the implementation varies command parameters, including target networks and ports, probe types, command arguments, and payload sizes. This allows us to observe traffic under a wide range of attacker behaviors. More details of the implementation are presented in Appendix A. Background traffic is generated by three types of activities: by enterprise clients using services on the HMI and EWS containers; by HMI-PLC communication, where HMIs periodically read process measurements from the PLCs and issue routine control commands; and by EWS-PLC communication, representing engineering operations performed by an operator. All non-periodic background activities are generated by independent Poisson processes. VI-B Estimating the observation function in the emulation environment In the emulation environment, we perform operation in periods of 3030 seconds wall clock time. During such a period, the attacker performs an attack action according to its strategy. Also, the defender performs a defensive action based on the observation data collected during the period. This data consists of network statistics obtained from the IDS, namely, o=(g,oh0,oh1,oh2,op0,op1)o=(g,o^h_0,o^h_1,o^h_2,o^p_0,o^p_1) (Eq. (5)). To estimate the observation function Z (Eq. (6)) of the POMDP, we collect monitoring observation data from 4000040000 periods, which amounts to 1414 days. Effectively estimating Z(o,s,d)=Pr(Ot+1=o∣St+1=s,Dt=d)Z(o,s,d)= (O_t+1=o S_t+1=s,D_t=d) is not feasible since it would require approximately 10810^8 observations (||||=52500|S||D|=52500 possible conditioning combinations of (s,d)(s,d) and assuming about 20002000 observations needed for each combination). To work with 4000040000 observation measurements, we simplify the observation function Z to Z by setting ^(o,a)=Pr(Ot+1=o∣At=a) Z(o,a)= (O_t+1=o A_t=a), where a denotes the attacker action. Fig. 3 shows empirical distributions for selected attacker actions. Fig. 3: Empirical distribution of IP packet counts measured on the switch in Fig. 1. The top figure relates to the total traffic passed through the switch; the middle figure relates to the traffic between EWS-0 and entities in the enterprise subnet; the lower figure relates to the traffic between PLC-0 and entities in the supervisory subnet. Different colors refer to measurements from different attacker actions. VI-C Running the POMDP simulation Simulating a POMDP episode starts the system in state s=(U,U,U,W,W,(∅,∅))s=( U, U, U, W, W,( , )). At each time step, the attacker and the defender perform an action following their respective strategies. The system state is then updated following the transition function T specified in §IV. The observation that the defender makes during the time step is sampled from Z. Further, the cost incurred for this time step is computed using the cost function C. The episode ends when the T time steps have been executed. The trajectories of observations, actions, and costs per period are used to update the defender strategies during training. VII Computing the Defender Strategies in the Simulation Environment and Evaluating Them in the Emulation Environment We train BF-PPO and two k-Obs-PPO variants, with k=1k=1 and k=4k=4 in the simulation environment. We then evaluate the obtained defender strategies in the emulation environment. We compare the proposed methods with two baselines, a threshold-based defender strategy and an idealized PPO baseline, where we assume the defender has full observation of the system states and the attacker actions. The training and evaluation are conducted on an Apple M3 Pro processor. The hyperparameters for the learning methods are listed in Appendix B. The source code is available at [45]. VII-A Evaluation setup VII-A1 Attacker strategies We train and evaluate the methods against three attacker strategies: Opportunistic The attacker prioritizes reaching process tampering as quickly as possible. It selects actions that advance the attack along the shortest available path toward tampering with a process. Explorative The attacker prioritizes gathering knowledge about the infrastructure before tampering with a process. It scans supervisory hosts, exploits available entry points, and inspects host–process control relationships before tampering with the processes. Adaptive This attacker adapts to defender actions by avoiding recently reset components and switching to alternative attack paths when possible. VII-A2 Baselines MDP-PPO The method applies PPO whereby the system state can be observed by the defender. This baseline method represents an idealized setting and a cost lower bound for the PPO-based methods proposed in this paper. Threshold strategy At each time step t, the defender computes a cost score RhtR^t_h for each host and a cost score RptR^t_p for each process. If the score of any process exceeds its respective threshold, the defender following this strategy resets the process with the highest score. Otherwise, if the score of any host exceeds its respective threshold, the defender resets the host with the highest score. The thresholds are determined by the reset costs σd _d (see Tab. I). The cost scores are given by the formulas in Appendix C. VII-A3 Evaluation process We train the defender strategies using BF-PPO, 11-Obs-PPO, and 44-Obs-PPO, as well as the baseline method MDP-PPO in the simulation environment. For each method, we perform four training runs with different seeds. A training run consists of 500500 iterations, where each iteration contains 100100 episodes with time horizon 100100. After each iteration, we evaluate the current policy to obtain a point in the learning curve. We finally evaluate the learned defender strategies in the emulation environment. The strategy obtained by each method is evaluated against the three attacker strategies for 2020 episodes with time horizon 100100. Evaluating a defender strategy against an attacker strategy takes about 1717 hours on an Apple M3 Pro processor. Fig. 4: The learning curves for the solution methods proposed in §V (BF-PPO – green curves, 11-Obs-PPO – blue curves, and 44-Obs-PPO – orange curves) and the baselines (MDP-PPO – red curves and Threshold strategy – dashed lines). The solution methods computed effective defender strategies for three different attacker strategies (Opportunistic, Explorative and Adaptive). The learning curves show the average and the confidence intervals of 120120 episodes. The best performing solution method with respect to convergence speed and cost is BF-PPO. Fig. 5: Evaluation of the learned defender strategies for the proposed solution methods, BF-PPO, 11-Obs-PPO and 44-Obs-PPO. The green box plots relate to evaluation results in the emulation environment. For comparison, we add the results from the simulation environment, presented by red box plots. The baseline refers to the learned defender strategies under the assumption of full observability of the system states. VII-B Evaluating the methods in the simulation environment Fig. 4 shows the learning curves of the proposed methods and the baselines against three attacker strategies. BF-PPO, 11-Obs-PPO, and 44-Obs-PPO, are represented by the green, blue, and orange curves, respectively. MDP-PPO and Threshold strategy are shown by red curves and dashed lines, respectively. Each plot corresponds to a attacker strategy and shows the natural logarithm of the cumulative cost during training. The learning curves show that 11-Obs-PPO and 44-Obs-PPO converge to higher cumulative costs than BF-PPO for all attacker strategies. They also exhibit larger variation. 11-Obs-PPO tends to perform better than 44-Obs-PPO, which suggests that, in this setting, increasing the observation history does not provide sufficient additional information to improve performance. BF-PPO converges faster and to lower cumulative costs than the k-Obs-PPO variants for all attacker strategies. Its learning curves are also close to those of MDP-PPO, the full-observability baseline. This indicates that using particle filter and marginal beliefs provides a solution for learning effective defender strategies. Further, Fig. 4 also shows that BF-PPO consistently outperforms the Threshold strategy. VII-C Evaluating the defender strategies in the emulation environment Fig. 5 shows the evaluation results of the defender strategies in the emulation environment (green color) and compares them with the results from the simulation environment (red color). The box plots show the distributions of cumulative costs over 2020 evaluation episodes for BF-PPO, 11-Obs-PPO and 44-Obs-PPO. The dashed line shows the average cumulative cost of MDP-PPO, the full-observability baseline method. The plots show that BF-PPO outperforms the k-Obs-PPO variants for all attacker strategies. Its cumulative costs are also close to those of MDP-PPO, which is consistent with the simulation results in Fig. 4. BF-PPO shows a gap between the performance in the emulation environment and the simulation environment. We expect that the defender strategy performs less well in the emulation environment than in the simulation environment. This is due to the fact that the formal model we have presented is an approximation of the more realistic emulated system, and the defender strategy has been optimized for the simulation environment. Overall, we find that BF-PPO is the strongest proposed method for our use case and the considered attacker strategies. VIII Conclusion and Future Work Based on the POMDP model of the use case, we have defined an optimal defender strategy and proposed two solution methods for learning the approximation of this strategy. The solution methods allow us to learn the defender strategies at a reasonable cost and with low sample complexity. The evaluation in the emulation environment shows that effective defender strategies can be learned with these methods. In particular, BF-PPO approaches the performance of a baseline with full observability. As for future work, we plan to evaluate the proposed model and solution methods on an industrial testbed in collaboration with an industrial partner, for an intrusion response use case similar to the one studied in this paper. This evaluation will allow us to study our approach to modeling and computing defender strategies in a different and realistic environment and further to include safety constraints. IX Acknowledgment This work has been supported by the DARPA CASTLE program through project ORLANDO and by the WASP NEST program through project AIRR. The authors thank KTH researchers Kim Hammar and Xiaoxuan Wang for their constructive comments. Appendix A Emulation configurations: Tab. I and Tab. I B Training hyperparameters: Tab. IV TABLE I: Configuration of the emulated physical components Component Services Vulnerabilities HMI0,HMI1HMI_0,HMI_1 HTTP Weak credentials EWS0EWS_0 SSH, Telnet, SMB SSH/Telnet weak credentials, CVE-2017-7494 PLC0,PLC1PLC_0,PLC_1 ModbusTCP, HTTP – Tank0,Tank1Tank_0,Tank_1 ModbusTCP – TABLE I: Implementation of attacker actions in the emulation Attacker action Implementation Scan network ICMP scan Scan host TCP/UDP/OS scan, lightweight probes Exploit host HMI: HTTP brute-force, SQL injection, EWS: SSH/Telnet brute-force, SambaCry Inspect host HMI: HTTP probes, EWS: ModbusTCP control Tamper process HMI: PLC parameter modifications via web interface, EWS: Craft & Upload PLC programs, PLC parameter control via ModbusTCP TABLE IV: Hyperparameters for the training of defender strategies Parameter Value # particles (BF-PPO) M 500500 # hidden layer, # neurons, random seeds 2,128,[77,108,433,841]2,128,[77,108,433,841] l.r. α, γ, GAE λ, ENT-COEF, clip ϵε 0.0001,0.99,0.95,0.001,0.20.0001,0.99,0.95,0.001,0.2 Optimiser (parameters) Adam (OPENβ1=0.9,β2=0.99) _1=0.9, _2=0.99) C Threshold strategy’s cost score formulas Eq. (11) defines the formulation of the cost score for each host and process at time t, where btb_t denotes the belief state. Rht=∑s∈Shbth(s)σh(s);Rpt=∑s∈Spbtp(s)σp(s)R^t_h= _s∈ S^hb_t^h(s) _h(s); R^t_p= _s∈ S^pb_t^p(s) _p(s) (11) References [1] Waterfall Security Solutions and ICS Strive (2024) 2024 threat report: ot cyberattacks with physical consequences. Technical Report Waterfall Security Solutions. Cited by: §I. [2] Dragos, Inc. (2025) 2025 ot/ics cybersecurity report: a year in review. Technical Report Dragos Inc.. Cited by: §I. [3] A. Di Pinto, Y. Dragoni, and A. Carcano (2018) TRITON: the first ics cyber attack on safety instrument systems. Proc. Black Hat USA 2018, p. 1–26. Cited by: §I. [4] D. D. McCarthy (2025) PRECURSOR analysis report: industroyer2 and wiper malware targeting ukrainian energy provider 2022. Technical report Idaho National Laboratory (INL), Idaho Falls, ID (United States). Cited by: §I. [5] M. Dobler, M. Hellwig, N. Lopes, K. Oakley, and M. Winterburn (2025) Systematic review and characterisation of malicious industrial network traffic datasets: m. dobler et al.. International Journal of Information Security 24 (5), p. 208. Cited by: §I. [6] G. M. Makrakis, C. Kolias, G. Kambourakis, C. Rieger, and J. Benjamin (2021) Industrial and critical infrastructure security: technical analysis of real-life security incidents. IEEE Access 9, p. 165295–165325. External Links: ISSN 2169-3536, Document Cited by: §I. [7] M. Rodda and V. Mavroudis (2025) Analysis of publicly accessible operational technology and associated risks. arXiv preprint arXiv:2508.02375. Cited by: §I. [8] P. Ackerman (2021) Industrial cybersecurity: efficiently monitor the cybersecurity posture of your ics environment. Packt Publishing Ltd. Cited by: §I. [9] K. Stouffer, M. Pease, C. Tang, T. Zimmerman, V. Pillitteri, S. Lightman, A. Hahn, S. Saravia, A. Sherule, and M. Thompson (2023) Guide to operational technology (ot) security. NIST Special Publication (SP), National Institute of Standards and Technology, Gaithersburg, MD. External Links: Document Cited by: §I. [10] International Society of Automation (2026) IEC 62443 Series of Standards. Note: https://w.isa.org/standards-and-publications/isa-standards/isa-iec-62443-series-of-standards Cited by: §I. [11] J. Vaudey, S. Mocanu, G. Delaval, and E. Rutten (2025) Reconfiguration of firewall filter rules as a response to industrial control system intrusion. In 2025 IEEE Conference on Communications and Network Security (CNS), Vol. , p. 1–6. External Links: Document Cited by: §I, §I. [12] A. F. Murillo Piedrahita, V. Gaur, J. Giraldo, Á. A. Cárdenas, and S. J. Rueda (2018) Leveraging software-defined networking for incident response in industrial control systems. IEEE Software 35 (1), p. 44–50. External Links: Document Cited by: §I, §I. [13] A. F. M. Piedrahita, V. Gaur, J. Giraldo, A. A. Cardenas, and S. J. Rueda (2018) Virtual incident response functions in control systems. Computer Networks 135, p. 147–159. External Links: ISSN 1389-1286, Document Cited by: §I, §I. [14] A. Babar, T. Halabi, and M. Zulkernine (2026) Autonomous and adaptive cyber incident detection and response in industrial cyber-physical systems using hierarchical reinforcement learning. ACM Trans. Cyber-Phys. Syst. 10 (1). External Links: ISSN 2378-962X, Document Cited by: §I, §I. [15] Y. Yue, D. Zhao, Y. Zhou, L. Xu, Y. Tang, and H. Peng (2025) An intrusion response approach based on multi-objective optimization and deep q network for industrial control systems. Expert Systems with Applications 272, p. 126664. External Links: ISSN 0957-4174, Document Cited by: §I, §I. [16] X. Li, C. Zhou, Y. Tian, and Y. Qin (2019) A dynamic decision-making approach for intrusion response in industrial control systems. IEEE Transactions on Industrial Informatics 15 (5), p. 2544–2554. External Links: Document Cited by: §I, §I, §I. [17] H. Chen, Y. Lai, J. Liu, and H. Wanyan (2024) Interpretable cross-layer intrusion response system based on deep reinforcement learning for industrial control systems. IEEE Transactions on Industrial Informatics 20 (7), p. 9771–9781. External Links: Document Cited by: §I, §I. [18] S. Xu, Z. Xie, C. Zhu, X. Wang, and L. Shi (2023) Enhancing cybersecurity in industrial control system with autonomous defense using normalized proximal policy optimization model. In 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS), Vol. , p. 928–935. External Links: Document Cited by: §I, §I. [19] L. Chen, Y. Lai, P. Zhao, B. Xie, and Y. Zhang (2026) Safety-aware intrusion response system based on safe reinforcement learning for cyber-physical systems. IEEE Transactions on Consumer Electronics 72 (2), p. 2711–2723. External Links: Document Cited by: §I, §I. [20] J. Mern, K. B. Hatch, R. Silva, J. S. Brush, and M. J. Kochenderfer (2021) Reinforcement learning for industrial control network cyber security orchestration. ArXiv abs/2106.05332. Cited by: §I, §I. [21] Y. Lai, P. Zhao, and Z. Wang (2026) MFIR: model-free intrusion response for partially observable industrial control systems. IEEE Internet of Things Journal (), p. 1–1. External Links: Document Cited by: §I, §I. [22] P. Yao, Z. Jiang, B. Yan, Q. Yang, and W. Wang (2024) Bayesian and stochastic game joint approach for cross-layer optimal defensive decision-making in industrial cyber-physical systems. Information Sciences 662, p. 120216. External Links: ISSN 0020-0255, Document Cited by: §I, §I. [23] K. Zhong, Z. Yang, G. Xiao, X. Li, W. Yang, and K. Li (2022) An efficient parallel reinforcement learning approach to cross-layer defense mechanism in industrial control systems. IEEE Transactions on Parallel and Distributed Systems 33 (11), p. 2979–2990. External Links: Document Cited by: §I, §I. [24] T. J. Williams (1994) The purdue enterprise reference architecture. Computers in Industry 24 (2), p. 141–158. External Links: ISSN 0166-3615, Document Cited by: §I, §I. [25] V. Krishnamurthy (2016) Partially observed markov decision processes: from filtering to controlled sensing. Cambridge University Press. Cited by: §I, §IV-A, §IV-B8. [26] S. Thrun, W. Burgard, and D. Fox (2005) Probabilistic robotics. MIT Press. External Links: ISBN 9780262201629, LCCN 2005043346 Cited by: §I, §V-B. [27] E. Miehling, M. Rasouli, and D. Teneketzis (2018) A pomdp approach to the dynamic defense of large-scale cyber networks. IEEE Transactions on Information Forensics and Security 13 (10), p. 2490–2505. External Links: Document Cited by: §I. [28] D. H. Le and R. Stadler (2025) Learning optimal defender strategies for cage-2 using a pomdp model. In 2025 21st International Conference on Network and Service Management (CNSM), Vol. , p. 1–9. External Links: Document Cited by: §I. [29] L. Zhang, T. Zhu, F. K. Hussain, D. Ye, and W. Zhou (2023) A game-theoretic method for defending against advanced persistent threats in cyber systems. IEEE Transactions on Information Forensics and Security 18 (), p. 1349–1364. External Links: Document Cited by: §I. [30] K. Hammar and R. Stadler (2020) Finding effective security strategies through reinforcement learning and self-play. In 2020 16th International Conference on Network and Service Management (CNSM), Vol. , p. 1–9. External Links: Document Cited by: §I. [31] K. Hammar, N. Dhir, and R. Stadler (2024) Optimal defender strategies for cage-2 using causal modeling and tree search. arXiv preprint arXiv:2407.11070. Cited by: §I. [32] J. Nyberg, P. Johnson, and A. Méhes (2022) Cyber threat response using reinforcement learning in graph-based attack simulations. In 2022 IEEE/IFIP Network Operations and Management Symposium (NOMS), Vol. , p. 1–4. External Links: Document Cited by: §I. [33] K. Hammar, T. Alpcan, and E. C. Lupu (2025) Incident response planning using a lightweight large language model with reduced hallucination. External Links: 2508.05188 Cited by: §I. [34] A. Wilson, R. Menzies, N. Morarji, D. Foster, M. C. Mont, E. Turkbeyler, and L. Gralewski (2024) Multi-agent reinforcement learning for maritime operational technology cyber security. External Links: 2401.10149 Cited by: §I. [35] K. Hammar (2024) Optimal security response to network intrusions in it systems. PhD dissertation, Kungliga Tekniska högskolan, Stockholm. Cited by: §I. [36] B. Al-Sada, A. Sadighian, and G. Oligeri (2024) MITRE att&ck: state of the art and way forward. ACM Comput. Surv. 57 (1). External Links: ISSN 0360-0300, Document Cited by: §I, §IV-B2. [37] E. J. Sondik (1978) The optimal control of partially observable markov processes over the infinite horizon: discounted costs. Operations Research 26 (2), p. 282–304. External Links: ISSN 0030364X, 15265463 Cited by: §V. [38] D. Burago, M. de Rougemont, and A. Slissenko (1996) On the complexity of partially observed markov decision processes. Theoretical Computer Science 157 (2), p. 161–183. External Links: ISSN 0304-3975, Document Cited by: §V. [39] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. External Links: 1707.06347 Cited by: §V. [40] D. Merkel (2014) Docker: lightweight linux containers for consistent development and deployment. Vol. 2014, Belltown Media, Houston, TX. External Links: ISSN 1075-3583 Cited by: §VI-A. [41] Nokia SR Linux Labs (2026) Containerlab. Note: Container-based networking lab orchestrator External Links: Link Cited by: §VI-A. [42] T. Alves and T. H. Morris (2018) OpenPLC: an IEC 61,131–3 compliant open source industrial controller for cyber security research. Computers & Security 78, p. 364–379. External Links: Document Cited by: §VI-A. [43] (2006) MODBUS Messaging on TCP/IP Implementation Guide. Modbus Organization. Note: Accessed: 2026-06-02 External Links: Link Cited by: §VI-A. [44] B. Pfaff, J. Pettit, T. Koponen, E. J. Jackson, A. Zhou, J. Rajahalme, J. Gross, A. Wang, J. Stringer, P. Shelar, K. Amidon, and M. Casado (2015) The design and implementation of open vswitch. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation, NSDI’15, USA, p. 117–130. External Links: ISBN 9781931971218 Cited by: §VI-A. [45] D. H. Le (2026) Intrusion Response in OT systems (implementation). Note: https://github.com/duchuyle108/viper Cited by: §VII.