Paper deep dive
ADORN: Adaptive Drift handling for Open RAN using Reinforcement Learning
Ashit Kumar Subudhi, Bhargav Chirumamilla, Shubham Vaishnav, Mduduzi C. Hlophe, Praveen Kumar Donta, Andrea Fumagalli, Venkateswarlu Gudepu, Koteswararao Kondepu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/10/2026, 5:55:30 AM
Summary
The paper introduces ADORN, a Q-learning-based reinforcement learning framework designed to handle model drift in Open RAN networks caused by dynamic traffic variations. It formulates the retraining decision as a Markov Decision Process (MDP), where an RL agent learns an optimal policy to balance forecasting accuracy and computational cost. The approach utilizes a multi-expert LSTM ensemble to mitigate catastrophic forgetting and demonstrates reduced retraining overhead compared to greedy and random baselines while maintaining SLA compliance.
Entities (12)
Relation Signals (11)
ADORN → employs → Q-learning
confidence 97% · This work proposes a Q-learning-based adaptive retraining approach that formulates the retraining decision as a Markov Decision Process (MDP)...
Dynamic Traffic → causes → Drift
confidence 96% · Dynamic traffic variations in Open Radio Access Networks (O-RAN) lead to drift, which degrades the performance of Artificial Intelligence/Machine Learning (AI/ML) models.
ADORN → formulates → Markov Decision Process
confidence 96% · The proposed approach formulates the retraining decision as a sequential decision-making problem using a Markov Decision Process (MDP)...
Drift → degrades → AI/ML Models
confidence 95% · Drift caused by dynamic fluctuations in user traffic produces false insights and inaccurate decisions, leading to poor network management...
Open RAN → uses → AI/ML Models
confidence 95% · The O-RAN Alliance introduces RAN Intelligent Controllers (RICs) that enable data collection across the RAN through standardized open interfaces and incorporate intelligence using Artificial Intelligence and Machine Learning (AI/ML) algorithms.
ADORN → incorporates → LSTM Ensemble
confidence 94% · The proposed approach incorporates a multi-expert Long Short-Term Memory (LSTM) ensemble to mitigate catastrophic forgetting and improve robustness across diverse traffic conditions.
RL Agent → optimizes → Retraining Policy
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Dynamic traffic variations in Open Radio Access Networks (O-RAN) lead to drift, which degrades the performance of Artificial Intelligence/Machine Learning (AI/ML) models. Traditional retraining approaches maintain forecasting accuracy but incur high computational cost and may lead to violations of Service Level Agreements (SLAs). This work proposes a Q-learning-based adaptive retraining approach that formulates the retraining decision as a Markov Decision Process (MDP), where a Reinforcement Learning (RL) agent learns a policy that balances forecasting accuracy and retraining cost. The proposed approach incorporates a multi-expert Long Short-Term Memory (LSTM) ensemble to mitigate catastrophic forgetting and improve robustness across diverse traffic conditions. Experimental results show that the proposed approach effectively reduces retraining overhead compared to greedy and random baselines, while maintaining system performance within predefined limits.
Tags
Links
- Source: https://arxiv.org/abs/2607.08443v1
- Canonical: https://arxiv.org/abs/2607.08443v1
Trouble viewing inline? Open PDF directly →
Full Text
29,872 characters extracted from source content.
Expand or collapse full text
ADORN: Adaptive Drift handling for Open RAN using Reinforcement Learning Ashit Kumar Subudhi1, Bhargav Chirumamilla3, Shubham Vaishnav5, Mduduzi C. Hlophe2, Praveen Kumar Donta5, Andrea Fumagalli4, Venkateswarlu Gudepu4, Koteswararao Kondepu1 Abstract Dynamic traffic variations in Open Radio Access Networks (O-RAN) lead to drift, which degrades the performance of Artificial Intelligence/Machine Learning (AI/ML) models. Traditional retraining approaches maintain forecasting accuracy but incur high computational cost and may lead to violations of Service Level Agreements (SLAs). This work proposes a Q-learning-based adaptive retraining approach that formulates the retraining decision as a Markov Decision Process (MDP), where a Reinforcement Learning (RL) agent learns a policy that balances forecasting accuracy and retraining cost. The proposed approach incorporates a multi-expert Long Short-Term Memory (LSTM) ensemble to mitigate catastrophic forgetting and improve robustness across diverse traffic conditions. Experimental results show that the proposed approach effectively reduces retraining overhead compared to greedy and random baselines, while maintaining system performance within predefined limits. I Introduction The advent of fifth-generation and beyond (B5G) networks marks a significant leap forward in telecommunications and enables a wide range of new services — network slicing, autonomous vehicles, Augmented and Virtual Reality (AR/VR), and e-Health. The B5G networks support high data rates, ultra-low latency, and connectivity for a massive number of devices, which are critical requirements for addressing diverse and dynamic user service demands. However, traditional Radio Access Network (RAN) architectures rely on proprietary hardware with closed and embedded interfaces, which limits flexibility and restricts configurability to meet B5G network requirements [5]. The O-RAN Alliance introduces RAN Intelligent Controllers (RICs) that enable data collection across the RAN through standardized open interfaces and incorporate intelligence using Artificial Intelligence and Machine Learning (AI/ML) algorithms. However, AI/ML model performance relies on the characteristics of the training data. The arrival of new and previously unseen user traffic patterns introduces user data traffic changes, which degrade AI/ML model performance, referred to as drift. Drift caused by dynamic fluctuations in user traffic produces false insights and inaccurate decisions, leading to poor network management, severe service interruptions, inefficient resource allocation, and overall network performance degradation. Such effects result in violations of Service-Level Agreements (SLAs) [1], along with reduced network reliability, increased operational costs, and degraded user experience. Such consequences can be catastrophic for time-critical applications — smart ambulances — where drift-induced performance degradation leads to failures in meeting stringent throughput and response-time requirements. Drift handling for B5G networks primarily relies on performance monitoring, statistical analysis, and data-driven approaches. Threshold-based approaches detect drift when AI/ML model performance metrics — accuracy, precision, recall, or error rate — exceed predefined thresholds, with traditional approaches including Drift Detection Method (DDM) and Early Drift Detection Method (EDDM) [3]. Statistical approaches such as the Fisher score evaluate changes in feature user traffic to identify drift without relying on model error rates [9]. Existing approaches rely on predefined thresholds, require large data samples, and struggle to adapt to highly dynamic, complex network environments. Selecting appropriate threshold values remains challenging: low thresholds increase computational overhead due to frequent retraining, while high thresholds degrade model performance and lead to SLA violations. Data stream-based approaches also suffer from limited adaptability, as static or periodic evaluations fail to capture the continuously evolving nature of real-world data. In [6], the authors introduced a drift adaptation and anomaly detection algorithm for Recurrent Neural Networks (RNNs), where the prediction model is updated incrementally without directly detecting drift. However, the adoption of Reinforcement learning (RL)-based approaches enables adaptive decision-making by learning when to trigger model retraining based on observed system states and has shown potential to improve the trade-off between prediction performance and retraining frequency in dynamic environments. The RL improves threshold-based retraining approaches by enabling adaptive decision-making, rather than static retraining rules, when drift is detected. The RL aims to maximize the cumulative reward, while making proactive and adaptive retraining decisions, unlike conventional static decision-making. Whereas the proposed RL-based approach learns an adaptive retraining policy that dynamically decides when to update the model based on observed system states. Q-learning, a model-free RL approach, is well-suited to the uncertain and dynamic nature of B5G systems, as it does not rely on a system model. In addition, it enables effective exploration-exploitation balances, enabling the proposed approach to dynamically adapt to different traffic patterns while ensuring reliable performance [2]. To overcome the shortcomings of threshold-based decision-making schemes combined with traditional predictive or unsupervised methods for model retraining, this paper proposes an RL-based approach for drift handling in B5G networks. The main contributions of the proposed ADORN approach are summarized as follows: • An RL-based Q-learning approach for drift handling. • A state representation using statistical features of user traffic. • An adaptive Long Short-Term Memory (LSTM) ensemble for dynamic model selection. • An evaluation using Quality of Service (QoS) prediction use case with real-time and synthetic datasets. • A performance comparison of the proposed work with Greedy and Random baseline approaches. Figure 1: System Model I System Model and RL Problem Formulation The proposed system model is based on the O-RAN architecture, defining RICs, i.e., the Near-Real Time (Near-RT RIC) and the Non-Real Time (Non-RT RIC), which enables the intelligence as illustrated in Fig. 1. The RICs enable autonomous optimization of O-RAN by functioning at different timescales, depending on the AI/ML model inference position. For instance, the Non-RT RIC corresponds to the operations that has granularity of >1sec>1\ sec — Offline AI/ML or RL agent training, policy creation and enforcement, and many other; whereas the Near-RT RIC handles functionalities — load balancing, handover, scheduling policy, RAN slicing and many other — that has time scale between 10ms10\ ms and <1sec<1\ sec. Inside the Non-RT RIC, the AI/ML models or RL agents are trained and deployed as rApps. The O-RAN architecture provides multiple interfaces — O1, A1, and E2 —, to facilitate data collection and communication among the RAN components (i.e., central unit (O-CU), distributed unit (O-DU), and radio unit (O-RU)). The AI/ML model management block within the Non-RT RIC plays a key role in drift handling utilizing the proposed RL-based approach, which leverages an RL agent. The trained RL agent generates optimal policies to determine the drift occurrence and communicates with the AI/ML model management block whenever drift is detected. The RL agent can also be deployed as an rApp to further enhance its integration within the RIC. To address drift in B5G networks due to dynamic changes in user traffic, the proposed work employs an ensemble of LSTM models that continuously evaluate performance under evolving network conditions. The proposed approach formulates the retraining decision as a sequential decision-making problem using a Markov Decision Process (MDP) and trains an RL agent to select retraining actions based on observed system states, with the objective of maximizing predictive accuracy while minimizing computational cost. The proposed approach feeds normalized Mean Absolute Error (nMAE) along with statistical features into the RL agent to distinguish normal variations from drift and to balance performance degradation with the number of retraining actions, and the proposed approach defines the RL problem using state, action, and reward components. I-A Preliminaries The proposed approach defines the problem as a MDP represented by the tuple (,,,R,γ)(S,A,P,R,γ), where S denotes the state space, A denotes the action space, (s′∣s,a)P(s s,a) defines the state transition dynamics, R(s,a)R(s,a) represents the reward function, and γ is the discount factor. The elements of the MDP are defined as follows: I-A1 State Space The state space is based on statistical characteristics of incoming traffic data, and the framework represents each state as a discretized 2-tuple: ≜(mean,variance)∈ s_t ( mean, variance ) (1) where S denotes the set of all possible states, and sts_t denotes the system state at time t. I-A2 Action Space The action space is defined as a binary decision to optimize the retraining policy of the specialized LSTM ensemble. At each discrete decision point, the reinforcement learning agent selects an action ata_t as follows: ≜1,Retrain the model0,Remain the same (Idle) a_t cases1, & $ Retrain the model$\\ 0, & $ Remain the same (Idle)$ cases (2) where at=1a_t=1 triggers immediate model retraining to restore predictive accuracy upon drift detection, and at=0a_t=0 avoids unnecessary retraining and preserves computational resources. I-A3 Reward Function The reward function is R(s,a):×→ℝR(s,a):S×A to balance predictive accuracy and computational cost. The reward function incorporates sensitivity to both the presence and magnitude of drift through a magnitude-aware piecewise formulation. Let ete_t denote the nMAE of a particular traffic window, and let θ denote the predefined drift threshold. The drift magnitude as Δt=et−θ _t=e_t-θ. The reward RtR_t for an action at∈0,1a_t∈0,1 is given by: Rt(st,at)=w1|Δt|+C1,if at=1,Δt>0w2|Δt|+C2,if at=0,Δt≤0−w3|Δt|−C3,if at=1,Δt≤0−w4|Δt|−C4,if at=0,Δt>0R_t(s_t,a_t)= casesw_1| _t|+C_1,&if a_t=1,\ _t>0\\ w_2| _t|+C_2,&if a_t=0,\ _t≤ 0\\ -w_3| _t|-C_3,&if a_t=1,\ _t≤ 0\\ -w_4| _t|-C_4,&if a_t=0,\ _t>0 cases (3) where w1,w2,w3,w4w_1,w_2,w_3,w_4 denote weighting coefficients that control sensitivity to drift magnitude, and C1,C2,C3,C4C_1,C_2,C_3,C_4 denote operational constants that represent fixed rewards or penalties associated with each decision. I-B Objective Function and Learning Strategy The proposed approach aims to achieve a balance between high predictive accuracy and low cumulative retraining actions by maximizing the expected cumulative discounted reward: J(π)=τ∼πθ[∑t=0∞γtr(st,at)]J(π)=Eτ πθ [ _t=0^∞γ^tr(s_t,a_t) ] (4) where τ=(s0,a0,r0,s1,a1,…)τ=(s_0,a_0,r_0,s_1,a_1,…) denotes the trajectory of states, actions, and rewards generated under policy πθ _θ, E denotes the expectation over traffic scenarios, and γ∈[0,1]γ∈[0,1] denotes the discount factor. The proposed approach employs a Q-learning algorithm [7] to learn an optimal policy through interaction with the environment. The Q-learning algorithm updates the action-value function Q(st,at)Q(s_t,a_t) using the Bellman update rule: Q(st,at)←Q(st,at)+α[rt+γmaxa′Q(s′,a′)−Q(st,at)]Q(s_t,a_t)← Q(s_t,a_t)+α [r_t+γ _a Q(s ,a )-Q(s_t,a_t) ] (5) where, α denotes the learning rate, s′s denotes the next state after taking action ata_t, a′a denotes the set of possible actions in state s′s , and maxa′Q(s′,a′) _a Q(s ,a ) denotes the maximum expected future reward. This learning process enables the agent to learn a policy that triggers retraining only when the long-term gain in prediction accuracy outweighs the associated computational cost, thereby ensuring efficient and reliable operation in dynamic B5G networks. TABLE I: Summary of Key Notations and Symbols Symbol Description W Incoming 5G Traffic Stream (Temporal Window) ℰi\E_i\ Ensemble of Specialized LSTM Models S State Space (discretized traffic statistics) ata_t Action (at=0a_t=0: Idle, at=1a_t=1: Retrain) ete_t Real-time Forecast Error (nMAE) θ Drift Allowance Threshold (SLA Boundary) Δt _t Drift Magnitude (et−θe_t-θ) Rt(s,a)R_t(s,a) Instantaneous Reward Function Q(s,a)Q(s,a) State-action value function (Q-value) α Learning rate γ Discount factor ϵε Exploration probability for the ϵε-greedy policy wi,Ciw_i,C_i Scaling weights and constant offsets for the reward function π∗π^* Optimized Drift Orchestration Policy 0: 5G Traffic Stream W, Specialist Ensemble ℰi\E_i\, drift threshold θ 0: Optimized Orchestration Policy π∗π^* 1: Initialize: Q-table Q(s,a)←0Q(s,a)← 0, Learning rate α, Discount factor γ 2: for each training episode m∈1,…,Mm∈\1,…,M\ do 3: for each discrete decision-making point t∈1,…,Tt∈\1,…,T\ do 4: Observe system state t∈s_t 5: Route incoming data to state-matched LSTM specialist ℰstE_s_t 6: Compute forecast and evaluate nMAE ete_t 7: Calculate drift magnitude Δt=et−θ _t=e_t-θ 8: Select action at∈0,1a_t∈\0,1\ using ϵε-greedy strategy 9: if at=1a_t=1 then 10: Update weights of ℰstE_s_t to restore accuracy 11: if Δt>0 _t>0 then 12: Rt(st,at)←w1|Δt|+C1R_t(s_t,a_t)← w_1| _t|+C_1 13: else 14: Rt(st,at)←−w3|Δt|−C3R_t(s_t,a_t)←-w_3| _t|-C_3 15: end if 16: else 17: if Δt≤0 _t≤ 0 then 18: Rt(st,at)←w2|Δt|+C2R_t(s_t,a_t)← w_2| _t|+C_2 19: else 20: Rt(st,at)←−w4|Δt|−C4R_t(s_t,a_t)←-w_4| _t|-C_4 21: end if 22: end if 23: Observe subsequent system state s′s 24: Update Q-value: Q(st,at)←Q(st,at)+α[Rt+γmaxa′Q(s′,a′)−Q(st,at)]Q(s_t,a_t)← Q(s_t,a_t)+α [R_t+γ _a Q(s ,a )-Q(s_t,a_t) ] 25: end for 26: end for Algorithm 1 RL-Based Adaptive Drift Handling I-C Proposed Algorithm Algorithm 1 takes the incoming user traffic stream W, the specialist LSTM ensemble ℰiE_i, and the drift threshold θ as inputs and outputs an optimized drift orchestration policy π∗π^*. The proposed approach processes the incoming traffic stream in a sequential manner and, at each decision step, the proposed approach observes the system state s_t based on discretized statistical features such as mean and variance (see line 4), which capture the current operating condition of the network. Then a state-matched LSTM specialist ℰstEs_t is chosen from the ensemble to generate predictions for the current traffic window and evaluates the prediction accuracy by computing the nMAE ete_t (see lines 5–6). Afterwards, the drift magnitude Δt=et−θ _t=e_t-θ is computed to quantify the deviation of the model performance from the acceptable SLA boundary (see line 7), where a positive value indicates the occurrence of drift. The RL agent selects an action at∈0,1a_t∈0,1 using an ϵε-greedy strategy (see line 8), where the strategy balances exploration and exploitation to improve policy learning over time. The action at=1a_t=1 triggers retraining of the selected LSTM model to restore predictive accuracy, whereas the action at=0a_t=0 skips retraining to preserve computational resources. When the agent selects the retraining action, the proposed approach updates the weights of the corresponding LSTM specialist using the most recent traffic data (see lines 9–13), enabling the model to adapt to the new user data traffic. The proposed approach computes the reward RtR_t based on both the selected action and the drift magnitude (see lines 10–18), where the reward function assigns positive rewards to correct decisions, such as retraining under drift or skipping retraining under stable conditions, and assigns penalties to incorrect decisions, such as unnecessary retraining or failure to retrain during drift. After performing the action, the proposed approach observes the subsequent system state s′s and updates the Q-value using the Bellman update rule (lines 19–20), enabling the agent to iteratively refine action-value estimates and improve decision-making. Through repeated interaction with dynamic and varying traffic conditions, the learning process enables the agent to converge toward an optimal policy with the long-term benefit of improved prediction accuracy outweighs the associated computational cost. The proposed adaptive decision-making approach ensures efficient resource utilization while maintaining reliable AI/ML model performance in B5G networks. I Performance Evaluation This section presents the experimental setup used to evaluate the proposed approach. The evaluation aims to assess how effectively the proposed approach manages drift in real-time traffic scenarios while balancing predictive accuracy and computational cost. The proposed approach employs a specialist LSTM ensemble to address catastrophic forgetting observed in single-model approaches. The ensemble consists of multiple state-matched models, each capturing specific traffic patterns, enabling the system to adapt to new traffic conditions while retaining previously learned knowledge. Thus, improves robustness to dynamic traffic changes and enhancing predictive performance under dynamic network conditions. The experiment is conducted across eight distinct traffic scenarios generated from the Colosseum traffic dataset [4], which provides realistic and time-varying network conditions. Each traffic scenario exhibits significantly different mean and variance characteristics, thereby defining the system’s state space. Each time step in the dataset includes features — the number of active users, downlink (DL) traffic rate, and uplink (UL) traffic rate. The proposed approach selects the DL traffic rate as the primary prediction target, as it directly affects user QoS and plays a critical role in O-RAN resource management. The proposed approach pre-processes the dataset by selecting DL traffic corresponding to specific user counts and time steps to capture meaningful traffic variations. Figure. 2 illustrates the experimental workflow of the proposed Q-Learning-based ADORN approach. Figure 2: Proposed Q-Learning based ADORN approach The RL environment defines state space based on statistical traffic features — mean and variance. Specifically, the continuous state space is discretized by rounding, mapping the environment into 8 distinct states. The action space consists of two actions: retrain and idle. The reward function guides the agent to learn an optimal trade-off between prediction accuracy and cumulative retraining actions. Table I summarizes the Q-learning hyperparameters, and Table I summarizes the LSTM ensemble configuration. The agent trains over 300 episodes, each consisting of sequential observations of traffic conditions. The proposed approach randomly shuffles traffic segments at the beginning of each episode to eliminate temporal bias and ensure that the learned policy depends only on statistical state representation rather than sequence ordering, which improves generalization and robustness under varying traffic conditions. TABLE I: Q-Learning Hyperparameters Parameter Value Learning Rate (α) 0.1 Discount Factor (γ) 0.9 Initial Exploration (ϵε) 1.0 Epsilon Decay Rate 0.95 Drift Threshold (θ) 0.25 Total Episodes 300 Weighing Coefficient (w1,w2,w3,w4w_1,w_2,w_3,w_4) 300, 100, 200, 500 Operational Constants (C1,C2,C3,C4C_1,C_2,C_3,C_4) 50, 20, 50, 50 TABLE I: LSTM Ensemble Hyperparameters Parameter Value Sequence Length 5 Hidden Units 64 Number of Layers 2 Dropout Rate 0.2 Training Epochs 150 Optimizer Adam Activation Function ReLU To evaluate the effectiveness of the proposed approach, two baseline policies that represent extreme operational strategies — a Greedy Baseline and a Random Baseline — are considered for comparison. • Greedy Baseline: Prioritizes prediction accuracy without considering the number of retraining actions. The policy selects the retraining action at every decision step, which minimizes drift impact but incurs high computational overhead. • Random Baseline: Selects actions randomly based on a uniform distribution. The policy does not incorporate system state or traffic dynamics and therefore lacks adaptive decision-making capability. To assess the performance of the proposed approach, three key performance metrics are considered, which are as follows: • Normalized Mean Absolute Error (nMAE): It measures forecasting accuracy by quantifying the difference between predicted and actual traffic values. nMAE=1N∑t=1N|yt−y^t|y¯nMAE= 1N _t=1^N |y_t- y_t| y where, yty_t denotes the actual traffic value at time step t, y^t y_t denotes the predicted value, y¯ y denotes the mean of the actual values, N is the total number of samples. nMAE also serves as an indicator for drift detection based on the pre-defined 25% threshold [8]. • Cumulative Reward: It evaluates the effectiveness of the learned policy by aggregating rewards over time, which reflects how well the proposed approach balances prediction accuracy and cumulative retraining actions. • Cumulative Retraining Actions: It quantifies resource efficiency by counting the number of retraining actions, which highlights the ability of the proposed approach to reduce computational and operational overhead compared to baseline approaches. I-A Results Figure. 3 depicts the cumulative retraining actions of the considered approaches across 300 training episodes. The curves are smoothed using a moving average with a window size of 10 episodes. The greedy baseline maintains a constant 8 retraining actions per episode, as the policy always selects the retraining action without considering computational overhead. The random baseline exhibits highly fluctuating behavior, with the smoothed cumulative retraining actions varying approximately 3.0 and 5.4 actions per episode (averaging 4.1 actions), due to stochastic decision-making without awareness of traffic dynamics. The proposed Q-learning-based approach demonstrates a consistent learning trend across episodes, with the number of retraining actions gradually decreasing from approximately 4–6 in the initial episodes to about 1–3 in the later episodes, indicating that the agent learns to avoid unnecessary retraining while maintaining predictive performance. Figure 3: Cumulative Retraining Actions vs Episodes Figure 4: Normalized Mean Absolute Error vs Episodes Figure. 4 presents the nMAE of the considered approaches across 300 training episodes, along with the predefined drift tolerance threshold. The curves are smoothed using a moving average with a window size of 10 episodes. The Greedy baseline achieves the lowest nMAE, gradually decreasing to approximately 0.150.15–0.160.16 (15%15\%–16%16\%) by the end of training due to continuous retraining at every step. The Random baseline stabilizes around 0.180.18–0.190.19 (18%18\%–19%19\%), but lacking stability due to stochastic decision-making. The proposed Q-learning-based approach converges to a stable nMAE of approximately 0.170.17–0.180.18 (17%17\%–18%18\%), remaining consistently below the drift threshold throughout training. Although the Greedy baseline achieves better accuracy, the proposed approach maintains comparable predictive performance while significantly reducing retraining cost, as observed in Fig. 3. Figure 5: Cumulative Reward vs Episodes Figure. 5 depicts the cumulative reward as a function of the number of training episodes. During the early exploration phase (i.e., episodes 11–1515), the proposed approach exhibits negative rewards, fluctuating between approximately −132-132 and −17-17. From around episode 2020, the reward consistently remains positive, fluctuating between −5-5 and 321321, which indicates that the agent begins to learn effective retraining policies. Beyond episode 150150, the reward stabilizes further, remaining within the range of 8080 to 184184, demonstrating policy convergence. The Greedy baseline accumulates consistently negative rewards due to redundant retraining actions that incur penalties from unnecessary computational cost. The Random baseline exhibits highly unstable reward behavior, with frequent penalties caused by both false alarms and unnecessary retraining decisions. The proposed approach effectively learns a balanced retraining policy, outperforming the baseline methods in terms of both decision efficiency and long-term reward. Figure 6: Mean Square Error vs Epochs Figure. 6 illustrates the effectiveness of the proposed multi-expert ensemble architecture by presenting the Mean Squared Error (MSE) of eight specialized LSTM models over 140 training epochs. Each model corresponds to a specific traffic state. The results show a rapid decline in MSE during the initial 10–20 epochs, where most models reduce their initial errors from approximately 0.250.25–0.670.67 to below 0.10.1, indicating fast convergence during early training. As training progresses, all models continue to converge smoothly, reaching stable MSE values between 0.010.01 and 0.020.02 after approximately 60–100 epochs. All eight models exhibit consistent convergence behavior. The proposed approach effectively mitigates catastrophic forgetting by assigning each model to a specific traffic state, instead of relying on a single generalized model. IV Conclusions and Future Work The proposed work introduces ADORN, a Q-learning-based adaptive drift handling approach for O-RAN that formulates the retraining decision as a sequential decision-making problem, enabling intelligent, selective model updates based on real-time system conditions. Experimental results demonstrate an effective balance between forecasting accuracy and computational efficiency. Moreover, the observed reward convergence confirms that the Q-learning agent learns a stable and efficient policy under non-stationary traffic conditions. The discretized state space restricts scalability in highly dynamic environments with continuous traffic variations, and the use of tabular Q-learning limits generalization to unseen states. Future work will address these limitations by exploring Deep Reinforcement Learning (DRL) to enable learning over large continuous state spaces for drift management in large-scale O-RAN deployments. V Acknowledgement This work has been partially supported by TTDF “SMARTRIC6G: Smart Drift-Handling Enabler for RAN Intelligent Controllers in 6G Networks (TTDF/6G/422)” project. References [1] V. Gudepu, V. R. Chintapalli, P. Castoldi, L. Valcarenghi, B. R. Tamma, and K. Kondepu (2024) The Drift Handling Framework for Open Radio Access Networks: An Experimental Evaluation. Computer Networks, p. 110290. Cited by: §I. [2] S. Liu, F. Bronzino, P. Schmitt, A. N. Bhagoji, N. Feamster, H. G. Crespo, T. Coyle, and B. Ward (2023) LEAF: Navigating Concept Drift in Cellular Networks. Proceedings of the ACM on Networking 1 (CoNEXT2), p. 1–24. Cited by: §I. [3] J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang (2018) Learning under Concept Drift: A Review. IEEE transactions on knowledge and data engineering 31 (12), p. 2346–2363. Cited by: §I. [4] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia (2022) ColO-RAN: Developing machine learning-based xApps for open RAN closed-loop control on programmable experimental platforms. IEEE Transactions on Mobile Computing 22 (10), p. 5787–5800. Cited by: §I. [5] M. Polese, L. Bonati, S. D’oro, S. Basagni, and T. Melodia (2023) Understanding O-RAN: Architecture, interfaces, algorithms, security, and research challenges. IEEE Communications Surveys & Tutorials 25 (2), p. 1376–1411. Cited by: §I. [6] S. Saurav, P. Malhotra, V. TV, N. Gugulothu, L. Vig, P. Agarwal, and G. Shroff (2018) Online anomaly detection with concept drift adaptation using recurrent neural networks. In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, CODS-COMAD ’18, p. 78–87. Cited by: §I. [7] R. S. Sutton and A. G. Barto (2018) Reinforcement learning: An introduction second edition. Adaptive computation and machine learning: The MIT Press, Cambridge, MA and London. Cited by: §I-B. [8] H. Zhang, X. Chen, M. Hu, and V. Sugumaran (2024) Multilayer Concept Drift Detection Method Based on Model Explainability. IEEE Access 12 (), p. 190791–190808. Cited by: 1st item. [9] K. Zhang, A. T. Bui, and D. W. Apley (2020) Concept Drift Monitoring and Diagnostics of Supervised Learning Models via Score Vectors. arXiv preprint arXiv:2012.06916. Cited by: §I.