Paper deep dive
Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference
Jiming Su, Hantao Hua, Lujia Yin, Yiping Yao, Feng Zhu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/8/2026, 2:05:29 AM
Summary
The paper introduces AutoThread, a hybrid adaptive thread-tuning method designed to mitigate simulation execution bottlenecks in high-performance Reinforcement Learning (RL) inference systems. AutoThread utilizes a Physics-Informed Neural Operator (PINO) to predict optimal thread counts based on workload characteristics and incorporates an M/M/1 queueing model to constrain predictions. It further employs load-aware online fine-tuning to adjust thread counts dynamically, achieving significant improvements in speedup and throughput over static strategies and other baselines like XGBoost and Reinforcer.
Entities (9)
Relation Signals (6)
AutoThread → appliedto → Reinforcement Learning
confidence 95% · AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference
AutoThread → uses → Physics-Informed Neural Operator
confidence 95% · AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor
AutoThread → uses → M/M/1 queueing model
confidence 95% · incorporates a finite-source M/M/1 queueing model to constrain and guide prediction
AutoThread → appliedto → Discrete Event Simulation
confidence 90% · AutoThread ... for SiL-RL simulation acceleration
AutoThread → outperforms → XGBoost
confidence 90% · achieves average throughput of 1.7x ... that of XGBoost
AutoThread → outperforms → Reinforcer
confidence 90% · achieves average throughput of ... 1.8x that of ... Reinforcer
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations. Existing multithreaded strategies struggle to match thread resources before or during execution, causing resource contention, scheduling overhead, and reduced throughput. Through empirical analysis, we identify the ratio of task execution time to scheduling time as the key factor determining the optimal thread count. Building on this insight, we propose AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference. AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor and incorporates a finite-source M/M/1 queueing model to constrain and guide prediction, enabling fast and accurate estimation under dynamic workloads. It further performs load-aware online fine-tuning to compensate for prediction errors and refine resource allocation. Experiments show that AutoThread improves average speedup by 18.4\% over static strategies, achieves average throughput of 1.7x and 1.8x that of XGBoost and Reinforcer, respectively, and reduces execution time by up to 83.8\% compared with state-of-the-art methods. Our code and dataset are publicly available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.06025v1
- Canonical: https://arxiv.org/abs/2608.06025v1
Trouble viewing inline? Open PDF directly →
Full Text
39,854 characters extracted from source content.
Expand or collapse full text
Hybrid-Adaptive Thread Tuning to Mitigate Simulation Execution Bottlenecks in High-Performance Reinforcement Learning Inference Jiming Su1 Hantao Hua2 Lujia Yin1 Yiping Yao1&Feng Zhu111footnotemark: 1 1College of Systems Engineering, National University of Defense Technology 2College of Computer Science and Technology, National University of Defense Technology jimings, ht_hua, yinlujia10, ypyao, zhufeng@nudt.edu.cn Abstract In simulation-in-the-loop decision-making systems, reinforcement learning (RL) inference is often constrained by simulator-side execution overhead, where workloads are highly dynamic and sensitive to runtime thread configurations. Existing multithreaded strategies struggle to match thread resources before or during execution, causing resource contention, scheduling overhead, and reduced throughput. Through empirical analysis, we identify the ratio of task execution time to scheduling time as the key factor determining the optimal thread count. Building on this insight, we propose AutoThread, a hybrid adaptive thread-tuning method for mitigating simulation bottlenecks in RL inference. AutoThread employs a Physics-Informed Neural Operator (PINO) as a thread-count predictor and incorporates a finite-source M/M/1 queueing model to constrain and guide prediction, enabling fast and accurate estimation under dynamic workloads. It further performs load-aware online fine-tuning to compensate for prediction errors and refine resource allocation. Experiments show that AutoThread improves average speedup by 18.4% over static strategies, achieves average throughput of 1.7× and 1.8× that of XGBoost and Reinforcer, respectively, and reduces execution time by up to 83.8% compared with state-of-the-art methods. Our code and dataset are publicly available at https://github.com/suchenjm/AutoThread. 1 Introduction As artificial intelligence is increasingly used for decision-making in complex dynamic systems, reinforcement learning with simulation-in-the-loop (SiL-RL) has become a core paradigm for wireless communication network optimization and swarm intelligence controlFeriani and Hossain (2021); Hu et al. (2024). Its operational efficiency, however, directly limits real-time decision-making. SiL-RL performance depends on three components: simulation-side model computation (e.g., social network dynamic evolution modelsWu et al. (2022)), agent-side algorithm inference (e.g., Deep Q-Network decision-makingZhang et al. (2021)), and their communication/synchronization. As simulation models grow increasingly complex (e.g., high-precision channel interference modelsEldeeb and Alves (2025) in communication networks or high-fidelity aerodynamic modelsRiboldi et al. (2025) in UAV swarms), the simulation side becomes the critical bottleneckMorgado et al. (2025), increasing end-to-end latency and impairing real-time decisions. This paper therefore accelerates simulation-side performance to break this bottleneck and support efficient SiL-RL systems. Figure 1: Diagram of the DES environment execution process Existing SiL-RL simulation environments typically adopt Discrete Event Simulation (DES) and use multithreading on multicore processors for parallel accelerationBelsare et al. (2022). They mainly employ two architectures: centralized queues with thread pools and distributed queues with work stealingRisco-Martín et al. (2022). The centralized queue, with dynamic load balancing and lower synchronization overhead, is well suited to SiL-RL inference, where agent activation rates and task loads fluctuate across simulation cyclesLin et al. (2025). This work therefore focuses on simulation acceleration under the centralized queue architecture. However, centralized queues are highly sensitive to the number of worker threads (WTs): all WTs share one task queue, and task acquisition and distribution are handled serially by the schedulerSamadi et al. (2024). Thus, the scheduler may become a serial bottleneck: as WTs increase, scheduler contention, lock waiting, and busy waiting rise sharply, negating or even reversing parallel gains. Selecting WTs according to workload characteristics is therefore key to improving simulation-side performanceLi et al. (2025). First, the optimal WT count depends on workload intensity and hardware factors such as core frequency, memory bandwidth, and cache hierarchy, which vary across platforms and are hard to know a priori. Second, SiL-RL workloads are highly dynamic; agent activation rates, task execution times, and interaction intensities fluctuate across simulation stages, making fixed WT counts suboptimal. Runtime noise and external disturbances further obscure transient interference versus persistent workload shifts. Together, these factors prevent precise WT allocation before or during runtime. To address this, we propose AutoThread, a hybrid adaptive thread-tuning method for SiL-RL simulation acceleration. The core idea is to construct a WT count predictor where a neural network is guided and constrained by a theoretical performance model. By introducing a dynamic load-aware adaptive fine-tuning strategy, we achieve significant improvements in simulation performance, thereby supporting high-performance inference in RL decision-making systems. The main contributions of this paper are as follows: • We open-source a large-scale multithreaded trajectory dataset for DES environment. Through empirical analysis, we derive three findings regarding the relationship between WT count and simulation performance. • We develop a Physics-Informed Neural Operator (PINO) thread predictor, utilizing a queueing model to constrain and guide the training process. • We propose AutoThread, which leverages PINO-based thread prediction and load-aware online fine-tuning to achieve fast and adaptive WT count allocation. 2 Empirical Study We first introduce the dataset, then empirically reveal the relationship between WT count and execution performance to guide AutoThread’s model construction and tuning strategy. 2.1 Dataset We constructed the multithreaded trajectory dataset for the DES environment by collecting over 10,000 multithreaded execution samples across both Intel and AMD CPU hardware platforms. The dataset includes two application cases: PCS: The Personal Communication System(PCS) Carothers et al. (2002) is a benchmark DES environment that simulates wireless communication processes within a network topology and is widely used to test decision algorithms in communication networks. In this application, each agent represents a communication node, continuously generating five types of communication tasks. UAV: This is a case study based on the DES paradigm, simulating the autonomous navigation and obstacle avoidance of multiple unmanned aerial vehicles (UAVs) within a specific area. The paths are derived from real-world flight dataFlightAware (2025). Each agent represents a UAV, involving both perception and navigation tasks. To emulate high-fidelity simulation, we embedded array dot products and iterative power computations into each task, producing execution times between 10μs10\, and 800μs800\, . With profiling tools, we captured each sample’s execution trajectories under various WT counts. The data include hardware utilization, application logs, and micro-architectural features, totaling 34 variables. To our knowledge, this is the first public DES trajectory dataset for RL systems and among the largest single-node thread-level trace sets; we release it to support studies on multithreaded behavior and DES acceleration. 2.2 Empirical Observations We conduct three experiments to characterize how workload, hardware, and micro-architectural metrics jointly affect performance. Our analysis yields the following observations. 2.2.1 Dependence of Optimal WTs on Workload Intensity Figure 2: Variation of speedup with WT count under different different application parameters, where t denotes the average task execution time and n denotes the number of agents Figure 3: Hotspot function distribution and thread time consumption in DES environment Figure 2 shows speedup versus WT count under different agent populations and task execution times. In UAV, each agent always generates two fixed tasks with execution time t; in PCS, each agent generates a variable number of tasks whose execution times follow a Poisson distribution with mean t. The optimal WT count is positively correlated with average task execution time: higher workload intensity increases the optimal WT count because longer tasks enlarge parallelizable regions and let more threads improve performance. Conversely, with fixed execution time, the optimal WT count remains nearly unchanged regardless of agent scale(Figure 2(b)). The variance in Figure 2(d) arises because changing the number of PCS agents alters the task-queue execution-time distribution and thus its mean. 2.2.2 Contention-Induced Waiting as the Cause of Degradation We conduct detailed analysis on a subset of samples using VTune. Figure 3 displays the distribution of hotspot functions and thread time consumption. As shown in Figure 3(a), the proportion of time spent on lock operations rises rapidly as the WT count increases. Once the lock-wait time exceeds the task execution time, system performance begins to degrade. Figure 3(b) further shows that the critical point at which busy waiting starts to increase sharply around the optimal WT count is insensitive to inactive wait times such as inactive synchronization and preemption. Figure 4: Variation of DES environment’s micro-architectural Metrics with WT count Although busy waiting time is key to the optimal WT count, measuring it online is challengingJancauskas et al. (2019). We therefore use lightweight metrics to indirectly reflect busy-waiting status, as shown in Fig 4. Figure 4(a) shows that usr_cpuusr\_cpu peaks near the optimal WT count, where CPU cycles are most efficiently used for parallel tasks. Once the WT count exceeds the optimum, system_cpusystem\_cpu (Figure 4(b)) rises sharply, indicating a shift from effective computation to OS scheduling and thread waiting. Since busy-waiting threads still occupy physical cores, aggregate CPU utilization does not accurately reflect the optimal WT count (see Figure 4(c)). Figure 4(d) further shows that the context switch rate reaches a local minimum near the optimum, suggesting that an appropriate thread count reduces unnecessary scheduling switches. In contrast, cache miss rate (Figure 4(e)) and memory consumption (Figure 4(f)) affect DES performance but not optimal WT selection. 2.2.3 Linear Interference Under Multicore Concurrency Figure 5 records the task execution time and task scheduling time for an individual working thread under varying numbers of active threads. The results show that as the number of active threads increases, both the execution and scheduling times for a single thread grow linearly. This indicates that the performance degradation of individual threads in a multithreaded environment can be approximated by a linear model, which captures the aggregate pressure exerted by all threads on a single thread’s runtime. The slope and intercept of these linear functions depend on hardware characteristics (e.g., core frequency, cache size) and workload intensity. Figure 5: Performance interference among multiple WTs 3 Methodology Based on the empirical findings in Section 2, we propose AutoThread. AutoThread consists of two core components: a PINO predictor and a dynamic tuner. This approach adopts a two-stage optimization strategy of online prediction followed by dynamic adjustment: during the online prediction stage, it rapidly estimates the optimal initial thread count based on the characteristics of the current workload; during the dynamic adjustment stage, it performs fine-grained adaptive thread tuning according to runtime feedback. Figure 6: Overall workflow of AutoThread 3.1 Model-Informed PINO Predictor Performance Model. In typical SiL-RL inference workflows, the simulator must process a large number of computation tasks generated by multiple agents within each simulation step. We decompose the execution of a single simulation cycle into three sequential phases: task execution, task scheduling, and synchronization. WT reconfiguration is performed at the granularity of simulation cycles, during which the WT count remains fixed. During the task execution phase, WTs process assigned tasks in parallel. Since the system must wait for all threads to finish before synchronization, the total execution time is determined by the slowest thread: Texe(k)=⌈Nk⌉∗1μeT_exe(k)= Nk * 1 _e where N is the total number of tasks within the current simulation step; k is the number of WTs; and μe _e denotes the average service rate of tasks within the current simulation step. Scheduling is inherently serial because the shared scheduler serves at most one WT at a time. We model the scheduler as an M/M/1 server and WTs as customers in a finite-source closed queueing system, where a WT requests scheduling only after finishing its current task, yielding an effective arrival rate λ=kμeλ=k _e. When the system is stable, steady-state Continuous-Time Markov Chain (CTMC) analysis gives: Tsch=N∗(kμb(1−P0)−1kμe), T_sch=N ( k _b(1-P_0)- 1k _e), P0=1∑i=0k!(k−i)!(kμeμb)i P_0= 1 _i=0^k k!(k-i)!( k _e _b)^i where μb _b denotes the scheduler service rate and PoP_o is the idle probability of the scheduler. When kμek _e exceeds μb _b, the scheduler becomes the dominant bottleneck and the total time is bounded by the scheduler capacity: Texe(k)+Tsch(k)=Nμb+1μeT_exe(k)+T_sch(k)= N _b+ 1 _e This captures the saturation regime under oversubscription. The synchronization phase (e.g., global state update and clock advance by the master thread) is treated as constant with respect to k when focusing on WT selection. Based on Section 2.2.3, resource contention causes performance degradation that grows linearly with active WT count: μb(k) _b(k) =1a1k+tbase1, = 1a_1k+t_base1, μe(k) _e(k) =1a2k+tbase2 = 1a_2k+t_base2 where a1a_1 and a2a_2 denote contention intensity coefficients for scheduling and execution, respectively, and tbase1t_base1, tbase2t_base2 represent baseline times under single-thread execution. These parameters are determined by hardware platform and application context. Combining the above, the total execution time of a single simulation cycle can be expressed as: T(k)=⌈Nk⌉∗1μe+N∗(kμb(1−P0)−1kμe),if kμe≤μbNμb+1μe,if kμe>μb.T (k )= \ matrix Nk 1 _e+N ( k _b (1-P_0 )- 1k _e ),if k _e≤ _b\\ N _b+ 1 _e,if k _e> _b.\ \\ matrix . (1) The above equation shows that execution time is a piecewise-continuous, non-smooth function of the WT count k. Numerical analysis reveals that the optimal configuration occurs at k∗≈μb/μek^*≈ _b/ _e, indicating optimal performance when scheduler service rate balances with task execution rate. PINO Predictor. While μe _e can be estimated online from probe-collected execution times, directly measuring μb _b is difficult due to microsecond-scale sensitivity and measurement overhead. We therefore introduce a Physics-Informed Neural Operator (PINO) to fuse the above model with historical traces for fast online prediction. We define a neural operator θ:a→k∗=θ(a)G_θ:a→ k^*=G_θ (a ), which maps system state parameters a to the optimal WT number k∗k^*. We construct the feature vector a=N,μe,kobs,usr_cpu,system_cpu,VSZ,cswcha=\N, _e,k_obs,usr\_cpu,system\_cpu,VSZ,cswch\ from both application and hardware features. Although virtual memory usage (VSZ) has limited impact on thread decisions, excessive memory consumption may trigger resource contention or even abnormal program termination; thus, it is included as an input to enhance robustness. As illustrated in Figure 6, PINO predictor employs a multi-output head cross-attention architecture. Encoders separately process application and hardware features, cross-attention captures their deep interactions, and FC layers decode fused features into optimal WT count k∗k^* and contention coefficients a1a_1, b1b_1. Physical loss: ℒphy=‖T(N,k,μb,μe)−Tobs‖2+α‖μbμe−k∗‖2L_phy=\|T(N,k, _b, _e)-T_obs\|_2+α\| _b _e-k^*\|_2 Supervised loss and Regularization term: ℒsup=‖k∗−kobs‖2,ℒreg=γ‖θ‖2L_sup=\|k^*-k_obs\|_2, _reg=γ\|θ\|_2 where T represents the theoretical performance model (Equation 1), μb _b is derived from the contention model as μb=a1k+b1 _b=a_1k+b_1, and TobsT_obs and kobsk_obs denote the observed task queue execution time and iteratively measured optimal WT number, respectively. The weight α normalizes contributions of the two physical components. The overall training objective is ℒloss=ℒphy+λℒsup+ℒregL_loss=L_phy+ _sup+L_reg. We incorporate Residual-Based Attention (RBA) Anagnostopoulos et al. (2024) to dynamically adjust λ between physical and supervised losses. In out-of-distribution scenarios, RBA automatically increases the physical-loss weight, strengthening reliance on the performance model and improving prediction robustness. 3.2 Adaptive Dynamic Tuner Based on the trained PINO predictor, we design an adaptive dynamic tuner adopting a three-tier progressive adjustment strategy. The tuner aims to establish a closed-loop control process of prediction–execution–feedback–readjustment, so as to mitigate adverse effects caused by prediction errors, system noise, and measurement bias. The complete workflow is presented in Algorithm 1. Load-Aware Fast Prediction. During each execution cycle, the system automatically records the average service time of the current task queue tcurtcur using probe pointer and computes the average processing time of the queue TcurTcur (lines 1-3). When a change in tcurt_cur exceeding a preset threshold is detected, the tuner immediately invokes PINO to predict the optimal WT count and update it (lines 4–6). This mechanism ensures fast responsiveness to sudden load changes. CPU-Aware Periodic Adjustment. To avoid continuous prediction overhead and handle gradually varying loads, a periodic checking mechanism is introduced. The check interval is dynamically adaptive: if prediction was triggered in the previous cycle, a short interval C1C_1 is used to verify and correct predictions quickly; otherwise, a longer interval C2C_2 is adopted to reduce monitoring overhead (line 7). At a checkpoint, if the observed system_cpusystem\_cpu exceeds the usr_cpuusr\_cpu, the algorithm proactively reduces the number of WTs to mitigate contention (lines 8–16). This mechanism effectively maps CPU indicators of resource contention into direct signals for thread adjustment, thereby preventing performance degradation due to oversubscription. Throughput-Driven Fine-tuning. When the system is detected to be in a high-contention state, the tuner switches to a fine-tuning phase. In this phase, throughput of the task queue is directly optimized as the adjustment objective. If reducing threads leads to improved throughput, the adjustment is accepted and further attempts are made; otherwise, the change is rolled back and the phase is terminated (lines 17–25). Considering that undersubscription has a less detrimental effect on performance than the severe contention caused by oversubscription, the fine-tuning phase only considers reducing threads. Moreover, to avoid erroneous adjustments due to random fluctuations or transient anomalies, the tuner exits this phase only if throughput fails to improve over three consecutive simulation steps or if the thread count reaches the minimum allowed value (line 20). Algorithm 1 Three-Level Progressive Thread Adjustment 1:Input: δ (load change threshold), C1C_1 (short period), C2C_2 (long period) 2:Output: WT count k for each simulation step 3:while simulation not finished do 4: tcur←ProbeTaskTime()t_cur← ProbeTaskTime() 5: Tcur←ExecuteCycle()T_cur← ExecuteCycle() 6: if |tcur−tprev|/(tprev+ϵ)>δ|t_cur-t_prev|/(t_prev+ε)>δ and tuning=0tuning=0 then 7: k←PINO-Predictk← PINO-Predict 8: predicted←Truepredicted← True 9: else if predicted and (tickmodC1=0)(tick C_1=0) then 10: (system_cpu,usr_cpu)←GetCpuMetrics()(system\_cpu,usr\_cpu)← GetCpuMetrics() 11: if usr_cpu<system_cpuusr\_cpu<system\_cpu then 12: k←k−1k← k-1, tuning←1tuning← 1 13: end if 14: predicted←Falsepredicted← False 15: else if ¬predicted and (tickmodC2=0)(tick C_2=0) then 16: if usr_cpu<usr_cpuusr\_cpu<usr\_cpu then 17: k←k−1k← k-1, tuning←1tuning← 1 18: end if 19: else if tuning≥1tuning≥ 1 and k>kmink>k_ then 20: if Tcur<TprevT_cur<T_prev then ⊳ Throughput improved 21: k←k−1k← k-1, tuning←1tuning← 1 22: else if tuning<3tuning<3 then ⊳ Tolerate fluctuation 23: tuning←tuning+1tuning +1 24: else⊳ Rollback and exit tuning 25: k←k+1k← k+1, tuning←0tuning← 0 26: end if 27: end if 28: tprev←tcurt_prev← t_cur, Tprev←TcurT_prev← T_cur 29: Synchronize(); NextTick() 30:end while 4 Evaluation In this section, we first introduce the experimental setup and current thread tuning methods. Subsequently, we evaluate the proposed method against these baselines in terms of system acceleration performance and adaptability. Finally, we conduct ablation studies to quantify the contributions of each optimization module and verify the robustness of our method with respect to various hyperparameters. 4.1 Setup The training set is derived from averaged statistics over the entire execution under constant-load scenarios, where the execution time of each task remains unchanged throughout the simulation horizon. We split the dataset into training/testing/validation sets with a ratio of 0.7/0.2/0.1 to offline train different predictors. During training, each feature vector is taken as the average over the whole execution; since the load is constant, the optimal thread count label for each sample can be obtained by sweeping different thread counts and recording the total execution time. In the testing phase, we use dynamic workloads to evaluate the speedup and adaptivity of each method. Specifically, within each simulation step, task execution times follow a Poisson distribution parameterized by t, while the parameter t varies across steps as a square wave with frequency rohroh (switching among [t,1.5t,2t,1.5t,2t]). Each method adjusts based on instantaneous values of the feature variables at a single time point, so as to emulate a practical deployment setting with offline training and online testing. Experiments are conducted on two multicore architectures: AMD EPYC 9734 (112 Cores/224 Threads) and Intel Xeon Gold 6338 (32 Cores/64 Threads). The baselines include ADAPT-TCosta et al. (2019) based on runtime feedback control; OtterLuan et al. (2022) employing search strategies; Thread ReinforcerPusukuri et al. (2011) utilizing system resource utilization; XgboostAkash et al. (2021) as an advanced representative of machine learning-based prediction. Additionally, a fixed thread count configuration was used as a control experiment, denoted as Static. All methods were implemented on the Repast HPC systemZhu et al. (2017). Each test case was executed at least five times to obtain optimal measurement data. Figure 7: Runtime comparison of different methods on AMD platform for UAV Figure 8: Adaptiveness comparison on PCS and UAV Figure 9: Results of ablation experiments 4.2 Performance Experiments We run the UAV benchmark on the AMD platform with the execution horizon set to 5000 and the load threshold δ=0.3δ=0.3, and record the total runtime from end to-end under different thread adjustment schemes; the results are shown in Fig. 7. Compared to ADAPT-T, AutoThread saves up to 83.8% of runtime in different scenarios. Over 20 test scenarios, AutoThread achieves an average runtime of 149.32 s and delivers 10.3% and 17.2% higher speedup than the state-of-the-art Otter and Reinforcer, respectively. Notably, we set the thread count of the fixed strategy to 20, i.e., the average output of all tuning schemes. Even when accounting for the additional overhead introduced by frequent thread adjustments, AutoThread still attains, on average, an 18.4% higher speedup than the fixed strategy with prior knowledge, with a maximum performance improvement of 62.6%. 4.3 Adaptiveness Experiments We run the PCS and UAV applications on both Intel and AMD platforms under different load-variation intensities (frequency rohroh, amplitude t) , and record the system throughput over environment steps to evaluate the adaptability of different schemes, where throughput denotes the number of tasks processed per millisecond. Figure 8 shows that, in practical scenarios, when competing methods successively exhibit non-convergent oscillations and extrapolation failures, AutoThread still maintains strong parallel speedup. On the PCS case, AutoThread attains an average throughput of 536.83, improving over ADTPT-T and Xgboost by 37.7% and 2.3%, respectively, while remaining below Reinforcer’s 718.55. Under the UAV case with larger workload fluctuations, AutoThread’s average throughput is 1.8× and 1.7× that of Reinforcer and Xgboost, respectively. 4.4 Ablation Experiments We ablate the PINO predictor and the Adaptive Dynamic Tuner to investigate the performance contribution of each component. The compared variants include: using only the physics-informed neural operator network prediction (PINO); using the dynamic tuner with the maximum thread count (Dyn); applying the Dyn tuner on top of the Xgboost output (Xgboost+Dyn); and pairing the ADAPT-T tuner with the output of PINO (ADAPT+PINO). Figure 9 reports the average throughput of different schemes when running the UAV application on the ADM platform. The results show that both the PINO predictor and the Adaptive Dynamic Tuner effectively improve runtime performance. Across 10 test scenarios, the Dyn technique increases the task processing speed of Xgboost by 87.8% on average, with a maximum improvement of 55.2×. PINO increases the task processing speed of the ADAPT-T tuner by 3.9× on average. Moreover, AutoThread improves average throughput by 1.3× and 2.0× over Xgboost+Dyn and ADAPT+PINO, respectively, demonstrating the combined benefits of the two techniques. 4.5 Robustness Experiment We further examine the impact of parameters δ, C1C_1 and C2C_2 on AutoThread’s performance. The parameter δ represents the maximum tolerable degree of workload change for thread adjustment. An overly small value leads to frequent thread changes due to random hardware jitter, whereas an overly large value prevents timely adaptation to workload changes. AutoThread’s robustness to δ is shown in Figure 10. Benefiting from the theoretical constraints of the physical model and the error-correction capability of the dynamic tuner, AutoThread provides stable speedup even under workload inputs with substantial randomness. In contrast, an excessively large δ weakens the predictor’s sensitivity to workload changes, resulting in performance degradation. Figure 10 shows the impact of different C1/C2C_1/C_2 combinations on system performance. We find that a smaller C1C_1 enables faster response; however, an overly small C2C_2 causes frequent periodic checks and increases system overhead, while an overly large C2C_2 may cause the system to miss critical tuning opportunities. The optimal ratio is approximately 2–4, within which short-cycle validation and long-cycle monitoring achieve a good balance—responding quickly to workload changes while avoiding excessive monitoring overhead. Figure 10: Robustness analysis of δ and C1/C2C_1/C_2 parameter 4.6 Discussion The experimental results clearly demonstrate the advantages of AutoThread in sustaining parallel speedup and substantially reducing execution time, particularly in environments with fluctuating workloads. The key strength of AutoThread lies in the deep integration of data-driven representational capacity with the structural constraints imposed by physical models: on the one hand, it learns latent prior patterns from runtime data to enable rapid prediction under dynamic workloads; on the other hand, by introducing interpretable theoretical models as inductive biases and training constraints, it anchors the prediction process in consistent mechanistic logic, thereby significantly improving robustness and the achievable performance upper bound while preserving interpretability and transferability. This form of mechanism-guided learning coupled with data-corrected inference is central to AutoThread’s superior effectiveness. Specifically, first, guided by a performance model, PINO enforces queueing-theoretic constraints on the neural network’s predictive outputs, endowing the predictions with strong generalization and physical plausibility; consequently, it can provide reliable initial estimates of thread counts even under workload conditions whose distributions differ from those seen during training. Second, the dynamic tuning mechanism employs a three-level strategy to perform online corrections to the predicted results, while accounting for the overhead of monitoring and adjustment. This enables the system to rapidly converge to a near-optimal thread configuration when the workload changes, thereby accelerating the simulation environment’s runtime efficiency. 5 Related Work Many WT tuning methods have been proposed across domains; however, compared with this study, they insufficiently address dynamic workload fluctuations or generalizability. Static modeling methods set fixed thread counts through empirical heuristics or offline analysis, such as using a multiple of physical coresMontesano et al. (2024) or estimating optimum from the ratio of thread creation to maintenance overhead Luo et al. (2025). Although simple and efficient, these strategies struggle in complex, volatile environments because they cannot adjust to workload fluctuations in real time. Machine learning has enabled data-driven thread-configuration prediction, including Reinforcement LearningXu et al. (2012), Ensemble Learning Chao et al. (2025), and online search optimizationStetsenko and Dyfuchyna (2019), with applications in large-scale cluster management. However, these methods often rely on high-quality training data; under dynamic scenarios absent from training, pure data-driven models may generalize poorly. Another common category uses adaptive regulation based on online feedback control Albalawi (2025). These methods incrementally adjust WT count, one thread at a time, using metrics such as request arrival rates and service times until performance no longer improves. Although practical for cloud services, such feedback-driven regulation often suffers from slow convergence, latency, and performance oscillations in large thread spaces or under frequent workload variations Bahadur et al. (2025). 6 Conclusion This paper proposes AutoThread, a thread-count tuning method for ABS systems to accelerate RL inference. The core contributions are as follows: (1) we open-source a multithreaded trajectory dataset collected from a DES simulation environment, and empirically derive three key insights; (2) we design a PINO-based thread predictor and establish a theoretical model relating the number of WTs to runtime, which is used to constrain and guide model training; and (3) we develop an efficient hybrid thread adjustment method that incorporates a load-aware adaptive tuning strategy on top of PINO’s predictions. Experimental results show that AutoThread delivers excellent parallel speedup and strong workload adaptivity. The proposed approach is portable and can be applied to accelerate other systems based on a shared event queue. Future work will explore batch-sample thread tuning methods for accelerating RL training. Acknowledgments We thank the anonymous IJCAI reviewers for their insightful comments. This work was partially supported by the National Natural Science Foundation of China (NSFC) under Grant No.62503491. Contribution Statement Jiming Su and Hantao Hua contributed equally to this work. Feng Zhu is the corresponding author. References L. Akash, D. Fernando, M. Jayasinghe, C. Keppitiyagama, and K. Thangarajah (2021) Machine learning based thread pool tuning via program analysis. In 2021 IEEE 23rd Int Conf on High Performance Computing & Communications; 7th Int Conf on Data Science & Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys), p. 648–653. Cited by: §4.1. N. S. Albalawi (2025) Dynamic scheduling strategies for cloud-based load balancing in parallel and distributed systems. Journal of Cloud Computing 14 (1), p. 33. Cited by: §5. S. J. Anagnostopoulos, J. D. Toscano, N. Stergiopulos, and G. E. Karniadakis (2024) Residual-based attention in physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 421, p. 116805. Cited by: §3.1. F. Bahadur, Z. Ahmad, and A. Algarni (2025) PoolRunner: an extensible performance testing simulation tool for thread-pool middleware. IEEE Access. Cited by: §5. S. Belsare, E. D. Badilla, and M. Dehghanimohammadabadi (2022) Reinforcement learning with discrete event simulation: the premise, reality, and promise. In 2022 Winter Simulation Conference (WSC), p. 2724–2735. Cited by: §1. C. D. Carothers, D. Bauer, and S. Pearce (2002) ROSS: a high-performance, low-memory, modular time warp system. Journal of parallel and distributed computing 62 (11), p. 1648–1669. Cited by: §2.1. Z. Chao, F. Gu, J. Huang, W. Li, J. Ye, H. Li, and X. Li (2025) Memory-efficient and adaptive heterogeneous framework for gate-level fault simulation. ACM Transactions on Design Automation of Electronic Systems 30 (5), p. 1–27. Cited by: §5. N. Costa, M. Jayasinghe, A. Atukorale, S. Abeysinghe, S. Perera, and I. Perera (2019) Adapt-t: an adaptive algorithm for auto-tuning worker thread pool size in application servers. In 2019 IEEE Symposium on Computers and Communications (ISCC), p. 1–6. Cited by: §4.1. E. Eldeeb and H. Alves (2025) Offline and distributional reinforcement learning for wireless communications. arXiv preprint arXiv:2504.03804. Cited by: §1. A. Feriani and E. Hossain (2021) Single and multi-agent deep reinforcement learning for ai-enabled wireless networks: a tutorial. IEEE Communications Surveys & Tutorials 23 (2), p. 1226–1252. Cited by: §1. FlightAware (2025) FlightAware - flight tracker / flight status. FlightAware, Eleven Greenway Plaza, Suite 2900, Houston, Texas 77046, USA. Note: https://uk.flightaware.com/Accessed: 2025-10-25 Cited by: §2.1. K. Hu, M. Li, Z. Song, K. Xu, Q. Xia, N. Sun, P. Zhou, and M. Xia (2024) A review of research on reinforcement learning algorithms for multi-agents. Neurocomputing 599, p. 128068. Cited by: §1. V. Jancauskas, T. Piontek, P. Kopta, and B. Bosak (2019) Predicting queue wait time probabilities for multi-scale computing. Philosophical Transactions of the Royal Society A 377 (2142), p. 20180151. Cited by: §2.2.2. C. Li, J. Wang, J. Shi, L. Liu, and S. Zhang (2025) ADWTune: an adaptive dynamic workload tuning system with deep reinforcement learning. Complex & Intelligent Systems 11 (4), p. 192. Cited by: §1. H. Lin, X. Yu, K. Zhao, H. Bao, Z. Zhan, T. Hu, W. Liu, Z. Yin, X. Li, and W. Liu (2025) HAP: hybrid adaptive parallelism for efficient mixture-of-experts inference. arXiv preprint arXiv:2508.19373. Cited by: §1. G. Luan, P. Pang, Q. Chen, S. Xue, Z. Song, and M. Guo (2022) Online thread auto-tuning for performance improvement and resource saving. IEEE Transactions on Parallel and Distributed Systems 33 (12), p. 3746–3759. Cited by: §4.1. X. Luo, W. Lin, J. Li, F. Chen, H. Zhong, and K. Li (2025) An extensible thread throttling method for multiple openmp parallel programs. ACM Trans. Embed. Comput. Syst.. External Links: ISSN 1539-9087, Link, Document Cited by: §5. F. Montesano, R. Marotta, and F. Quaglia (2024) Spatial/temporal locality-based load-sharing in speculative discrete event simulation on multi-core machines. ACM Transactions on Modeling and Computer Simulation 35 (1), p. 1–31. Cited by: §5. Â. Morgado, K. Ota, M. Dong, and N. Pombo (2025) Evaluating end-to-end autonomous driving architectures: a proximal policy optimization approach in simulated environments. Autonomous Intelligent Systems 5 (1), p. 14. Cited by: §1. K. K. Pusukuri, R. Gupta, and L. N. Bhuyan (2011) Thread reinforcer: dynamically determining number of threads via os level monitoring. In 2011 IEEE International Symposium on Workload Characterization (IISWC), p. 116–125. Cited by: §4.1. C. E. Riboldi, M. Tomasoni, et al. (2025) Formation flight of fixed-wing uavs: dynamic modeling, guidance design, and testing in realistic scenarios. Aerospace 12 (3), p. 1–49. Cited by: §1. J. L. Risco-Martín, K. Henares, S. Mittal, L. F. Almendras, and K. Olcoz (2022) A unified cloud-enabled discrete event parallel and distributed simulation architecture. Simulation Modelling Practice and Theory 118, p. 102539. Cited by: §1. M. Samadi, S. Royuela, L. M. Pinho, T. Carvalho, and E. Quiñones (2024) Time-predictable task-to-thread mapping in multi-core processors. Journal of Systems Architecture 148, p. 103068. Cited by: §1. I. V. Stetsenko and O. Dyfuchyna (2019) Thread pool parameters tuning using simulation. In International Conference on Computer Science, Engineering and Education Applications, p. 78–89. Cited by: §5. Z. Wu, Q. Zhou, Y. Dong, J. Xu, A. H. Altalhi, and F. Herrera (2022) Mixed opinion dynamics based on degroot model and hegselmann–krause model in social networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53 (1), p. 296–308. Cited by: §1. C. Xu, J. Rao, and X. Bu (2012) URL: a unified reinforcement learning approach for autonomic cloud management. Journal of Parallel and Distributed Computing 72 (2), p. 95–105. Cited by: §5. C. Zhang, S. R. Kuppannagari, and V. K. Prasanna (2021) Parallel actors and learners: a framework for generating scalable rl implementations. In 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), p. 1–10. Cited by: §1. F. Zhu, Y. Yao, W. Tang, and J. Tang (2017) A hierarchical composite framework of parallel discrete event simulation for modelling complex adaptive systems. Simulation Modelling Practice and Theory 77, p. 141–156. Cited by: §4.1.