Paper deep dive
Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving
Mehdi Azarafza, Faezeh Pasandideh, Ali Ehteshami Bejnordi, Stefan Henkler, Achim Rettberg
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 8/21/2026, 2:51:40 AM
Summary
The paper proposes a hybrid autonomous driving framework that integrates Large Language Models (LLMs) with Reinforcement Learning (RL) and rule-based control. An orchestrator coordinates specialized agents (Safety, Situation, Planning, Behavior) with different Automotive Safety Integrity Levels (ASIL). LLMs are used offline to refine RL reward functions and generate common-sense driving rules, while runtime safety is enforced by deterministic agents, mitigating LLM latency and hallucination risks. The system is evaluated in the CARLA simulator.
Entities (13)
Relation Signals (13)
Multi-Agent Orchestration Framework → evaluatedin → CARLA
confidence 98% · evaluated in highly randomized CARLA scenarios
Behavior Agent → hassafetylevel → ASIL
confidence 95% · Behavior Agent: rated at ASIL-A
Safety Agent → hassafetylevel → ASIL
confidence 95% · Safety Agent: rated at ASIL-D
Situation Agent → hassafetylevel → ASIL
confidence 95% · Situation Agent: rated at ASIL-C
Planning Agent → hassafetylevel → ASIL
confidence 95% · Planning Agent: rated at ASIL-B
Large Language Models → usedfor → Common-Sense Reasoning
confidence 95% · LLM common-sense reasoning applied throughout the framework
YOLOv11 → usedin → Perception
confidence 95% · YOLOv11 model ... processes camera images to detect and classify traffic participants
Multi-Agent Orchestration Framework → uses → Large Language Models
confidence 95% · LLM common-sense reasoning applied throughout the framework
Multi-Agent Orchestration Framework → uses →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios. While reinforcement learning and rule-based methods can provide effective control and safety mechanisms, their performance may degrade in situations requiring contextual reasoning. Large Language Models (LLMs) have demonstrated strong capabilities in understanding multimodal information and generating contextual reasoning, however, their use for direct vehicle control can introduce latency and hallucination risks. To address these limitations, a hybrid framework is proposed. This system uses an orchestrator to coordinate PPO-trained reinforcement learning and PID control, with LLM common-sense reasoning applied throughout the framework. LLM reasoning is further employed iteratively to refine the RL reward function for dynamic driving environments. The proposed framework is evaluated in highly randomized CARLA scenarios under diverse environmental and traffic conditions. The results demonstrate the potential of integrating LLM-based reasoning with conventional autonomous driving methods while retaining structured control and safety mechanism.
Tags
Links
- Source: https://arxiv.org/abs/2608.20129v1
- Canonical: https://arxiv.org/abs/2608.20129v1
Trouble viewing inline? Open PDF directly →
Full Text
37,761 characters extracted from source content.
Expand or collapse full text
Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving Mehdi Azarafza Affiliation: Department of Computer Science Affiliation: Hamm-Lippstadt University Affiliation: of Applied Sciences, Germany Email: mehdi.azarafza@hshl.de Faezeh Pasandideh Affiliation: Department of Computer Science Affiliation: Hamm-Lippstadt University Affiliation: of Applied Sciences, Germany Email: Faezeh.Pasandideh@hshl.de Ali Ehteshami Bejnordi Affiliation: Department of Computer Science Affiliation: Hamm-Lippstadt University Affiliation: of Applied Sciences, Germany Email: ali.ehteshami-bejnordi@hshl.de Stefan Henkler Affiliation: Department of Computer Science Affiliation: Hamm-Lippstadt University Affiliation: of Applied Sciences, Germany Email: stefan.henkler@hshl.de Achim Rettberg Affiliation: Department of Computer Science Affiliation: Hamm-Lippstadt University Affiliation: of Applied Sciences, Germany Email: achim.rettberg@hshl.de Abstract Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios. While reinforcement learning and rule-based methods can provide effective control and safety mechanisms, their performance may degrade in situations requiring contextual reasoning. Large Language Models (LLMs) have demonstrated strong capabilities in understanding multimodal information and generating contextual reasoning, however, their use for direct vehicle control can introduce latency and hallucination risks. To address these limitations, a hybrid framework is proposed. This system uses an orchestrator to coordinate PPO-trained reinforcement learning and PID control, with LLM common-sense reasoning applied throughout the framework. LLM reasoning is further employed iteratively to refine the RL reward function for dynamic driving environments. The proposed framework is evaluated in highly randomized CARLA scenarios under diverse environmental and traffic conditions. The results demonstrate the potential of integrating LLM-based reasoning with conventional autonomous driving methods while retaining structured control and safety mechanisms. 1 Introduction In recent years, autonomous vehicles have advanced significantly. However, the complexity of real-world environments remains a major challenge for safe and reliable driving Feng et al. 2025. Despite recent progress, Reinforcement Learning (RL) remains limited by its dependence on reward-driven optimization, which provides no formal guaranties of safe behavior and often restricts generalization beyond trained environments Zhang et al. 2024. Rule-based systems are constrained by predefined logic and struggle in dynamic conditions. In contrast, Large Language Models (LLMs) demonstrate better flexibility and stronger cross-domain generalization by integrating diverse contextual information Zhang et al. 2024 , although their use in real-time control remains challenging. Furthermore, human driving requires not only vehicle control, but also visual perception and deductive reasoning, the ability to infer hidden or future states, perform counterfactual reasoning, and distinguish between visually similar but semantically different scenarios Kothawade et al. 2021. These capabilities highlight the need for autonomous driving systems that can combine low-level vehicle control with contextual reasoning and decision-making under diverse and uncertain driving conditions. Table 1 presents a comparison of these methods. To address these challenges, a hybrid approach is proposed that combines rule-based methods and reinforcement learning while leveraging the reasoning capability of LLMs through an orchestrator. Instead of using the LLM as a direct decision-maker in real time (due to latency and potential hallucination issues) we employ it as an advisory component and as a reward mechanism for the RL model. This design enables improved reasoning support while maintaining real-time performance and system reliability. Figure 1 illustrates the high-level system architecture, showing the organization of its components. Figure 1: High-level system architecture. The hybrid architecture is structured in multiple layers, beginning with the perception layer implemented in CARLA Dosovitskiy et al. 2017, where camera and LiDAR sensors are employed for environmental sensing. The knowledge base layer incorporates the LLM as both a reasoning module and a reward model, along with the mimic of ISO-based configuration. This is followed by a multi-agent layer, and finally the driving mode layer, which generates the control actions. Table 1: Comparison of decision-making methods in autonomous driving. The table highlights the strengths and limitations of LLM-, RL-, rule-based, and hybrid approaches across key decision-making criteria. Criteria LLMa RLb Rule-based Hybrid Decision-making ✓ ✓ ✓ ✓ Flexibility ✓ × × ✓ Generalization ✓ × × ✓ Real-time Response × ✓ ✓ ✓ Common-Sense Reasoning ✓ × × ✓ aLarge Language Model, bReinforcement Learning. 2 Related Work Integrating LLMs into autonomous driving RL policies as direct decision-makers risks hallucination and unsafe execution. To mitigate this, HCRMP Chen et al. 2025 introduces LLM-Hinted RL, restricting the LLM to generating semantic state hints while bridging low-frequency reasoning and high-frequency control via memory caching and contextual anchoring. However, HCRMP struggles during dynamic traffic density shifts and lacks real-world validation beyond idealized CARLA simulations. Alternatively, Li et al. Li et al. 2025 adopt a hierarchical architecture where the LLM generates long-term goals and meta-actions, while a low-level RL agent executes continuous control (steering and acceleration). They align policy updates via a Goal Gradient-based Transfer mechanism and refine rewards through human preferences. Nevertheless, LLM-generated goals remain vulnerable to unsafe hallucinations in edge cases, and the framework’s evaluation is constrained by a small human evaluation pool and limited scenario diversity. Diverging from LLM-centric designs, Choi and Kim Choi and Kim 2025 reformulate the intrinsic RL risk signal without LLMs. They introduce Safety Potential, a dense reward-shaping metric based on predicted path overlap and temporal weighting. While effective in reducing collisions in CARLA, this approach is limited to longitudinal control (relying on a rule-based Pure Pursuit controller for steering), depends on ground-truth simulator states, and underpredicts risk during complex diagonal vehicle interactions. Closest to hybrid safety integration, LSADQN Ren and Xing 2026 screens experience data via physical criteria (e.g., TTC), invoking an LLM (GPT-4) only for ambiguous samples to enforce margin-based contrastive regularization in value learning. Although it achieves < 1% collision rates in highway-env, invoking the LLM online still introduces computational overhead and latency bottlenecks. Furthermore, its heuristic safety thresholds lack formal guarantees, urban applicability, and multi-agent validation. Existing literature presents a fundamental trade-off: online LLM integration introduces latency and hallucination risks, whereas pure RL models suffer from single-dimensional control, over-reliance on ground-truth states, and a lack of auditable safety guarantees aligned with industry standards (e.g., ISO 26262/ASIL). We address these limitations with an Orchestrator-based multi-agent framework that completely decouples the LLM from real-time execution. The LLM operates strictly offline for reward refinement (with human validation) and static rule extraction. During runtime, risk is dynamically evaluated by a multi-factored Safety Agent and Situation Agent . Crucially, the Safety Agent possesses a deterministic ASIL-D-compliant veto, triggering emergency overrides when risk thresholds are breached. At the control level, two independent PPO networks learn joint longitudinal and lateral control from raw camera (YOLOv11) and LiDAR inputs, ensuring end-to-end traceability and industry-aligned safety. 3 Methodology The proposed framework operates in two domains: an offline development phase and a runtime execution phase. In the offline phase, LLM-based reasoning guides reward function refinement, while in runtime a multi-agent architecture with orchestrator controls the vehicle. A high-level overview of the proposed framework is provided in Appendix A. At runtime, camera and LiDAR data are processed in the perception layer to generate a unified perception state comprising object detections, distances, lane geometry, and environmental context. This state serves as shared input to four specialized agents operating in parallel: a Safety Agent (ASIL-D) for collision risk and rule compliance assessment, a Situation Agent (ASIL-C) for contextual analysis, a Planning Agent (ASIL-B) for target speed selection, and a Behavior Agent (ASIL-A) for comfort optimization. Each agent produces an action recommendation with associated confidence and reasoning trace. A central decision arbitrator performs emergency checks and applies ASIL-priority–based fusion, ensuring that higher safety levels dominate in case of conflict. The final decision specifies steering and speed commands with full traceability. Control execution is handled by rule-based planners paired with PID controllers or PPO-trained RL policies, with safety constraints overriding learned behavior when necessary. During development, the LLM (GPT 5.2)OpenAI 2025 summarizes and analyzes logged episodes to identify failure patterns and recommend reward adjustments. Human validation precedes retraining, forming an iterative improvement cycle. A knowledge base of approximately 100 common-sense rules from Claude Anthropic 2025 supports both runtime safety enforcement and reasoning. In the following subsections. We present a detailed description of each layer and component of the proposed framework. 3.1 Perception The perception layer integrates three complementary modules to generate a comprehensive representation of the environment. A YOLOv11 model Jocher and Qiu 2024, trained in Roboflow for CARLA-relevant object detection, processes camera images to detect and classify traffic participants (including vehicles, pedestrians, cyclists and traffic lights) along with associated confidence scores. The lane detection module employs edge detection and polynomial fitting to extract lane boundaries, calculating center offsets for lateral positioning and curvature estimation. The intersection detector evaluates road geometry by estimating lane widths and identifying horizontal markings such as crosswalks. Outputs from all three modules are fused into a unified perception state that captures both dynamic objects and static road structures, forming the foundational input for subsequent reasoning layers. 3.2 Agents The multi-agent reasoning layer employs four specialized agents operating in parallel, each assigned a distinct safety integrity level. To support compliance with the relevant ISO standards, Claude Sonnet and Opus 4.5Anthropic 2025 were employed as advisory models for safety and regulatory alignment. In addition, a set of structured assumptions was adopted to operationalize the integration of ISO requirements within the proposed multi-agent framework. These assumptions were established based on LLM-generated recommendations and human review, and reflect the design choices made in our implementation. The Safety Agent: rated at ASIL-D as the highest criticality level, serves as the primary guardian of vehicle safety. It estimates time-to-collision (TTC) from consecutive front-distance measurements and relative velocity. When an approaching object is detected, TTC is computed from the closing distance and relative velocity, an ego-velocity-based estimate is used. Collision risk is then classified using multiple TTC thresholds: emergency (<1.5<1.5 s), warning (<3.0<3.0 s), caution (<5.0<5.0 s), and low risk otherwise. The resulting risk level is used by the orchestrator to prioritize safety-critical actions. This risk is further adjusted based on absolute distance thresholds, taking the maximum of TTC-based risk and distance-based risk where Rcollision=max(Rcollision,0.9)R_collision= (R_collision,0.9) when front distance falls below emergency brake distance. The agent integrates a common-sense reasoning engine containing over thousand driving rules. From these LLM-generated rules, based on CARLA capabilities, only a subset can be applied, therefore, approximately 100 rules were selected for implementation. These rules cover traffic signals, pedestrians, weather conditions, and defensive driving principles. The agent evaluates the current perception state against these rules and selects the most critical triggered rule for decision-making. It performs holistic risk assessment combining multiple factors: Rholistic=min(CLOSE0.5⋅Rcollision+0.1⋅Nfaults+0.2⋅Lperception+Rtraffic+Rintersection,1.0) splitR_holistic= (&0.5· R_collision+0.1· N_faults\\ &+0.2· L_perception+R_traffic+R_intersection,1.0 ) split (1) where NfaultsN_faults represents the count of sensor faults, LperceptionL_perception denotes perception limitation score, Rtraffic=0.15R_traffic=0.15 when traffic light is red, and Rintersection=0.1R_intersection=0.1 when at an intersection. Based on this assessment, the agent determines appropriate safe states: normal operation when Rholistic≤0.4R_holistic≤ 0.4, degraded operation when 0.4<Rholistic≤0.70.4<R_holistic≤ 0.7, minimal risk condition when Rholistic>0.7R_holistic>0.7, and safe stop when Rcollision>0.9R_collision>0.9. The Situation Agent: rated at ASIL-C, provides environmental context assessment and evaluates Operational Design Domain (ODD) compliance based on speed constraints, weather, lighting, road type, sensor health, and traffic density, to the extent these factors can be implemented within CARLA. Current conditions are classified according to the number of detected ODD violations: normal, edge case , degraded, and out of ODD. The situation risk is then determined from collision probability, the number of ODD violations, scenario category, and traffic conditions. maximum risk of 1.0. The Planning Agent: rated at ASIL-B, handles strategic decision-making including action planning and turn decisions. It maintains a road model tracking current lane position and intersection status, plans actions based on current scenario, and employs rule-based turn decisions using distance-weighted scoring with exploration factors and turn history penalties. The agent supports optional RL-based learned turn decisions when learning mode is explicitly enabled, though this capability is disabled by default in the current implementation to ensure deterministic behavior for safety validation. The Behavior Agent: rated at ASIL-A, optimizes driving quality and comfort. It assesses driving quality through smoothness metrics based on steering variance calculated from a sliding window of recent center offsets. Each agent produces a recommendation containing target speed, steering offset, stop command, driving mode suggestion, and confidence score. 3.3 Orchestrator The orchestrator manages multi-agent recommendations, with the decision arbiter resolving conflicts using ASIL-weighted priority fusion. For each agent i, the effective weight is calculated as: wi=wbase,i⋅mASIL,i⋅ci⋅bconflict,iw_i=w_base,i· m_ASIL,i· c_i· b_conflict,i (2) where wbasew_base represents base weights (Safety: 2.5, Situation: 1.5, Planning: 1.2, Behavior: 0.8), mASILm_ASIL represents ASIL multipliers (ASIL-D: 2.0, ASIL-C: 1.5, ASIL-B: 1.2, ASIL-A: 1.0), cic_i is the agent’s confidence score, and bconflict=1.3b_conflict=1.3 for agents that won conflict resolution or 1.0 otherwise. When agents conflict on stop decisions, any ASIL-D or ASIL-C agent recommending stop with confidence above threshold has veto power that forces immediate stop regardless of other agent opinions. The proposed multi-agent orchestration framework comprises a priority-weighted arbiter and four specialized sub-agents (see Appendix, Figure 6). Table 2 summarizes the control strategies evaluated in this work, covering baseline PID control and configurations incorporating reinforcement learning for curve and speed control. The hybrid strategy combines PID and RL components to provide control across both lateral and longitudinal behaviors. Table 2: Comparison of control strategies used in autonomous driving. Strategy Steer Curve Speed Reasoning Base PID PID N/A Common sense Curve PID RL N/A Common sense Speed PID N/A RL Common sense Hybrid PID+RL RL RL Common sense 3.4 Reinforcement learning In this work, reinforcement learning was applied as part of the control module in three modes. Separate RL models were used for speed control and curve handling, while a combination of both was utilized in the hybrid mode. Two independent PPO-based actor-critic networks Schulman et al. 2017 handle control, respectively. Both employ the same hidden architecture and identical training hyperparameters. The orchestrator invokes them independently at each control tick, and the safety agent retains override authority over both outputs. The steering controller outputs δ∈[−1,1]δ∈[-1,1], the same value of steering control the vehicle steering from CARLA CARLA Simulator, for lane keeping and curve following. The reward signal penalises lane deviation and steering discontinuities. The key state variables used by the RL controllers are provided in Appendix C. 4 LLM Integration This section describes the integration of LLMs into two components of the proposed framework. First, an LLM is employed as a reward refinement mechanism for the RL-based curve controller. The initial RL model is trained and evaluated without LLM guidance, after which its outputs are analyzed by GPT-5.2 to identify potential improvements. Second, Claude is incorporated into the common-sense reasoning component. The following subsections describe these two applications in detail. 4.1 LLM as a Reward Refinement Mechanism for RL An initial RL model (V0) was trained without LLM guidance using data collected from CARLA over 1,000 episodes. The episodes were conducted in highly dynamic environments without predefined scenarios, allowing vehicles and pedestrians to appear at varying locations and under different traffic conditions. This setup was intended to expose the controller to various situations, including configurations that may not have been adequately represented during training. The reward function was iteratively refined using GPT-5.2 feedback based on summarized curve-steering episodes. The initial refinement improved some steering metrics but reduced overall progress, motivating further adjustment of the reward scale and penalty terms. The complete iterative refinement process is provided in Appendix D. 4.2 LLM as a Common-Sense Reasoner A common-sense reasoning module was implemented using Claude as the LLM to generate driving rules for diverse environmental, traffic, and road-user conditions. The generated rules covered scenarios including traffic-light states, weather, night-time driving, vehicle following distance, pedestrians, cyclists, and intersections. The rules were subsequently mapped to executable conditions and actions within CARLA. Rules involving actions that were not supported or observable in the simulator, such as horn and flashing-light behavior, were excluded, while rules with corresponding executable conditions and actions were retained. Approximately 100 executable rules were generated and integrated into CARLA. The resulting rule set provides context-dependent driving behavior beyond basic traffic-light handling. For example, red-light conditions can trigger stopping or gradual speed reduction based on distance, while adverse weather can increase caution and reduce speed. Similarly, close vehicle following can trigger an emergency stop, whereas following in rain can require increased spacing. Pedestrian-related rules include stopping at crosswalks, yielding at intersections, and reducing speed for pedestrians under reduced-visibility conditions. Additional rules address cyclists, night-time driving, and weather-dependent intersection behavior. The generated rules were integrated into the Safety Agent, which was assigned the highest priority within the orchestration framework. When a corresponding condition is detected, the Orchestrator can invoke the common-sense reasoner and forward the selected action to the CARLA control system. This integration allows contextual rules generated by the LLM to be translated into executable safety-oriented behavior while retaining the Safety Agent’s priority over other control decisions. (a) (b) Figure 2: Examples of common-sense reasoning in CARLA. (a) Rule 53: speed reduction under heavy rain. (b) Rule 75: speed reduction when cross traffic cannot be observed Figure 2 illustrates two examples of common-sense reasoning under driving conditions in CARLA. Figure 2(a) shows Rule 53, which requires a substantial speed reduction under heavy rain and results in the Safety Agent reducing the ego vehicle’s speed. Figure 2(b) shows Rule 75, where limited visibility of cross traffic triggers a speed reduction for the ego vehicle. Additional examples are provided in Appendix E. Figure 7(a) shows a short following distance that triggers a slow-down command (Rule 156), with the reasoning linked to an approaching storm (Rule 93). Figure 7(b) shows a stop signal at a red traffic light in heavy rain. 5 Evaluation Figure 3 compares the learning dynamics of V0–V3 using trailing 50-episode statistics. Although all variants exhibit non-monotonic behavior, V3 provides the strongest end-of-training balance between tracking accuracy, steering stability, and curve progress. At episode 1,000, V3 achieves the lowest median lane error (0.0514) and the lowest curve-oscillation rate among the non-degenerate variants (45.89%), while maintaining the highest median curve exposure (193.5 steps). V2 attains comparable exposure (187.5 steps) but produces higher lane error (0.0849), greater steering-change magnitude (0.3708), and a higher oscillation rate (47.99%). V1 remains comparatively stable but reaches only 101.5 curve steps and exhibits the highest terminal oscillation rate (49.69%). V0 shows a sharp late reduction in steering changes and oscillations; however, this coincides with its curve exposure collapsing to 21.5 steps, indicating reduced driving opportunity rather than genuine stabilization. Overall, the results demonstrate that V3 achieves the most favorable balance between steering quality and sustained curve progress across the evaluated variants. Figure 3: Training behavior of the four curve-steering reward variants. Figure 4 summarizes the end-of-training behavior of V0–V3 over the final 200 episodes, where boxes represent the median and interquartile range, whiskers indicate the 10th–90th percentiles, individual points denote episode-level observations, and diamonds in panel (c) show exposure-weighted curve-oscillation rates. V3 achieves the lowest median lane error at 0.0627, corresponding to an improvement of approximately 15–20% relative to V0, V1, and V2, whose medians are 0.0780, 0.0733, and 0.0770, respectively. In terms of steering-change magnitude, V3 obtains a median of 0.3119, improving upon V1 and V2 while remaining slightly higher than V0. A similar pattern is observed for curve oscillations: V3 records an exposure-weighted rate of 45.27%, compared with 49.18% for V1 and 47.61% for V2, whereas V0 reaches a lower rate of 43.18%. However, panel (d) shows that V0 operates under substantially lower curve exposure, with a median of 113 curve steps per episode, compared with 160 for V3. V3 also exceeds V1 and V2, which achieve medians of 99 and 135.5 curve steps, respectively. Consequently, the lower steering-change and oscillation values observed for V0 must be interpreted together with its reduced curve traversal. Overall, V3 provides the strongest combined performance, achieving the lowest lane error, improved steering stability relative to V1 and V2, and the greatest sustained curve exposure among all evaluated variants. Figure 4: End-of-training assessment of tracking accuracy, steering stability, and curve progression across the four reward variants. 6 Conclusion In this work, a hybrid autonomous driving framework is developed by combining a common-sense reasoner, PID controllers, real-time object detection, and reinforcement learning. Four specialized agents and dedicated controllers were coordinated through an orchestrator, with decision authority retained by the orchestration framework rather than being directly delegated to the LLM. The framework was evaluated in CARLA under diverse and highly randomized scenarios, including different weather conditions, with the reasoning layer playing a particularly important role in decision-making under adverse conditions. LLM-guided reward refinement was also employed iteratively to improve the RL controller. Across successive RL versions, improvements in the balance between tracking accuracy, steering stability, and curve progress were observed, demonstrating the potential of combining LLM reasoning with conventional autonomous driving approaches. Future work can investigate more advanced agentic AI architectures and explore how greater decision-making authority can be safely delegated to individual agents within constrained control settings. Acknowledgments and Disclosure of Funding The authors acknowledge the use of LLMs as a core methodological component of this work. In particular, LLMs were employed to explore common-sense reasoning capabilities for autonomous driving, to support alignment with relevant ISO standards to the extent possible within CARLA, and as reward models within the proposed framework, with Claude and GPT used as part of the methodological contribution. The authors used generative AI tools strictly for grammar checking and copy-editing to improve the readability of the manuscript. The core methodology, structure, experimental design, analysis, and conclusions were developed entirely by the human authors. This paper is the result of preliminary work by Hamm-Lippstadt University of Applied Sciences, Germany, on the EdgeAI-Trust project “Decentralized Edge Intelligence: Advancing Trust, Safety, and Sustainability in Europe.” This work was supported by the EdgeAI-Trust project under grant agreement No 101139892, funded by the Chips Joint Undertaking (Chips JU). The Chips JU receives support from the European Union’s Horizon Europe research and innovation program and from National Authorities and project AKI4KMU (Automatisiertes KI-Framework für KMU) supported by the Ministry of Economic Affairs, Industry, Climate Action and Energy of the State of North Rhine-Westphalia. Co-funded by the European Union. Grant number EFRE-20800498. References Anthropic [2025] Anthropic. Introducing claude opus 4.5: Redefining frontier intelligence for agents and coding, November 2025. URL https://w.anthropic.com/news/claude-opus-4-5. [2] CARLA Simulator. Python api reference. CARLA Documentation. URL https://carla.readthedocs.io/en/latest/python_api/. Accessed August 10, 2026. Chen et al. [2025] Zhiwen Chen, Hanming Deng, Zhuoren Li, Huanxi Wen, Guizhe Jin, Ran Yu, and Bo Leng. Hcrmp: A llm-hinted contextual reinforcement learning framework for autonomous driving. In Advances in Neural Information Processing Systems, volume 38, 2025. Choi and Kim [2025] Jinho Choi and Shiho Kim. Predictive risk-aware reinforcement learning for autonomous vehicles using safety potential. Electronics, 14(22):4446, 2025. doi: 10.3390/electronics14224446. Dosovitskiy et al. [2017] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun. Carla: An open urban driving simulator. In Proceedings of the Conference on Robot Learning (CoRL), pages 1–16, 2017. Feng et al. [2025] B. Feng, Z. Mei, B. Li, J. Ost, F. Ghilotti, R. Girgis, A. Majumdar, and F. Heide. Verdi: Vlm-embedded reasoning for autonomous driving. arXiv preprint arXiv:2505.15925, 2025. Jocher and Qiu [2024] G. Jocher and J. Qiu. Ultralytics yolov11, 2024. Kothawade et al. [2021] Suraj Kothawade, Vinaya Khandelwal, Kinjal Basu, Huaduo Wang, and Gopal Gupta. Auto-discern: Autonomous driving using common sense reasoning. arXiv preprint arXiv:2110.13606, 2021. Li et al. [2025] Lin Li, Runjia Tan, Jianwu Fang, Jianru Xue, and Chen Lv. Llm-augmented hierarchical reinforcement learning for human-like decision-making of autonomous driving. Expert Systems with Applications, 294:128736, 2025. doi: 10.1016/j.eswa.2025.128736. OpenAI [2025] OpenAI. Introducing gpt-5.2, December 2025. URL https://openai.com/index/introducing-gpt-5-2/. Ren and Xing [2026] Haochen Ren and Yuelin Xing. Llm-guided deep reinforcement learning with contrastive safety regularization for autonomous driving. Discover Artificial Intelligence, 6(1):169, 2026. doi: 10.1007/s44163-025-00812-w. Schulman et al. [2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. Zhang et al. [2024] Y. Zhang, S. Mao, T. Ge, X. Wang, A. de Wynter, Y. Xia, W. Wu, T. Song, M. Lan, and F. Wei. Llm as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.01230, 2024. Appendix A System Architecture Figure 5 provides a detailed overview of the hierarchical system architecture of the proposed method. Figure 5: Hierarchical system architecture of the proposed method. Appendix B Multi-Agent Orchestration Figure 6: High-level overview of the proposed multi-agent orchestration framework Appendix C Reinforcement Learning State Vectors Table 3: Key state variables used by the longitudinal and lateral RL controllers. Controller State Variable Value/Scale Speed Speed / Target Speed v/50v/50 Front Distance d/50d/50 Front-Left / Front-Right Distance d/30d/30 Traffic Light One-hot ∈ℝ4 ^4 Steering Center Offset clip(o/2,−1,1)clip(o/2,-1,1) Curvature clip(10κ,−1,1)clip(10κ,-1,1) Speed v/50v/50 Front Distance d/50d/50 Front-Left / Front-Right Distance d/30d/30 Appendix D LLM-Guided RL Reward Refinement This section presents the iterative LLM-guided refinement of the RL reward function across successive model versions. Part of LLM-guided reward refinement for curve steering Prompt for V0: You are evaluating SUMMARIZED curve steering episodes for RL reward. This is ONLY for CURVE STEERING RL (not speed control). • Focus on: steering quality, lane_error, smoothness, and curve tracking. • Ignore: speed metrics (handled by separate RL model). Key Metrics: • steering_oscillations: number of steering sign changes (higher = more oscillatory). • steering_smoothness: average |steering change||steering change| per step (lower = smoother). • curve_steering_oscillations: steering sign changes only during is_curve=True. • avg_lane_error: average absolute lane deviation. • outcome: collision, off_road, good_progress, or completed. GPT-5.2 Evaluation: Worst Problem: curve_tracking Reward Adjustments: • lane_keeping = 0.75 • smoothness = 0.80 • curve_tracking = 0.85 • safety = 1.00 • progress = 0.25 Priority Fixes: 1. Strongly penalize steering oscillations during is_curve and add a penalty on steering jerk (delta-steer) to suppress rapid sign changes and dithering in bends. 2. Increase lane-centering penalties with a non-linear term, applying stronger weighting during curves. 3. Add an explicit curve-anticipation/heading-alignment shaping term to reduce late corrections that cause overshoot and run-off. V1: Implementation of LLM Feedback The following reward modifications were implemented based on the V0 evaluation: • Added an oscillation penalty with stronger weighting during curves. • Added a steering jerk penalty based on the second derivative. • Changed the lane-deviation penalty from linear to quadratic. • Added a curve-anticipation reward using a far waypoint at 12 m. • Applied the LLM-recommended weights: lane = 0.75, smoothness = 0.80, curve = 0.85, safety = 1.00, progress = 0.25. Part of V1 Evaluation and V2 Reward Refinement GPT-5.2 Evaluation: Root Cause: Reward scale/shape imbalance: penalty terms (especially jerk/oscillation and quadratic lane error) appear unbounded or too heavily weighted, dominating the return and pushing the policy toward behaviors that reduce penalty accumulation (low progress, early crash/off-road, or timid/oscillatory micro-corrections) rather than learning a stable steering law that matches curvature. V2 Reward Adjustments: • lane_keeping = 0.60 • smoothness = 0.25 • curve_tracking = 0.90 • safety = 1.00 • progress = 0.80 Priority Fixes: 1. Rescale and cap penalties 2. Replace/augment oscillation counting with a bounded frequency/energy proxy 3. Improve curve-tracking reward signal summary: V1 slightly reduced lane deviation and oscillation counts but is overall worse because it removed all successes and significantly reduced route progress while producing extremely negative returns. The dominant issue is reward magnitude imbalance: new penalties swamp the learning signal and encourage avoidance/early failure rather than stable curve tracking. V2 should keep curve-specific shaping but heavily cap/normalize penalties and increase bounded curve-tracking and progress rewards, with curvature-adaptive lookahead. V2: Implementation of V1 LLM Feedback The following reward modifications were implemented based on the V1 evaluation: • Capped all penalty terms using Huber loss: max_lane = 2.0, max_jerk = 0.5, max_oscillation = 1.0. • Increased the progress weight from 0.25 to 0.80. • Reduced the smoothness weight from 0.80 to 0.25. • Changed the oscillation penalty from count-based to a bounded rate-based measure in [0,1][0,1]. • Introduced curvature-adaptive lookahead, using 6 m for sharp curves and 12 m for gentle curves. • Added a curve-survival bonus for maintaining progress through curves without failure. • Applied the V2 weights: lane = 0.60, smoothness = 0.25, curve = 0.90, safety = 1.00, progress = 0.80. Part of V2 Evaluation and V3 Reward Refinement GPT-5.2 Evaluation: Root Cause: The reward still does not create a strong, well-conditioned gradient toward the correct steady-state steering for curvature: curve tracking signals (lane/heading/curvature consistency) are either too weak, too saturated by caps, or not aligned with the control objective, so the agent can earn decent return via progress/survival while exhibiting oscillatory, reactive steering that eventually drifts off-road on sharper/longer bends. V2 Reward Adjustments: • lane_keeping = 0.75 • smoothness = 0.45 • curve_tracking = 1.00 • safety = 1.00 • progress = 0.55 Priority Fixes: 1. Make curve tracking ‘non-optional’ 2. Replace/augment oscillation shaping with action-derivative shaping that targets true instability 3. Rebalance progress vs quality summary: V1 fixed reward-shaping intentions but over-penalized (large negative returns) and reduced progress without improving outcomes. V2’s capping and rebalancing restored learnability and increased long runs and good_progress frequency, but the policy still fails primarily from oscillatory, reactive curve steering and accumulated lateral drift. V3 should keep bounded penalties but strengthen curve-specific lane/heading/curvature-consistency shaping and gate progress/survival on curve-tracking quality to eliminate ‘progress despite bad tracking’ solutions. V3: Implementation of V2 LLM Feedback The following reward modifications were implemented based on the V2 evaluation: • Added a curvature-consistency reward using a feedforward steering target based on waypoint angle. • Added a counter-steer penalty for steering opposite to the curve direction when |wp_angle|>10∘| wp\_angle|>10 . • Gated progress using progress×curve_quality_factorprogress×curve\_quality\_factor, where quality=exp(−k(lane_err+heading_err))quality= (-k(lane\_err+heading\_err)). • Added a conditional survival bonus applied only when lane_error<0.35lane\_error<0.35 and heading_error<0.3heading\_error<0.3. • Introduced a two-stage lane penalty consisting of Huber loss near the center and a boundary hinge for offset >0.5>0.5. • Added a curve-weighted heading-alignment reward. • Increased the smoothness weight from 0.25 to 0.45. • Increased the lane-keeping weight from 0.60 to 0.75. • Increased the curve-tracking weight from 0.90 to 1.00, making it the highest priority. • Reduced the progress weight from 0.80 to 0.55 while gating it on tracking quality. • Applied the V3 weights: lane = 0.75, smoothness = 0.45, curve = 1.00, safety = 1.00, progress = 0.55. Appendix E Additional Common-Sense Reasoning Examples Additional examples of context-dependent rules generated by the common-sense reasoner are provided to illustrate its behavior under combined driving conditions. (a) (b) Figure 7: Additional examples of common-sense reasoning in CARLA. (a) Short following distance with an approaching storm. (b) Red traffic light under heavy rain.