Paper deep dive
ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation
Zihe Wang, Yihuan Wang, Haiyang Yu. Zhiyong Cui, Xiaojian Liao, Chengcheng Wang, Yonglin Tian, Yongxin Tong
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/22/2026, 5:43:01 AM
Summary
ExpressMind is a domain-specific multimodal large language model (MLLM) designed for intelligent expressway operations. It addresses limitations in existing rule-based systems and general LLMs by integrating a full-stack expressway dataset, a dual-layer pre-training paradigm, a Graph-Augmented RAG framework for dynamic knowledge retrieval, and an RL-aligned Chain-of-Thought (RL-CoT) mechanism to improve incident response reasoning.
Entities (5)
Relation Signals (4)
ExpressMind → integrates → Graph-Augmented RAG
confidence 100% · a graph-based retrieval-augmented generation (RAG) is proposed to enhance the extraction of key traffic scene characteristics
ExpressMind → isbuiltupon → Qwen
confidence 100% · ExpressMind, built upon the Qwen foundation model
ExpressMind → uses → RL-CoT
confidence 100% · we develop a RL-aligned Chain-of-Thought (RL-CoT) mechanism that enforces consistency
RL-CoT → utilizes → GRPO
confidence 95% · employ the Group Relative Policy Optimization (GRPO) algorithm to mine underlying logical patterns
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The current expressway operation relies on rule-based and isolated models, which limits the ability to jointly analyze knowledge across different systems. Meanwhile, Large Language Models (LLMs) are increasingly applied in intelligent transportation, advancing traffic models from algorithmic to cognitive intelligence. However, general LLMs are unable to effectively understand the regulations and causal relationships of events in unconventional scenarios in the expressway field. Therefore, this paper constructs a pre-trained multimodal large language model (MLLM) for expressways, ExpressMind, which serves as the cognitive core for intelligent expressway operations. This paper constructs the industry's first full-stack expressway dataset, encompassing traffic knowledge texts, emergency reasoning chains, and annotated video events to overcome data scarcity. This paper proposes a dual-layer LLM pre-training paradigm based on self-supervised training and unsupervised learning. Additionally, this study introduces a Graph-Augmented RAG framework to dynamically index the expressway knowledge base. To enhance reasoning for expressway incident response strategies, we develop a RL-aligned Chain-of-Thought (RL-CoT) mechanism that enforces consistency between model reasoning and expert problem-solving heuristics for incident handling. Finally, ExpressMind integrates a cross-modal encoder to align the dynamic feature sequences under the visual and textual channels, enabling it to understand traffic scenes in both video and image modalities. Extensive experiments on our newly released multi-modal expressway benchmark demonstrate that ExpressMind comprehensively outperforms existing baselines in event detection, safety response generation, and complex traffic analysis. The code and data are available at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.16495v1
- Canonical: https://arxiv.org/abs/2603.16495v1
Trouble viewing inline? Open PDF directly →
Full Text
69,510 characters extracted from source content.
Expand or collapse full text
ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation Zihe Wang Beihang University Beijing, China by2313310@buaa.edu.cn Yihuan Wang Beihang University Beijing, China yhuanwang@buaa.edu.cn Haiyang Yu Beihang University Beijing, China hyyu@buaa.edu.cn Zhiyong Cui ∗ Beihang University Beijing, China zhiyongc@buaa.edu.cn Xiaojian Liao Beihang University Beijing, China liaoxj@buaa.edu.cn Chengcheng Wang Shandong Hi-speed Group Co., Ltd Jinan, China wangchengcheng@sdhsg.com Yonglin Tian Institute of automation, Chinese Academy of Sciences Beijing, China tyldyx@mail.ustc.edu.cn Yongxin Tong Beihang University Beijing, China yxtong@buaa.edu.cn Abstract The current expressway operation relies on rule-based and isolated models, which limits the ability to jointly analyze knowledge across different systems. Meanwhile, Large Language Models (LLMs) are increasingly applied in intelligent transportation, advancing traffic models from algorithmic to cognitive intelligence. However, gen- eral LLMs are unable to effectively understand the regulations and causal relationships of events in unconventional scenarios in the expressway field. Therefore, this paper constructs a pre-trained mul- timodal large language model (MLLM) for expressways, Express- Mind, which serves as the cognitive core for intelligent expressway operations. This paper constructs the industry’s first full-stack ex- pressway dataset, encompassing traffic knowledge texts, emergency reasoning chains, and annotated video events to overcome data scarcity. This paper proposes a dual-layer LLM pre-training para- digm based on self-supervised training and unsupervised learning. Additionally, this study introduces a Graph-Augmented RAG frame- work to dynamically index the expressway knowledge base. To enhance reasoning for expressway incident response strategies, we develop a RL-aligned Chain-of-Thought (RL-CoT) mechanism that enforces consistency between model reasoning and expert problem-solving heuristics for incident handling. Finally, Express- Mind integrates a cross-modal encoder to align the dynamic fea- ture sequences under the visual and textual channels, enabling it to understand traffic scenes in both video and image modalities. ∗ Corresponding author Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference acronym ’X, Woodstock, NY © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-X-X/2018/06 https://doi.org/X.X Extensive experiments on our newly released multi-modal express- way benchmark demonstrate that ExpressMind comprehensively outperforms existing baselines in event detection, safety response generation, and complex traffic analysis. The code and data are available at: https://wanderhee.github.io/ExpressMind/. CCS Concepts • Computing methodologies→ Artificial intelligence. Keywords Large Language Models, Intelligent Expressway Operations, Pre- training Paradigm, Chain-of-Thought, Multimodal Understanding ACM Reference Format: Zihe Wang, Yihuan Wang, Haiyang Yu, Zhiyong Cui, Xiaojian Liao, Chengcheng Wang, Yonglin Tian, and Yongxin Tong. 2026. ExpressMind: A Multimodal Pretrained Large Language Model for Expressway Operation. In Proceedings of Make sure to enter the correct conference title from your rights confirma- tion email (Conference acronym ’X). ACM, New York, NY, USA, 13 pages. https://doi.org/X.X 1 Introduction With the continuous advancement of intelligent transportation systems (ITS), expressway operation is evolving from the reactive and rule-based paradigm towards intelligent agents endowed with deep cognitive reasoning capabilities. Breakthroughs in artificial intelligence [36], particularly the emergence of Large Language Models (LLMs) with exceptional computational and reasoning abil- ities [38], are profoundly shaping the intelligent transformation across ITS industries. However, the expressway sector still lacks a domain-specific LLM which is capable of deeply adapting to com- plex expressway operational needs. At the current stage, advancing foundation models’ core capabilities such as expressway knowledge integration, multimodal scene understanding, and autonomous in- cident reasoning remains a critical bottleneck, necessitating a fun- damental methodological shift. arXiv:2603.16495v1 [cs.AI] 17 Mar 2026 Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. Video Understanding Expressway Konwledge QA Incident Strategy Report ExpressMind Text Data Video Data E：Applications Expert Annotation A：DataSet B：Benchmark C：Model D：Knowledge Base Figure 1: Overview of ExpressMind. Obviously, most open-source LLMs [20] lack a deep understand- ing of publicly inaccessible specialized knowledge in the express- way domain, such as technical standards and regulations. Dynamic information and professional terminology cannot be promptly fed into LLMs and, thus, their reasoning and decision-making processes struggle to guarantee safety and efficiency requirements. Further- more, existing methods exhibit shortcomings in key visual feature extraction and traffic-related reasoning within multimodal scenar- ios. Therefore, current approaches fail to meet the dynamic and precise operation requirements of expressway, lacking an intelli- gent central hub capable of multi-task collaborative cognition and deep industry understanding. However, constructing an expressway domain-specific multi- modal large model as such intelligent central hub faces consider- able difficulties and unique challenges. The highly heterogeneous and complex multimodal data in this field is the first hurdle. The unstructured data such as real-time monitoring videos and semi- structured data including incident records, making it extremely difficult to achieve effective alignment and fusion between different modalities. Additionally, the expressway domain has strict require- ments on safety and accuracy. The desired foundation model needs to accurately grasp professional knowledge such as traffic engineer- ing principles and emergency disposal specifications, which brings great challenges to the in-depth integration of domain knowledge and the design of multimodal architecture. More importantly, the scarcity of high-quality labeled multimodal data in the expressway field, coupled with the privacy and security constraints, further increases the difficulty of model training and optimization, becom- ing a key obstacle to building a high-performance domain-specific multimodal large model. To address these challenges, this paper introduces ExpressMind, a domain multimodal LLM for expressway operation. We construct the first full-stack expressway dataset and propose a two-stage pre- training paradigm for the internalization of expressway-domain knowledge. This study also develops a Reinforcement Learning (RL)- based Chain-of-Thought (CoT) alignment mechanism to strengthen domain reasoning. Furthermore, a visual-enhanced cross-modal encoder is incorporated and a graph-based retrieval-augmented generation (RAG) is proposed to enhance the extraction of key traf- fic scene characteristics and dynamic knowledge. The integration of these modules as a whole build the foundation of the multi- modal pretrained LLM to process multi-source expressway data and provide efficient expressway operation decision support. The overview of ExpressMind is illustrated in Figure. 1, and its five core contributions are summarized as follows: • Full-stack Expressway dataset: This study constructs the first industry’s full-stack expressway dataset spanning text cognition, logical reasoning, and visual perception, including three specialized subsets: traffic knowledge texts, emergency response reasoning, and event video scene understanding. • RL-aligned CoT Reasoning: We design a RL-based express- way strategy alignment strategy in LLM training, which can significantly enhance the model’s logical reasoning and self- correction capabilities. •Graph-Augmented Retrieval: A graph RAG-based dy- namic knowledge base is established for critical expressway information retrieval and indexing. •Multimodal Alignment mechanism: A Visual-Prior Align- ment mechanism is designed by enforcing alignment and reweighting of visual tokens to enhance the understanding of visual features. •Multi-modal Benchmark: The multi-modal Benchmark for evaluating LLMs within the expressway domain is released, encompassing four evaluation subsets: basic knowledge com- prehension, video incident detection, safety response gener- ation, and traffic analysis reporting. 2 Relatedwork Traffic related foundation model: Current mainstream LLM ar- chitectures are primarily categorized into three paradigms: Encoder- only (e.g., BERT [6]), Encoder-Decoder (e.g., GLM [8]), and Decoder- only (e.g., GPT [1], LLaMA [7]). To adapt general-purpose models to vertical domains, methodologies such as Supervised Fine-Tuning (SFT) [22] and Parameter-Efficient Fine-Tuning (PEFT) [12] have been introduced. Within the transportation sector, models such as TransGPT [28] and TrafficGPT [37] advances traffic safety analysis via domain-adaptive training. Furthermore, UrbanGPT [17] inte- grates spatio-temporal encoders with LLMs through instruction tuning for general urban analysis. However, a domain LLM for expressway tasks has yet to emerge. MLLMs: CLIP [24] established the foundation of multimodal learn- ing by aligning the representation spaces of images and texts via contrastive learning. To endow LLMs with visual comprehension capabilities, LLaVA [19] introduces a linear projection layer to map visual features into token embeddings processable by language models. BLIP-2 [16] proposes the Q-former architecture to extract text-relevant features from frozen visual encoders.Following this re- search trajectory, the Qwen-VL series [4] is subsequently proposed to align visual and linguistic representations, In traffic scenarios, MLLMs have been applied to tasks such as accident analysis (Traffi- cLens [2], MoTIF [30]) and anomaly detection (Anomaly-OneVision [32]) by facilitating semantic understanding of surveillance footage. ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY Express-QA 1. Incident Description: At [Time], a rear-end collision ... on the [position], .... 2. Causal Inference: Upon analysis, [x] is determined to ... for the accident, ..... 3. Response Strategy Formulation: First, establish traffic control, ... 4. Strategy Evaluation: The decision to close Lane 1 is .... Answer “At [Time], ...occurred on the [position], ....Upon analysis,[x] is determined to bear primary liability for the accident, ....” Pre-training ExpressMind Question “What incident occurred on the expressway and what is the correct response strategy?” Expressway Konwledge RAG Double-layer pre-training LLM Operation Data Regulatory Standard Textbook Literature Initial data Algorithm Unsupervised Self-supervised Express-VQA Position embedding Visual Tokens Adaptive Visual Encoding Visual Encoder Tokenizer Visual Prior Alignment VPA ... ... ... Feature alignment Naive Dynamic Resolution Express-IncidentCoT RL-Enhanced Reasoning Group Sampling Caculate Reward UpdateModel Figure 2: The Overall Framework of ExpressMind. Reinforcement Learning for Reasoning: Reinforcement Learn- ing from Human Feedback [22] has emerged as the prevailing par- adigm for aligning LLMs with human intent. Algorithms such as DPO [25], CPO [31], and GRPO [26] leverage preference infor- mation inherent in CoT processes to further optimize reasoning trajectories and training efficiency. To mitigate reward hacking, DreamPRM [5] introduces a domain re-weighting mechanism. Re- cently, integrating the semantic comprehension of LLMs with the decision-making capabilities of RL has moved to the forefront of transportation research. Traffic-R1 [40] and LLMLight [15] employ RL to enhance the generalization of LLMs in signal control tasks. AgentsCoMerge [13] uses RL with ramp and density rewards for traffic optimization, Time-LLM [14] transforms time-series into text via RL, achieving SOTA in traffic forecasting. While RL has been applied in related traffic tasks, its use for enhancing reasoning in the expressway domain remains unexplored. 3 Methodology This study proposes ExpressMind, a domain-specific MLLM tailored for expressway operation. The overall framework is illustrated in Figure. 2 and the key components are introduced as follows: 3.1 Task-oriented Domain Data Profiling To address the domain-specific tasks depicted above, which include Expressway Knowledge QA, Video Understanding, and Incident Strategy Report, this study collects four distinct types of data to sup- port the complete training pipeline of ExpressMind, as illustrated in Figure 3. •Textual Data: To establish the model’s fundamental domain understanding, textual data, including policy documents, expert knowledge, and SFT QA pairs, are used in the 3.2 Pre-training Stage. •Incident CoT Data: To refine reasoning trajectories via reinforcement learning, incident CoT data comprises inci- dent descriptions, causal reasoning, response strategies, and evaluations. It is employed during the 3.3 RL Alignment Stage. •Dynamic Knowledge Base: To ensure model responses remain aligned with the latest operational scenarios, it con- tains real-time traffic conditions, incident reports, and traffic flow data, providing 3.4 real-time retrieval augmentation across all training stages. •Multimodal Data: To achieve video-language understand- ing, data such as accident images and congestion videos are introduced in the 3.5 Cross-modal Alignment Stage to achieve video-language understanding. A: Text Data Unsuprevised Traning data Transportation Polciy Transportation Law Professional Textbooks SFT QA Pairs Expressway Management Intelligent Road ITS Konwledge Transport Infrastructure C： Video Data D: Dynamic Knowledge Base Expert Annotation OCR B: Incident CoT Data Incident Report Expert Annotation Incident Description Causal Inference Response Strategy Strategy Evaluation Pre-training RAG RL-alignment Multimodal Real-time Conditions Domain Vocabulary Graph-based Text Indexing Retrieved Content Video Image Text Multimodal Dataset VQA Object Detection Prompt Engineering Scene Understanding Accident Identification ParkingCongestionWeather ... LLM LLM Figure 3: Task-oriented Domain Data Profiling. 3.2 Training Paradigm of Pretrained LLM To ensure the model acquires high-quality foundational knowledge for expressway scenarios, we constructed a dedicated dataset con- taining unlabeled text and self-supervised QA pairs, with all data undergoing rigorous deduplication and standardization. Express- Mind, built upon the Qwen foundation model, adopts a two-phase Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. pre-training strategy: the first phase establishes fundamental sce- nario knowledge, and the second phase adapts the model to handle complex domain-specific tasks, respectively. Stage 1: Unsupervised Training. In this phase, model parameters 휃are optimized by minimizing the negative log-likelihood loss. Given an input sequence푥=푥 1 ,푥 2 , . . .,푥 푇 derived from domain- specific corpora, the pre-training loss functionL 푃푇 is formulated as: L 푃푇 (휃)=− 푇 ∑︁ 푡=1 log푃(푥 푡 | 푥 <푡 ;휃)(1) where푥 <푡 denotes the context sequence preceding time step푡, and 푃(푥 푡 | 푥 <푡 ;휃)represents the conditional probability of the model predicting the next token given the current parameters. Stage 2: Full-Parameter Supervised Fine-Tuning. Following the ac- quisition of foundational domain knowledge, full-parameter SFT is conducted to align the model with specific tasks and instructions in the expressway transportation domain. During training, To ensure the model focuses on response generation, a masked loss strategy is employed by introducing a binary mask vector푀, where푀 푡 =0 corresponds to instruction tokens and푀 푡 =1 to response tokens. Consequently, the loss function for supervised fine-tuning, denoted asL 푆퐹푇 , is formulated as: L 푆퐹푇 (휃)=− 1 Í 푇 푡=1 푀 푡 푇 ∑︁ 푡=1 푀 푡 · log푃(푥 푡 | 푥 <푡 ;휃)(2) This two-stage training equips the model with an in-depth mas- tery of expressway domain knowledge, providing a basis for the following alignment and reasoning tasks. 3.3 RL for Expressway Strategy Alignment Although LLMs have acquired fundamental domain cognition through full-parameter pre-training, when dealing with unseen complex expressway accident scenarios, their generated response strategies often fail to establish a complete logical chain from scene analysis to strategy formulation and evaluation. It notably lacks deep logical deduction and fails to ensure strategic optimality. As shown in Figure 4, to enforce the "Perception-Analysis-Decision-Reflection" cognitive loop and address its reasoning bottleneck, we leverage a CoT dataset derived from real-world expressway emergency re- sponses and employ the Group Relative Policy Optimization (GRPO) algorithm to mine underlying logical patterns, thereby significantly enhancing the model’s reasoning capabilities. The core mechanism of GRPO involves sampling a group of candidate outputs표 1 ,표 2 , . . .,표 퐺 for a given query푞and computing gradients by evaluating the relative scores within the group. Here, 푞is from the set of all possible queries푄. The objective function is formulated as follows: 퐽 퐺푅푃푂 (휃)= E 푞∼푃(푄),표 푖 퐺 푖=1 ∼휋 휃 표푙푑 [ L ] (3) L= 1 퐺 퐺 ∑︁ 푖=1 min ( 푟 푖 퐴 푖 , clip(푟 푖 , 1−휖, 1+휖)퐴 푖 ) − 훽퐷 퐾퐿 (휋 휃 ||휋 푟푒푓 ) (4) Structure Reward Knowledge Reward Realism Reward Policy LLM Incident Txt Group Sampling Reward Model Advantage Estimation Policy Update Reference Model Expert COT data Incident Description Causal Inference Response Strategy Strategy Evaluation Figure 4: Schematic of the RL-based Reasoning Enhance- ment. where퐴 푖 denotes the advantage function. The term훽퐷 퐾퐿 acts as a regularization constraint to mitigate catastrophic forgetting during the reinforcement learning process, explicitly ensuring that the model’s linguistic generation remains aligned with the standardized traffic terminology acquired during the SFT phase. Through this mechanism, the algorithm effectively reduces the variance of gradient estimation, ensuring that the model prioritizes learning the relative superiority of strategies over absolute scores. Therefore, this enables stable policy iteration within the complex reasoning space of traffic incident disposal. To steer the model towards structured accident response Logic, we design a multi-dimensional reward푅 푡표푡푎푙 = 휆 1 푅 푠푡푟푢푐푡 +휆 2 푅 푘푛표푤 + 휆 3 푅 푠푒푚 , comprising three decoupled terms: •Structural Integrity (푅 푠푡푟푢푐푡 ): To enforce the "Perception- Analysis-Decision-Reflection" cognitive loop, we employ a gated counting mechanism. The reward accumulates only if the four stage-specific tags푆 1..4 appear in a strict monotonic order: 푅 푠푡푟푢푐푡 = 4 ∑︁ 푘=1 I(푆 푘 ∈ 푂) ! ·I ( idx(푆 1 )< idx(푆 2 )< idx(푆 3 )< idx(푆 4 ) ) (5) • Domain Alignment (푅 푘푛표푤 ): We maximize the coverage of stage-specific expert terminologyV 푘 while penalizing linguistic degradation via a perplexity (PPL) constraint: 푅 푘푛표푤 = 1 퐾 퐾 ∑︁ 푘=1 휔 푘 |푆 푘 ∩V 푘 | |푆 푘 | −휂· ReLU PPL(푂)−휏 푝푙 (6) •Semantic Consistency (푅 푠푒푚 ): To ensure strategic optimal- ity, we compute the cosine similarity between the model’s decision logic and a reference setD 푟푒푓 containing expert records and teacher traces in the embedding space: 푅 푠푒푚 = max 푑∈D 푟푒푓 cos 휙(푆 2 ⊕ 푆 3 ),휙(푑) (7) ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY The overall algorithm process for RL reasoning alignment is pre- sented in Table 1. At its core, the study generates decision-making strategies equipped with complete and verifiable expressway emer- gency response processes. This explicit reasoning trace enhances the interpretability and reliability of the model’s outputs. Imple- mentation details regarding the expert vocabularyV 푘 and hyper- parameters are provided in Appendix B. Algorithm 1 Traffic Incident Strategy Alignment via GRPO Require: unstructured traffic incident text description 푞 ∼ 푃(푄) Ensure: optimized traffic disposal policy 휋 ∗ 1:Initialize policy model휋 휃 , reference model휋 푟푒푓 , expert data- baseD 푟푒푓 2: for training epoch 푡= 1 to푇 do 3:// Step 1: Candidate Response Sampling 4:Sample 퐺 candidate responses from current policy: 5: 표 푖 퐺 푖=1 ∼ 휋 휃 푡−1 (· | 푞) 6:// Step 2: Multi-dimensional Reward Evaluation 7: for each candidate response 표 푖 do 8:Compute reward: 푟 푖 = Í 푗 휆 푗 푅 푗 (표 푖 ) 9:where 푅 ∈ 푅 푠푡푟푢푐푡 ,푅 푘푛표푤 ,푅 푠푒푚 10: end for 11:// Step 3: Advantage Normalization within Group 12:Compute group mean: 휇 group = 1 퐺 Í 퐺 푖=1 푟 푖 13:Compute group standard deviation: 14: 휎 group = √︃ 1 퐺 Í 퐺 푖=1 (푟 푖 − 휇 group ) 2 15: for each candidate response 표 푖 do 16:Compute advantage: 퐴 푖 = 푟 푖 −휇 group 휎 group +휖 17: end for 18:// Step 4: Policy Update via GRPO Objective 19:Update parameters by maximizing GRPO objective: 20: 휃 푡 ← argmax 휃 퐽 GRPO (휃;퐴 푖 ,휋 휃 푡−1 ,휋 푟푒푓 ) 21: end for 22: return optimized policy 휋 ∗ ← 휋 휃 푇 3.4 Knowledge Graph-Augmented Retrieval The static parameters of LLMs cannot capture dynamic informa- tion and professional vocabulary. Therefore, this paper constructs a expressway knowledge base to assist LLMs in learning these knowl- edge. Traffic knowledge is unstructured data so that a graph-based RAG must be adopted to retrieve the knowledge base. This study employs LightRAG [11] to enhance the ability to update incremen- tal knowledge which introduces a dual-layer retrieval mechanism by constructing a structured graph index During the indexing phase, an unstructured traffic corpus퐷is transformed into a structured and incrementally updatable knowl- edge graph b 퐺=( b 푉, b 퐸). Here, each node b 푉represents a standardized traffic term, and each edge b 퐸 encodes semantic relations. This paper designs an entity and relation extraction module to identify the entities specific to the transportation field and their interrelationships. The proposed LLM profiling is the generation of a structured key-value pair(퐾,퐿)for every node푣 ∈ 푉and edge 푒 ∈ 퐸, where the key퐾serves as a normalized identifier for effi- cient retrieval, and the value퐿is an LLM-generated summary that integrates multi-source definitions and usage contexts. Deduplica- tion merges redundant nodes and edges of different text fragments through semantic similarity comparison. Given a new document 퐷 ′ , its corresponding subgraph b 퐺 ′ =( b 푉 ′ , b 퐸 ′ )is generated indepen- dently and merged into the existing graph via set union operations b 푉 ∪ b 푉 ′ , b 퐸∪ b 퐸 ′ . In the retrieval and generation phase, a dual-level retrieval par- adigm is employed to jointly capture concrete facts and abstract concepts. Given an initial non-standardized response ˆ 푞 raw , an LLM first extracts two types of keywords: local terms푘 (푙) (e.g., "long queue”, "red-green light") and global semantic cues푘 (푔) (e.g., "traffic congestion", "signal control"). Two parallel retrieval paths are then activated: Low-level retrieval focuses on exact or near-exact matching by computing the similarity between the embedding of푘 (푙) and the key embedding of entity nodes: 푠 low (푣)= sim 푒(푘 (푙) ),푒(퐾 푣 ) (8) where푒(·)denotes an embedding function,sim(·)is typically cosine similarity, and 퐾 푣 ,퐾 푒 denote the retrieval keys of node 푣 and edge 푒, respectively. High-level retrieval operates at the conceptual level by matching 푘 (푔) against topic keys associated with relation edges: 푠 high (푒)= sim 푒(푘 (푔) ),푒(퐾 푒 ) (9) All retrieved structured descriptions and associated text snip- pets are concatenated into a unified context퐶, which is fed into the LLM to perform term-level replacement along with ˆ 푞 raw . The final normalized output ˆ 푞 norm is generated according to the RAG formulation:푝( ˆ 푞 norm | ˆ 푞 raw ,퐶) ∝ exp 푓 LLM ( ˆ 푞 raw ;퐶])), where푓 LLM denotes the LLM internal scoring function. 3.5 Multimodal Understanding with VPA End-to-end Multimodal understanding of expressway is a critical task for expressway supervision. This paper combines a visual encoder with ExpressMind to form a MLLM with video understand- ing capabilities. In order to enhance the ability of visual feature extraction, this paper introduces a novel visual encoding archi- tecture integrated with a Visual-Prior Alignment (VPA) mecha- nism, as illustrated in Figure 5. The feature of the visual encoder is I=I 1 ,I 2 , . . .,I 푡 ,I∈ R 푁 푣 ×푑 푣 , where푁 푣 is the total number of visual tokens and푑 푣 is the dimension of each visual feature vector. We design a cross-modal projection network and use layer nor- malization to stabilize the training process as shown in the follow- ing equation: H 푣 = LayerNorm ( W 2 · GELU ( W 1 · I+ b 1 ) + b 2 ) (10) where,퐻 푣 ∈ R 푁 푣 ×푑 LLM , W 1 ∈ R 푑 푣 ×푑 ℎ and W 2 ∈ R 푑 ℎ ×푑 LLM are learnable weight matrices, b 1 and b 2 are bias terms, and푑 ℎ denotes the intermediate hidden dimension. The output dimension푑 LLM matches the hidden size of the LLM. This paper employs MRoPE to address the issue of uneven fre- quency allocation in video understanding. It allocates feature chan- nels to the temporal, height, and width axes in a fine-grained polling manner, ensuring that each positional axis is encoded with a full Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. DeepStack MRoPE T=24 Pretrained LLM Time Height Width VPA Feature alignment Feature Projection Attention Calculation 퐀 퐀 Weight allocation optimization Enhance visual feature Visual Encoder Visual Tokens LLM Block 1 LLM Block 2 LLM Block N H=20 W=20 Figure 5: Multimodal Encoding Framework. frequency spectrum ranging from high to low frequencies. Fur- thermore, this study introduces the DeepStack[21] mechanism to extract feature maps and perform cross-layer fusion at different depths, which enables the language decoder to access a complete visual hierarchy spanning from pixel-level to semantic-level infor- mation. The visual features may suffer from feature attenuation during the sequence alignment after encoding due to the compression of sequence length. This paper innovatively proposes the VPA mechanism to overcome this problem. It introduces learnable cross- modal attention reweighting to achieve dynamic alignment of visual and language features. VPA explicitly enhances the computational weight of visual tokens in multimodal fusion, establishing a visual- priority inductive preference. Given the projected visual features H 푣 and the text embeddings U∈ R 퐿 푡 ×푑 LLM from the instruction prompt,퐿 푡 is the text sequence length. The formula for adjusting the learnable cross-modal attention weights in VPA is as follows: ˆ H 푣 = softmax H 푣 W 푝 U ⊤ √ 푑 LLM · H 푣 (11) where W 푝 ∈ R 푑 LLM ×푑 LLM is a learnable projection matrix. This method increases the weight of visual features when aligning visual tokens and text, establishing the inductive bias of visual priors during the feature fusion process. The enhanced visual features ˆ H 푣 are concatenated with the text embeddings U to form a unified multimodal input sequence: Z= ˆ H 푣 ; U ∈ R (푁 푣 +퐿 푡 )×푑 LLM (12) The combined representation Z is fed into the LLM and generates natural language description of video, which achieves understand- ing of traffic scenarios. 4 Experiments 4.1 Dataset This study released a comprehensive open-source expressway dataset comprising four specialized sub-datasets and a standardized bench- mark, for the training and evaluation of ExpressMind, focusing on multi-modal capabilities such as expressway domain knowl- edge understanding, incident response strategy reasoning, video understanding and incident detection. 1: Express-Insight contains over 7 million tokens of high- quality text where the content serves as a domain-specific corpus for unsupervised pre-training. The dataset is derived from web- crawled resources including traffic laws, expressway policy docu- ments, and theoretical books on Smart expressways and Intelligent Transportation Systems. 2: Express-QA contains over 870,000 samples where QA pairs are obtained through a quality-aware generation and refinement process using DeepSeek-V3 with designed prompts. 3: Express-IncidentCoT contains 1,786 incident response strat- egy Chain-of-Thought samples derived from real-world incident reports of Shandong Expressway, structured into a four-stage cog- nitive chain: [Incident Description]→[Causal Inference]→[Re- sponse Strategy Formulation]→ [Strategy Evaluation]. 4: Express-VQA a multi-modal dataset for expressway visual reasoning, integrating 1627 surveillance videos from expressways in Shandong and Guangdong, China. The average duration of the videos exceeds 2 minutes. There are over 3,200 pairs of VQA pairs. There are also 12 sets of surveillance video from Tianjin that cover two consecutive days. The data is collected across 70 roads at 1920×1080 resolution, encompassing diverse times and weather conditions to evaluate model robustness against seven core traffic anomalies such as accidents, congestion, and construction. 4.2 Experiment Setup The experimental environment is configured with the following specifications: All experimental workflows—including model train- ing, testing, and inference—were executed on a server node equipped with 8 NVIDIA H20 GPUs. The software environment is built upon Python 3.10+, PyTorch 2.4.0+, and CUDA 12.4+. The transformers, tokenizers, torchvision, and opencv-python libraries are employed for processing text, model, and image data, respectively. Stable ver- sions of DeepSpeed, Accelerate, PEFT, and Flash-Attention 2 are utilized to facilitate efficient distributed fine-tuning, thereby ensur- ing high training efficiency and stability. Under this configuration, the end-to-end training of the framework required approximately 700 gpu hours. In particular, the hyperparameter configurations for each training stage are detailed in Appendix A. 4.3 Quantitative Results 4.3.1 Pre-training Results. The results presented in Table 1 sum- marize the performance of ExpressMind across three specialized tasks (totaling 20,000 test questions): Expressway Laws & Reg- ulations QA, Smart Expressway Knowledge QA, and Intelligent Transportation System Knowledge QA. For each task, We used a set of multi-dimensional metrics, including Accuracy, F1-Score, Embedding Similarity, and GPT-Score, to benchmark our model againstestablished open-source baseline models such as Qwen-32B [33], Llama-3.3-70B [9], and the DeepSeek-R1-Distill series [10]. De- spite its specialized focus, ExpressMind consistently outperforms these LLM across all evaluation dimensions. Notably, in the Express- way Laws & Regulations QA task, our model achieves a peak MCQ ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY Table 1: Pre-training results on QA tasks. Models MCQ [%]True/False [%]Fill-in-the-Blank [%]Short Answer [%] AccRecF1AccRecF1EmbF1GPT-ScoreF1 Expressway Laws & Regulations QA Qwen-32B97.996.597.296.195.896.493.785.375.682.1 Baichuan-32B91.388.589.989.588.289.488.878.269.374.5 DeepSeek-R1-Distill-Qwen-32B96.196.397.496.596.096.192.484.176.682.4 DeepSeek-R1-Distill-Llama-70B97.096.296.597.095.896.495.988.785.987.8 Llama-3.3-70B97.596.096.896.596.296.896.689.486.187.2 GLM-4-32B91.789.490.590.189.590.390.480.571.476.4 ExpressMind-14B98.497.998.198.297.598.397.490.586.888.5 Smart Expressway Knowledge QA Qwen-32B96.595.295.895.594.895.783.178.576.479.5 Baichuan-32B89.787.588.688.287.088.187.777.568.473.8 DeepSeek-R1-Distill-Qwen-32B95.496.1 94.896.795.596.295.688.983.186.0 DeepSeek-R1-Distill-Llama-70B95.996.195.594.896.295.794.188.484.286.1 Llama-3.3-70B96.895.996.395.895.496.195.788.584.086.9 GLM-4-32B90.688.789.690.289.190.489.379.870.975.2 ExpressMind-14B96.796.296.497.597.097.696.489.285.487.8 Intelligent Transport System Knowledge QA Qwen-32B94.393.593.994.894.295.291.784.271.280.5 Baichuan-32B88.686.887.786.585.986.487.576.868.972.5 DeepSeek-R1-Distill-Qwen-32B93.494.193.892.792.593.295.687.183.585.5 DeepSeek-R1-Distill-Llama-70B93.994.594.592.893.292.794.187.484.286.1 Llama-3.3-70B94.994.194.595.094.595.194.287.684.785.8 GLM-4-32B90.188.589.389.488.889.788.678.969.774.1 ExpressMind-14B95.6 95.195.396.596.096.895.988.784.986.5 2 4 6 8 10 Safety Compliance Logical Consistency Actionability Cause Depth Preventive Insight Qwen-32B Baichuan-32B DeepSeek-R1-Distill-Qwen-32B DeepSeek-R1-Distill-Llama-70B Llama-3.3-70B GLM-4-32B ExpressMind (Pre-train) ExpressMind (Pre-train+RL) Figure 6: Performance of ExpressMind with RL Alignment. accuracy of 98.4% and a Short Answer F1-score of 88.5%, surpass- ing the strongest baseline, Llama-3.3-70B, by a significant margin. Furthermore, regarding the GPT-Score, which evaluates deep se- mantic understanding, ExpressMind maintains a high average of 85.7%, demonstrating superior competitiveness against reasoning- distilled models like DeepSeek-R1-Distill-Llama-70B. These results emphasize ExpressMind’s expert-level proficiency and its ability to provide precise, logically coherent responses within the specialized expressway transportation domain. Qwen-32BBaichuan-32BDeepSeek-R1- Distill-Qwen-32B DeepSeek-R1- Distill-Llama-70B Llama-3.3-70BGLM-4-32BExpressMind (Pre-train) ExpressMind (Pre-train+RL) Model 5 10 15 20 25 Reasoning time (ms) 13.4 ms 17.5 ms 12.3 ms 15.1 ms 13.6 ms 16.2 ms 10.8 ms 13.2 ms Figure 7: The Reasoning Time of the RL Alignment. 4.3.2 RL Alignment Results. The results presented in Figure 6 in- dicate that ExpressMind (Pretrain+RL) consistently outperforms existing generalist baseline methods in the specialized task of ex- pressway incident management strategy generation on five metrics, detailed in appendix A. Specifically, in domain-critical metrics such as Safety Compliance and Actionability, ExpressMind (Pretrain+RL) achieves scores in the range of 8.0–9.0, which is notably higher than general baselines like Llama-3.3-70B and Qwen-32B. The ablation study between the two ExpressMind configurations underscores the decisive impact of RL alignment: the base pretrained model without RL tuning yields the weakest performance, whereas its RL- aligned counterpart demonstrates a significant improvement. This enhancement is directly attributable to the model’s better-aligned Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. Incident Response Strategy Generation Incident Description üIncident Type : Illegal Stop üIncident Level : Regular üDetailed Information : Two male drivers are communicating outside their vehicles ... leading to traffic delays. Response Strategy Report Time : x year x/x Road Section : x-x-x+x-West Position : The incident is located on the emergency lane (right side of the frame), near the guardrail area. Weather : Foggy weather, low visibility, and poor lighting conditions. Strategy : The incident mainly affects the emergency lane ... vehicles on the main lanes are driving normally. Reflection : It is recommended to promptly contact relevant authorities for on-site verification and handling. Figure 8: An Example of Incident Response Strategy Generation. chain-of-thought reasoning with the required procedural knowl- edge. As shown in Figure 7, ExpressMind (Pre-train+RL) demon- strates exceptional deterministic performance in terms of reasoning efficiency. Experimental results indicate that its average inference latency is reduced to 13.2 ms, achieving a 24.6% acceleration com- pared to models like Baichuan-32B. More importantly, the tightly clustered distribution in the box plot reveals minimal latency jitter during instruction processing. This combination of low latency and high stability ensures reliable performance for time-sensitive applications, such as millisecond-level response requirements in smart expressway scenarios. Regulatory UnderstandingTechnical SpecificationProfessional Vocabulary 0.0 0.2 0.4 0.6 0.8 F1-Score 0.73 0.61 0.780.78 0.75 0.88 0.87 0.78 0.91 ExpressMind(no RAG)ExpressMind(RAG)ExpressMind(LightRAG) Figure 9: The Ablation Experiment of RAG. 4.3.3 Knowledge Retrieval Ablation Study. This section verifies the improvement in the generation capability of ExpressMind resulting from the use of the expressway knowledge base. We leverage Ligh- tRAG to query this repository. The evaluation dataset comprises 200 queries addressing complex traffic regulation comprehension and 100 queries focused on technical specifications. Accuracy Precision Analysis Time Space Level 4 6 8 10 ExpressMind-VL Qwen3-VL MoTIF InternVL3.5 MiniCPM-V 4.5 VideoLLaMA3 (a) Shandong Expressway Accuracy Precision Analysis Time Space Level 4 6 8 10 ExpressMind-VL Qwen3-VL MoTIF InternVL3.5 MiniCPM-V 4.5 VideoLLaMA3 (b) Guangdong Expressway Figure 10: Results of Traffic Incident Detection. We employ the F1-Score to evaluate the factual accuracy of the generated responses. As illustrated in Figure 9, the experimental results demonstrate that ExpressMind exhibits exceptional reason- ing capabilities and robust retrieval performance. The Expressway knowledge base can increase the occurrence probability of profes- sional vocabularies by 16.7%. 4.3.4 Scene Understanding Comparison. Built upon the Express- Mind backbone and a sophisticated cross-modal encoder, the MLLM, ExpressMind-VL, demonstrates exceptional proficiency in under- standing traffic videos. In this study, we evaluate its performance against a range of leading MLLMs, including VideoLLaMA 3 [35], MiniCPM-V 4.5 [34], InternVL 3.5 [29], and Qwen3-VL [3]. To quantitatively evaluate the quality of the generated descrip- tions, we employ four standard automated metrics: BLEU-4 [23], ROUGE-L [18], CIDEr [27], and BERTScore [39]. Each metric as- sesses a distinct dimension of linguistic fidelity, ranging from lex- ical overlap to semantic similarity. To ensure reproducibility, we ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY (a) Detection Accuracy(b) Detection Recall Figure 11: Results of Traffic Incident Detection. conducted a comprehensive evaluation using a dataset of 670 ex- pressway surveillance videos input into all comparative models. As presented in Table 2, the experimental results demonstrate that ExpressMind-VL significantly outperforms other generalist models in the descriptive accuracy of expressway scenes. The model exhibits a superior capability to interpret complex traffic scenar- ios, validating the effectiveness of our domain-specific multimodal alignment. Table 2: Automated evaluation of quantitative results. ModelLLMBLEU-4ROUGE-LCIDErBERTScore MoTIFLLaMA2-7B82.5987.4569.9588.43 VideoLLaMA3Qwen 2.5-7B79.6385.3267.4285.33 MiniCPM-V 4.5LLaMA 3-8B81.9887.1369.7688.04 InternVL3.5Qwen 3-38B84.5388.9471.6788.97 Qwen3-VLQwen 3-32B84.8589.3072.9689.18 ExpressMind-VLExpressMind-14B 85.2489.2573.3689.28 Accurate detection and comprehensive analysis of traffic in- cidents are paramount for intelligent expressway operation. To rigorously evaluate these capabilities, we benchmark ExpressMind- VL against the high-performing Qwen3-VL-32B using a curated dataset of 200 traffic incident videos. The quantitative evaluation employs a multi-dimensional metric suite: Accuracy and Precision for event classification, and F1-scores to assess the semantic fi- delity of descriptive texts concerning event severity (Level), causal analysis, spatiotemporal context (Time & Space), and congestion status (Queue). As illustrated in Figure 10, experimental results demonstrate that ExpressMind-VL exhibits significantly superior recognition and reasoning capabilities in traffic incident detection and analysis compared to the baseline. Furthermore, a represen- tative example of Incident Response Strategy Generation, which translates these analytical insights into actionable decisions, is vi- sualized in Figure 11. To evaluate the practical performance of ExpressMind-VL in real-world expressway scenarios, we assessed its detection capa- bility for six core types of traffic incidents on the Express-VQA dataset. As shown in Figure 11, the accuracy and recall rates of ExpressMind-VL have both exceeded 90% across all tasks, includ- ing abnormal parking, pedestrian intrusion, non-motorized vehicle detection, traffic congestion, extreme weather, and traffic incidents. The high precision and recall of ExpressMind-VL can be attributed to the multi-level technical strategies integrated during its develop- ment. Pre-training on high-quality video-text pairs established a robust foundation for cross-modal semantic alignment. As shown (a) Accuracy(b) Recall Figure 12: Ablation Results of VPA. in Figure 12, the VPA mechanism explicitly enhances the contribu- tion of visual features, ensuring the dominance of dynamic visual cues in the reasoning process. Furthermore, standardized prompt engineering reformulates the detection task into a structured text generation problem, enabling the model to naturally incorporate prior knowledge such as traffic rules into logical reasoning. 4.4 Deployment Analysis The proposed ExpressMind has already been deployed in the Shan- dong Expressway Cloud Brain system. It demonstrates domain- specific comprehension in professional knowledge question-answering and the generation of emergency response strategies for traffic inci- dents. Furthermore, addressing user needs for customized ExpressMind- VL functionality, we have developed an ExpressMind-VL-based ex- pressway incident monitoring and management platform for both Shandong and Zhejiang expressways. This study designs standard- ized prompt engineering and a video stream detection mechanism, enabling the model to retain 10 seconds of video upon detecting a traffic incident while simultaneously generating a structured incident analysis report. Through this approach, the system can automatically identify traffic incidents, analyze Traffic scenes, and produce response plans based on real-time conditions. Ultimately, ExpressMind-VL achieves a fully autonomous "perception-analysis- decision" processing pipeline for expressway traffic incidents, serv- ing as an intelligent central hub of expressway. The demonstration of the system application is described in the Appendix D. 5 Conclusion This study introduces ExpressMind, the first domain-specific MLLM designed for expressway scenarios. It is built through multiple tech- nical innovations: a two-stage pretraining paradigm for domain knowledge internalization, a GRPO-enhanced RL framework for safety-critical reasoning alignment, a graph-augmented RAG mech- anism for real-time spatiotemporal knowledge retrieval, and a VPA multimodal alignment module for deep video understanding. To support this work, we have open-sourced the first training dataset covering domain knowledge, incident CoT strategy reasoning, and multimodal incident detection VQA. The ExpressMind has been applied in top-tier expressway groups, serving as a representative application case of large models in the expressway domain. In future work, we will focus on three key improvements: en- hancing multimodal spatiotemporal reasoning for dynamic sce- nario understanding, strengthening chain-of-thought alignment in long-text analysis, and advancing model lightweighting for edge deployment. Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. References [1]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). [2] Md Adnan Arefeen, Biplob Debnath, and Srimat Chakradhar. 2024. TrafficLens: Multi-Camera Traffic Video Analysis Using LLMs. In 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 3974–3981. [3]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. 2025. Qwen3-VL Technical Report. arXiv:2511.21631 [cs.CV] https://arxiv.org/abs/2511.21631 [4]Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al.2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025). [5]Qi Cao, Ruiyi Wang, Ruiyi Zhang, Sai Ashish Somayajula, and Pengtao Xie. 2025. DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning. arXiv preprint arXiv:2505.20241 (2025). [6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. doi:10.18653/ v1/N19-1423 [7] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv e-prints (2024), arXiv–2407. [8]Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al.2024. Chatglm: A fam- ily of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793 (2024). [9] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al.2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024). [10] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025). [11]Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. 2024. Lightrag: Simple and fast retrieval-augmented generation. arXiv preprint arXiv:2410.05779 (2024). [12] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3. [13]Senkang Hu, Zhengru Fang, Zihan Fang, Yiqin Deng, Xianhao Chen, Yuguang Fang, and Sam Tak Wu Kwong. 2025. Agentscomerge: Large language model empowered collaborative decision making for ramp merging. IEEE Transactions on Mobile Computing (2025). [14]Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al.2023. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728 (2023). [15] Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, and Hui Xiong. 2025. Llmlight: Large language models as traffic signal control agents. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 2335–2346. [16]Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. PMLR, 19730–19742. [17]Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. 2024. Urbangpt: Spatio-temporal large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5351–5362. [18]Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81. [19] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual in- struction tuning. Advances in neural information processing systems 36 (2023), 34892–34916. [20]Qian Ma, Hongliang Chi, Hengrui Zhang, Kay Liu, Zhiwei Zhang, Lu Cheng, Suhang Wang, Philip S Yu, and Yao Ma. 2025. Overcoming pitfalls in graph contrastive learning evaluation: Toward comprehensive benchmarks. ACM SIGKDD Explorations Newsletter 27, 2 (2025), 97–106. [21]Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, and Yu-Gang Jiang. 2024. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms. Advances in Neural Information Processing Systems 37 (2024), 23464–23487. [22] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730–27744. [23] Matt Post. 2018.A Call for Clarity in Reporting BLEU Scores. arXiv:1804.08771 [cs.CL] https://arxiv.org/abs/1804.08771 [24]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748–8763. [25]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems 36 (2023), 53728–53741. [26]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al.2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024). [27]Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition. 4566–4575. [28] Peng Wang, Xiang Wei, Fangxu Hu, and Wenjuan Han. 2024. Transgpt: Multi- modal generative pre-trained transformer for transportation. In 2024 international conference on computational linguistics and Natural Language processing (CLNLP). IEEE, 96–100. [29]Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al.2025. Internvl3. 5: Ad- vancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265 (2025). [30]Zihe Wang, Haiyang Yu, Changxin Chen, Zhiyong Cui, Yufeng Bi, Yilong Ren, Zijian Wang, Delan Kong, Jing Tian, Shoutong Yuan, et al.2025. MoTIF: An end-to-end multimodal road traffic scene understanding foundation model. Com- munications in Transportation Research 5 (2025), 100227. [31]Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024. Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation. arXiv preprint arXiv:2401.08417 (2024). [32] Jiacong Xu, Shao-Yuan Lo, Bardia Safaei, Vishal M Patel, and Isht Dwivedi. 2025. Towards zero-shot anomaly detection and reasoning with multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference. 20370–20382. [33]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025). [34]Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, et al.2025. Minicpm-v 4.5: Cooking efficient mllms via architecture, data, and training recipe. arXiv preprint arXiv:2509.18154 (2025). [35]Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al.2025. Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106 (2025). [36]Jianqing Zhang, Xinghao Wu, Yanbing Zhou, Xiaoting Sun, Qiqi Cai, Yang Liu, Yang Hua, Zhenzhe Zheng, Jian Cao, and Qiang Yang. 2025. Htfllib: A compre- hensive heterogeneous federated learning library and benchmark. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 5900–5911. [37]Siyao Zhang, Daocheng Fu, Wenzhe Liang, Zhao Zhang, Bin Yu, Pinlong Cai, and Baozhen Yao. 2024. Trafficgpt: Viewing, processing and interacting with traffic foundation models. Transport Policy 150 (2024), 95–105. [38] Tianlong Zhang, Xiaoxi He, Yuxiang Wang, Yi Xu, Rendi Wu, Zhifei Wang, and Yongxin Tong. 2025. FedMetro: Efficient Metro Passenger Flow Prediction via Federated Graph Learning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 5215–5224. [39] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019). [40] Xingchen Zou, Yuhao Yang, Zheng Chen, Xixuan Hao, Yiqi Chen, Chao Huang, and Yuxuan Liang. 2025. Traffic-r1: Reinforced llms bring human-like reasoning to traffic signal control systems. arXiv preprint arXiv:2508.02344 (2025). ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY A ExpressMind Setup and Metrics A.1 Hyperparameter Settings In this section, we detail the hyperparameter configurations for ExpressMind. To facilitate clear presentation and reproducibility, the specific settings for these phases are reported separately in Table 3. Table 3: Hyperparameter Settings for Pre-training and GRPO Algorithm Pre-training Hyperparameters Parameter NameValue Model ArchitectureQwen-14B DeepSpeed Strategy ZeRO Stage 3 Precisionbfloat16 Max Sequence Length8192 OptimizerAdamW Weight Decay 0.1 Gradient Clipping1.0 Learning Rate (lr)1e-5 Lr SchedulerCosine Warmup Ratio 0.05 Batch Size256 Epochs 3 GRPO Algorithm Hyperparameters Parameter NameValue Learning Rate (lr)1e-5 Lr Scheduler Cosine Warmup Ratio0.05 Group Size 16 KL Coefficient0.04 Clip Range 0.2 Visual Encoder Hyperparameters Parameter NameValue image resolution224 patch size14 hidden layers 24 attention heads16 A.2 GPT-Score To evaluate the performance of short-answer questions, we employ GPT-Score powered by GPT-4o. Functioning as a virtual domain expert, the model compares the Predicted Answer against the Ques- tion and Standard Ground Truth. It assesses factual accuracy and logical completeness to assign a quantitative score ranging from 0 to 100. The system prompt used for this evaluation is defined as Figure 13: system_prompt: You are a 'Highway Operations and Management Expert' with 20 years of experience. Your task is to evaluate the accuracy of the candidate's (AI model) response, akin to grading a professional examination paper. You must conduct the scoring strictly according to the following Evaluation Standards: 1. Strict Ground Truth Adherence: Do not merely check for semantic alignment. You must explicitly verify the precision of key data, regulatory articles, and handling procedures against the provided Ground Truth. 2. Key Point Verification: Extract core entities from the Ground Truth (e.g., speed limit values, emergency hotlines like 12122, procedural order like 'turn on hazard lights first') and verify if the candidate's response includes them. 3. Safety Penalty: Any errors involving traffic safety must result in severe penalty points. 4. Scoring: Assign a quantitative score from 0 to 100 based on factual accuracy and logical completeness. Figure 13: The System Prompt of GPT-Score. A.3 SFT-sysprompt During the SFT phase, to ensure consistency in model outputs and adherence to specific interaction protocols, a unified system prompt was integrated at the beginning of each training sample. As illustrated in Figure , this prompt defines the model’s core identity, task boundaries, and response style. SystemPrompt=You are a Senior Engineer and Researcher in Intelligent Transportation Systems (ITS). You specialize in the bridge between classical traffic flow theory and AI- driven autonomous stacks. 1. Expertise Pillars: Standards：x,Intelligence: Spatio-temporal forecasting, RL-based decision making, and trajectory prediction. 2. Execution Guidelines: No Fluff: Skip all introductory pleasantries (e.g., "I'd be happy to help"). Start with the answer. 3. Reliability & Safety: Probabilistic Language: Use "95% confidence," "probabilistic bounds," or "asymptotic stability" instead of "perfect" or "absolute." 4. Interaction Templates: Policy Requests: Focus on Liability, Privacy, and Safety Standards.Engineering Requests: Prioritize real-time constraints, redundancy, and hardware deployment. Figure 14: The System Prompt of SFT. A.4 Evaluation Using LLM-as-a-Judge A.4.1 RL Alignment. To evaluate the strategies generated by the LLM for the expressway incident response task, this study utilizes an "LLM-as-a-Judge" framework. Which assesses the strategies based on the following five key dimensions: • Safety Compliance: Evaluates whether the plan prioritizes safety by explicitly including necessary safety distances, on- site protection, and personnel evacuation instructions to fundamentally prevent secondary accident risks. •Preventive Insight: Validates whether the report strictly follows and completes the four-stage cognitive chain, which proceeds from [Incident Description] — [Causal Infer- ence] — [Response Strategy Formulation] — [Strategy Evaluation], ensuring a logically coherent and closed-loop process. •Logical Consistency: Validates whether each cause iden- tified in the Causal Inference section is addressed in the Strategy Formulation section. •Actionability: Evaluates whether the accident response strategy is concise, clear, and non-redundant, ensuring high information density and high executability of the generated strategy for on-site personnel. •Cause Depth: Examines the accuracy and depth of the inci- dent cause analysis, as well as the correct use of professional terminology for expressway incident response, to ensure the foundational knowledge required for effective response planning. A.4.2 Scene Understanding. To systematically evaluate the MLLMs’ scene understanding capabilities on expressway surveillance videos, this study adopts the "LLM-as-a-Judge" evaluation framework and designs the following six core dimensions: • Accuracy: Evaluates the overall correctness of the model’s description of incidents, objects, and states in the video, serv- ing as a baseline indicator of comprehensive performance. Conference acronym ’X, June 03–05, 2018, Woodstock, NYWang et al. •Level: Evaluates the model’s ability to provide an overall summary and qualitative assessment of the video content, judging whether it can go beyond local details and accurately summarize the core events and overall situation. • Precision: Evaluates the model’s accuracy in fine-grained recognition tasks, including precise descriptions of specific targets such as vehicle types, traffic signs, construction facil- ities, and human behaviors. • Space: Evaluates the model’s understanding of static and dynamic spatial relationships between key entities (vehicles, personnel, facilities) in the scene, such as relative positions, lane occupancy, and driving directions. •Analysis: Evaluates the model’s ability to infer causality, impact, and potential risks of incidents, such as analyzing accident causes, predicting congestion spread, or assessing the effectiveness of response measures. •Time: Evaluates the model’s understanding of event se- quencing and dynamic evolution processes, such as judging the order of actions and the continuity of state changes. These six dimensions are evaluated by the LLM acting as a judge according to structured instructions, and their scores col- lectively constitute a systematic and comprehensive evaluation of the model’s multimodal scene understanding capabilities. BImplementation Details of Reward Functions In this section, we provide the granular implementation details of the Structure-Knowledge-Semantics reward mechanism, including the construction of the expert vocabulary, the specific configura- tions of the embedding models, and the hyperparameter settings used in our experiments. B.1 Structural Integrity Constraints (푅 푠푡푟푢푐푡 ) To enable precise parsing of the model’s CoT, we defined four special control tokens corresponding to the standard traffic incident disposal workflow. The Structural Integrity Reward푅 푠푡푟푢푐푡 performs strict string matching to verify the existence and order of these tokens. Table 4: Definition of Reasoning Stage Delimiters Stage Index (푘)Stage NameControl Token (푆 푘 ) 1Perception[Incident Description] 2Analysis[Causal Inference] 3Decision[Response Strategy Formulation] 4Reflection[Strategy Evaluation] The index functionidx(푆 푘 )returns the character position of the first occurrence of token푆 푘 in the generated string푂. If a token is missing,idx(푆 푘 )= ∞. The sequence checkI(idx(푆 1 )< ·< idx(푆 4 )) ensures the reasoning flow is logically valid. B.2 Domain Knowledge Alignment (푅 푘푛표푤 ) The domain alignment reward relies on a curated Stage-Specific Expert Vocabulary (V 푘 ). Vocabulary Construction. We constructedV 푘 by mining high- frequency professional terms from a corpus of 500+ real-world ex- pressway traffic emergency plans and national standard documents (e.g., GB/T 29100-2012). We utilized TF-IDF to extract keywords and manually filtered them to ensure relevance to each specific reasoning stage. Examples are shown in Table 5. Table 5: Examples of Expert VocabularyV 푘 for Each Stage StageRepresentative Keywords (Translated) 푆 1 : PerceptionMulti-vehicle pileup, Hazardous chemical leakage, Occupying emergency lane, Visibility range, Traffic volume saturation, Fire spreading 푆 2 : AnalysisSecondary accident risk, Chain reaction, Brake failure, Fatigue driving, Lane capacity reduction, Danger radius 푆 3 : DecisionRemote diversion, Upstream interception, Green wave control, Air-ground coordination, Break-bulk transport, Gating control 푆 4 : EvaluationResidual congestion, Rescue efficiency, Public sentiment moni- toring, Secondary damage assessment Perplexity Computation. To calculate the PPL penalty termPPL(푂), we utilize a frozen version of the Supervised Fine-Tuned (SFT) model as the reference. The threshold휏 푝푙 is dynamically set to the 95th percentile of the PPL distribution observed on the validation set, preventing the model from generating incoherent keyword lists. B.3 Semantic Consistency (푅 푠푒푚 ) The semantic consistency reward evaluates the strategic quality of the generated response. •Embedding Model (휙): We employ BGE-M3 (BAAI Gen- eral Embedding), a state-of-the-art multilingual embedding model, to map text into dense vectors. We specifically use the [CLS] token embedding. •Focus Scope: We extract only the content within the푆 2 (Analysis) and푆 3 (Decision) segments for embedding, de- noted as푆 2 ⊕푆 3 . This isolates the core logic from the generic description (푆 1 ) or formatting text. • Reference Set (D 푟푒푓 ): For each query in the training batch, we retrieve top-퐾(퐾=5) most similar historical expert cases from our vector database to form the local reference setD 푟푒푓 . This ensures the reward is calculated against contextually relevant ground truth. C Benchmark This research constructs a comprehensive evaluation benchmark for expressway operation, comprising three distinct dimensions: The Textual Knowledge QA benchmark consists of 20,000 pairs of QA data. It focuses on the core dimensions of express- way operation, such as functional positioning and land-use control mechanisms. Additionally, it incorporates expert knowledge for emergency driving, providing precise instructions for handling ve- hicle instability, such as correcting rear-wheel skids on hazardous road surfaces. An example of this dataset’s structure and content is illustrated in Figure 15. The Incident CoT benchmark includes 300 samples designed to evaluate complex decision-making and causal reasoning. Each entry follows a structured logical flow. This simulates the chain of command in a smart expressway operarion center, moving from ini- tial accident reporting to liability determination and the evaluation ExpressMind: A Multimodal Pretrained Large Language Model for Expressway OperationConference acronym ’X, June 03–05, 2018, Woodstock, NY Question1: What are the key points of urban rail transit planning compilation? Answer1:Urban rail transit planning focuses on six core dimensions: research scope and timeframe, functional positioning, and network/station layout. It further addresses intermodal integration with other transit systems, prioritized research areas, and land-use control mechanisms to ensure the feasibility and implementation of the network. Question2: How should you handle a situation where the rear wheels of a vehicle skid to the right on a muddy road? Answer2: To correct a rear-wheel skid to the right, steer to the right. Avoid sudden braking or accelerating; instead, gently turn the wheel in the direction of the skid while lightly tapping the brakes. This helps the vehicle regain traction and realign. Remember the rule: steer with a rear skid and against a front skid to stay on path. Text Konwledge QA Figure 15: Example of Textual Knowledge QA. of response strategies like lane closures. The structured reasoning process is shown in Figure 16. Incident CoT Context 1. Incident Description: At [Time], a rear-end collision involving ... occurred on the [position], .... 2. Causal Inference: Upon analysis, [x] is determined to bear primary liability for the accident, ..... 3. Response Strategy Formulation: First, establish traffic control, ... 4. Strategy Evaluation: The decision to close Lane 1 is appropriate. Figure 16: Example of the Incident CoT Structured Data. The Express-VQA is a multi-modal benchmark designed for ex- pressway scene understanding, consisting of 670 real-world video segments captured from expressway surveillance systems. It eval- uates the model’s, particularly the LLM’s, ability to understand and reason about expressway scenes, specifically in identifying and analyzing six typical expressway incidents: traffic accidents, con- gestion, road construction, abnormal parking, pedestrian intrusion, and debris clearance. Visual examples of these categories and the corresponding annotation style are provided in Figure 17. Express-VQA Expressway Viedo Q: What traffic incident occurred in the video? A: In this video, the traffic volume is small in the upward direction, while it is large in the downward direction. In the downward direction, x . Q: Where did this traffic incident occur? A: The incident occurred at a moderate distance from the camera and occupied the emergency lane in the downward direction of the road, without affecting the normal lanes. Figure 17: Visualization of the Traffic Incident VQA Data. D Application The ExpressMind-VL intelligent operation system has been de- ployed in practical applications for multi-task scenarios on express- ways. The visualization system, as shown in the figure 18, includes functions such as releasing warning information, summarizing traf- fic conditions, describing video events, and generating handling rec- ommendations. We have deployed the system on the expressways in Shandong and Zhejiang provinces. In the intelligent management of Shandong expressways, ExpressMind-VL classifies the types and severity levels of traffic surveillance videos, and generates analytical reports along with handling strategies for traffic incidents. For the intelligent management of Guangdong expressways, ExpressMind- VL detects traffic events based on real-time video streams and pro- duces structured textual descriptions for comprehension. The code, data, benchmark and the demonstration of the system application are available at: https://wanderhee.github.io/ExpressMind/. Figure 18: Application of EpxressMind-VL.