Paper deep dive
Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving
Zehao Wang, Huaide Jiang, Shuaiwu Dong, Yuping Wang, Hang Qiu, Jiachen Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/27/2026, 1:39:38 AM
Summary
Drive My Way (DMW) is a personalized Vision-Language-Action (VLA) framework for autonomous driving that aligns vehicle behavior with long-term user habits and real-time natural language instructions. It utilizes a novel Personalized Driving Dataset (PDD) and employs Group Relative Policy Optimization (GRPO) to fine-tune driving policies, ensuring safety, comfort, and efficiency while adapting to individual driver styles.
Entities (5)
Relation Signals (3)
Drive My Way → evaluatedon → Bench2Drive
confidence 100% · Extensive closed-loop experiments on Bench2Drive benchmark demonstrate that DMW improves style instruction adaptation
Drive My Way → uses → Personalized Driving Dataset
confidence 100% · DMW learns user embeddings that encode long-term driving behaviors from a newly collected personalized driving dataset
Drive My Way → implements → Group Relative Policy Optimization
confidence 95% · We employ the Group Relative Policy Optimization (GRPO) to further finetune the personalization ability
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. However, existing end-to-end autonomous driving systems either optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision-Language-Action (VLA) driving framework that aligns with users' long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide additional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver's own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.25740v1
- Canonical: https://arxiv.org/abs/2603.25740v1
Trouble viewing inline? Open PDF directly →
Full Text
66,789 characters extracted from source content.
Expand or collapse full text
Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving Zehao Wang 1 , Huaide Jiang 1 , Shuaiwu Dong 1 , Yuping Wang 1,2 , Hang Qiu 1 , Jiachen Li 1,† 1 University of California, Riverside 2 University of Michigan Abstract Human driving behavior is inherently personal, which is shaped by long-term habits and influenced by short-term intentions. Individuals differ in how they accelerate, brake, merge, yield, and overtake across diverse situations. How- ever, existing end-to-end autonomous driving systems ei- ther optimize for generic objectives or rely on fixed driving modes, lacking the ability to adapt to individual preferences or interpret natural language intent. To address this gap, we propose Drive My Way (DMW), a personalized Vision- Language-Action (VLA) driving framework that aligns with users’ long-term driving habits and adapts to real-time user instructions. DMW learns a user embedding from our personalized driving dataset collected across multiple real drivers and conditions the policy on this embedding during planning, while natural language instructions provide addi- tional short-term guidance. Closed-loop evaluation on the Bench2Drive benchmark demonstrates that DMW improves style instruction adaptation, and user studies show that its generated behaviors are recognizable as each driver’s own style, highlighting personalization as a key capability for human-centered autonomous driving. Our data and code are available at https://dmw-cvpr.github.io/. 1. Introduction End-to-end autonomous driving has emerged as a powerful paradigm that directly learns to map raw multi-modal sen- sor inputs to driving trajectories or control actions [1–4]. Recent advances in unified architectures and foundation- model-based frameworks have demonstrated impressive performance in open-loop benchmarks through imitation of expert trajectories [5–7]. However, these systems often op- timize for generic safety and efficiency objectives, over- looking the individuality and context-dependent nature of human driving behaviors. In practice, driving is inherently personal; individuals differ in how assertively they accel- erate, brake, or overtake depending on the situation, pur- † Corresponding author. zehao.wang1@email.ucr.edu, jiachen.li@ucr.edu Throttle: 0.5 Brake: 0.0 Steer: -0.3 Throttle: 0.0 Brake: 0.8 Steer: 0.0 Current Speed: 3 m/s Brake now,stop until there is no car from opposite. 36-year-old school teacher,16years of driving... Maintain speed, straddle the center line and take over. Driver A 27-year-old sales manager, 5 years of driving... Driver B Drive My Way Figure 1. Drive My Way (DMW) achieves end-to-end personal- ized driving via both long-term preference alignment and short- term style instruction adaptation. pose, or emotional state (e.g., commuting, leisure, or emer- gency travel). Existing autonomous driving systems typi- cally provide only a few preset modes (e.g., “sport”, “com- fort”, or “eco”) or manual parameter adjustments, which fail to capture subtle and evolving passenger preferences [8– 10]. More importantly, such rigid presets cannot interpret intuitive, natural language instructions, such as “I’m tired” or “I’m late for work”. Therefore, achieving long-term, context-adaptive personalization is crucial for enhancing user trust, comfort, and satisfaction [11, 12]. Existing research on personalized autonomous driving mainly falls into two categories. First, data-driven meth- ods extract predefined driving styles from human demon- strations and learn style-conditioned policies using behav- ior cloning or inverse reinforcement learning (IRL) [11, 13]. While effective at mimicking human-like styles, these approaches require diverse large-scale datasets and scale poorly to a growing population of users with heterogeneous preferences. Surmann et al. [14] mitigate this limitation through multi-objective reinforcement learning, enabling runtime adjustment of preference weights. However, such methods cannot handle real-time human interaction via nat- ural language or adapt to user instructions on the fly. Sec- 1 arXiv:2603.25740v1 [cs.RO] 26 Mar 2026 ond, language-driven approaches leverage large language models (LLMs) for instruction-based personalization [15]. For example, Talk2Drive [16] employs GPT-4 to interpret passenger commands and generate high-level driving de- cisions, reducing takeover rates in simplified intersection and parking scenarios involving limited traffic participants. Despite promising results, existing language-based meth- ods remain constrained to simple, low-interaction settings. They have not been systematically investigated in complex, dynamic driving contexts, such as merging, overtaking, or emergency braking, where trade-offs among safety, com- fort, and efficiency must adapt to dynamic scenes. Fully leveraging the reasoning and generalization capabilities of foundation models for such context-aware adaptations re- mains largely unexplored. Additionally, current personal- ization methods based on LLMs [16] or vision-language models (VLMs) [17] do not consider a driver’s long-term habits or cumulative experiences, which are diverse and continuously evolving. To address these issues, we propose DMW, a novel Vision-Language-Action (VLA) framework that integrates visual observations, contextual user profiles, and real-time natural language instructions to generate adaptive, person- alized driving actions, as shown in Fig. 1. DMW learns user embeddings that encode long-term driving behaviors from a newly collected personalized driving dataset comprising profiles, trajectories, and privileged information from thirty drivers across twenty realistic scenarios in CARLA [18]. To enable human interaction and real-time adaptation, we fur- ther apply reinforcement fine-tuning to the driving policy using rewards derived from safety, comfort, and efficiency objectives, whose weights are dynamically adjusted. Ex- tensive closed-loop experiments on Bench2Drive [19] show that DMW effectively adapts to different style instructions while maintaining safety. Further user studies demonstrate that its driving behavior aligns with individual user prefer- ences and expresses distinct driving behaviors. Our main contributions are summarized as follows: • We propose DMW, a novel personalized end-to-end au- tonomous driving framework that integrates contextual user embeddings to align policy behaviors with individual driving preferences. DMW also enables real-time human interaction and adaptability through reinforcement fine- tuning conditioned on natural language instructions. • We construct the first multi-modal personalized driving dataset (PDD) collected from thirty real drivers across diverse traffic scenarios in CARLA, which provides a valuable dataset for developing and evaluating human- centered driving models. • We conduct extensive evaluations on the Bench2Drive benchmark [19], complemented by personalization met- rics and user studies, which demonstrate the effectiveness of DMW in adapting its driving behavior to individual user preferences while maintaining a balanced trade-off among safety, efficiency, and comfort. 2. Related Work Foundation Models for Autonomous Driving. Existing end-to-end autonomous driving models [20–22] still strug- gle in closed-loop benchmarks [23, 24] due to reliance on imitation learning and limited generalization [25, 26]. Foundation models (LLMs, VLMs) have recently emerged [27–36] and existing work (e.g., DriveVLM [37], GPT- Driver [38], AlphaDrive [39]) leverages their world knowl- edge and semantic reasoning to interpret complex traffic scenarios, producing high-level decisions and language ra- tionales for their actions. In parallel, VLA models have demonstrated the ability to directly translate raw sensory in- puts and linguistic instructions into fine-grained actions [40, 41] or low-level control signals through action tokens [22] or action decoders [42]. While these approaches have made substantial progress toward language-conditioned driving, their capacity for personalization and adaptation to individ- ual driving preferences remains largely underexplored. Personalization in Autonomous Driving. The pursuit of personalized autonomous driving remains an active area of research, with the key challenge lying in aligning individual driving preferences. Recent studies like [11] learn style- conditioned driving policies, but overlook fine-grained in- dividual differences. MAVERIC [43] explores the learn- ing of a latent space of diverse and socially-aware driv- ing behaviors, offering greater flexibility compared to pre- defined style categories. Meanwhile, LLMs have enabled more intuitive human-vehicle interaction [8, 15, 17, 44], al- lowing systems to respond to personalized commands in real time. Recent works [16] leverage memory modules based on retrieval-augmented generation (RAG) to person- alize decisions by retrieving user-specific context. How- ever, these approaches rely on explicit human feedback and accurate retrieval, producing contexts that are largely de- scriptive rather than behavioral and fail to capture implicit, long-term driving tendencies. Moreover, most of them are evaluated only in open-loop settings, making it difficult to assess how well personalization transfers to closed-loop de- cision making and influences real-time driving behavior. 3. Personalized Driving Dataset To learn and evaluate personalized driving policies, we con- struct a new Personalized Driving Dataset (PDD), as illus- trated in Fig. 2. PDD captures the driving environments and the personal context of human drivers across diverse, highly interactive, safety-critical driving situations. Unlike prior datasets that rely on synthetic instruction-action pairs with limited behavioral variation [45] or those that infer style la- bels retrospectively using coarse VLM-based reasoning or heuristic labeling [11], our dataset records human driving behaviors during complex real-time interactions, which al- lows us to study how different individuals make decisions when faced with similar situations. We recruit thirty drivers with diverse backgrounds and 2 Driving Data RGB Image Human ActionExpert Speed 4 Scenario Types 20 Routes 30 Drivers Ego Status Surrounding Vehicles Profile Data Age Driving Experience Driving Preference ... Driving Style A 36-year-old school teacher with the following driving background: Driving Experience: 16 years of driving, typically 5–10 hours per week. Driving Style: Calm. Driving Purposes: Commuting to school, weekend hiking trips with family. ... Figure 2. An overview of the Personal Driving Dataset, which consists of the driving data and structured driver profile data. levels of driving experience. Before data collection, each participant completes a structured questionnaire covering demographic information, driving history, and typical driv- ing purposes such as commuting or leisure travel. These profiles provide a semantic context that connects personal background with long-term driving habits. Each participant then performs a standardized set of twenty driving scenar- ios in the CARLA, spanning four scenario types, includ- ing overtaking, merging into traffic, handling intersections, and navigating pedestrian crossings or encountering vehi- cle cut-ins. All drivers operate the vehicle using a Logitech G-series steering wheel and pedal setup to enable naturalis- tic control inputs. PDD records ego-vehicle motion states, scene perception of surrounding vehicles, pedestrians, cy- clists, and roadside hazards, as well as traffic context such as signal states, stop signs, route geometry, and speed lim- its. To support comparison across drivers and scenarios, we record an expert target speed using PDM-Lite [46]; the de- viation between human-driven speed and this target serves as a dense descriptor of behavioral style under varying con- ditions. Overall, PDD offers a rich and behavior-sensitive foundation for modeling personalized driving, which sup- ports preference-conditioned policy learning. 4. Problem Formulation We aim to learn a personalized driving policy that aligns with a driver’s long-term driving behavior while adapting to real-time preference instructions. LetM = 1,...,M denote the set of drivers, where M is the number of drivers. For each driver m ∈ M, the collected driving dataset is D m = (s m t ,a m t ) T m t=1 , where s m t and a m t denote the envi- ronmental state and the executed action of driver m at time step t, and T m is the trajectory length. The user profile ob- tained is P m , which summarizes the driver’s background, habits, and driving experience. The complete dataset across all drivers isD = S m∈M D m . We formalize personalized driving as a Markov Decision Process (MDP), defined by the tuple (S,A,O,T ,R,γ). S represents the state space, which includes environmental in- formation. A ⊂ R 3 is the continuous action space consist- ing of throttle, brake, and steering. At time t, the agent ob- serves o t = (I t , q t ,I t ,g t ,P m )∈O, where I t ∈ R H×W×3 is the front-view image, q t is the ego-vehicle state, I t is a language instruction expressing short-term preference (e.g., “I’m in a rush”), g t is the navigation target (e.g., route way- points), and P m identifies the context of the active driver.T denotes the transition in the environment. R is the reward function that evaluates the action. γ is the discount factor. The objective is to learn a driving policy π θ (a t | o t ) that maximizes the expected cumulative discounted re- ward: max θ E π θ h P T t=0 γ t R(s t ,a t ) i . To achieve person- alization, the policy must infer the driver’s long-term be- havioral tendencies from their historical driving data D m and relate these behaviors to their profile P m . During ex- ecution, the policy adapts its decisions based on the speci- fied user embedding z m p , which is encoded from the driver’s profile P m , and the current instruction I t , ensuring safe and effective driving behavior that reflects each driver’s charac- teristic style and situational preferences. 5. Method We focus on personalized adaptation of driving policies to accommodate different users and their short-term driving preferences. The overall diagram of DMW is provided in Fig. 3. Given the camera observations and navigation goals, the model fuses the driver’s long-term preference with user instructions to produce adaptive, personalized actions. 5.1. VLA Backbone We employ SimLingo [42] as our VLA backbone due to its ability to perform language grounding and planning. SimLingo is built on InternVL2-1B [47, 48], which inte- grates the InternViT-300M-448px vision encoder with the Qwen2-0.5B language model, offering computational effi- ciency compared to larger alternatives [49]. Supervised by privileged expert demonstrations from PDM-Lite [46] and auxiliary reasoning signals, the backbone predicts tempo- ral and geometric waypoints, which are then converted into target speed and steering commands. 5.2. Personalization via Reinforcement Fine-tuning Despite the strong planning capability of the VLA back- bone, imitation learning alone tends to capture generalized behaviors rather than user-specific driving styles. We em- ploy the Group Relative Policy Optimization (GRPO) to further finetune the personalization ability [50]. By sam- pling a group of outputs and computing normalized group advantages, GRPO inherently promotes policy specializa- tion. To generate a rich set of candidate actions for the GRPO policy update, we introduce a residual decoder. As shown in Fig. 3, learnable residual query tokens are inte- grated into the language model along with vision, language, navigation, user embedding, and motion query tokens. The resulting residual features are decoded by an MLP followed 3 Motion Predictor Residual Decoder Drive My Way Qwen2-0.5B User Instruction “I'm really in a rush.” “I feel car sick.” “ Keep steady,go carefully.” “I am being late to work.” “Let’s be patient and cautious.” ... Instruction Encoder Language Token Vision Encoder Vision Token Motion Query Residual Query ... Target Token Navigation Encoder Long Term Preference Encoder User Embedding Route Target: [50.023, 31.981] Driver Profile A Driving Preference Gender Driving Experience Choice 1Choice 2Choice 3 ............ ... Figure 3. An overview of the DMW framework with a pretrained VLA backbone. The model takes in front-view camera images, instruc- tions, route target points, and user profile as inputs, while the motion predictor outputs route and speed waypoints, which derive the base action (throttle, steer angle). The residual decoder outputs a discrete residual applied to the base to produce the final personalized action. by a categorical action head to produce two discrete residual adjustments: (i) speed change and (i) steering change. The combined action is then passed through a PID con- troller to produce the final a t = a base t + a ∆ t , where a base t is derived from the predicted waypoints and a ∆ t is the per- sonalized residual. This design allows the policy to pre- serve safe planning while adapting expressively to different drivers and situational intentions among multiple feasible planning trajectories (i.e., different choices of action). 5.3. Long-term Preference Learning and Alignment User Embedding Learning. To model the underlying driv- ing style of different users and align VLA with personalized preferences, we introduce a long-term preference encoder that learns user embeddings from profile. To relate the se- mantic profile context with corresponding driving behavior, we adopt a contrastive learning mechanism [51] to learn a shared latent space Z . As illustrated in Fig. 4, the long- term preference encoder f p (·) takes a profile P m of driver m from the total set of drivers M and outputs a user embed- ding z m p ∈ Z . f p (·) includes a DeBERTaV3 [52] text pro- cessor followed by a projection head. The route processor f b (·) is a temporal encoder with multi-head self-attention that processes a past trajectory window of length k at the current timestep t, ξ m t = (I m t−k:t , q m t−k:t ,a m t−k:t ), which consists of sequential front-view camera images I m t−k:t , ego- vehicle states q m t−k:t and driver’s actions a m t−k:t from D m in PDD. This outputs a behavior embedding z m b,t ∈ Z that summarizes the driver’s actual driving tendencies. To align the embeddings from the two encoders, we employ the In- foNCE [51] contrastive objective that encourages z m p and z m b,t from the same driver to be close while pushing embed- dings from different drivers apart: L m t =− log exp sim(z m p ,z m b,t )/τ P M j=1 exp sim(z j p ,z m b,t )/τ ,(1) where z j p is the user embedding of all other drivers, sim(·,·) denotes the cosine similarity and τ is the temperature. Preference Alignment. After obtaining the user embed- ding z m p , we condition the VLA policy on this embedding and further adapt it through reinforcement fine-tuning. The user embedding acts as a latent personalization prior, guid- ing the policy toward decision patterns consistent with the driver’s characteristic style. To align the policy with individual driving preferences while enhancing behavioral diversity, for each target driver m, we augment trajectories by conditioning the user em- bedding on both their own and another driver’s data u ∈ M, u ̸= m. We want driver m and driver u to behave dif- ferently in order to effectively distinguish different drivers, so for a given driver m, we always select u who has the least similarity with m regarding user embedding: u = arg min x∈M sim(z m p ,z x p ). The augmented action ̃a m t is es- timated by scaling the original human action a m t according to the ratio of their route-level action statistics, which cap- ture each driver’s action variability. Let the route-level aver- age action of driver m and u be ̄a m and ̄a u respectively, then we calculate the augmented action as: ̃a m t = ̄a m ̄a u · a m t . The reward is then formulated as a behavioral similarity between the model’s sampled action and the reference or augmented action. When the policy is conditioned on the target profile P m : R(s m t ,a t ) = d(a t ,a m t ). When conditioned on P u : R(s m t ,a t ) = d(a t , ̃a m t ), where d(·) measures similarity in action space, encouraging the policy to adapt decisions to each user embedding while learning a smooth manifold of diverse driving styles. 5.4. Human-Vehicle Personalized Interaction Style Instruction. In complex and dynamically evolving traffic scenarios, effective personalization requires the au- tonomous agent to infer not only the long-term driving pref- erence but also the context-adaptive preferences that vary across situations. To capture these nuances, we construct a 4 Driving History of Driver B Contrastive Learning Driver BProfile A 27-year-old sales manager with the following driving background: Driving Experience: 5 years of driving, typically more than 20 hours per week. Driving Purposes: ... Habits: ... Driver AProfile A 36-year-old school teacher with the following background: Driving Experience: 16 years of driving, typically 5– 10 hours per week. Driving Purposes: ... Habits:... Long Term Preference Encoder Driving History of Driver A v t Route Processor Driver Similarity Matrix Negative Pair Positive Pair A B C ABC ... ... v t Figure 4. The contrastive learning mechanism on the long-term preference encoder and route processor. style instruction set spanning twenty distinct driving scenar- ios. These instructions incorporate style intent and scene- specific semantics. Each scenario includes nine stylistic in- structions that cover three driving styles expressed at three levels of directness, following the degrees of implicitness defined in prior work [16]. This design enables the model to interpret both explicit commands and subtle linguistic cues that reflect real-time user intent across diverse con- texts, thereby achieving context-aware adaptation. Style-Aware Reward Adaptation. To translate user intents into safe and personalized objectives, we design a reward function that integrates driving performance metrics with instruction-dependent style alignment. The overall driving reward is formulated as a weighted combination of safety, efficiency, and comfort terms: R(s t ,a t ) = w s ·R safety +w e ·R efficiency +w c ·R comfort , (2) where w s , w e , and w c are importance weights, and R safety , R efficiency , and R comfort denote the corresponding compo- nents. The safety reward penalizes risky interactions based on Time-to-Collision (TTC): R safety = I safety TTC t ≥ β safety , where I safety is the binary indicator enforces a min- imum instruction-dependent safety threshold β safety . The efficiency reward encourages maintaining a speed consis- tent with the desired style: R efficiency = exp(−α · |v t − v pref |), where v t is the actual speed derived from action a t , v pref is the instruction-dependent preferred speed, and α is a penalty coefficient controlling sensitivity to de- viation.The comfort reward evaluates whether a t re- mains within smoothness limits: R comfort = I comf a steer t | < β lat and|a acceleration t | < β long , where I comf is a binary in- dicator, β lat and β long are instruction-dependent lateral and longitudinal comfort thresholds, respectively. To encourage personalized interaction with the user and Preference Alignment via GRPO Fine-tuning Preference Reward Generation with LLM “Let’s be patient.” Accident on the Side 0.3 0.4 0.3 S: 0.1 E: 0.9 C: 0.2 R: 0.29 S: 0.5 E: 0.4 C: 0.5 R: 0.48 S: 1.0 E: 0.2 C: 0.3 R: 0.63 Comfort: 0.2 Efficiency: 0.3 Safety:0.5 Motion Predictor Vision Encoder Qwen2-0.5B LoRA Residual Decoder Reasons Figure 5. The fine-tuning process and reward generation for short- term instruction alignment. adapt to real-time preferences, we map the reward param- eters based on both the instruction and the driving con- text. Specifically, for each style command I t , and sce- nario description, this mapping produces a set of weights (w s ,w e ,w c ), a TTC threshold β safety , a desired speed v pref , and comfort thresholds β lat , β long . Instructions associated with an aggressive style correspond to a higher w e and v pref to prioritize efficiency over other metrics. This formula- tion provides a more distinct optimization objective for each style rather than relying on a generic combination. To enable scalable and reliable weight adjustment, we employ a multi-stage approach illustrated in Fig. 5. First, we leverage the reasoning capabilities of advanced LLMs (e.g., GPT-5) to infer reward weights and thresholds from the scenario description and the instruction with a style S ∈ Conservative, Neutral, Aggressive, and the inferred parameters are initialized within predefined upper bounds for each style to maintain balanced trade-offs among met- rics. Second, the generated parameters are refined through expert review to ensure that the final reward functions re- flect the intended command semantics while preserving safe driving behaviors across diverse traffic conditions. This adaptation bridges the gap between subtle preferences in language and dynamic style rewards. 6. Experiments We conduct extensive closed-loop experiments to validate the effectiveness of the DMW framework in both user align- ment and instruction preference adaptation. Our evaluation is designed to answer three key research questions: • RQ1: Long-term Driving Alignment. Can our policy align with specific driving behavior when conditioned on 5 Table 1. Bench2Drive closed-loop driving metrics with different style instructions. We compare SimLingo and StyleDrive under different style instructions with our policy fine-tuning with fixed rewards weights (DMW-Vanilla) and style-aware rewards weights (DMW). MethodStyleDSSREfficiencyComfortSpeedAccelerationLCHeadwayTT SimLingo [42]Aggressive78.5665.83247.6018.617.665.390.7525.9925.35 Neutral78.1565.85241.4424.677.375.220.7527.8131.41 Conservative78.1865.56238.7726.997.215.290.7029.1233.02 StyleDrive [11]Aggressive75.6860.89256.7116.797.235.590.7424.9527.76 Neutral76.2662.13249.0721.356.985.430.6623.6229.12 Conservative77.0261.96242.1823.676.825.390.7027.1929.98 DMW-VanillaAggressive82.1970.97253.1015.867.865.290.7826.4619.69 Neutral81.9670.63247.7719.217.665.170.7726.6323.16 Conservative81.4871.05246.8021.877.755.370.7526.9022.51 DMWAggressive79.5067.36281.5621.627.726.010.7026.3726.93 Neutral82.0370.95244.9828.676.345.430.6127.6040.75 Conservative82.7271.56237.0634.626.185.260.6030.0547.38 learned user embeddings? • RQ2: Short-term Instruction Adaptation. How well does the policy align with various styles of real-time lan- guage commands under different scenarios? • RQ3: Driving Performance. Does the introduction of personalization maintain or compromise driving perfor- mance, particularly regarding safety and success rate in complex scenarios? 6.1. Experimental Setup Implementation Details. For user embedding learning, both the long-term preference encoder and the route pro- cessor are trained using the AdamW optimizer [53] with a weight decay of 1e-3 and a learning rate of 1e-4. After convergence, the long-term preference encoder is frozen, and we fine-tune the full motion predictor and residual de- coder while adopting the LoRA adapter [54] for parameter- efficient adaptation of Qwen2-0.5B [49]. The full model is trained on eight NVIDIA RTX A6000 GPUs with a per- GPU batch size of 8. For GRPO, we generate 4 responses per input for policy gradient updates. Scenarios and Baselines. We evaluate closed-loop perfor- mance using Bench2Drive [19]. Unlike the original evalu- ation, which continues the episode after a collision, we ter- minate the route immediately once a collision occurs to em- phasize the safety-critical assessment. To validate the effect of personalization, we compare DMW with SimLingo [42] under various styles of instruction, focusing on whether the model can effectively adjust its driving tendency accord- ing to user-specific preferences. Additionally, to strengthen comparisons with prior personalization-focused work, we implement a StyleDrive-like [11] baseline by mapping each instruction to a style condition (i.e., Aggressive, Neutral, Conservative) and injecting it into the policy. Meanwhile, we adopt MORL-PD [14] as a baseline since it conditions the policy on a preference vector (speed/comfort) at runtime to modulate behavior. By deriving a per-user preference vector from driving metrics of test drivers and conditioning a multi-objective RL (MORL) policy, we compare it with DMW for long-term preference alignment. Evaluation Metrics. We employ metrics that assess both driving performance and stylistic alignment. The Driv- ing Score (DS), Success Rate (SR), Efficiency (Effic.), and Comfort are adopted from Bench2Drive [19]. To quan- tify the personalization of the model, we measure the mean value of driving metrics, including average Speed in m/s, Acceleration (Acce.) in m/s 2 , Lane Change Counts (LC), Headway in m, and Travel Time (T) in s. We introduce the Alignment Score (AS) for user studies to evaluate how well the policy aligns with individual driving preferences. User Study for Long-Term Preference Alignment. To test whether the driving behaviors of DMW align with each driver’s preference. We introduce an Alignment Score (AS) by first clustering all drivers using their historical logs, then generating roll-outs condition on each driver’s profile, and finally calculating the accuracy across test routes where roll-out is correctly classified into their target cluster. Ad- ditionally, we recruit ten evaluators and ask them to rate the similarity of the driver’s own logs and corresponding model roll-outs on a 1-10 scale. To test the zero-shot generalizabil- ity of DMW, we perform preference alignment on 25 drivers from PDD and evaluate the behavior alignment on both 25 in-distribution (ID) and 5 out-of-distribution (OOD) drivers. For each scenario type, we choose three test routes. 6.2. Main Results Adaptation to Style Instruction. Table 1 presents the closed-loop evaluation results on the Bench2Drive. The results show that fine-tuning alone already improves both SR and DS, likely due to the newly introduced safety re- ward. Furthermore, when conditioned on conservative in- structions, DMW achieves the greatest gains in DS and SR while maintaining comparable efficiency, reflecting im- proved reliability in cautious driving. Under the aggressive instructions, our policy yields a substantial efficiency gain of 18.77% compared to the 3.70% in SimLingo, and 6.00% 6 Table 2. Driving metrics with and without style instructions, for each driver profile with a different long-term preference. Note that for Alignment Score (AS) and Ratings, we compute their value regardless of the style since it only measures long-term alignment. DriverScenarioStyleDSSpeedEfficiencyAccelerationHeadwayASRatings D1 Emergency Brake None ↓ Aggressive 95.10→ 84.028.06→ 9.58150.94→ 167.126.31→ 6.3918.94→ 17.111.008.8 D298.64→ 96.714.61→ 4.9796.02→ 99.214.94→ 5.3523.10→ 21.180.678.3 D3 97.88→ 95.935.39→ 6.34109.83→ 121.065.63→ 6.1621.61→ 19.221.008.0 D494.02→ 93.157.76→ 8.35145.22→ 159.566.03→ 6.9221.03→ 18.190.677.8 D1 Merging None ↓ Conservative 90.05→ 94.389.32→ 8.76270.84→ 260.105.15→ 4.6896.44→ 101.160.678.4 D286.94→ 91.726.37→ 5.76196.82→ 175.314.29→ 4.16118.92→ 121.031.008.1 D396.85→ 95.946.98→ 6.20205.41→ 195.774.46→ 4.24111.26→ 114.050.677.6 D4 97.31→ 97.888.78→ 8.63261.52→ 259.525.01→ 4.9188.76→ 100.931.008.2 D1 Overtaking None ↓ Neutral 97.56→ 97.947.70→ 7.94220.19→ 223.137.28→ 7.3714.69→ 14.531.009.0 D2 98.23→ 98.415.50→ 5.56167.40→ 167.805.65→ 5.7822.23→ 22.041.008.6 D397.14→ 96.626.34→ 6.13191.02→ 183.846.18→ 6.0120.79→ 22.190.677.5 D480.61→ 82.199.63→ 9.51271.98→ 265.847.11→ 6.8217.81→ 18.031.008.1 D1 Traffic Sign None ↓ Conservative 90.82→ 96.679.08→ 8.49408.74→ 401.287.12→ 6.3731.44→ 32.521.008.7 D297.92→ 98.355.79→ 4.91193.26→ 183.115.72→ 5.5135.02→ 36.911.008.2 D3 89.31→ 92.026.88→ 5.37237.05→ 231.465.96→ 5.9134.54→ 36.511.008.1 D495.91→ 97.328.91→ 6.74418.02→ 379.686.54→ 5.8131.19→ 31.770.677.9 in StyleDrive, with only a 3.89% reduction in DS relative to the conservative condition. DMW significantly outper- forms SimLingo in style adaptation, which highlights that DMW accurately captures the intended behaviors expressed by the instructions while preserving overall performance. Meanwhile, StyleDrive induces smaller metric shifts and achieves a lower DS than DMW, likely due to its fixed style condition, which further highlights the benefit of our style- sensitive adaptation. The consistent reduction in accelera- tion and lane-change counts under conservative ones further validates that DMW maintains distinct stylistic behaviors. To further illustrate the behavioral diversity induced by different instructions, Fig. 6 visualizes how our pol- icy reacts under aggressive and conservative instructions in safety-critical scenarios. When facing a road blockage, the aggressive instruction prefers an immediate overtake (red speed curve). Conversely, the conservative command causes the agent to decelerate and either stop or maintain a low following speed until the opposite lane is clear (blue curve). This behavioral divergence arises from the style- aware weighting in our fine-tuning reward, which dynami- cally adjusts the balance between safety and efficiency ac- cording to the given instruction. In hard-braking scenar- ios, the agent tends to maintain a tighter following distance when objects appear ahead or merge into the lane under ag- gressive instructions, while the agent slows down and yields more noticeably when conditioned on conservative ones. This distinction is largely governed by the style-aware TTC threshold, which is adapted to be more permissive for ag- gressive ones and more restrictive for conservative ones, directly tuning the model’s safety margins. The complete driving clips are provided in the supplementary materials. Long-term Preference Alignment. We report metrics and user study results for two seen drivers (D1, D2) and two unseen drivers (D3, D4).Table 2 shows that the pol- icy conditioned on each profile exhibits consistent motion statistics across different scenarios. For instance, Drivers Table 3. Long-term preference alignment. D: Driver ID. Alignment ScoreAverage Ratings Methods IDOODIDOOD D1D2D3D4D1 D2D3 D4 MORL-PD [14]0.42 0.580.25 0.335.1 6.23.9 3.5 DMW0.92 0.920.83 0.838.7 8.37.8 8.0 1 and 4 exhibit higher efficiency and acceleration, reflect- ing a more aggressive driving preference, whereas Drivers 2 and 3 show lower acceleration and larger headway across all categories, indicating a preference that prioritizes safety and comfort. Furthermore, Driver 2 shows the most cau- tious style with the lowest acceleration, while Driver 3, though still conservative, exhibits slightly higher speeds and greater acceleration, suggesting a willingness to acceler- ate more readily in scenarios such as Overtaking and Traf- fic Sign. While each driver’s driving metrics vary across scenarios, the driving behavior remains distinct among the drivers, suggesting that the policy aligns with individual drivers’ long-term driving preferences across different sce- narios. Furthermore, Table 3 shows higher AS and ratings for DMW, indicating that more roll-outs are classified into the target cluster where their driver belongs, and evalua- tors can consistently recognize driving behaviors that reflect the corresponding driver’s driving habits. While MORL- PD also induces style shifts via its preference-weight condi- tioning, it achieves a lower alignment, especially on unseen drivers, which suggests that our user embedding better cap- tures semantic context from the profile, leading to stronger generalization. In addition, consistent with Table 1, language style in- structions can provide an additional adaptation, shifting the driving towards the short-term preference. 6.3. Ablation Study Adaptive Average Pooling. We ablate the masked Adap- tive Average Pooling (AAP) module used in the preference 7 Something on the roadside block the way... Something leading to a hard-brake... Action under aggressive instructions: Speed up and overtake immediately. Action under conservative instruction: Wait or follow until safe to overtake. Action under aggressive instructions: Maintain a shorter safe distance. Action under conservative instruction: Reaching a lower top speed. “Quickly swerve into the adjacent lane to pass the hazard without braking.” “Please wait for a safe chance; I don‘t like rushing past hazards.” “Don't lose speed - assert our position but avoid collisions.” “Let merging car/crossing person go; give them space, keep the ride calm.” Conservative Aggressive Conservative Aggressive Conservative Aggressive Conservative Aggressive Figure 6. Driving preference under aggressive and conservative instructions. Red waypoints denote distance parametrized (every 1 m) navigation path and green waypoints denote time parametrized (every 0.25 s) trajectory. Table 4. Averaged driving metrics using long-term preference encoder with or without Adaptive Average Pooling (AAP). DriverEncoderDS Speed Effic. Acce. HeadwayAS Ratings D1 w/o AAP 82.74 4.52 161.38 5.8450.110.676.1 D288.02 4.21 163.07 5.8847.560.585.9 D3 87.65 4.49 133.12 5.8141.630.254.6 D480.13 4.46 172.92 5.7938.940.505.3 D1 w/ AAP 93.38 8.54 262.68 6.4640.380.928.7 D295.43 5.57 163.38 5.1549.820.928.3 D395.30 6.40 185.83 5.5647.050.837.8 D491.96 8.77 274.19 6.1739.700.838.0 encoder, which compresses textual profile features into a fixed-length sequence aligned with the temporal features of the route processor. Replacing this module with an unmasked global mean reduces the expressiveness of user embeddings. As shown in Table 4, driving policy con- ditioned on the user embeddings from the preference en- coder with AAP yields a higher diversity of motion statis- tics across different user profiles. We also observe a higher AS, demonstrating that the model can more consistently re- produce each driver’s behavioral traits. We attribute the im- provement to its ability to preserve semantically important embeddings while mitigating the bias from irrelevant seg- ments. We also observe that in denser interaction scenarios, the policy may shift toward conservative behavior for safety, which can reduce style expressiveness for neutral drivers. Style-aware Reward Adaptation. We examine the effect of fixed versus style-aware adaptive reward weights on driv- ing performance and behavioral diversity. The results are in Table 1. When the reward components are fixed (i.e., w s = 0.35, w e = 0.35, w c = 0.30), only the style- dependent TTC and comfort thresholds remain to induce stylistic variation. DMW-Vanilla achieves higher overall DS and SR, indicating effective optimization of generic ob- jectives. However, the inability of dynamic adaptation to efficiency and comfort weights causes aggressive and con- servative instructions to exhibit similar driving, revealing a reduced sensitivity to style. In contrast, DMW adap- tively adjusts reward weights and thresholds based on both context and instruction style, maintaining driving perfor- mance while producing clear behavioral distinctions. Ag- gressive instructions lead to higher speeds and shorter head- ways, whereas conservative ones favor smoother control and larger safety margins. This demonstrates the necessity of adaptive weighting to achieve both reliable and personal- ized driving behaviors. 7. Conclusion We introduced DMW, a personalized VLA driving frame- work that aligns long-term driver preferences and adapts to short-term style instructions. Through learning user em- beddings from the curated Personalized Driving Dataset and conditioning on the end-to-end driving policy, our method captures and aligns with individual drivers’ behav- iors. Extensive experiments on Bench2Drive demonstrate that DMW achieves more distinct adaptation than existing baselines. User studies further highlight the potential of per- sonalized VLA systems to bridge the gap toward human- centered autonomous driving, enabling vehicles that adapt their behavior to individual user preferences. Our approach is currently validated within the CARLA simulator. Future work will focus on bridging the sim-to-real gap by deploy- ing and evaluating the DMW framework on real vehicles. We also plan to expand our dataset to encompass a wider diversity of driving behaviors to enable more robust and generalizable preference alignment. 8 Acknowledgment We gratefully acknowledge Janice Nguyen, Shubham Der- hgawen, Jonathan Setiabudi, Alexander Totah, Pranav Gowrishankar, and Pratheek Sunilkumar for their support in data collection. References [1] Sicong Jiang, Zilin Huang, Kangan Qian, Ziang Luo, Tianze Zhu, Yang Zhong, Yihong Tang, Menglin Kong, Yunlong Wang, Siwen Jiao, Hao Ye, Zihao Sheng, Xin Zhao, Tuopu Wen, Zheng Fu, Sikai Chen, Kun Jiang, Diange Yang, Seongjin Choi, and Lijun Sun. A survey on vision-language- action models for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 4524–4536, October 2025. 1 [2] Litian Gong, Fatemeh Bahrani, Yutai Zhou, Amin Banay- eeanzade, Jiachen Li, and Erdem Bıyık. Autofocus-il: Vlm- based saliency maps for data-efficient visual imitation learn- ing without extra human annotations. In International Con- ference on Robotics and Automation (ICRA), 2026. [3] Yuping Wang, Shuo Xing, Cui Can, Renjie Li, Hongyuan Hua, Kexin Tian, Zhaobin Mo, Xiangbo Gao, Keshu Wu, Sulong Zhou, et al.Generative ai for autonomous driving:Frontiers and opportunities.arXiv preprint arXiv:2505.08854, 2025. [4] Bernard Lange, Masha Itkina, Jiachen Li, and Mykel Kochenderfer.Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving. In Proceedings of Robotics: Science and Systems, 2025. 1 [5] Rui Zhao, Yuze Fan, Ziguo Chen, Fei Gao, and Zhenhai Gao. Diffe2e: Rethinking end-to-end driving with a hy- brid diffusion-regression-classification policy. In The Thirty- Ninth Annual Conference on Neural Information Processing Systems, 2025. 1 [6] Jinning Li, Jiachen Li, Sangjae Bae, and David Isele. Adap- tive prediction ensemble: Improving out-of-distribution gen- eralization of motion forecasting. IEEE Robotics and Au- tomation Letters, 10(2):1553–1560, 2024. [7] Maneekwan Toyungyernsub, Esen Yel, Jiachen Li, and Mykel J Kochenderfer. Predicting future spatiotemporal oc- cupancy grids with semantics for autonomous driving. In 2024 IEEE Intelligent Vehicles Symposium (IV), pages 2855– 2861. IEEE, 2024. 1 [8] Genghua Kou, Fan Jia, Weixin Mao, Yingfei Liu, Yucheng Zhao, Ziheng Zhang, Osamu Yoshie, Tiancai Wang, Ying Li, and Xiangyu Zhang. Padriver: Towards personalized au- tonomous driving. In 2025 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2025. 1, 2 [9] Manabu Nakanoya, Junha Im, Hang Qiu, Sachin Katti, Marco Pavone, and Sandeep Chinchali. Personalized fed- erated learning of driver prediction models for autonomous driving, 2021. [10] Jiachen Li, David Isele, Kanghoon Lee, Jinkyoo Park, Kikuo Fujimura, and Mykel J Kochenderfer. Interactive au- tonomous navigation with internal state inference and in- teractivity estimation. IEEE Transactions on Robotics, 40: 2932–2949, 2024. 1 [11] Ruiyang Hao, Bowen Jing, Haibao Yu, and Zaiqing Nie. Styledrive: Towards driving-style aware benchmarking of end-to-end autonomous driving.In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4627– 4635, 2026. 1, 2, 6 [12] Udita Ghosh, Dripta S. Raychaudhuri, Jiachen Li, Konstanti- nos Karydis, and Amit K. Roy-Chowdhury. Reducing oracle feedback with vision-language embeddings for preference- based rl. In IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2026. 1 [13] Anqing Jiang, Yu Gao, Yiru Wang, Zhigang Sun, Shuo Wang, Yuwen Heng, Hao Sun, Shichen Tang, Lijuan Zhu, Jinhao Chai, et al. Irl-vla: Training an vision-language- action policy via reward world model.arXiv preprint arXiv:2508.06571, 2025. 1 [14] Hendrik Surmann, Jorge De Heuvel, and Maren Bennewitz. Multi-objective reinforcement learning for adaptable person- alized autonomous driving. In 2025 European Conference on Mobile Robots (ECMR), pages 1–8. IEEE, 2025. 1, 6, 7 [15] Yunsheng Ma, Can Cui, Xu Cao, Wenqian Ye, Peiran Liu, Juanwu Lu, Amr Abdelraouf, Rohit Gupta, Kyungtae Han, Aniket Bera, James M. Rehg, and Ziran Wang. Lampilot: An open benchmark dataset for autonomous driving with lan- guage model programs. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 15141– 15151, 2024. 2 [16] Can Cui, Zichong Yang, Yupeng Zhou, Yunsheng Ma, Juanwu Lu, Lingxi Li, Yaobin Chen, Jitesh Panchal, and Zi- ran Wang. Personalized autonomous driving with large lan- guage models: Field experiments. In 2024 IEEE 27th Inter- national Conference on Intelligent Transportation Systems (ITSC), pages 20–27. IEEE, 2024. 2, 5 [17] Can Cui, Zichong Yang, Yupeng Zhou, Juntong Peng, Sung- Yeon Park, Cong Zhang, Yunsheng Ma, Xu Cao, Wenqian Ye, Yiheng Feng, et al. On-board vision-language mod- els for personalized autonomous vehicle motion control: System design and real-world validation.arXiv preprint arXiv:2411.11913, 2024. 2 [18] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, pages 1–16, 2017. 2 [19] Xiaosong Jia, Zhenjie Yang, Qifeng Li, Zhiyuan Zhang, and Junchi Yan.Bench2drive: Towards multi-ability bench- marking of closed-loop end-to-end autonomous driving. Ad- vances in Neural Information Processing Systems, 37:819– 844, 2024. 2, 6 [20] Xiaosong Jia,Yulu Gao,Li Chen,Junchi Yan, Patrick Langechuan Liu, and Hongyang Li. Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving. In ICCV, 2023. 2 [21] Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driv- ing: Towards scalable decoders for end-to-end autonomous driving. In CVPR, 2023. [22] Zewei Zhou, Tianhui Cai, Seth Z. Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma.Autovla: A vision- language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. In 9 Advances in Neural Information Processing Systems, 2025. 2 [23] Carla autonomous driving leaderboard, 2025. 2 [24] Yuping Wang, Xiangyu Huang, Xiaokang Sun, Mingxuan Yan, Shuo Xing, Zhengzhong Tu, and Jiachen Li. Uniocc: A unified benchmark for occupancy forecasting and prediction in autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 25560– 25570, 2025. 2 [25] Haohan Chi, Huan-ang Gao, Ziming Liu, Jianing Liu, Chenyu Liu, Jinwei Li, Kaisen Yang, Yangcheng Yu, Zeda Wang, Wenyi Li, Leichen Wang, Xingtao Hu, Hao Sun, Hang Zhao, and Hao Zhao. Impromptu vla: Open weights and open data for driving vision-language-action models. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2025. 2 [26] Haohong Lin, Yunzhi Zhang, Wenhao Ding, Jiajun Wu, and Ding Zhao. Model-based policy adaptation for closed-loop end-to-end autonomous driving. In Workshop on Foundation Models Meet Embodied Agents at CVPR 2025, 2025. 2 [27] Zhiwen Fan, Jian Zhang, Renjie Li, Junge Zhang, Runjin Chen, Hezhen Hu, Kevin Wang, Huaizhi Qu, Dilin Wang, Zhicheng Yan, et al. Vlm-3r: Vision-language models aug- mented with instruction-aligned 3d reconstruction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 2 [28] Yuan Gao, Mattia Piccinini, Yuchen Zhang, Dingrui Wang, Korbinian Moller, Roberto Brusnicki, Baha Zarrouki, Alessio Gambi, Jan Frederik Totz, Kai Storms, Steven Pe- ters, Andrea Stocco, Bassam Alrifaee, Marco Pavone, and Johannes Betz. Foundation models in autonomous driving: A survey on scenario generation and scenario analysis. IEEE Open Journal of Intelligent Transportation Systems, 2026. [29] Piyush Gupta, Sangjae Bae, Jiachen Li, and David Isele.Scale-plan: Scalable language-enabled task plan- ning for heterogeneous multi-robot teams. arXiv preprint arXiv:2603.08814, 2026. [30] Yunsong Zhou, Linyan Huang, Qingwen Bu, Jia Zeng, Tianyu Li, Hang Qiu, Hongzi Zhu, Minyi Guo, Yu Qiao, and Hongyang Li. Embodied understanding of driving sce- narios. In European Conference on Computer Vision, pages 129–148. Springer, 2024. [31] Xiaopan Zhang, Zejin Wang, Zhixu Li, Jianpeng Yao, and Jiachen Li. Commcp: Efficient multi-agent coordination via llm-based communication with conformal prediction. In IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2026. [32] Mingxuan Yan, Yuping Wang, Zechun Liu, and Jiachen Li. Rdd: Retrieval-based demonstration decomposer for planner alignment in long-horizon tasks. In Proceedings of the 39th Annual Conference on Neural Information Processing Sys- tems (NeurIPS), 2025. [33] Xiaopan Zhang, Hao Qin, Fuquan Wang, Yue Dong, and Ji- achen Li. Lamma-p: Generalizable multi-agent long-horizon task allocation and planning with lm-driven pddl planner. In 2025 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 10221–10221. IEEE, 2025. [34] Trishna Chakraborty,Udita Ghosh,Xiaopan Zhang, Fahim Faisal Niloy, Yue Dong, Jiachen Li, Amit Roy- Chowdhury, and Chengyu Song. Heal: An empirical study on hallucinations in embodied agents driven by large lan- guage models.In Findings of the Association for Com- putational Linguistics: EMNLP 2025, pages 21226–21243, 2025. [35] Sayak Nag, Udita Ghosh, Calvin-Khang Ta, Sarosij Bose, Jiachen Li, and Amit K Roy-Chowdhury. Conformal pre- diction and mllm aided uncertainty quantification in scene graph generation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 11676–11686, 2025. [36] Zhikai Zhao, Chuanbo Hua, Federico Berto, Kanghoon Lee, Zihan Ma, Jiachen Li, and Jinkyoo Park. Trajevo: Trajectory prediction heuristics design via llm-driven evolution. In Pro- ceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2026. 2 [37] Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, XianPeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. In Conference on Robot Learning, pages 4698–4726. PMLR, 2025. 2 [38] Jiageng Mao, Yuxi Qian, Junjie Ye, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt. arXiv preprint arXiv:2310.01415, 2023. 2 [39] Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xing- gang Wang. Alphadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reason- ing. arXiv preprint arXiv:2503.07608, 2025. 2 [40] Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. Orion: A holistic end-to- end autonomous driving framework by vision-language in- structed action generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 24823–24834, October 2025. 2 [41] Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, and Xing Wei. Futuresight- drive: Thinking visually with spatio-temporal cot for au- tonomous driving. In Advances in Neural Information Pro- cessing Systems, 2025. 2 [42] Katrin Renz, Long Chen, Elahe Arani, and Oleg Sinavski. Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11993– 12003, 2025. 2, 3, 6, 1 [43] Mariah L Schrum, Emily Sumner, Matthew C Gombolay, and Andrew Best.Maveric: A data-driven approach to personalized autonomous driving. IEEE Transactions on Robotics, 40:1952–1965, 2024. 2 [44] Xu Han, Xianda Chen, Zhenghan Cai, Pinlong Cai, Meixin Zhu, and Xiaowen Chu. From words to wheels: Automated style-customized policy generation for autonomous driving. arXiv preprint arXiv:2409.11694, 2024. 2 [45] Ziye Qin, Siyan Li, Chuheng Wei, Guoyuan Wu, Matthew J Barth, Amr Abdelraouf, Rohit Gupta, and Kyungtae Han. Investigating personalized driving behaviors in dilemma zones: Analysis and prediction of stop-or-go decisions. IEEE Robotics and Automation Letters, 2025. 2 [46] Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with 10 graph visual question answering. In European Conference on Computer Vision, 2024. 3 [47] Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101, 2024. 3 [48] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024. 3 [49] An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. eprint arXiv: 2407.10671, 2024. 3, 6 [50] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 3 [51] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 4 [52] Pengcheng He, Jianfeng Gao, and Weizhu Chen.De- BERTav3: Improving deBERTa using ELECTRA-style pre- training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Repre- sentations, 2023. 4 [53] Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations (ICLR), 2019. 6 [54] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 6 11 Drive My Way: Preference Alignment of Vision-Language-Action Model for Personalized Driving Supplementary Material A. Additional Quantitative Results To further validate whether DMW produces behaviors con- sistent with human expectations, we report the detailed user study results on both long-term preference alignment and short-term adaptation to style instructions. A.1. Long-term Preference Alignment Building on the results across scenario types in Table 2, the averaged metrics in Table 5 further show clear differences in driver-specific behavior. Drivers with higher average speeds exhibit consistent behavior across scenarios, whereas more cautious drivers tend to maintain larger headways and lower accelerations. These trends are consistent with the cross- scenario patterns observed in Table 2, suggesting that the policy aligns with persistent behavioral traits beyond indi- vidual routes. A.2. Adaptation on Style Instruction Beyond long-term preferences, users may also express short-term intentions via style instructions depending on sit- uational context. To complement objective metrics in Table 1, and further evaluate whether the policy adapts to these in- structions, we conduct a user study in which human evalua- tors rate trajectories generated under different style prompts for the same scenario. Each evaluator is shown short video clips rendered under conservative, neutral, and aggressive instructions and asked to judge whether the resulting behav- ior matches the intended style. Evaluators score each trajectory on a 0-10 scale accord- ing to three criteria: (i) how well the behavior follows the instruction, (i) efficiency, comfort, and smoothness of drive, and (i) perceived safety. As shown in Table 6, across four representative scenario types, both StyleDrive [11] and DMW outperform SimLingo baseline [42], indicating the benefit of style-aware driving adaptation. StyleDrive [11] demonstrates improved alignment with short-term style in- structions compared to SimLingo, but still falls short of DMW. In contrast, DMW consistently achieves the high- est ratings across all styles and scenarios. Evaluators con- sistently observe that under aggressive instructions, DMW produces higher speeds, shorter following distances, and more decisive accelerations, while conservative instructions yield smoother control profiles and larger safety margins. These results confirm that the proposed policy not only re- sponds to explicit style instructions, but does so in a more sensitive manner. Table 5. Driving metrics across all scenario types. DriverDSSpeed Effic. Acce. HeadwayAS Ratings D193.388.54262.68 6.4640.380.928.7 D2 95.435.57163.38 5.1549.820.928.3 D395.306.40185.83 5.5647.050.837.8 D4 91.968.77274.19 6.1739.700.838.0 D592.587.48229.93 5.5839.080.837.9 D6 97.716.16190.74 6.0842.410.757.4 D796.117.16218.25 6.3640.120.837.9 D8 95.285.82188.27 5.5443.950.928.2 D996.165.60178.97 5.8445.070.677.0 D1094.208.38258.28 6.8940.261.008.6 A.3. Additional Qualitative Results Fig. 7 visualizes how DMW responds to aggressive and conservative instructions across safety-critical scenarios. In the lost-of-control scenario, where the ego-vehicle risks los- ing control due to the bad road conditions, the aggressive in- struction prioritizes efficiency: it maintains a higher speed and attempts to pass the unstable area quickly. In contrast, the conservative instruction leads the agent to maintain a smoother trajectory, and preserve vehicle stability. This di- vergence highlights how the policy adapts its balance be- tween efficiency and comfort based on the given instruction. A similar pattern emerges in the oncoming-vehicle intru- sion scenario. When another vehicle invades the ego lane, the aggressive instruction causes the agent to accelerate and execute a decisive rightward maneuver at relatively high speed. Meanwhile, the conservative instruction prompts early caution: the agent reduces speed, yields space sooner, and performs a safer avoidance. In another scenario involving a parked vehicle blocking the lane, the agent must decide when to safely overtake. Un- der an aggressive instruction, corresponding to personal re- quirements such as “I’m in a hurry” or “I’m running late”. The policy seeks the earliest viable gap and initiates the overtake quickly to reduce waiting time. In contrast, under a conservative instruction, the agent remains patient, yield- ing until the oncoming lane is fully clear, especially under low-visibility conditions. In the scenario where the agent needs to make a left turn at an unsignalized junction, under the aggressive instruc- tion, the agent initiates the turn earlier once it identifies a tighter and feasible opening, minimizing delay. In con- trast, the conservative instruction causes the agent to wait patiently for a safer gap before turning. This cautious be- havior reflects the emphasis on safety and low-risk in com- plex intersection negotiations. Together, these examples il- 1 Table 6. User study ratings (0-10) evaluating how well trajectories match intended instructions. Five evaluators (E1-E5) rate trajecto- ries from SimLingo [42], StyleDrive [11], and DMW. ScenarioModelStyleE1 E2 E3 E4 E5 Emergency Brake SimLingo [42] Conservative7.4 6.7 6.4 7.3 6.5 Neutral7.0 7.3 7.5 7.0 7.1 Aggressive6.5 7.6 7.2 6.6 7.4 StyleDrive [11] Conservative8.2 7.5 7.3 8.0 7.2 Neutral7.8 8.0 8.2 7.6 7.9 Aggressive7.2 8.4 7.9 7.1 8.2 DMW Conservative9.0 8.1 8.0 8.8 7.9 Neutral8.4 8.6 9.1 8.1 8.5 Aggressive7.8 9.2 8.3 7.6 9.0 Merging SimLingo [42] Conservative7.2 6.4 6.3 7.2 6.3 Neutral6.8 7.0 7.3 6.7 6.8 Aggressive6.2 7.5 6.9 6.1 7.3 StyleDrive [11] Conservative8.0 7.3 7.2 7.9 7.1 Neutral7.6 7.8 8.1 7.4 7.7 Aggressive7.0 8.3 7.6 6.9 8.1 DMW Conservative8.9 7.9 7.7 8.7 7.6 Neutral8.2 8.4 9.0 8.0 8.3 Aggressive7.5 9.1 8.1 7.4 8.9 Overtaking SimLingo [42] Conservative7.3 6.3 6.2 7.2 6.2 Neutral6.9 7.1 7.4 6.8 6.9 Aggressive6.1 7.5 7.0 6.0 7.3 StyleDrive [11] Conservative8.1 7.2 7.1 8.0 7.0 Neutral7.7 7.9 8.2 7.5 7.8 Aggressive6.9 8.4 7.7 6.8 8.2 DMW Conservative9.1 7.7 7.5 8.8 7.6 Neutral8.3 8.5 9.0 8.0 8.4 Aggressive7.4 9.3 8.2 7.2 9.2 Traffic Sign SimLingo [42] Conservative7.4 7.0 6.9 7.3 6.8 Neutral7.1 7.3 7.5 7.0 7.1 Aggressive6.8 7.6 7.2 6.7 7.4 StyleDrive [11] Conservative8.3 7.8 7.6 8.1 7.7 Neutral7.9 8.1 8.3 7.7 8.0 Aggressive7.4 8.6 8.0 7.2 8.4 DMW Conservative8.9 8.3 8.1 8.6 8.2 Neutral8.5 8.7 8.8 8.2 8.6 Aggressive8.0 9.0 8.4 7.8 8.9 lustrate DMW’s ability to adapt in real-time according to short-term style instruction. B. Personalized Driving Dataset We provide a more detailed description of the collected per- sonalized driving datasets from thirty real drivers. B.1. Scenarios In Town 12, we collect twenty routes that cover a di- verse set of representative driving scenarios under vary- ing weather and illumination conditions. The scenarios are summarized below (descriptions are taken from https: //leaderboard.carla.org/scenarios/): • Accident / ParkedObstacle / ConstructionObstacle: Bad road conditions / Opposite vehicle invades Action under aggressive instructions: Maintains higher speeds throughout. Action under conservative instruction: Reacts with caution. “Let’s move fast and maintain speed.” “Let’s keep it easy and smooth.” Parked obstacle / Turn at non-signalized junction Action under aggressive instructions: Overtake at the earliest possible opening. Action under conservative instruction: Wait until a larger safe margin. “I’m okay with taking tighter gaps today. I’m running late.” “Better to let others go first, patience will keep us safest here.” Figure 7.Driving preference under aggressive and conserva- tive instructions. Red waypoints denote distance parametrized (every 1 m) navigation path and green waypoints denote time parametrized (every 0.25 s) trajectory. An obstacle (e.g., a construction zone, an accident, or a parked vehicle) is blocking the ego lane. The ego vehicle must change lanes into traffic moving in the same direc- tion to bypass the obstacle. • SignalizedJunctionLeftTurn / NonSignalizedJunction- LeftTurn: The ego vehicle performs an unprotected left turn at an intersection (can occur at both signalized and unsignalized intersections). • CrossingBicycleFlow: The ego vehicle must execute a turn at an intersection while yielding to bicycles crossing perpendicular to its path. • StaticCutIn: Another vehicle cuts into the ego lane from a queue of stationary traffic. It must decelerate, brake, or change lanes to avoid a collision. 2 • NonSignalizedJunctionRightTurn / SignalizedJunc- tionRightTurn / VanillaNonSignalizedTurn: The ego vehicle makes a right turn at an intersection while yield- ing to crossing traffic. • InterurbanActorFlow: The ego vehicle leaves the in- terurban road by turning left, crossing a fast traffic flow. • BlockedIntersection: While performing a maneuver, the ego vehicle encounters a stopped vehicle on the road and must perform an emergency brake or an avoidance ma- neuver. • HazardAtSideLane: A slow-moving hazard (e.g., bicy- cle) partially obstructs the ego vehicle’s lane. The ego vehicle must either brake or carefully bypass the hazard (bypassing on the lane with traffic in the same direction). • ParkingCutIn: A parked vehicle exits a parallel parking space into the ego vehicle’s path. The ego vehicle must slow down to allow the parked vehicle to merge into traf- fic. • VehicleOpensDoorTwoWays: The ego vehicle needs to avoid a parked vehicle with its door opening into the lane. • DynamicObjectCrossing:A pedestrian suddenly emerges from behind a parked vehicle and enters the lane. The ego vehicle must brake or take evasive action to avoid hitting the pedestrian. • EnterActorFlow: A flow of cars runs a red light in front of the ego when it enters the junction, forcing it to react (interrupting the flow or merging into the flow). These ve- hicles are ’special’ ones, such as police cars, ambulances, or firetrucks. • HighwayExit: The ego vehicle must cross a lane of mov- ing traffic to exit the highway at an off-ramp. • ControlLoss: The ego vehicle loses control due to bad conditions on the road and it must recover, coming back to its original lane. • MergerIntoSlowTraffic: The ego-vehicle merge into a slow traffic on the off-ramp when exiting the highway. B.2. Auxiliary Information To enable reliable and interpretable driving preference anal- ysis, we extract environmental information and motion statistics from driving logs. Concretely, we record data at 5 Hz, including: • Camera Sensor. A forward-facing RGB camera with a resolution of 1024×512 and a wide 110 ◦ field of view serves as the primary visual sensor. • Ego-state and Control Signals.We store full ego- vehicle kinematics, including linear acceleration, angular velocity, speed, and the world-frame pose (location, rotation). Human control commands, including throt- tle, brake, steering, gear, hand-brake status, and reverse flag, together with the speed limit and lane-level attributes such as lane ID, lane type, lane width, and whether the ego vehicle is currently inside a junction. • Surrounding Agents. We store a detailed description of the leading vehicle (front vehicleinfo: ID, type, 3D position, velocity, speed, and color), along with all nearby dynamic agents in other vehicles and walkers. For each surrounding actor, we log its trans- form, velocity, bounding-box extent, and other metadata. Nearby traffic lights and stop signs are also recorded, cap- turing both their spatial relation to the ego and whether they are currently influencing the ego vehicle. • Expert Supervision.Each timestep is paired with privileged control from the PDM-Lite expert, includ- ing throttle, brake, steer, and the corresponding target speed. • Route Geometry. We additionally provide route infor- mation in the form of two upcoming waypoints along the global route, transformed into the ego frame as target point and targetpointnext. All actor-level annotations are saved as JSON files syn- chronized with the image index. 3