Paper deep dive
VLALight: Lightweight Vision-Language-Action Models for Emergency-Aware Traffic Signal Control
Kemou Jiang, Maonan Wang, Xingchen Zou, Jiayue Zhu, Yuhang Fu, Sicheng Wang, Xi Chen, Yirong Chen, Zhiyong Cui
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/28/2026, 3:25:36 AM
Summary
The paper introduces VLALight, a lightweight end-to-end Vision-Language-Action (VLA) framework for emergency-aware traffic signal control (TSC). It addresses limitations of cascaded VLM approaches by directly mapping multi-view intersection observations and textual instructions to discrete signal phases using a compact 0.5B parameter model. The system utilizes a simulation-grounded data generation pipeline to create high-fidelity, labeled training data that replicates real-world topologies and includes emergency vehicles. VLALight achieves superior emergency vehicle service and real-time performance on local hardware.
Entities (10)
Relation Signals (8)
VLALight → solves → Traffic Signal Control
confidence 95% · we propose VLALight, a lightweight end-to-end vision-language-action framework that directly maps intersection observations and signal-phase information to discrete signal actions.
VLALight → uses → SigLIP
confidence 95% · directional camera images and the text instruction are encoded by the frozen DINOv2 and SigLIP visual encoders
VLALight → uses → DINOv2
confidence 95% · directional camera images and the text instruction are encoded by the frozen DINOv2 and SigLIP visual encoders
VLALight → outperforms → VLMLight
confidence 90% · reducing pooled emergency waiting time by 21.1% over the cascaded VLMLight
VLALight → uses → LoRA
confidence 90% · adapted with LoRA through the VLA-Adapter paradigm
VLALight → usesdatafrom → TransimHub
confidence 90% · we replicate six real-world intersections in the TransimHub simulation platform
VLALight → usesdatafrom → OpenStreetMap
confidence 85% · we first obtain the raw road-network data from OpenStreetMap
VLALight → usesrenderingengine → Blender
confidence 85% · We therefore render every scene with Blender’s high-fidelity rendering engine
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Traffic signal control (TSC) is essential for mitigating urban congestion. Recent advances in vision-language models (VLMs) enable richer interpretation of intersection scenes, opening new opportunities for visual-context-aware TSC. However, the loose coupling and repeated information conversion between modules can lead to the loss of fine-grained visual details, while sequential inference introduces substantial latency. To address these limitations, we propose VLALight, a lightweight end-to-end vision-language-action framework that directly maps intersection observations and signal-phase information to discrete signal actions. To handle the multi-view nature of TSC, VLALight combines multiple directional camera views into a unified visual input and uses textual instructions to establish their correspondence with traffic movements and signal phases. This design enables direct action prediction with a compact 0.5 B-parameter model, without intermediate image-to-text descriptions or handcrafted traffic-state representations. Experiments show that VLALight delivers the best emergency-vehicle service of all compared methods, reducing pooled emergency waiting time by 21.1% over the cascaded VLMLight while running in real time on local hardware and generalizing to unseen intersection topologies and traffic-flow patterns.
Tags
Links
- Source: https://arxiv.org/abs/2609.30709v1
- Canonical: https://arxiv.org/abs/2609.30709v1
Trouble viewing inline? Open PDF directly →
Full Text
49,450 characters extracted from source content.
Expand or collapse full text
VLALight: Lightweight Vision-Language-Action Models for Emergency-Aware Traffic Signal Control Kemou Jiang Maonan Wang Xingchen Zou Jiayue Zhu Yuhang Fu Sicheng Wang Xi Chen Yirong Chen Zhiyong Cui* Abstract Traffic signal control (TSC) is essential for mitigating urban congestion. Recent advances in vision-language models (VLMs) enable richer interpretation of intersection scenes, opening new opportunities for visual-context-aware TSC. However, the loose coupling and repeated information conversion between modules can lead to the loss of fine-grained visual details, while sequential inference introduces substantial latency. To address these limitations, we propose VLALight, a lightweight end-to-end vision-language-action framework that directly maps intersection observations and signal-phase information to discrete signal actions. To handle the multi-view nature of TSC, VLALight combines multiple directional camera views into a unified visual input and uses textual instructions to establish their correspondence with traffic movements and signal phases. This design enables direct action prediction with a compact 0.5 B-parameter model, without intermediate image-to-text descriptions or handcrafted traffic-state representations. Experiments show that VLALight delivers the best emergency-vehicle service of all compared methods, reducing pooled emergency waiting time by 21.1% over the cascaded VLMLight while running in real time on local hardware and generalizing to unseen intersection topologies and traffic-flow patterns. 1School of Transportation Science and Engineering, Beihang University, Beijing, China 2The Chinese University of Hong Kong (Shenzhen), Shenzhen, China 3The Chinese University of Hong Kong, Hong Kong SAR, China 4The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China 5Shanghai Artificial Intelligence Laboratory, Shanghai, China *Corresponding author Introduction Urban traffic congestion remains a major challenge in modern cities. Intersections are the bottlenecks of road networks, and their signal strategies largely determine network-wide efficiency and safety. Traffic signal control (TSC) methods have evolved from fixed-time plans to adaptive rule-based strategies such as SOTL (Gershenson 2005) and Max Pressure (Varaiya 2013) and, more recently, deep reinforcement learning (DRL) (Wei et al. 2021). These controllers react to real-time measurements and perform well under steady demand, but they almost universally represent the traffic state with compact hand-crafted statistics such as queue lengths, lane occupancy, and vehicle counts. Because such statistics discard fine-grained spatial and interaction information, these controllers can misjudge the situation or degrade sharply in abnormal scenarios such as accidents, sudden congestion, or emergency-vehicle priority requests, where scene understanding matters most. Large language models (LLMs) and vision-language models (VLMs) have recently been introduced to inject semantic understanding into TSC (Lai et al. 2025; Wang et al. 2025a; Zou et al. 2026), yet they adopt a cascaded “perception, reasoning, and control” pipeline in which traffic images are first converted into textual descriptions, over which an LLM then reasons to produce control suggestions; the vision-to-text conversion again compresses spatial detail, and multi-stage inference adds latency that is problematic for real-time control. Both lines of work thus share a common bottleneck: the visual scene is compressed before decisions are made. Vision-language-action (VLA) models offer a direct way to remove this bottleneck. By tokenizing actions and treating control as a generative prediction problem, models such as RT-2 (Zitkovich et al. 2023), Octo (Octo Model Team et al. 2024), and OpenVLA (Kim et al. 2024) unify perception, language understanding, and action execution in a single end-to-end network, preserving full visual detail up to the decision layer while inheriting web-scale perceptual and linguistic priors. These properties are exactly what emergency-aware traffic control demands: the controller must see the whole scene, understand topology and intent expressed in language, and act directly. Yet VLA research has so far focused on robot manipulation (Yu et al. 2025; Ye et al. 2025; Zhang et al. 2026); in this work, we bring the VLA paradigm to traffic signal control. In TSC scenarios, however, VLA models face several new challenges. First, the observation structure differs. A robot policy consumes a temporal stream from a single ego-centric view, whereas a traffic controller must process multiple directional views at once, and each phase decision affects only the traffic movements visible in some of the views; the model must therefore relate the views to each other and parse the lane-level structure within each view. Second, intersection topologies vary. Real intersections differ in shape, topology, and lane count, yet a single policy must serve all of them, and ideally unseen ones as well, which places a higher demand on generalization. Third, the control objective is event-driven. Beyond serving routine traffic efficiently, the controller must also spot special vehicles such as ambulances and fire engines in dense ordinary traffic, rare events that demand an immediate shift of phase priority in their favor; such long-tail events are precisely where compact state statistics and efficiency-driven training are weakest. Real-time operation imposes a further practical constraint on model size and adaptation cost. We address these challenges with VLALight, a lightweight end-to-end VLA framework for intersection signal control. Because real surveillance footage paired with control labels is difficult to obtain at scale, we first build a simulation-grounded data-generation pipeline that replicates the lane-level topology of real intersections, designs diverse traffic-flow patterns with injected emergency vehicles, and renders high-fidelity camera images with verified expert labels. On the modeling side, VLALight stitches the directional camera views of an intersection into a single panoramic observation, mirroring how a human operator reads a surveillance wall, and pairs it with a synthesized textual instruction that establishes the correspondence between camera views, traffic movements, and signal phases. Conditioned on these inputs, a compact 0.5B multimodal backbone, adapted with LoRA through the VLA-Adapter paradigm (Wang et al. 2026) and augmented with a small set of learnable phase queries, directly decides the next phase in a single forward pass, with no intermediate image-to-text descriptions or handcrafted traffic-state representations at any stage. Figure 1 illustrates the overall architecture. Our main contributions are: Figure 1: Overview of VLALight. • We build a simulation-grounded visual data-generation pipeline for TSC that replicates real-world intersection topologies, synthesizes diverse traffic flows with injected emergency vehicles, collects expert labels under human verification, and renders high-fidelity multi-view images, yielding a verified multi-intersection dataset for emergency-aware signal control. • We design a unified multi-directional visual representation and a TSC-native phase-classification action space that enable direct action prediction with a compact multimodal model. • We propose VLALight, a lightweight end-to-end vision-language-action framework for traffic signal control that maps intersection observations and signal-phase information directly to discrete signal actions, running in real time on local hardware. Related Work Traffic signal control. Control strategies have progressed from fixed-time plans to adaptive rule-based strategies and, more recently, learning-based policies. Fixed-time controllers derive cycle lengths and green splits from historical traffic statistics (Webster 1958), whereas adaptive methods such as SOTL (Gershenson 2005) and Max Pressure (Varaiya 2013) react to real-time detector measurements. Deep reinforcement learning has further advanced the field, spanning value-based methods (Clifton and Laber 2020; van Hasselt et al. 2016; Huang et al. 2018), policy-gradient algorithms (Schulman et al. 2017), and multi-agent coordination frameworks (Wei et al. 2019b; Chen et al. 2020; Wei et al. 2018; Wei et al. 2019a; Oroojlooy et al. 2020; Wang et al. 2024b). Figure 2: Architecture of VLALight. (a) The simulation-grounded traffic data-generation pipeline: road network topology extraction, traffic flow design and expert label collection, and realistic intersection image rendering. (b) Post-training and inference: the directional camera images and the text instruction are encoded by the frozen DINOv2 and SigLIP visual encoders and the compact VLM backbone, which directly outputs the next signal phase. LLMs and VLMs for traffic control. To bring semantic reasoning into TSC, recent studies have turned to LLMs and VLMs (Nie et al. 2025). Early systems encode structured detector states as text and prompt an LLM for decisions (Lai et al. 2025; Masri et al. 2025; Wang et al. 2024a; Da et al. 2024), while more recent designs add visual perception, coupling visual encoders with language reasoning for signal control (Wang et al. 2025a; Zou et al. 2026). Vision-language-action models. VLA models offer an end-to-end alternative by casting control as generative action prediction. RT-1 (Brohan et al. 2023), RT-2 (Zitkovich et al. 2023), Octo (Octo Model Team et al. 2024), and OpenVLA (Kim et al. 2024) demonstrated this paradigm at scale in robotic manipulation, and subsequent work has improved efficiency (Pertsch et al. 2025; Yu et al. 2025), reasoning (Zhao et al. 2025; Ye et al. 2025; Zhang et al. 2026), action representations (Black et al. 2025; Chi et al. 2025; Zhao et al. 2023), and safety (Zhang et al. 2025). A parallel line of compact architectures, including RoboFlamingo (Li et al. 2024), UniVLA (Bu et al. 2025), and OpenVLA-OFT (Kim et al. 2025), reduces model size or latency, and a recent survey (Yu et al. 2025) provides a comprehensive overview of this fast-growing literature. Method This section first formulates emergency-aware traffic signal control as a vision-language-action decision problem, then describes the simulation-grounded pipeline that generates the training data, and finally presents the VLALight policy. Problem Formulation An intersection connects incoming and outgoing approaches, each carrying one or more lanes; a movement carries vehicles from an incoming approach to an outgoing one, and movements are grouped into phases, each a set of non-conflicting movements that receive the green signal simultaneously. We consider intersection control with a phase set of size K, and time is discretized into decision steps of length Δt t. At each step t, the agent observes a stitched surveillance image It∈ℝH×W×3I_t ^H× W× 3 covering all approaching directions, together with a natural-language instruction L that describes the intersection topology and the meaning of each signal phase, and directly outputs a discrete phase at∈0,…,K−1a_t∈\0,…,K-1\, equivalently a one-hot vector t∈0,1Ka_t∈\0,1\^K. The goal is to learn a policy πθ(at∣It,L) _θ(a_t I_t,L) that minimizes the average travel and waiting times of all vehicles while remaining responsive to emergency-vehicle priority requests. The following sections present the two designs that adapt the VLA paradigm to this task, namely a multi-directional visual representation and a TSC-native action space, together with the overall VLALight architecture illustrated in Figure 2; a detailed comparison with VLA models in other domains is deferred to the supplementary material (Section E). Synthesizing Labeled Intersection Scenes from Real Topology Real intersection footage paired with control labels is difficult to obtain at scale: raw surveillance video raises privacy concerns and is rarely released, and the signal decisions that were executed while the cameras rolled are equally hard to access, so usable supervision labels do not exist in the real world. We therefore generate training data in simulation while anchoring every stage to the real world, through three steps: replicating real road-network topology, designing traffic flows and collecting expert labels, and rendering realistic camera images (details in the supplementary material, Section A). Road Network Topology Extraction To make the simulated scenes faithful to real deployment conditions, we replicate six real-world intersections in the TransimHub simulation platform (Wang et al. 2025b) with their lane-level topology exactly preserved. For each site, we first obtain the raw road-network data from OpenStreetMap, and then refine it against Google satellite imagery and street-view photos, correcting lane counts, lane functionalities, and traffic organization until the simulated network matches reality. The selected intersections cover both four-way cross and T-junction layouts and differ in lane count and phase design, so that the collected data capture diverse signal-control scenarios instead of overfitting to one specific intersection layout. Expert Label Collection For each intersection we design a regular flow that reproduces a typical day with moderate, smoothly varying volumes, and collect the expert demonstrations under it; additional demand patterns used only for evaluation are introduced in the Experiments section. We inject emergency vehicles (police cars, ambulances, and fire engines) with predefined types, entry times, and paths. A control label is then attached to every decision step according to the scenario. Under routine demand, where the objective is to discharge queues and minimize congestion, each step is labeled by a maximum-pressure expert (Varaiya 2013), which selects the phase with the highest pressure, defined as the aggregated difference between incoming and outgoing vehicle counts over the movements it serves. Max pressure is blind to emergency vehicles, so whenever an emergency vehicle is present, we instead query a multimodal LLM with the visual scene to select the phase that serves its movement. All labels are finally verified by traffic signal control experts, yielding the dataset =(I,L,∗)D=\(I,L,a^*)\ of stitched images, instructions, and one-hot expert actions. The demonstrations thus teach the policy both to balance pressure and to prioritize emergency vehicles. Realistic Image Rendering Raw simulator frames suffer from low resolution, blurry textures, and missing detail, and policies trained on them transfer poorly to real scenes. We therefore render every scene with Blender’s high-fidelity rendering engine, producing realistic lighting, materials, and vehicle appearance that narrow the sim-to-real visual gap. Each intersection is instrumented with up to four cameras, one per approach, placed at roadside surveillance viewpoints rather than a bird’s-eye view, matching how intersections are actually monitored. Post-Training and Inference VLALight follows the VLA-Adapter paradigm (Wang et al. 2026): a frozen pretrained VLM is augmented with a small set of lightweight trainable modules, among them a set of NaN_a learnable phase queries =jj=1NaQ=\q_j\_j=1^N_a, and predicts the next phase in a single forward pass. To bridge the domain differences identified above, we first present the two key designs, the multi-directional visual representation and the TSC-native action space, and then walk through the complete policy-learning pipeline. Mutli-view Visual A traffic controller sees the world very differently from a robot. In practice, intersections are rarely observable from a bird’s-eye view; they are monitored by roadside cameras, one per approach, and a signal decision must coordinate traffic movements that are visible across different views. More importantly, choosing a phase is a trade-off among competing directions: granting green time to the movements in one view withholds it from all others, so a sound decision must weigh the demand visible in every view against each other rather than reading any view in isolation. This is why human operators watch the camera feeds arranged side by side on a surveillance wall: only a joint view of all directions supports such cross-view weighing. We give the model the same joint view, composing a single panoramic observation by stitching the directional views into a 2×22× 2 grid: It=Stack2×2(Ct(i)i=03)∈ℝH×W×3,I_t=Stack_2× 2 (\C_t^(i)\_i=0^3 ) ^H× W× 3, (1) where Ct(i)∈ℝHc×Wc×3C_t^(i) ^H_c× W_c× 3 is the i-th camera view and missing views are zero-padded. Encoding the views jointly is essential here: the cross-view trade-off above must be made within one shared representation, and a compact backbone has limited ability to relate separately encoded views. Following multi-camera VLAs that place all camera streams in one sequence before the language model, the stitched image is resized and normalized as a whole and encoded in a single pass by two frozen pretrained encoders, with DINOv2 contributing self-supervised geometric features and SigLIP providing language-aligned semantic features (Oquab et al. 2024; Zhai et al. 2023; Radford et al. 2021). With the ViT/14 backbones at their native input resolution, the stitched image yields 256256 patch tokens arranged in a 16×1616× 16 spatial grid, and the two encoders’ features are concatenated patch-wise. Because the grid inherits the 2×22× 2 layout of the stitched image, each 8×88× 8 token block keeps its directional identity, so the instruction can ground every token block to a direction, its movements, and its phase. All 256256 tokens enter the language model as one sequence within a single forward pass of the 0.5B backbone, where self-attention contextualizes every visual token by all views at once: =[dino∥siglip],=MLPproj(), gatheredv= [\,v^dino\,\|\,v^siglip\, ], =MLP_proj(v), gathered (2) where dinov^dino and siglipv^siglip are the patch features of the stitched image from the two encoders, ∥\| denotes concatenation along the feature dimension, and Z holds the NpN_p visual tokens projected to the LLM hidden size. The instruction L is a concise textual description of the intersection, covering the approaching direction monitored by each camera and the traffic movement enabled by each phase index, inserted into a fixed prompt template; exposing topology through text rather than architecture lets a single policy serve intersections with different layouts and lane counts by only changing the instruction. TSC-Native Action Space In signal control, the decision itself is the choice of a phase: at each step the controller selects the next phase from a small discrete set rather than emitting continuous signals. We therefore cast the decision as phase classification, formulated as token prediction in the same representation space as vision and language (Zitkovich et al. 2023; Kim et al. 2024). Concretely, each action dimension is tied to a reserved token from the end of the LLM vocabulary (Yang et al. 2024), and during training these positions are occupied by a shared set of learnable phase queries, decoupling the action representation from the tokenizer vocabulary. The action head maps the phase-query hidden states to the K phase logits ^∈ℝK a ^K, and the predicted phase is p^t=argmaxja^t,j p_t= *argmax_j\, a_t,j. At deployment, the prediction passes through a minimum-green dwell constraint: once activated, a phase is held for a minimum duration before any switch is permitted, mirroring real signal-timing rules and preventing the policy from oscillating between locally attractive phases across decision steps. Policy Learning Algorithm 1 gives the complete training and inference procedure, and all hyperparameters are listed in the supplementary material (Section B.2). A frozen pretrained VLM, comprising dual vision encoders and a compact language model, is augmented with four lightweight trainable modules: a multilayer projector, LoRA adapters, the phase queries Q, and a residual-MLP action head. At each decision step, the projected visual tokens Z are placed right after the beginning-of-sequence embedding so that every later position can attend to the full visual context under the causal mask, followed by the instruction tokens and the phase queries: =[BOS;1;…;Np;1;…;NL;1;…;Na;EOS],E=[e_BOS;Z_1;…;Z_N_p;e_1;…;e_N_L;\\ q_1;…;q_N_a;e_EOS], (3) where BOSe_BOS and EOSe_EOS are the beginning- and end-of-sequence embeddings, 1,…,NpZ_1,…,Z_N_p are the NpN_p projected visual tokens, 1,…,NLe_1,…,e_N_L are the embeddings of the NLN_L instruction tokens, and 1,…,Naq_1,…,q_N_a are the NaN_a learnable phase queries; only the phase-query positions are supervised. Let (l)H^(l) denote the hidden states of the l-th LLM layer. A zero-initialized action state (0)x^(0) is refined through residual blocks paired one-to-one with the LLM layers; each block performs a tri-branch attention that fuses, under a single softmax, keys and values from three sources (the action state itself, the phase-query hidden states, and the visual hidden states) with rotary position embeddings applied within each branch: Attn()=softmax([x;a;tanh(g)v]⊤dh) (x)=softmax\! ( q\,[k_x;\,k_a;\, (g)\,k_v] d_h ) (4) ⋅[x;a;v], ·\,[v_x;\,v_a;\,v_v], where q is a linear projection of the action state x; (x,x)(k_x,v_x), (a,a)(k_a,v_a), and (v,v)(k_v,v_v) are the key-value pairs computed from the action state, the phase-query hidden states, and the visual hidden states, respectively; and dhd_h is the head dimension. The visual branch is gated by tanh(g) (g) with a learnable scalar g initialized to zero, so the head relies on action features early in training and gradually incorporates visual context as the gate opens. After a residual connection and a feed-forward layer, a final linear projection maps the refined action state to the phase logits, ^=o(L)+o, a=W_o\,x^(L)+b_o, (5) where (L)x^(L) is the action state after the L residual blocks, and oW_o and ob_o are the output-projection weight and bias. Training is behavioral cloning of the verified expert labels: given the one-hot ground-truth action ∗a^*, we minimize the cross-entropy over the K phases, ℒCE=−∑j=0K−1aj∗logexp(a^j)∑k=0K−1exp(a^k),L_CE=- _j=0^K-1a_j^* ( a_j) _k=0^K-1 ( a_k), (6) updating only the lightweight modules θ=θproj,θLoRA,,θheadθ=\ _proj, _LoRA,Q, _head\ while the vision encoders and the base LLM remain frozen, preserving the visual-linguistic priors of large-scale pretraining (Liu et al. 2023; Li et al. 2023; Radford et al. 2021) with a minimal number of trainable parameters. Algorithm 1 VLALight Training and Inference Input: Expert demonstrations =(I,L,∗)D=\(I,L,a^*)\ (stitched image, instruction, one-hot expert action) Parameters: Vision encoder E, LLM fLLMf_LLM, projector gprojg_proj, action head fheadf_head, phase queries Q, adapter parameters θ Output: Policy πθ _θ 1: Initialize θ←θproj,θhead,,θLoRAθ←\ _proj, _head,Q, _LoRA\ 2: Freeze E and fLLMf_LLM 3: while not converged do 4: Sample a batch (Ii,Li,∗i)i=1B\(I^i,L^i,a^*i)\_i=1^B of size B from D 5: i=E(Ii)v^i=E(I^i) 6: i=gproj(i)Z^i=g_proj(v^i) 7: i=[BOS;i;Tokenizer(Li);;EOS]E^i=[e_BOS;Z^i;Tokenizer(L^i);Q;e_EOS] 8: ^i=fhead(fLLM(i,θ)) a^i=f_head (f_LLM(E^i;θ) ) 9: Update θ by minimizing ℒCE(^i,∗i)L_CE( a^i,a^*i) 10: end while 11: for each evaluation step t do 12: Observe (It,Lt)(I_t,L_t) 13: t=[BOS;E(It);Tokenizer(Lt);;EOS]E_t=[e_BOS;E(I_t);Tokenizer(L_t);Q;e_EOS] 14: ^t=fhead(fLLM(t,θ)) a_t=f_head (f_LLM(E_t;θ) ) 15: Execute p^t=argmaxja^t,j p_t= *argmax_j\, a_t,j subject to the minimum-green dwell constraint 16: end for 17: return πθ _θ Experiments Type Method Beihuan Beishahe Changjianglu Gaojiaoyuan Pinganli Zhijingdao ETT↓ EWT↓ ETT↓ EWT↓ ETT↓ EWT↓ ETT↓ EWT↓ ETT↓ EWT↓ ETT↓ EWT↓ Trad. FixTime 59± 19‡ 19± 12† 44± 2.9 12± 2.6 37± 4.1† 1.6± 2.8† 57± 6.7 22± 7.2 51± 7.6 16± 7.8 63± 24 28± 20 Webster 62± 17 21± 12 43± 2.6 11± 2.6 37± 4.0⋆ 1.4± 2.6⋆ 57± 7.0 22± 7.1 51± 7.8 16± 7.8 63± 24 28± 20 SOTL 59± 9.5† 21± 9.5‡ 44± 3.8 11± 3.5 41± 4.3 5.0± 4.4 54± 4.6 19± 3.3 46± 15 11± 12 61± 15 26± 14 MaxPressure 77± 21 35± 17 42± 5.9 9.3± 5.1 45± 10 9.4± 8.6 59± 10 25± 10 45± 8.3 9.7± 8.0 67± 20 33± 17 RL IntelliLight 124± 25 69± 24 41± 5.1 8.8± 3.9 40± 3.2 3.6± 2.5 52± 6.9‡ 16± 5.9‡ 43± 5.3‡ 8.0± 4.3‡ 65± 18 29± 16 PressLight 112± 21 52± 11 54± 5.6 19± 4.2 41± 4.6 4.6± 4.1 91± 13 45± 7.8 69± 9.1 29± 7.8 89± 26 47± 20 AttendLight 72± 15 30± 13 40± 6.3† 6.9± 5.4† 41± 2.2 5.0± 2.7 83± 10 42± 7.8 58± 16 22± 13 80± 24 42± 21 UniTSA 62± 7.0 22± 5.6 41± 4.0‡ 7.8± 3.7‡ 40± 8.2 4.8± 6.4 53± 10 18± 8.7 41± 8.5† 6.1± 6.8† 50± 8.0† 16± 5.5† VLM VLMLight (32B) 65± 13 25± 9.9 42± 2.9 8.7± 3.4 40± 3.9 3.5± 2.6 48± 4.0† 14± 3.9† 44± 9.8 9.2± 8.7 49± 6.2⋆ 14± 3.9⋆ VLMLight (3B) 154± 12 70± 14 65± 11 24± 7.8 53± 13 12± 7.6 113± 10 62± 6.5 61± 22 22± 18 112± 70 62± 55 Ours VLALight (0.5B) 57± 13⋆ 17± 9.8⋆ 38± 1.7⋆ 6.6± 2.2⋆ 38± 1.4‡ 2.4± 1.7‡ 44± 4.2⋆ 9.6± 3.3⋆ 36± 4.2⋆ 3.0± 2.7⋆ 55± 16‡ 20± 13‡ Table 1: Per-city comparison on the regular flow (mean± over 4 seeds, seconds). The top three per column are boldfaced and ranked ⋆/†/‡. Experimental Settings Datasets and Simulation We evaluate VLALight on the six real-world intersections introduced in the Method section, rendered in TransimHub (Wang et al. 2025b); Figure 3 pairs each real intersection with its simulated counterpart. A single model is trained and evaluated across all sites in closed-loop simulation, on the regular flow used to collect the expert demonstrations and three test flows (off-peak, rush-hour, and fluctuating), with four independent seeds per city-flow pair; full dataset details are given in the supplementary material (Section B.1). Figure 3: The six real-world intersections in our dataset. For each site, the satellite view of the real intersection (left) is paired with its rendered simulation (right). Evaluation Metrics We report the mean and standard deviation of four metrics: Average Travel Time (ATT) and Average Waiting Time (AWT) for overall traffic efficiency, and Emergency Travel Time (ETT) and Emergency Waiting Time (EWT) for emergency-vehicle service, all in seconds (lower is better). Compared Methods We compare against nine controllers from three families: four rule-based methods, namely FixTime, Webster (Webster 1958), SOTL (Gershenson 2005), and MaxPressure (Varaiya 2013); four RL-based methods, namely IntelliLight (Wei et al. 2018), PressLight (Wei et al. 2019a), AttendLight (Oroojlooy et al. 2020), and UniTSA (Wang et al. 2024b); and the VLM-based VLMLight (Wang et al. 2025a), whose fast branch is a pretrained UniTSA/PPO controller and whose slow branch invokes cloud-scale VLM/LLM agents. VLMLight is evaluated under the same protocol with two backbone sizes: the cloud configuration 32B and a 3B variant. VLALight uses a 0.5B backbone and runs entirely locally. Baseline configurations are detailed in the supplementary material (Section B.3). Traffic Efficiency Comparison Table 1 reports the per-city comparison under the regular flow; complete breakdowns of all four metrics, organized by intersection and by traffic flow, are provided in the supplementary material (Section C), and generalization to the test flows is analyzed below. VLALight provides the best emergency-vehicle service of all compared methods: it attains the lowest ETT and EWT in four of the six cities and, pooled over all six cities, improves on the cloud-scale VLMLight (32B) by 7.0%7.0\% in ETT and 21.1%21.1\% in EWT. The two exceptions are Beijing Changjianglu and Tianjin Zhijingdao, where the rule-based controllers or VLMLight (32B) rank ahead. Figure 4 shows a concrete example: when an emergency vehicle approaches in one camera view, VLALight switches to the phase serving its movement at the next decision step, letting the vehicle pass without stopping; an extended case study is provided in the supplementary material (Section D). These results show that an end-to-end 0.5B policy can match or surpass the emergency-vehicle service of a cloud-scale cascaded system. Figure 4: An emergency-priority example in TransimHub. At time T an emergency vehicle approaches the intersection in one camera view; VLALight switches to the phase 3, and at time T+1T+1 the vehicle has crossed the intersection. Inference Latency and Deployment Analysis Beyond control quality, real-time deployment hinges on inference latency, which Table 2 summarizes. In routine steps VLMLight can fall back on its fast RL branch, but emergency-aware service is exactly where its slow branch fires: the cascaded design then chains several agent calls for a single decision, costing 10.910.9 s with the cloud 32B backbone and 4.24.2 s on average with a locally deployed 3B backbone, both close to or beyond the 55 s decision interval. VLALight instead outputs the action in a single local forward pass, averaging 224224 ms per decision on one RTX 4090 GPU, which supports fully on-premises deployment without streaming surveillance feeds to the cloud. VLALight is therefore the only compared controller that stays comfortably within the decision budget with emergency awareness always enabled. Method Model size Latency/decision VLMLight 32B (cloud) 10.9 s VLMLight 3B (local) 4.2 s VLALight 0.5B (local) 0.2 s Table 2: Inference latency per control decision in emergency scenarios, where VLMLight’s slow branch fires. Ablation Study As Figure 5 shows, we ablate the visual-input design by comparing our single stitched image against feeding the four camera views as four separately encoded images, which quadruples the visual-token count from 256256 to 1,0241,024. The single stitched input wins in every city and every seed under the regular flow, reducing pooled ATT from 107.0107.0 s to 52.152.1 s and ETT from 127.7127.7 s to 44.744.7 s. The quadrupled tokens inflate activation memory and limit the training batch without adding spatial information, and a compact 0.5B backbone struggles to relate separately encoded views through self-attention. The variant is also twice as slow at inference, at 450450 ms per decision compared with 224224 ms for the stitched input. The stitched input is thus superior in control quality, emergency response, memory footprint, and latency alike, which validates encoding the directional views jointly as one image. An ablation of the minimum-green dwell constraint is provided in the supplementary material (Section C). Figure 5: Ablation of the visual-input design: stitched vs. unstitched visual input. (a) Pooled ATT and ETT under the regular flow; (b) inference latency per decision. Generalization Analysis We assess generalization along two axes: unseen traffic-flow patterns and unseen intersection topologies. Generalization to unseen traffic flows. Figure 6: Generalization to unseen traffic flows: ETT and EWT on the regular flow and the three test flows. Figure 7: Generalization to unseen intersection topologies: transfer to two unseen intersections with few-shot and zero-shot adaptation. We further evaluate the same model, without any retraining, on the three test flows (off-peak, rush-hour, and fluctuating; per-flow results in the supplementary material, Section C.3). Pooled over the three test flows, VLALight retains the best emergency service, improving on VLMLight (32B) by 5.8%5.8\% in ETT and 20.8%20.8\% in EWT, a margin close to that on the regular flow. As Figure 6 shows, the advantage grows as conditions depart from the training distribution: on par under the off-peak flow, 16.5%16.5\% lower EWT under the rush-hour flow, and 33.1%33.1\% under the fluctuating flow. The policy thus transfers well to demand patterns it never encountered during training. Generalization to unseen topologies. We split the six intersections into four training sites and two unseen test sites, Beijing Beishahe and Tianjin Zhijingdao, and Figure 7 compares three variants on the two test sites: the fully trained model, a few-shot variant fine-tuned with only 10% of the target intersections’ data, and a zero-shot variant given no target data at all. The few-shot variant stays within 11–3%3\% of the fully trained model on ETT and EWT, and even slightly improves ATT and AWT. Without any target data, the zero-shot variant remains competitive: its ETT and EWT rise by only 4.3%4.3\% and 11.4%11.4\%, and ATT by 7.6%7.6\%. VLALight thus learns transferable visual-language priors rather than memorizing the training intersections. Conclusion We presented VLALight, a lightweight end-to-end vision-language-action framework for intersection traffic signal control. By stitching multiple directional camera views into a unified visual input grounded by textual instructions, a compact 0.5B model maps intersection observations directly to discrete signal actions in a single forward pass. Experiments on six real-world intersections under four traffic-flow patterns show that VLALight attains the strongest emergency-vehicle service among all compared controllers, with the lowest ETT and EWT in four of the six cities and a 21.1%21.1\% lower pooled EWT than the cloud-scale VLMLight, while answering each decision in 224224 ms on local hardware and generalizing to unseen intersection topologies and traffic-flow patterns. Several directions remain open for future work, which also mark the current limitations of VLALight: the policy is trained by behavior cloning of simulation-collected expert labels, and reinforcement fine-tuning is a natural next step; decisions are made from single frames, where temporal memory could help; and evaluation is limited to simulation, leaving real-world validation and formal safety guarantees to future studies. References Black et al. (2025) K. Black, N. Brown, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, L. Smith, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky π0 _0: A vision-language-action flow model for general robot control. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA. Cited by: Related Work. Brohan et al. (2023) A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, K. Lee, S. Levine, Y. Lu, U. Malla, D. Manjunath, I. Mordatch, O. Nachum, C. Parada, J. Peralta, E. Perez, K. Pertsch, J. Quiambao, K. Rao, M. Ryoo, G. Salazar, K. Sayed, J. Singh, S. Sontakke, A. Stone, C. Tan, H. Tran, V. Vanhoucke, S. Vega, Q. Vuong, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich RT-1: robotics transformer for real-world control at scale. In Proceedings of Robotics: Science and Systems, Cited by: Related Work. Bu et al. (2025) Q. Bu, Y. Yang, J. Cai, S. Gao, G. Ren, M. Yao, P. Luo, and H. Li UniVLA: learning to act anywhere with task-centric latent actions. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA. Cited by: Related Work. Chen et al. (2020) C. Chen, H. Wei, N. Xu, G. Zheng, M. Yang, Y. Xiong, K. Xu, and Z. Li Toward a thousand lights: decentralized deep reinforcement learning for large-scale traffic signal control. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, p. 3414–3421. Cited by: Related Work. Chi et al. (2025) C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song Diffusion policy: visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44 (10–11), p. 1684–1704. External Links: Document Cited by: Related Work. Clifton and Laber (2020) J. Clifton and E. B. Laber Q-learning: theory and applications. Annual Review of Statistics and Its Application 7 (1), p. 279–301. Cited by: Related Work. Da et al. (2024) L. Da, M. Gao, H. Mei, and H. Wei Prompt to transfer: sim-to-real transfer for traffic signal control with prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 82–90. Cited by: Related Work. Gershenson (2005) C. Gershenson Self-organizing traffic lights. Complex Systems 16 (1), p. 29–53. Cited by: Introduction, Related Work, Compared Methods. Huang et al. (2018) Y. Huang, G. Wei, and Y. Wang V-d d3qn: the variant of double deep q-learning network with dueling architecture. In 2018 37th Chinese Control Conference (C), p. 9130–9135. Cited by: Related Work. Kim et al. (2025) M. J. Kim, C. Finn, and P. Liang Fine-tuning vision-language-action models: optimizing speed and success. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA. Cited by: Related Work. Kim et al. (2024) M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn OpenVLA: an open-source vision-language-action model. In Proceedings of The 8th Conference on Robot Learning, Vol. 270, p. 2679–2713. Cited by: Introduction, Related Work, TSC-Native Action Space. Lai et al. (2025) S. Lai, Z. Xu, W. Zhang, H. Liu, and H. Xiong LLMLight: large language models as traffic signal control agents. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 2335–2346. External Links: Document Cited by: Introduction, Related Work. Li et al. (2023) J. Li, D. Li, S. Savarese, and S. Hoi BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, p. 19730–19742. Cited by: Policy Learning. Li et al. (2024) X. Li, M. Liu, H. Zhang, C. Yu, J. Xu, H. Wu, C. Cheang, Y. Jing, W. Zhang, H. Liu, H. Li, and T. Kong Vision-language foundation models as effective robot imitators. In The Twelfth International Conference on Learning Representations, Cited by: Related Work. Liu et al. (2023) H. Liu, C. Li, Q. Wu, and Y. J. Lee Visual instruction tuning. In Advances in Neural Information Processing Systems, Vol. 36, p. 34892–34916. Cited by: Policy Learning. Masri et al. (2025) S. Masri, H. I. Ashqar, and M. Elhenawy Large language models (llms) as traffic control systems at urban intersections: a new paradigm. Vehicles 7 (1), p. 11. External Links: Document Cited by: Related Work. Nie et al. (2025) T. Nie, J. Sun, and W. Ma Exploring the roles of large language models in reshaping transportation systems: a survey, framework, and roadmap. Artificial Intelligence for Transportation 1, p. 100003. Cited by: Related Work. Octo Model Team et al. (2024) Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y. L. Tan, L. Y. Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine Octo: an open-source generalist robot policy. In Proceedings of Robotics: Science and Systems, Cited by: Introduction, Related Work. Oquab et al. (2024) M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jégou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski DINOv2: learning robust visual features without supervision. Transactions on Machine Learning Research. Cited by: Mutli-view Visual. Oroojlooy et al. (2020) A. Oroojlooy, M. Nazari, D. Hajinezhad, and J. Silva AttendLight: universal attention-based reinforcement learning model for traffic signal control. In Advances in Neural Information Processing Systems, Vol. 33, p. 4079–4090. Cited by: Related Work, Compared Methods. Pertsch et al. (2025) K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine FAST: efficient action tokenization for vision-language-action models. In Proceedings of Robotics: Science and Systems, Los Angeles, CA, USA. Cited by: Related Work. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, p. 8748–8763. Cited by: Mutli-view Visual, Policy Learning. Schulman et al. (2017) J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: Related Work. van Hasselt et al. (2016) H. van Hasselt, A. Guez, and D. Silver Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence 30 (1). Cited by: Related Work. Varaiya (2013) P. Varaiya Max pressure control of a network of signalized intersections. Transportation Research Part C: Emerging Technologies 36, p. 177–195. External Links: Document Cited by: Introduction, Related Work, Expert Label Collection, Compared Methods. Wang et al. (2025a) M. Wang, Y. Chen, A. Pang, Y. Cai, C. S. Chen, Y. Kan, and M. Pun VLMLight: safety-critical traffic signal control via vision-language meta-control and dual-branch reasoning architecture. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: Introduction, Related Work, Compared Methods. Wang et al. (2024a) M. Wang, A. Pang, Y. Kan, M. Pun, C. S. Chen, and B. Huang LLM-assisted light: leveraging large language model capabilities for human-mimetic traffic signal control in complex urban environments. External Links: 2403.08337 Cited by: Related Work. Wang et al. (2025b) M. Wang, A. Pang, Y. Xie, M. Pun, Y. Chen, D. Wang, Y. Cai, Z. Ma, C. Xu, Y. Kan, K. Jiang, Z. Cui, L. Roullet, C. S. Chen, and M. Lepech TranSimHub: a unified air–ground simulation platform for multi-modal perception and decision-making. External Links: 2510.15365 Cited by: Road Network Topology Extraction, Datasets and Simulation. Wang et al. (2024b) M. Wang, X. Xiong, Y. Kan, C. Xu, and M. Pun UniTSA: a universal reinforcement learning framework for v2x traffic signal control. IEEE Transactions on Vehicular Technology 73 (10), p. 14354–14369. Cited by: Related Work, Compared Methods. Wang et al. (2026) Y. Wang, P. Ding, L. Li, C. Cui, Z. Ge, X. Tong, W. Song, H. Zhao, W. Zhao, P. Hou, S. Huang, Y. Tang, W. Wang, R. Zhang, J. Liu, and D. Wang VLA-adapter: an effective paradigm for tiny-scale vision-language-action model. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 18638–18646. Cited by: Introduction, Post-Training and Inference. Webster (1958) F. V. Webster Traffic signal settings. Road Research Technical Paper Technical Report 39, Road Research Laboratory, London. Cited by: Related Work, Compared Methods. Wei et al. (2019a) H. Wei, C. Chen, G. Zheng, K. Wu, V. V. Gayah, K. Xu, and Z. Li PressLight: learning max pressure control to coordinate traffic signals in arterial network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p. 1290–1298. Cited by: Related Work, Compared Methods. Wei et al. (2019b) H. Wei, N. Xu, H. Zhang, G. Zheng, X. Zang, C. Chen, W. Zhang, Y. Zhu, K. Xu, and Z. Li CoLight: learning network-level cooperation for traffic signal control. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, p. 1913–1922. Cited by: Related Work. Wei et al. (2021) H. Wei, G. Zheng, V. V. Gayah, and Z. Li Recent advances in reinforcement learning for traffic signal control: a survey of models and evaluation. ACM SIGKDD Explorations Newsletter 22 (2), p. 12–18. Cited by: Introduction. Wei et al. (2018) H. Wei, G. Zheng, H. Yao, and Z. Li IntelliLight: a reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p. 2496–2505. Cited by: Related Work, Compared Methods. Yang et al. (2024) A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2.5 technical report. External Links: 2412.15115 Cited by: TSC-Native Action Space. Ye et al. (2025) A. Ye, Z. Zhang, B. Wang, X. Wang, D. Zhang, and Z. Zhu VLA-r1: enhancing reasoning in vision-language-action models. External Links: 2510.01623 Cited by: Introduction, Related Work. Yu et al. (2025) Z. Yu, B. Wang, P. Zeng, H. Zhang, J. Zhang, L. Gao, J. Song, N. Sebe, and H. T. Shen A survey on efficient vision-language-action models. External Links: 2510.24795 Cited by: Introduction, Related Work. Zhai et al. (2023) X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 11975–11986. Cited by: Mutli-view Visual. Zhang et al. (2025) B. Zhang, Y. Zhang, J. Ji, Y. Lei, Y. Cai, J. Dai, Y. Chen, and Y. Yang SafeVLA: towards safety alignment of vision-language-action model via constrained learning. In Advances in Neural Information Processing Systems, Vol. 38. Cited by: Related Work. Zhang et al. (2026) J. Zhang, X. Chen, Q. Wang, M. Li, Y. Guo, Y. Hu, J. Zhang, S. Bai, J. Lin, and J. Chen VLM4VLA: revisiting vision-language-models in vision-language-action models. In The Fourteenth International Conference on Learning Representations, Cited by: Introduction, Related Work. Zhao et al. (2025) Q. Zhao, Y. Lu, M. J. Kim, Z. Fu, Z. Zhang, Y. Wu, Z. Li, Q. Ma, S. Han, C. Finn, A. Handa, T. Lin, G. Wetzstein, M. Liu, and D. Xiang CoT-vla: visual chain-of-thought reasoning for vision-language-action models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 1702–1713. Cited by: Related Work. Zhao et al. (2023) T. Z. Zhao, V. Kumar, S. Levine, and C. Finn Learning fine-grained bimanual manipulation with low-cost hardware. In Proceedings of Robotics: Science and Systems, Cited by: Related Work. Zitkovich et al. (2023) B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, W. Huang, S. Welker, K. Hausman, J. Orbay, et al. RT-2: vision-language-action models transfer web knowledge to robotic control. In Proceedings of The 7th Conference on Robot Learning, p. 2165–2183. Cited by: Introduction, Related Work, TSC-Native Action Space. Zou et al. (2026) X. Zou, Y. Yang, Z. Chen, X. Hao, Y. Chen, C. Huang, and Y. Liang Traffic-r1: reinforced LLMs bring human-like reasoning to traffic signal control systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), San Diego, California, United States, p. 21823–21838. Cited by: Introduction, Related Work.