Paper deep dive
A Deployed Hybrid Vehicle-in-the-Loop Platform for Validating Cooperative Perception
Anastasia Bolovinou, Giorgos Hadjipavlis, Markos Antonopoulos, Panagiotis Tachtalis, Konstantinos Petousakis, Konstantinos Lazaridis, Alexandros Siskos, Bill Roungas, Angelos Amditis
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/16/2026, 2:02:39 AM
Summary
This paper introduces a deployed hybrid Vehicle-in-the-Loop (ViL) platform that integrates a real instrumented vehicle with a CARLA-based digital twin via a V2X message pipeline. The system validates cooperative perception by fusing ETSI-compliant CAM/CPM messages into a GPU-accelerated probabilistic occupancy grid. Experiments across nominal, rain, and night conditions, alongside varying localization noise levels, demonstrate that cooperative perception significantly widens field-of-view coverage and improves occupied-cell recall. The study concludes that beyond moderate noise thresholds, localization uncertainty becomes the dominant error source, and outlines the platform's trajectory toward a Mediterranean operational design domain testing service aligned with European virtual homologation regulations.
Entities (8)
Relation Signals (7)
Hybrid Vehicle-in-the-Loop (ViL) platform → couples → CARLA digital twin
confidence 97% · We present a deployed hybrid Vehicle-in-the-Loop (ViL) platform that couples a real instrumented vehicle with a CARLA-based digital twin (DT) through a V2X message pipeline
Real vehicle → streams → ETSI-compliant CAM/CPM messages
confidence 96% · A real vehicle streams ETSI-compliant CAM/CPM messages into the DT
Cooperative Perception (CP) module → fuses → ETSI-compliant CAM/CPM messages
confidence 95% · where a GPU-accelerated Cooperative Perception (CP) module fuses them into a probabilistic occupancy grid during scenario runtime.
Cooperative Perception → improves → Field-of-view coverage
confidence 94% · The results show that CP substantially widens field-of-view (FoV) coverage and improves occupied-cell recall
Cooperative Perception → improves → Occupied-cell recall
confidence 94% · The results show that CP substantially widens field-of-view (FoV) coverage and improves occupied-cell recall
Localization uncertainty → becomes → Dominant error source
confidence 93% · beyond a moderate localization-noise threshold, positioning uncertainty, and not weather, becomes the dominant error source.
SYNERGIES project → funds → Hybrid Vehicle-in-the-Loop (ViL) platform
confidence 92% · This work was supported by the SYNERGIES project, funded by the European Union’s Horizon Europe research and innovation programme
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:European safety regulation now permits a large share of automated-driving homologation evidence to be produced virtually, provided a validated physical-virtual facility generates it. We present a deployed hybrid Vehicle-in-the-Loop (ViL) platform that couples a real instrumented vehicle with a CARLA-based digital twin (DT) through a V2X message pipeline, and we report its first integrated operation on a public-road-representative test track. A real vehicle streams ETSI-compliant CAM/CPM messages into the DT, where a GPU-accelerated Cooperative Perception (CP) module fuses them into a probabilistic occupancy grid during scenario runtime. We demonstrate the platform on a multi-vehicle double T-intersection scenario, characterise the CP workload across nominal, rain and night conditions and five localization-noise levels, and discuss the platform's current architectural limits and the engineering targets they define. The results show that CP substantially widens field-of-view (FoV) coverage and improves occupied-cell recall, and that beyond a moderate localization-noise threshold, positioning uncertainty, and not weather, becomes the dominant error source. We outline the platform's trajectory toward a Mediterranean operational design domain (ODD) testing service.
Tags
Links
- Source: https://arxiv.org/abs/2607.13806v1
- Canonical: https://arxiv.org/abs/2607.13806v1
Trouble viewing inline? Open PDF directly →
Full Text
18,826 characters extracted from source content.
Expand or collapse full text
A Deployed Hybrid Vehicle-in-the-Loop Platform for Validating Cooperative Perception Anastasia Bolovinou, Giorgos Hadjipavlis, Markos Antonopoulos, Panagiotis Tachtalis, Konstantinos Petousakis, Konstantinos Lazaridis,Alexandros Siskos, Bill Roungas, Angelos Amditis Institute of Communication and Computer Systems, Athens, Greece anastasia.bolovinou, giorgos.hadjipavlis, panagiotis.tachtalis, markos.antonopoulos, konstantinos.petousa, k.lazaridis, alexandros.siskos, vroungas, a.amditis@iccs.gr Abstract—European safety regulation now permits a large share of automated-driving homologation evidence to be pro- duced virtually, provided a validated physical-virtual facility generates it. We present a deployed hybrid Vehicle-in-the-Loop (ViL) platform that couples a real instrumented vehicle with a CARLA-based digital twin (DT) through a V2X message pipeline, and we report its first integrated operation on a public- road-representative test track. A real vehicle streams ETSI- compliant CAM/CPM messages into the DT, where a GPU- accelerated Cooperative Perception (CP) module fuses them into a probabilistic occupancy grid during scenario runtime. We demonstrate the platform on a multi-vehicle double T-intersection scenario, characterise the CP workload across nominal, rain and night conditions and five localization-noise levels, and discuss the platform’s current architectural limits and the engineering targets they define. The results show that CP substantially widens field-of-view (FoV) coverage and improves occupied-cell recall, and that beyond a moderate localization-noise threshold, positioning uncertainty, and not weather, becomes the dominant error source. We outline the platform’s trajectory toward a Mediterranean operational design domain (ODD) testing service. Index Terms—hybrid testing, vehicle-in-the-loop, cooperative perception, occupancy grid, V2X, CARLA, digital twin, ADAS validation, autonomous vehicles I. INTRODUCTION The EU General Safety Regulation [1] has made advanced driver-assistance systems mandatory across all new vehicle categories, while recent UN regulations for automated systems (UN R152, R157 and R171) [2]–[4] now accept virtual testing as valid homologation evidence, on the condition that the virtual results are validated against physical tests at the same facility. A testing facility that couples a real vehicle to a credible simulation, rather than a pure simulator, is therefore precisely what the regulatory framework requires. Hybrid testing is also a practical necessity. Driving sce- narios, including CP scenarios, may require more connected agents than an organization physically owns, and the mix of real and virtual participants varies from experiment to experiment with the available hardware, like the number of This work was supported by the SYNERGIES project, funded by the European Union’s Horizon Europe research and innovation programme under Grant Agreement No. 101146542. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or CINEA. Neither the European Union nor the granting authority can be held responsible for them. instrumented vehicles, road side units, extra traffic agents (connected or not connected) etc. By substituting virtual agents for unavailable real ones, a hybrid platform can exercise multi- agent scenarios that would otherwise be infeasible to stage physically, at a virtual/real ratio chosen per experiment. This paper describes such a facility as deployed infrastruc- ture. Our contribution is threefold: • A hybrid ViL platform that couples a real instru- mented vehicle to a CARLA digital twin (DT) through an ETSI-compliant Cooperative Awareness Messages (CAM)/Collective Perception Messages (CPM) pipeline [5], [6] extended from the CARLA ROS bridge. • A real-world demonstrator on our institution’s public- road-representative test track, in which three real vehicles (two sharing CPM) are reflected in the DT and consumed by a GPU-accelerated occupancy-grid module. • A characterisation of the CP workload across nominal, rain and night conditions, together with a statement of the platform’s current limits. We deliberately separate what is demonstrated functionally on the real vehicle from what is quantified in simulation; closing that gap with a quantitative sim-to-real fidelity metric is ongoing work. I. PLATFORM ARCHITECTURE The platform comprises three groups of components (Fig. 3): real assets, virtual assets and the ROS-based perception modules that run across both. Real assets. The core real asset is a fully instrumented vehicle carrying GNSS and an IMU for localization (sup- porting both standard GPS and differential GPS (D-GPS), the latter providing centimetre-level positioning usable as ground truth), a Velodyne VLP-16 LiDAR and a forward- facing RGB camera for perception, a NUC-class edge PC for on-board object detection, and an Nfiniity Cube EVK OnBoard Unit (OBU) providing C-V2X and ITS-G5 connectivity. The platform further supports additional connected vehicles and roadside units (RSUs) as real assets. Connected assets transmit their kinematic state as CAM and their detections as CPM [5], [6], both triggered every 100 ms. While the platform is currently tested in a V2X CP configuration, the ViL coupling itself is agnostic to the communication layer; assets and the DT can equally be linked over a direct vehicle-to-cloud arXiv:2607.13806v1 [cs.RO] 15 Jul 2026 Fig. 1. Twin-construction pipeline: (a) the source road network in OpenStreetMap is (b) converted to an OpenDRIVE (.xodr) logical map and (c) imported into CARLA as a drivable 3D scenario. Fig. 2. Custom twin assets. Top: the real vehicle is reconstructed as a 3D mesh and imported as a calibrated, instrumented CARLA model. Bottom: custom terrain, buildings and vegetation (left) and retroreflective cat’s-eye road studs under night lighting (right). connection. Finally, the platform also supports a driver-in-the- loop configuration, in which a human drives the real vehicle while the DT and its virtual agents react in real time, closing the loop through the human as well as the sensors. Virtual assets. The DT is not merely a drivable map but a complete model of the road environment, its static and dynamic assets, and the simulated agents that populate it, all kept consistent with the real world. A CARLA [7] instance of this environment ingests the live stream through a custom middleware layer that extends the CARLA ROS bridge, parsing ETSI messages into DT-compatible objects. Virtual agents participate in V2X through the VaN3Twin framework (ns-3 + OpenCDA) [8], [9], so that simulated and real entities both contribute to the shared perception. The site is reconstructed through a repeatable pipeline (Fig. 1): the road network is extracted from OpenStreetMap, enriched, and converted into an OpenDRIVE (.xodr) logical map encoding lanes, junctions and connectivity, and imported into CARLA as a newly created unreal map (.umap file). The same toolchain reconstructs any OpenStreetMap-covered site as a DT, which is what makes the Mediterranean-ODD scenario library of Section VI tractable to scale beyond a single location. Validity also depends on the DT reproducing the features that drive sensor and perception behaviour, so the default CARLA asset set is augmented with site- and vehicle-specific models (Fig. GNSS/IMU, VLP-16 LiDAR, RGB camera NUC edge PC (on-board detection) Nfiniity Cube V2X modem Real assets (vehicle, RSUs) Middleware (extends CARLA ROS bridge) CARLA digital twin VaN3Twin agents (ns-3 + OpenCDA) Virtual assets (digital twin) Camera–LiDAR perception (supported) “Occupancy” CP (Docker· ROS 2 · JAX/CUDA) navmsgs/ OccupancyGrid ROS-based perception modules Off-board (edge/cloud) ETSI-compliant CAM/CPM (100 ms) fused CAM/CPM streams Fig. 3. End-to-end architecture of the hybrid ViL platform. 2): the real vehicle is reconstructed as a calibrated 3D mesh whose geometry and on-board sensor placement are modelled on the physical platform; the site’s buildings, terrain and vegetation are modelled to reproduce real occlusions and sight- lines; condition-specific features such as retroreflective cat’s- eye road studs recreate night-time appearance cues. These as- sets directly underpin the multi-condition behaviour of Section IV: the night results, in particular, are only meaningful if the DT renders the visual environment a vehicle would encounter after dark. ROS-basedperceptionmodules.Theplatform hosts its perception as ROS modules: a single-view camera–LiDAR perception module and the cooperative- perception “Occupancy” module (containerised; ROS2; GoogleJAXwithNVIDIACUDAviaXLAJIT), which fuses incoming CAM/CPM into a standardised nav_msgs/msg/OccupancyGrid – a probabilistic grid whose cells encode occupancy likelihood (occupied, free, or uncertain), regenerated for each batch of messages. Because both are ROS nodes consuming standardised messages, they deploy unchanged on the real vehicle or against the DT through the CARLA-ROS bridge; this paper evaluates the CP module (Section IV). I. REAL-WORLD DEMONSTRATOR We exercised the full chain on our institution’s test track using a double T-intersection scenario. Three real vehicles operated on the track (two connected and sharing CPM and a third, CAM-only vehicle broadcasting its position but not perception data) while the same scenario was reconstructed in the DT. Fig. 4 places, side by side, drone footage of the real intersection, its CARLA reconstruction, and the occupancy grid produced from the live messages, including a variation in which a virtual agent augments the real ones. This demonstrator validates the platform functionally: real CAM/CPM traffic is ingested, reflected in the DT and turned into a coherent occupancy grid end to end. We emphasise its scope honestly. Because the test-track vehicle cannot be synchronised frame-exactly with the simulation, repeated runs of the same initial conditions are required, and the perception metrics in Section IV are computed in the DT with controlled localization noise rather than against real-vehicle ground truth. The demonstrator is thus a proof of concept for the hybrid methodology; quantitative sim-to-real fidelity is the subject of follow-up work. The platform is not a one-off demonstrator but shared in- frastructure already exercised across EU projects. In EVENTS [10], it supports collective perception evaluation and the scaling-up of cooperative scenarios. In SUNRISE [11], the same hybrid setup is used for hybrid (virtual-physical) testing and the logging of key performance indicators feeding CCAM safety argumentation (video available on request). 1 The same CARLA-based simulation toolchain, including its driver-in- the-loop configuration, also underpins scenario-data generation in SYNERGIES [12], for the project’s federated scenario dataspace. IV. CP WORKLOAD AND MULTI-CONDITION BEHAVIOUR The platform itself produces no metric in isolation; we therefore test it through the CP module it runs. The derived occupancy-grid is evaluated, using the KPIs as a demo for what the platform is capable of, in the DT under three environmental conditions (nominal, rain, night), three coop- erative configurations (ego-only, three connected agents, all six agents), and five localization-noise levels (0, 0.5, 1.0, 1.5 and 2.0 meters). All metrics are reported as area-under-curve (AUC) over each scenario. Fig. 5 summarises the headline KPIs, from which three findings stand out. First, joint FoV coverage rises sharply with cooperation, from roughly one quarter of the area of interest for a single vehicle to the majority of it with all agents (Fig. 5a) and, notably, stays stable across the full localization-noise range, indicating that probabilistic fusion preserves spatial awareness even under positioning uncertainty. Second, occupied-cell recall improves with cooperation at low noise but all configurations (except ego-only) converge near the same value at 2 meters (Fig. 5b): beyond a moderate noise 1 Seedetailsat6:15inthefollowingvideo: https://w.youtube.com/watch?v=OClo941WAso threshold, localization uncertainty (not the number of agents or the weather) becomes the dominant error source. Third, the cooperative benefit is largest exactly when individual sensing is weakest, i.e., under night conditions, where adding agents recovers occupied-cell recall from a degraded single-vehicle level back toward its nominal value (shaded region in Fig. 5b). Occupied-cell precision shows the expected slight decrease as agents are added (overlapping noisy observations), an accept- able trade-off in a safety setting where missed obstacles matter more than conservative ones; free-space estimation remains reliable across all conditions. V. CURRENT LIMITATIONS AND ENGINEERING TARGETS With regard to the platform’s current limitations, the cou- pled pipeline executes on a fixed-interval cycle, where each cycle ingests the live CAM/CPM stream, updates the DT and regenerates the occupancy grid. Still, strict determinism cannot be guaranteed in the current configuration, because transient slowdowns of the simulation loop introduce timing irregularities that can alter the temporal alignment between real-vehicle message arrivals and simulation state updates. This is an architectural property of the CARLA-based co- simulation rather than a hardware limit. With regard to the engineering targets, the first one is raising the loop rate (the frequency of the complete ingest- update-regenerate cycle) and bounding its jitter (the worst- case deviation of the cycle interval), since bounded timing is a precondition for run-to-run reproducibility in a testing context. We are pursuing this through a co-simulation sched- uler that advances all components on a common clock and by isolating the grid generator from the simulator’s rendering loop. The second target is a GPU-versus-CPU benchmark of the occupancy module itself, to quantify the contribution of the JAX/CUDA implementation. The planned sim-to-real fidelity assessment builds directly on this: a D-GPS reference trajectory provides the positioning ground truth against which the DT’s synthetic localization noise can be replaced by measured real-vehicle error. Planned extensions broaden the loop further: remote operation of the real vehicle and an immersive driver interface via VR headset for in-the-loop scenario testing. VI. TOWARD A MEDITERRANEAN-ODD TESTING SERVICE No hybrid ViL facility currently operates in southeastern Europe, to our knowledge. Moreover, existing European facil- ities are optimised for established homologation procedures rather than hybrid V2X-in-the-loop testing. The platform’s planned differentiator is a validated scenario library encoding Mediterranean ODDs (high solar glare, high ambient temper- atures, dry-road friction profiles, dense mixed urban traffic) built on the architecture described here. Combined with a pro- gressive accreditation path (ISO 9001, then ISO/IEC 17025), this positions the platform as deployable testing infrastructure for suppliers and developers with the intention to comple- ment existing commercial proving grounds and EuroNCAP Fig. 4. Left: drone footage of the real double T-intersection. Right: CARLA DT reconstruction. Center bottom: occupancy grid produced from the live CAM/CPM stream (with one virtual agent added). Fig. 5. Top (a): Joint FoV coverage of CP across conditions. Bottom (b): Occupied-cell recall across different environmental conditions and localization noise. test sites, adding hybrid V2X-in-the-loop capability on our institution’s own 5G-enabled testbed. VII. CONCLUSION We presented a deployed hybrid ViL platform, its first integrated real-world demonstrator and a multi-condition char- acterisation of the CP workload it runs. The platform turns live V2X traffic into a probabilistic occupancy grid end to end, and the evaluation shows that cooperation widens coverage and improves recall while localization uncertainty dominates beyond a moderate threshold. Ongoing work targets quantita- tive sim-to-real fidelity, a higher and bounded loop rate and the Mediterranean-ODD scenario library. REFERENCES [1] European Parliament and Council of the European Union, “Regulation (EU) 2019/2144 on type-approval requirements for motor vehicles (General Safety Regulation),” Official Journal of the European Union, 2019. [2] UNECE, “UN regulation no. 152 – uniform provisions concerning the approval of motor vehicles with regard to the Advanced Emergency Braking System (AEBS) for M1 and N1 vehicles,” United Nations Economic Commission for Europe, 2020. [3] —, “UN regulation no. 157 – automated lane keeping systems (alks) extended to trucks, buses and coaches,” United Nations Economic Commission for Europe, 2022. [4] —, “UN regulation no. 171 – uniform provisions concerning the approval of vehicles with regard to Driver Control Assistance Systems (DCAS),” United Nations Economic Commission for Europe, 2024. [5] ETSI, Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 2: Specification of Cooperative Aware- ness Basic Service, European Telecommunications Standards Institute, ETSI EN 302 637-2. [6] —, Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Collective Perception Service, European Telecommunications Standards Institute, ETSI TS 103 324. [7] A. Dosovitskiy, G. Ros, F. Codevilla, A. L ́ opez, and V. Koltun, “CARLA: An open urban driving simulator,” in Proc. 1st Annual Conf. on Robot Learning (CoRL), 2017, p. 1–16. [8] R. Pegurri, D. Gasco, F. Linsalata, M. Rapelli, E. Moro, F. Raviglione, and C. Casetti, “VaN3Twin: The multi-technology V2X digital twin with ray-tracing in the loop,” IEEE Transactions on Wireless Communica- tions, vol. 25, p. 16 812–16 827, 2026. [9] R. Xu, Y. Guo, X. Han, X. Xia, H. Xiang, and J. Ma, “Opencda: an open cooperative driving automation framework integrated with co-simulation,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC). IEEE, 2021, p. 1155–1162. [10] EVENTS Project, “EVENTS: reliablE in-Vehicle pErception and decisioN-making in complex environmenTal conditionS,” Horizon Eu- rope project, Grant Agreement No. 101069614, https://events-project.eu. [11] SUNRISE Project, “SUNRISE: Safety assurance framework for con- nected and automated mobility systems,” Horizon Europe project, Grant Agreement No. 101069573, https://ccam-sunrise-project.eu/. [12] SYNERGIES Project, “SYNERGIES: Real and synthetic scenarios generated for the development, training, virtual testing and valida- tion of CCAM systems,” Horizon Europe project, Grant Agreement No. 101146542, https://synergies-ccam.eu/.