Paper deep dive
SARFusion: Scene-Aware Routing Fusion for Robust Camera-LiDAR 3D Object Detection
Yuting Zhao, Ziyi Zheng, Shuxiao Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/25/2026, 5:08:38 AM
Summary
The paper introduces SARFusion, a robust 3D object detection framework for autonomous driving that addresses the vulnerability of existing camera-LiDAR fusion methods to sensor corruptions and environmental changes. Unlike traditional methods that use fixed fusion pathways, SARFusion employs a scene-aware branch routing strategy. It maintains three parallel decoding branches (camera, LiDAR, and fusion) and uses a Scene Reliability Prior, estimated from global context and supervised by textual prompts, combined with local object-level evidence, to dynamically route each object query to the most reliable branch. This approach mitigates negative transfer from degraded modalities while preserving complementary cues. SARFusion achieves state-of-the-art performance on the nuScenes dataset, with 72.5 mAP and 74.4 NDS on the test set, demonstrating superior robustness under adverse weather and sensor corruption conditions.
Entities (10)
Relation Signals (9)
SARFusion → evaluatedon → nuScenes
confidence 98% · On the nuScenes test set, SARFusion achieves strong performance with 72.5 mAP and 74.4 NDS.
SARFusion → achievesscoreon → 74.4 NDS
confidence 95% · On the nuScenes test set, SARFusion achieves strong performance with 72.5 mAP and 74.4 NDS.
SARFusion → achievesscoreon → 72.5 mAP
confidence 95% · On the nuScenes test set, SARFusion achieves strong performance with 72.5 mAP and 74.4 NDS.
SARFusion → usesmodule → Scene-Aware Branch Routing
confidence 92% · SARFusion further incorporates object-level evidence to route each query to the most suitable branch.
SARFusion → usesmodule → Scene Reliability Prior
confidence 92% · Guided by a Scene Reliability Prior estimated from the global driving context, SARFusion further incorporates object-level evidence to route each query to the most suitable branch.
Yuting Zhao → affiliatedwith → Chinese Academy of Sciences
confidence 90% · Affiliation: Institute of Automation, Chinese Academy of Sciences, Beijing, China
Shuxiao Li → affiliatedwith → Chinese Academy of Sciences
confidence 90% · Affiliation: Institute of Automation, Chinese Academy of Sciences, Beijing, China
SARFusion → proposedby →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Camera-LiDAR fusion has become a prevailing paradigm for 3D object detection in autonomous driving. However, existing fusion detectors often establish strong inter-modality dependencies by decoding object queries from tightly coupled multimodal representations. Under corrupted driving conditions, such dependencies make the detector vulnerable to unreliable modalities, where degraded observations may interfere with reliable modality-specific evidence and lead to suboptimal predictions. Moreover, modality reliability can vary across both global driving scenes and individual object queries, requiring adaptive fusion decisions at a finer granularity. To bridge this gap, we reformulate robust camera-LiDAR fusion as a scene-aware branch routing problem and propose SARFusion, a robust 3D object detector. Instead of producing detections from a single fused representation, SARFusion decouples object-query decoding into three parallel reasoning branches: a camera branch, a LiDAR branch, and a camera-LiDAR fusion branch. Guided by a Scene Reliability Prior estimated from the global driving context, SARFusion further incorporates object-level evidence to route each query to the most suitable branch. This query-wise routing strategy alleviates harmful cross-modal interference while preserving the benefits of multimodal fusion when complementary cues are trustworthy. On the nuScenes test set, SARFusion achieves strong performance with 72.5 mAP and 74.4 NDS. Extensive analyses demonstrate its robustness under challenging conditions, including sensor corruptions and environmental changes.
Tags
Links
- Source: https://arxiv.org/abs/2609.29235v1
- Canonical: https://arxiv.org/abs/2609.29235v1
Trouble viewing inline? Open PDF directly →
Full Text
43,125 characters extracted from source content.
Expand or collapse full text
SARFusion: Scene-Aware Routing Fusion for Robust Camera-LiDAR 3D Object Detection Yuting Zhao Affiliation: Institute of Automation, Chinese Academy of Sciences, Beijing, China Affiliation: School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China Ziyi Zheng Affiliation: Wuhan College, Wuhan, China E-mail zhaoyuting2023,shuxiao.li@ia.ac.cn, 542197655@q.com Shuxiao Li †thanks: Corresponding author: shuxiao.li@ia.ac.cn Affiliation: Institute of Automation, Chinese Academy of Sciences, Beijing, China Abstract Camera-LiDAR fusion has become a prevailing paradigm for 3D object detection in autonomous driving. However, existing fusion detectors often establish strong inter-modality dependencies by decoding object queries from tightly coupled multimodal representations. Under corrupted driving conditions, such dependencies make the detector vulnerable to unreliable modalities, where degraded observations may interfere with reliable modality-specific evidence and lead to suboptimal predictions. Moreover, modality reliability can vary across both global driving scenes and individual object queries, requiring adaptive fusion decisions at a finer granularity. To bridge this gap, we reformulate robust camera-LiDAR fusion as a scene-aware branch routing problem and propose SARFusion, a robust 3D object detector. Instead of producing detections from a single fused representation, SARFusion decouples object-query decoding into three parallel reasoning branches: a camera branch, a LiDAR branch, and a camera-LiDAR fusion branch. Guided by a Scene Reliability Prior estimated from the global driving context, SARFusion further incorporates object-level evidence to route each query to the most suitable branch. This query-wise routing strategy alleviates harmful cross-modal interference while preserving the benefits of multimodal fusion when complementary cues are trustworthy. On the nuScenes test set, SARFusion achieves strong performance with 72.5 mAP and 74.4 NDS. Extensive analyses demonstrate its robustness under challenging conditions, including sensor corruptions and environmental changes. Keywords: 3D Detection Camera-LiDAR Fusion Robust Perception 1 Introduction Reliable 3D object detection is a fundamental capability for autonomous driving systems. Modern autonomous vehicles perceive their surroundings with multiple complementary sensors, among which cameras and LiDAR are two of the most widely used modalities. Cameras provide dense appearance and semantic information, which is beneficial for object recognition and distant-object perception. LiDAR directly measures depth and 3D geometry, providing accurate localization cues. By combining these complementary signals, camera-LiDAR fusion detectors have achieved strong performance on standard 3D detection benchmarks. [2, 50, 9] Despite this progress, robust multi-modal perception remains challenging in complex driving environments. Sensor observations can be degraded by adverse weather [3], illumination changes, motion blur, temporal inconsistency, spatial misalignment, camera failures, LiDAR beam dropping, incomplete echoes, and other corruptions [4, 32, 14, 1, 3, 11]. Under such conditions, the reliability of each modality can change significantly. A modality that provides useful evidence in clean scenes may become noisy or misleading when degraded. If a detector always follows the same fusion pathway, unreliable modality information can be injected into the final prediction, leading to negative transfer and reduced robustness. A common limitation of existing fusion detectors is that they treat fusion behavior as largely fixed [2, 30]. This design implicitly assumes that the same sensor combination is suitable for all driving scenes. However, modality reliability is strongly scene-dependent. Weather, illumination, and sensor status can change the overall usefulness of camera and LiDAR cues. For example, low-light or adverse weather may weaken visual observations, while LiDAR corruptions can reduce geometric reliability. This motivates the need for scene-aware reliability modeling rather than condition-agnostic fusion. Moreover, modality reliability is not only scene-dependent but also object-dependent. In the same frame, different objects may have different observation quality due to distance, occlusion, visibility, and point density [11, 22, 46]. A distant pedestrian may have sparse LiDAR returns but still preserve recognizable image semantics, while a nearby vehicle may be better localized by LiDAR geometry. Therefore, a single scene-level fusion decision is insufficient: forcing all objects in a frame to use the same sensor combination may still propagate locally degraded evidence. Since object queries naturally represent candidate instances in transformer-based 3D detectors, query-level branch selection provides an appropriate granularity for reliable fusion [45, 27, 2, 48].. Motivated by these observations, we propose SARFusion, a scene-aware branch routing framework for robust camera-LiDAR 3D object detection. SARFusion explicitly maintains three candidate branches: a camera branch, a LiDAR branch, and a camera-LiDAR fusion branch. These branches provide different sensing paths for object queries under varying sensor conditions. To guide branch selection, SARFusion first estimates a Scene Reliability Prior from global driving conditions such as weather and time. During training, the scene prior is aligned with textual scene prompts [34]. It then performs Scene-Aware Branch Routing, which combines the scene prior with local feature evidence around each object query to select a reliable branch. This design allows different objects in the same scene to rely on different sensor combinations, reducing negative transfer from degraded modalities while preserving useful cross-modal complementarity. We evaluate SARFusion on clean and corrupted autonomous-driving benchmarks. In nuScenes [6], SARFusion achieves 71.1 mAP and 73.7 NDS on the validation split, and 72.5 mAP and 74.4 NDS on the test split. Under adverse nuScenes-C conditions [11, 16], SARFusion improves detection performance under adverse weather and illumination changes, including fog, snow, rain and strong sunlight. Ablation studies verify the effectiveness of our core components. Our main contributions are summarized as follows: 1. We propose SARFusion, a scene-aware branch routing framework for robust camera-LiDAR 3D object detection, which dynamically selects among camera, LiDAR, and camera-LiDAR fusion branches instead of relying on a fixed fusion pathway. 2. We introduce a Scene Reliability Prior to model scene-dependent modality reliability from global driving conditions, providing condition-aware guidance under degraded environments. 3. We design Scene-Aware Branch Routing, which combines the scene reliability prior with local object-level evidence to perform query-level branch selection. 4. SARFusion achieved SOTA performance among existing camera-LiDAR fusion methods under various sensor failure conditions and extreme weather scenarios. 5. The code will be publicly available. 2 Related Work 2.1 3D Object Detection 3D object detection estimates object categories and 3D bounding boxes from onboard sensor observations. Camera-based methods infer 3D layouts from monocular, stereo, or surround-view images. Early methods rely on geometric priors, keypoints, or explicit depth estimation [8, 5, 29, 43]. Recent surround-view detectors lift image features into 3D or bird’s-eye-view (BEV) space [15, 26, 25], or use 3D queries to aggregate multi-view image features [45, 27]. These methods benefit from dense appearance cues, but remain limited by depth ambiguity, especially for distant, occluded, or weakly textured objects. LiDAR-based detectors exploit point clouds with accurate depth and geometry. Point-based methods operate on raw point sets [33, 37], while voxel- and pillar-based methods convert point clouds into regular representations for efficient feature extraction [53, 49, 23, 51]. Hybrid point-voxel methods further balance geometric precision and computational efficiency [36]. However, LiDAR points are sparse at long range and provide limited semantic appearance. They can also be degraded by adverse weather, beam sparsity, incomplete echoes, and sensor malfunction. These complementary limitations motivate camera-LiDAR fusion for more accurate and reliable 3D detection. 2.2 Multi-modal Sensor Fusion Camera-LiDAR fusion combines dense image semantics with accurate LiDAR geometry. Point-level methods project image semantics or features onto LiDAR points to enrich point-cloud representations [40, 41, 52]. They improve detection performance, but usually depend on accurate calibration and are sensitive to projection errors, spatial misalignment, and LiDAR degradation. Recent methods fuse modalities in intermediate feature spaces. BEV-based methods transform camera and LiDAR features into a shared BEV representation [30, 42], which provides an efficient fusion space but still relies on depth estimation and geometric projection. Other methods explore unified voxel representations, deformable feature alignment, and sparse multi-sensor representations [24, 9, 10, 47]. Transformer-based methods use object queries or modality tokens for cross-modal interaction [2, 50, 48]. TransFusion refines LiDAR proposals with image features, DeepInteraction performs iterative modality interaction, and CMT uses coordinate embeddings for implicit alignment. These methods achieve strong clean-set performance, but most of them use fixed fusion pathways or predefined interaction patterns. They rarely adapt the fusion behavior to scene-dependent and object-dependent modality reliability. 2.3 Robust Multi-Modality Fusion Robust multi-modal fusion has attracted increasing attention because 3D detectors can degrade severely under adverse weather, motion blur, temporal inconsistency, spatial misalignment, missing cameras, beam dropping, and incomplete echoes [46, 22, 3, 18, 17]. These corruptions change modality reliability at different granularities. Weather and illumination often affect the whole scene, while occlusion, distance, point density, and local sensor failures may affect individual objects within the same frame. Existing methods improve robustness through data augmentation, modality dropout, decoupled representations, or adaptive fusion strategies [48, 13, 42, 7, 31, 12, 16, 38, 35]. CMT adopts masked-modal training to handle missing modalities [48]. MEFormer reduces negative fusion with modality-agnostic decoding and prediction ensemble [7]. Recent expert-based methods further explore routing or selecting modality-specific experts under sensor failures [31]. These studies suggest that reducing rigid inter-modality dependence is important for robust perception. However, many existing robust fusion methods mainly focus on predefined corruptions, modality-level failures, or fixed robustness training strategies. Less attention has been paid to jointly modeling global scene reliability and query-level local observation quality. In realistic driving scenes, the reliable modality may vary not only with weather or illumination, but also across different objects due to distance, visibility, occlusion, and point density [21, 20, 39, 44]. Motivated by this observation, SARFusion formulates robust camera-LiDAR fusion as a scene-aware branch routing problem. It estimates a global scene reliability prior and combines it with local query evidence to select camera, LiDAR, or fusion branches for each object query [19, 28]. 3 Method 3.1 Overview The overall framework of SARFusion is shown in Fig. 1. Given multi-view camera images ℐ=Ivv=1VI=\I_v\_v=1^V and a LiDAR point cloud P, SARFusion first extracts camera features and LiDAR features using modality-specific encoders. The camera features are flattened into image tokens C∈ℝNC×DX_C ^N_C× D, and the LiDAR features are encoded into BEV tokens L∈ℝNL×DX_L ^N_L× D. We further concatenate them as multimodal context tokens CL=[C;L]X_CL=[X_C;X_L]. Figure 1: Overall architecture of the proposed SARFusion. Instead of producing a single fused representation, SARFusion explicitly maintains three candidate decoding branches: a camera branch, a LiDAR branch, and a Camera-LiDAR fusion branch. These branches provide complementary representation choices under different sensing conditions. The camera branch decodes object queries using camera tokens, the LiDAR branch uses LiDAR tokens, and the fusion branch attends to multimodal tokens. To adaptively select reliable branches, SARFusion first learns a Scene Reliability Prior from multimodal context to encode global sensing conditions. Then, for each object query, a Scene-Aware Branch Router combines this scene prior with local observation evidence around the query reference point and predicts branch probabilities over the three decoding branches. The query is routed to the selected branch for final decoding and prediction. 3.2 Scene Reliability Prior The reliability of camera and LiDAR observations is highly correlated with scene-level conditions, such as weather, illumination, sensor degradation, and temporal-spatial disturbance. Therefore, we introduce a Scene Reliability Prior to provide global reliability guidance for branch routing. Given multimodal context tokens CLX_CL, we use a lightweight Transformer to aggregate scene-level information. Specifically, a learnable scene token sq_s is prepended to the multimodal tokens, and the output scene token is projected to obtain the scene reliability prior: prior=LN(MLP(Transformer([s;CL])0)),c_prior=LN (MLP (Transformer([q_s;X_CL])_0 ) ), (1) where prior∈ℝDc_prior ^D summarizes the global sensing condition of the current scene. To encourage the scene prior to encode meaningful reliability-related information, we supervise it with textual scene prompts during training. For each training scene, a prompt describing the scene condition is constructed, e.g., “A weather condition driving scene at time of day.” (2) The prompt is encoded by a text encoder into a text embedding t. We then apply a symmetric contrastive loss between the scene prior and the text embedding: ℒsrp=ℒc→t+ℒt→c.L_srp=L_c→ t+L_t→ c. (3) This supervision encourages priorc_prior to preserve scene-level reliability cues. During inference, no text prompt or scene label is required; the scene prior is directly inferred from multimodal features. Figure 2: The illustration of SRP module 3.3 Scene-Aware Branch Routing While the Scene Reliability Prior captures global sensing conditions, modality reliability can also vary across object regions [11, 22, 46]. For example, LiDAR points may be sparse for distant objects, while camera observations may be locally occluded or corrupted. Therefore, SARFusion performs branch routing at the object-query level. For the i-th object query iq_i with reference point iref=(xi,yi,zi)p_i^ref=(x_i,y_i,z_i), we project the reference point onto the camera feature plane and the LiDAR BEV plane. Around each projected location, we construct a local attention window to collect nearby modality features. The union of camera and LiDAR local tokens forms a query-specific local region Ωi _i. We use a local attention mask iM_i to restrict attention to Ωi _i: i(n)=0,n∈Ωi,−∞,n∉Ωi.M_i(n)= cases0,&n∈ _i,\\ -∞,&n∉ _i. cases (4) The local observation evidence for query iq_i is then computed by masked attention: ilocal=Softmax(i⊤D+i),e^local_i=Softmax ( q_iK D+M_i )V, (5) where K and V are projected from multimodal context tokens CLX_CL. The branch router combines local observation evidence with the global scene prior: i=[ilocal;prior],h_i=[e^local_i;c_prior], (6) and predicts branch probabilities: i=Softmax(MLP(i)),i=[πi,C,πi,L,πi,F], π_i=Softmax (MLP(h_i) ), π_i=[ _i,C, _i,L, _i,F], (7) where C, L, and F denote the camera, LiDAR, and fusion branches, respectively. The selected branch is: mi∗=argmaxm∈C,L,Fπi,m.m_i^*= _m∈\C,L,F\ _i,m. (8) Accordingly, object queries are partitioned into three groups: m=i∣mi∗=m,m∈C,L,F.Q_m=\q_i m_i^*=m\, m∈\C,L,F\. (9) Each group is decoded by its corresponding branch. This design allows different object queries in the same scene to select different reliable representations. Figure 3: The illustration of SABR module. 3.4 Training Objective Training SARFusion involves three objectives: branch-wise detection supervision, scene prior supervision, and routing supervision. First, to ensure that each branch has independent detection ability, we apply detection losses to all three branches: ℒdet=ℒdetC+ℒdetL+ℒdetF,L_det=L_det^C+L_det^L+L_det^F, (10) where each detection loss consists of classification and box regression terms. Second, the Scene Reliability Prior is supervised by the contrastive scene-prompt loss ℒsrpL_srp defined above, which encourages the prior to encode global sensing conditions. Third, we supervise the branch router with reliability-aware routing targets. During training, modality degradation or dropout is applied to simulate unreliable sensing conditions. The routing target encourages queries to select the branch corresponding to the reliable modality: the LiDAR branch when camera observations are degraded, the camera branch when LiDAR observations are unreliable, and the fusion branch when both modalities are informative. The routing loss is: ℒroute=−1N∑i=1N∑m∈C,L,Fyi,mlogπi,m,L_route=- 1N _i=1^N _m∈\C,L,F\y_i,m _i,m, (11) where yi,my_i,m is the routing target for query iq_i. The full training objective is: ℒ=λdetℒdet+λsrpℒsrp+λrouteℒroute.L= _detL_det+ _srpL_srp+ _routeL_route. (12) In practice, we adopt a staged training strategy. We first train the three decoding branches with branch-wise detection supervision, then learn the Scene Reliability Prior with prompt-based contrastive supervision, and finally optimize the router with reliability-aware routing supervision. This strategy stabilizes branch learning and prevents the router from degenerating into a fixed branch preference. 4 Experiments 4.1 Experimental Setup Dataset and metrics. We evaluate SARFusion on the nuScenes 3D object detection benchmark, which provides synchronized multi-view camera images and LiDAR point clouds for autonomous driving scenes. Following the official protocol, the dataset is divided into 700 training scenes, 150 validation scenes, and 150 test scenes. We report the official mean Average Precision (mAP) and nuScenes Detection Score (NDS). mAP measures localization and classification accuracy, while NDS further summarizes multiple detection quality factors, including translation, scale, orientation, velocity, and attribute errors. Robustness evaluation. Besides the standard clean validation and test sets, we further evaluate different methods under representative degraded sensing conditions, including snow, rain, fog, and strong sunlight. These conditions affect camera and LiDAR observations in different ways. Snow, rain, and fog may weaken image visibility and disturb point-cloud measurements, while strong sunlight mainly changes image appearance and reduces the reliability of visual cues. Unless otherwise specified, all models are trained on the original training set and directly evaluated under degraded validation conditions without condition-specific fine-tuning. This protocol is used to examine whether the fusion strategy itself can adapt to scene-dependent sensing reliability. Figure 4: Comparison of camera-LiDAR observations under degraded sensing conditions. Implementation details. SARFusion follows a multi-branch camera-LiDAR detection framework. The camera stream extracts image features from surrounding-view images, and the LiDAR stream encodes point clouds into BEV features. Based on these representations, SARFusion maintains three candidate prediction branches: a camera branch, a LiDAR branch, and a camera-LiDAR fusion branch. The Scene Reliability Prior estimates global scene-level sensing conditions, while the Scene-Aware Branch Routing module combines this prior with local query evidence to select a suitable branch for each object query. All ablation experiments are conducted on the nuScenes validation split. We adopt a staged training strategy: the detection branches are first optimized to provide valid modality-specific and fused representations; the scene prior is then learned with scene-level supervision; finally, the routing module is jointly optimized with the detection objective. 4.2 Comparison with SOTA Methods Results on the clean benchmark. Table 1 compares SARFusion with representative camera-only, LiDAR-only, and camera-LiDAR fusion methods on the standard nuScenes validation and test sets. Camera-only methods are limited by the lack of explicit 3D geometry, while LiDAR-only methods provide stronger localization due to accurate depth and structure. Multi-modal methods further improve detection by combining image semantics with LiDAR geometry. SARFusion achieves 71.1 mAP and 73.7 NDS on the validation set, and 72.5 mAP and 74.4 NDS on the test set. Compared with representative camera-LiDAR fusion methods, SARFusion obtains strong performance on clean scenes. The improvement on clean data is moderate but meaningful, since most clean samples already benefit from conventional fused representations. This indicates that the proposed scene-aware routing strategy improves robustness without sacrificing standard detection accuracy. Table 1: Comparison with representative 3D object detection methods on the nuScenes validation and test sets. All results are reported in percentage. Modality Method Validation Test mAP NDS mAP NDS Camera FCOS3D [43] 34.3 41.5 35.8 42.8 PETR [27] 37.0 44.2 39.1 45.5 BEVDet [15] – – 42.2 48.2 LiDAR SECOND [49] 52.6 63.0 52.8 63.3 CenterPoint [51] 59.6 66.8 60.3 67.3 TransFusion-L [2] 65.1 70.1 65.5 70.2 Camera + LiDAR FUTR3D [9] 64.5 68.3 – – PointAugmenting [41] – – 66.8 71.0 UVTR [24] 65.4 70.2 67.1 71.1 AutoAlignV2 [10] 67.1 71.2 68.4 72.4 TransFusion [2] 67.5 71.3 68.9 71.6 MetaBEV [13] 68.0 71.5 – – BEVFusion [30] 68.5 71.4 70.2 72.9 DeepInteraction [50] 69.9 72.6 70.8 73.4 SparseFusion [47] 70.4 72.8 72.0 73.8 CMT [48] 70.3 72.9 72.0 74.1 Camera + LiDAR SARFusion 71.1 73.7 72.5 74.4 Results under degraded conditions. Table 2 reports the robustness comparison under representative weather and lighting conditions. Compared with clean scenes, degraded scenes introduce stronger modality imbalance. Visual appearance may become unreliable under rain, fog, or strong sunlight, while LiDAR observations can become sparse or noisy in adverse weather. These corruptions make tightly coupled fusion vulnerable, since degraded observations may interfere with reliable modality-specific evidence. SARFusion achieves the best performance under all evaluated clean and degraded conditions. Compared with the strongest competing results, SARFusion improves mAP by 0.8 on clean scenes, 1.3 under snow, 3.2 under rain, 7.8 under fog, and 2.0 under strong sunlight. The advantage becomes more evident under degraded conditions, supporting our motivation that robust camera-LiDAR fusion should adapt to scene-level and object-level reliability rather than relying on a single fixed fused representation. Figure 5: Qualitative detection results under representative adverse weather conditions, including fog, snow, and rain. Table 2: Robustness comparison on the nuScenes validation set under representative weather and lighting conditions. Method Clean Snow Rain Fog Strong Sunlight FCOS3D [43] 23.9 2.0 13.0 13.5 17.2 DETR3D [45] 34.7 5.1 20.4 27.9 34.7 PointPillars [23] 27.7 27.6 27.7 24.5 23.7 CenterPoint [51] 59.3 55.9 56.1 43.8 54.2 BEVFusion [30] 68.5 62.8 66.1 54.1 64.4 TransFusion [2] 66.4 63.3 65.4 53.7 55.1 DeepInteraction [50] 69.9 62.3 66.5 54.8 64.9 FUTR3D [9] 64.2 52.7 58.4 53.2 57.7 CMT [48] 70.3 63.5 62.6 61.4 66.3 SARFusion 71.1 64.8 69.7 69.2 68.3 4.3 Ablation Study Effectiveness of each component. Table 3 studies the contribution of the main components in SARFusion. The single fusion branch baseline already achieves strong performance, since it benefits from both camera semantics and LiDAR geometry. However, directly introducing multiple branches without an effective routing mechanism leads to a significant performance drop. This shows that simply adding modality-specific branches is not sufficient. Without query-level routing, the detector cannot determine which representation should be responsible for each object, and the branch predictions are not properly coordinated. After introducing Scene-Aware Branch Routing, the performance recovers to the level of the single fusion branch baseline. This result indicates that query-level branch selection is essential for making the multi-branch design effective. Finally, adding the Scene Reliability Prior further improves both mAP and NDS, showing that global scene reliability provides useful context for object-level routing decisions. Table 3: Component ablation of SARFusion on the nuScenes validation set. Setting mAP NDS Single fusion branch 70.7 73.1 Multi-branch w/o routing 52.4 62.8 + Scene-Aware Branch Routing 70.8 73.1 + Scene Reliability Prior 71.1 73.7 Effect of scene prior supervision. Table 4 evaluates the effect of supervising the Scene Reliability Prior. Without scene-level supervision, the prior is learned only through the final detection objective. In this case, the scene representation may encode information useful for detection, but it is not explicitly encouraged to describe global sensing reliability. With scene prior supervision, SARFusion achieves higher mAP and NDS. This improvement shows that explicitly aligning the scene prior with global driving conditions makes the routing decision more stable. The prior does not replace local evidence; instead, it calibrates local query evidence with scene-level reliability context. Table 4: Effect of scene prior supervision on the nuScenes validation set. Setting mAP NDS w/o scene prior supervision 69.6 72.1 w/ scene prior supervision 71.1 73.7 Effect of routing evidence. Table 5 compares different inputs for branch routing. Using only local query evidence already provides strong performance, because it captures object-level cues such as local image visibility, point density, and feature consistency around the query reference point. This is important for autonomous driving scenes, where degradation is often spatially non-uniform. Combining local evidence with the Scene Reliability Prior gives the best result. The two types of evidence are complementary: local evidence describes the reliability of a specific object query, while the scene prior describes the global sensing condition shared by the whole frame. Their combination enables SARFusion to make object-specific routing decisions under a scene-aware context. Table 5: Ablation of routing evidence on the nuScenes validation set. Routing Evidence mAP NDS Local evidence only 70.8 73.1 Scene prior + local evidence 71.1 73.7 Effect of training strategy. Table 6 compares direct end-to-end training with the staged training strategy. Direct training gives much lower performance, indicating that joint optimization of branch specialization, scene prior learning, and routing selection is difficult from scratch. In the early training stage, the router may make unstable choices before each branch becomes a reliable detector, which further weakens branch learning and may lead to routing collapse. The staged strategy substantially improves performance. By first training the candidate branches, each branch learns a meaningful representation for detection. The scene prior is then learned as a reliability-aware condition signal. Finally, the routing module is optimized on top of already informative branches and a stable scene prior. Table 6: Effect of training strategy on the nuScenes validation set. Training Strategy mAP NDS Direct training 65.7 69.6 Staged training 71.1 73.7 4.4 Qualitative Analysis Figure 6: Branch routing visualization under different sensing conditions. Routing behavior under different scenes. To further understand how SARFusion makes routing decisions, we analyze the distribution of selected branches under different sensing conditions. As shown in Figure 6, the fusion branch dominates in clean scenes, which is consistent with the strong performance of conventional fusion methods under normal conditions. This confirms that SARFusion does not unnecessarily avoid fusion when both modalities are reliable. When one modality becomes unreliable, the routing distribution changes accordingly. Under LiDAR failure, most queries are routed to the camera branch, while under camera failure, most queries are routed to the LiDAR branch. Under adverse weather or illumination, the model does not simply switch to a single modality. Instead, it keeps a considerable portion of queries in the fusion branch while increasing the use of modality-specific branches. This behavior indicates that SARFusion learns a reliability-aware routing policy rather than a fixed modality preference. Detection examples. Figure 5 presents qualitative comparisons under representative adverse weather conditions, including rain, fog, and snow. These conditions degrade image visibility and may disturb point-cloud observations, making tightly coupled camera-LiDAR fusion vulnerable to unreliable modality features. Compared with BEVFusion and CMT, SARFusion produces more complete and accurately localized detection results across the degraded scenes. In particular, competing fusion methods may miss objects or generate less precise boxes when corrupted observations interfere with reliable modality-specific evidence. By routing object queries to camera, LiDAR, or fusion branches according to scene-level and object-level reliability, SARFusion alleviates harmful cross-modal interference while preserving useful complementary cues. These qualitative results further support the robustness of scene-aware routing fusion under adverse weather conditions. 5 Conclusions We introduced SARFusion, a scene-aware branch routing framework for robust camera-LiDAR 3D object detection. Unlike fixed fusion methods, SARFusion maintains camera, LiDAR, and fusion branches, and combines a global Scene Reliability Prior with local query evidence to select reliable sensing paths for each object. This design reduces the influence of degraded modalities while preserving complementary semantic and geometric cues. Experiments on nuScenes and corrupted driving scenarios demonstrate the effectiveness of SARFusion: it achieves 72.5 mAP and 74.4 NDS on the nuScenes test set and consistently improves robustness under snow, rain, fog, and strong sunlight. Ablations further validate the benefits of scene reliability modeling and query-level routing. References [1] F. Albreiki, S. Abughazal, J. Lahoud, R. Anwer, H. Cholakkal, and F. Khan (2022) On the robustness of 3d object detectors. External Links: 2207.10205, Link Cited by: §1. [2] X. Bai, Z. Hu, X. Zhu, Q. Huang, Y. Chen, H. Fu, and C. Tai (2022) TransFusion: robust lidar-camera fusion for 3d object detection with transformers. External Links: 2203.11496, Link Cited by: §1, §1, §1, §2.2, Table 1, Table 1, Table 2. [3] T. Beemelmanns, Q. Zhang, C. Geller, and L. Eckstein (2024) MultiCorrupt: a multi-modal robustness dataset and benchmark of lidar-camera fusion for 3d object detection. External Links: 2402.11677, Link Cited by: §1, §2.3. [4] M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide (2020) Seeing through fog without seeing fog: deep multimodal sensor fusion in unseen adverse weather. External Links: 1902.08913, Link Cited by: §1. [5] G. Brazil and X. Liu (2019) M3D-rpn: monocular 3d region proposal network for object detection. External Links: 1907.06038, Link Cited by: §2.1. [6] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom (2020) NuScenes: a multimodal dataset for autonomous driving. External Links: 1903.11027, Link Cited by: §1. [7] J. Cha, M. Joo, J. Park, S. Lee, I. Kim, and H. J. Kim (2024) Robust multimodal 3d object detection via modality-agnostic decoding and proximity-based modality ensemble. External Links: 2407.19156, Link Cited by: §2.3. [8] X. Chen, K. Kundu, Z. Zhang, H. Ma, S. Fidler, and R. Urtasun (2016) Monocular 3d object detection for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2.1. [9] X. Chen, T. Zhang, Y. Wang, Y. Wang, and H. Zhao (2023) FUTR3D: a unified sensor fusion framework for 3d detection. External Links: 2203.10642, Link Cited by: §1, §2.2, Table 1, Table 2. [10] Z. Chen, Z. Li, S. Zhang, L. Fang, Q. Jiang, and F. Zhao (2022) AutoAlignV2: deformable feature aggregation for dynamic multi-modal 3d object detection. External Links: 2207.10316, Link Cited by: §2.2, Table 1. [11] Y. Dong, C. Kang, J. Zhang, Z. Zhu, Y. Wang, X. Yang, H. Su, X. Wei, and J. Zhu (2023) Benchmarking robustness of 3d object detection to common corruptions in autonomous driving. External Links: 2303.11040, Link Cited by: §1, §1, §1, §3.3. [12] F. Drews, D. Feng, F. Faion, L. Rosenbaum, M. Ulrich, and C. Gläser (2022) DeepFusion: a robust and modular 3d object detector for lidars, cameras and radars. External Links: 2209.12729, Link Cited by: §2.3. [13] C. Ge, J. Chen, E. Xie, Z. Wang, L. Hong, H. Lu, Z. Li, and P. Luo (2023) MetaBEV: solving sensor failures for bev detection and map segmentation. External Links: 2304.09801, Link Cited by: §2.3, Table 1. [14] M. Hahner, C. Sakaridis, D. Dai, and L. V. Gool (2021) Fog simulation on real lidar point clouds for 3d object detection in adverse weather. External Links: 2108.05249, Link Cited by: §1. [15] J. Huang, G. Huang, Z. Zhu, Y. Ye, and D. Du (2022) BEVDet: high-performance multi-camera 3d object detection in bird-eye-view. External Links: 2112.11790, Link Cited by: §2.1, Table 1. [16] Y. Huang, K. Yu, Q. Guo, F. Juefei-Xu, X. Jia, T. Li, G. Pu, and Y. Liu (2024) Improving robustness of lidar-camera fusion model against weather corruption from fusion strategy perspective. External Links: 2402.02738, Link Cited by: §1, §2.3. [17] Y. Ji, C. Chi, H. Tan, Y. Zhao, H. Lyu, E. Zhou, P. Wang, Z. Wang, X. Zheng, and S. Zhang (2026) From ill-posed to well-posed: a decomposition framework for reliable robot learning with foundation models. IEEE Transactions on Robotics. Cited by: §2.3. [18] Y. Ji, Y. Liu, Z. Zhang, Z. Zhang, Y. Zhao, X. Hao, G. Zhou, X. Zhang, and X. Zheng (2025) Enhancing adversarial robustness of vision-language models through low-rank adaptation. In Proceedings of the 2025 International Conference on Multimedia Retrieval, p. 550–559. Cited by: §2.3. [19] Y. Ji, Y. Liu, H. Tan, X. Huang, F. Huang, Y. Xu, C. Chi, Y. Zhao, H. Lyu, P. Co, et al. (2026) PRM-as-a-judge: a dense evaluation paradigm for fine-grained robotic auditing. arXiv preprint arXiv:2603.21669. Cited by: §2.3. [20] Y. Ji, H. Tan, C. Chi, Y. Xu, Y. Zhao, E. Zhou, H. Lyu, P. Wang, Z. Wang, S. Zhang, et al. (2025) MathSticks: a benchmark for visual symbolic compositional reasoning with matchstick puzzles. arXiv preprint arXiv:2510.00483. Cited by: §2.3. [21] Y. Ji, Y. Wang, Y. Liu, X. Hao, Y. Liu, Y. Zhao, H. Lyu, and X. Zheng (2025) VisualTrans: a benchmark for real-world visual transformation reasoning. arXiv preprint arXiv:2508.04043. Cited by: §2.3. [22] L. Kong, Y. Liu, X. Li, R. Chen, W. Zhang, J. Ren, L. Pan, K. Chen, and Z. Liu (2023) Robo3D: towards robust and reliable 3d perception against corruptions. External Links: 2303.17597, Link Cited by: §1, §2.3, §3.3. [23] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom (2019) PointPillars: fast encoders for object detection from point clouds. External Links: 1812.05784, Link Cited by: §2.1, Table 2. [24] Y. Li, Y. Chen, X. Qi, Z. Li, J. Sun, and J. Jia (2022) Unifying voxel-based representation with transformer for 3d object detection. External Links: 2206.00630, Link Cited by: §2.2, Table 1. [25] Y. Li, Z. Ge, G. Yu, J. Yang, Z. Wang, Y. Shi, J. Sun, and Z. Li (2022) BEVDepth: acquisition of reliable depth for multi-view 3d object detection. External Links: 2206.10092, Link Cited by: §2.1. [26] Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Yu, and J. Dai (2022) BEVFormer: learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. External Links: 2203.17270, Link Cited by: §2.1. [27] Y. Liu, T. Wang, X. Zhang, and J. Sun (2022) PETR: position embedding transformation for multi-view 3d object detection. External Links: 2203.05625, Link Cited by: §1, §2.1, Table 1. [28] Y. Liu, Y. Shen, R. Chen, J. Zhao, Y. Tian, Y. Zhang, T. Long, Z. Yin, Y. Wang, Z. Qin, et al. (2026) PRM-as-a-judge 1.5: a toolkit for robot process assessment. arXiv preprint arXiv:2608.14284. Cited by: §2.3. [29] Z. Liu, Z. Wu, and R. Tóth (2020) SMOKE: single-stage monocular 3d object detection via keypoint estimation. External Links: 2002.10111, Link Cited by: §2.1. [30] Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. Rus, and S. Han (2024) BEVFusion: multi-task multi-sensor fusion with unified bird’s-eye view representation. External Links: 2205.13542, Link Cited by: §1, §2.2, Table 1, Table 2. [31] K. Park, Y. Kim, D. Kim, and J. W. Choi (2025) Resilient sensor fusion under adverse sensor failures via multi-modal expert fusion. External Links: 2503.19776, Link Cited by: §2.3. [32] M. Pitropov, D. E. Garcia, J. Rebello, M. Smart, C. Wang, K. Czarnecki, and S. Waslander (2020) Canadian adverse driving conditions dataset. The International Journal of Robotics Research 40 (4-5), p. 681–690. External Links: ISSN 1741-3176, Link, Document Cited by: §1. [33] C. R. Qi, H. Su, K. Mo, and L. J. Guibas (2017) PointNet: deep learning on point sets for 3d classification and segmentation. External Links: 1612.00593, Link Cited by: §2.1. [34] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever (2021) Learning transferable visual models from natural language supervision. External Links: 2103.00020, Link Cited by: §1. [35] R. Sadeghian, N. Hooshyaripour, C. Joslin, and W. Lee (2025) Reliability-driven lidar-camera fusion for robust 3d object detection. External Links: 2502.01856, Link Cited by: §2.3. [36] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li (2021) PV-rcnn: point-voxel feature set abstraction for 3d object detection. External Links: 1912.13192, Link Cited by: §2.1. [37] S. Shi, X. Wang, and H. Li (2019) PointRCNN: 3d object proposal generation and detection from point cloud. External Links: 1812.04244, Link Cited by: §2.1. [38] S. Sural, N. Sahu, and R. Rajkumar (2024) ContextualFusion: context-based multi-sensor fusion for 3d object detection in adverse operating conditions. External Links: 2404.14780, Link Cited by: §2.3. [39] Y. Tian, Y. Ji, X. Zheng, Z. Qin, Y. Wang, X. Zheng, Y. Liu, S. Bai, Z. Li, L. Wang, et al. (2026) Spatial intelligence from a cognitive map perspective: a survey. Cited by: §2.3. [40] S. Vora, A. H. Lang, B. Helou, and O. Beijbom (2020) PointPainting: sequential fusion for 3d object detection. External Links: 1911.10150, Link Cited by: §2.2. [41] C. Wang, C. Ma, M. Zhu, and X. Yang (2021) PointAugmenting: cross-modal augmentation for 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 11794–11803. Cited by: §2.2, Table 1. [42] H. Wang, H. Tang, S. Shi, A. Li, Z. Li, B. Schiele, and L. Wang (2023) UniTR: a unified and efficient multi-modal transformer for bird’s-eye-view representation. External Links: 2308.07732, Link Cited by: §2.2, §2.3. [43] T. Wang, X. Zhu, J. Pang, and D. Lin (2021) FCOS3D: fully convolutional one-stage monocular 3d object detection. External Links: 2104.10956, Link Cited by: §2.1, Table 1, Table 2. [44] Y. Wang, Y. Ji, M. Cao, Y. Shen, R. Xiao, H. Lyu, S. Xie, E. Liu, K. Tian, T. Long, et al. (2026) Orca: the world is in your mind. arXiv preprint arXiv:2606.30534. Cited by: §2.3. [45] Y. Wang, V. Guizilini, T. Zhang, Y. Wang, H. Zhao, and J. Solomon (2021) DETR3D: 3d object detection from multi-view images via 3d-to-2d queries. External Links: 2110.06922, Link Cited by: §1, §2.1, Table 2. [46] S. Xie, L. Kong, W. Zhang, J. Ren, L. Pan, K. Chen, and Z. Liu (2023) RoboBEV: towards robust bird’s eye view perception under corruptions. External Links: 2304.06719, Link Cited by: §1, §2.3, §3.3. [47] Y. Xie, C. Xu, M. Rakotosaona, P. Rim, F. Tombari, K. Keutzer, M. Tomizuka, and W. Zhan (2023) SparseFusion: fusing multi-modal sparse representations for multi-sensor 3d object detection. External Links: 2304.14340, Link Cited by: §2.2, Table 1. [48] J. Yan, Y. Liu, J. Sun, F. Jia, S. Li, T. Wang, and X. Zhang (2023) Cross modal transformer: towards fast and robust 3d object detection. External Links: 2301.01283, Link Cited by: §1, §2.2, §2.3, Table 1, Table 2. [49] Y. Yan, Y. Mao, and B. Li (2018) SECOND: sparsely embedded convolutional detection. Sensors 18 (10). External Links: Link, ISSN 1424-8220, Document Cited by: §2.1, Table 1. [50] Z. Yang, J. Chen, Z. Miao, W. Li, X. Zhu, and L. Zhang (2022) DeepInteraction: 3d object detection via modality interaction. External Links: 2208.11112, Link Cited by: §1, §2.2, Table 1, Table 2. [51] T. Yin, X. Zhou, and P. Krähenbühl (2021) Center-based 3d object detection and tracking. External Links: 2006.11275, Link Cited by: §2.1, Table 1, Table 2. [52] T. Yin, X. Zhou, and P. Krähenbühl (2021) Multimodal virtual point 3d detection. External Links: 2111.06881, Link Cited by: §2.2. [53] Y. Zhou and O. Tuzel (2018) Voxelnet: end-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 4490–4499. Cited by: §2.1.