Paper deep dive
The Operator Mismatch Problem: Deploying BEV Perception with Portable GPU Compute
Rohit Verma, Anand V Bodas
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 10/10/2026, 1:48:52 AM
Summary
The paper introduces BEVPIPE, a framework designed to solve the 'operator mismatch problem' in deploying Bird's-Eye-View (BEV) perception models for autonomous driving. Standard inference runtimes handle dense operations well but fail to support sparse 3D convolutions and geometric scatter operations efficiently, often locking deployments to CUDA and PyTorch. BEVPIPE partitions the model into runtime-managed dense subgraphs and external portable GPU operator extensions (voxelizer, sparse encoder, BEV projector) using OpenCL/SYCL. This approach achieves a 19.5x speedup over conventional methods while maintaining 98.5% of the reference mean Average Precision (mAP) and enabling portability across different GPU hardware vendors.
Entities (15)
Relation Signals (12)
BEVPIPE → achievesspeedup → 19.5x
confidence 95% · BEVPIPE achieves a 19.5x end-to-end speedup over conventional deployments
BEVPIPE → containscomponent → Voxelizer
confidence 95% · three external operator extensions (voxelizer, sparse encoder, BEV projector)
BEVPIPE → containscomponent → BEV Projector
confidence 95% · three external operator extensions (voxelizer, sparse encoder, BEV projector)
BEVPIPE → containscomponent → Sparse Encoder
confidence 95% · three external operator extensions (voxelizer, sparse encoder, BEV projector)
BEVPIPE → retainsaccuracy → 98.5% mAP
confidence 95% · while retaining 98.5% of reference mAP
BEVPIPE → solves → Operator Mismatch Problem
confidence 95% · We present BEVPIPE... to solve the operator mismatch problem
BEVPIPE → uses → OpenCL
confidence 95% · BEVPIPE partitions the model into runtime-managed dense subgraphs and three external operator extensions... using portable GPU compute APIs
BEVPIPE → uses → SYCL
confidence 95% · Our experiments demonstrate... generalizes beyond OpenCL to SYCL
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern autonomous driving systems rely on bird's-eye-view (BEV) perception models that fuse camera and LiDAR inputs to detect objects in 3D space. These models are accurate, but they cannot be deployed through standard inference runtimes. The reason is an operator mismatch between dense convolutions (which runtimes handle well), sparse 3D convolutions (which runtimes cannot represent), and geometric scatter operations (which runtimes have no vocabulary for). Today, every sparse convolution library is CUDA-only and PyTorch-coupled, locking BEV deployment to a single vendor's hardware and a single execution framework. We present BEVPIPE, a framework for deploying multimodal BEV perception pipelines using portable GPU compute APIs and integrating them with production inference runtimes. BEVPIPE partitions the model into runtime-managed dense subgraphs and three external operator extensions (voxelizer, sparse encoder, BEV projector), connected through a shared GPU memory space. BEVPIPE achieves a 19.5x end-to-end speedup over conventional deployments while retaining 98.5% of reference mAP. We also showcase that BEVPIPE is portable across different GPU backends.
Tags
Links
- Source: https://arxiv.org/abs/2610.11504v1
- Canonical: https://arxiv.org/abs/2610.11504v1
Trouble viewing inline? Open PDF directly →
Full Text
43,355 characters extracted from source content.
Expand or collapse full text
The Operator Mismatch Problem: Deploying BEV Perception with Portable GPU Compute Rohit Verma Anand V Bodas Abstract Modern autonomous driving systems rely on bird’s-eye-view (BEV) perception models that fuse camera and LiDAR inputs to detect objects in 3D space. These models are accurate, but they cannot be deployed through standard inference runtimes. The reason is an operator mismatch between dense convolutions (which runtimes handle well), sparse 3D convolutions (which runtimes cannot represent), and geometric scatter operations (which runtimes have no vocabulary for). Today, every sparse convolution library is CUDA-only and PyTorch-coupled, locking BEV deployment to a single vendor’s hardware and a single execution framework. We present BEVPIPE, a framework for deploying multi-modal BEV perception pipeline using portable GPU compute APIs, integrated with production inference runtime. BEVPIPE partitions the model into runtime-managed dense subgraphs and three external operator extensions (voxelizer, sparse encoder, BEV projector), connected through a shared GPU memory space. BEVPIPE achieves a 19.5×19.5× end-to-end speedup over conventional deployments while retaining 98.5%98.5\% of reference mAP. We also showcase that BEVPIPE is portable across different GPU backends. Intel rohit1.verma@intel.com, anand.v.bodas@intel.com Introduction Autonomous vehicles perceive the world through sensors and cameras that see color and texture, and LiDAR scanners that measure precise 3D geometry. The most accurate perception systems fuse both modalities into a unified bird’s-eye view (BEV) representation, where objects are detected in the coordinate frame relevant to planning (Liu et al. 2023; Liang et al. 2022; Li et al. 2022). Although training these models is a solved problem, deploying them is not. The difficulty lies in a mismatch between what these models compute and what inference runtimes understand. A BEV perception pipeline interleaves three fundamentally different kinds of computation: Dense operations consisting of image convolutions, transformers, and fully-connected layers, that operate on regular tensors with static shapes; Sparse 3D convolutions, which are applied to the irregular set of occupied voxels in a point cloud, where the number of active locations and their connectivity change with every input; and Geometric scatter operations which include voxelization (hashing points into bins) and BEV projection (scattering depth-weighted features into a grid), that involve hash tables, atomic accumulation, and coordinate-dependent branching. When a runtime encounters an operator it cannot represent, it has three options: (a) fall back to a slow CPU reference implementation, (b) write a custom kernel plugin, or (c) reject the model entirely (NVIDIA Corporation 2024; Intel Corporation 2024). The existence of entire libraries such as SpConv (Contributors 2022), TorchSparse (Tang et al. 2022; Tang et al. 2023), MinkowskiEngine (Choy et al. 2019), built outside the inference stack is direct evidence that runtimes cannot handle this class of computation.11 1 Recent TensorRT versions support dynamic shapes via INetworkDefinition profiles, but tensors whose index sets change at runtime, as in sparse convolution, remain outside the scope of static graph compilation. This operator mismatch has a concrete consequence: state-of-the-art BEV perception models cannot be deployed through standard inference pathways without substantial custom engineering. And that engineering, today, is locked to a single hardware vendor. Every existing sparse convolution library is CUDA-only and PyTorch-coupled (Contributors 2022; Tang et al. 2022; Choy et al. 2019). BEVFusion’s own deployment uses TensorRT with custom CUDA plugins (Liu et al. 2023). If one wants to run BEV perception on Intel, AMD, or mobile GPUs, or through any runtime other than TensorRT, they are on their own. In this paper, we answer the question: Can a complete multimodal BEV perception pipeline be deployed efficiently using portable GPU compute APIs (OpenCL/SYCL) without CUDA and integrated with a production inference runtime? We present BEVPIPE, a system that does this by partitioning the pipeline between the runtime (which handles dense operations) and three portable OpenCL GPU extensions (which handle the rest). Answering this question involves addressing the below research questions. Q1. How should a heterogeneous perception pipeline be split between a runtime’s native operators and external custom kernels? Q2. How does the granularity of that split affect overhead? Q3. Where does execution time actually go in each operator class? The key contributions of this work are summarized below which address these questions. • We formalize the operator-mismatch problem as a partition boundary placement problem and demonstrate which partitioning approach works effectively. • We design BEVPIPE, an end-to-end BEV perception system suitable for portable GPU compute APIs and integrated with a production inference runtime, enabling deployment on Intel, AMD, and other hardwares without modification. • Our experiments demonstrate the effectiveness of the proposed system in terms of both performance and accuracy. We also showcase that the pipeline partitioning methodology generalizes beyond OpenCL to SYCL, enabling a single pipeline to target multiple GPU backends. Related Work The operator mismatch problem has been addressed from several directions, including specialized sparse convolution libraries, inference runtime extensions, compiler-based code generation, and direct BEV deployment efforts. Each addresses part of the problem but leaves a gap that motivates our work. Sparse Convolution Libraries Sparse 3D convolution (Graham 2015) and its submanifold variant (Graham and van der Maaten 2018) are implemented by three mature libraries: SpConv (Contributors 2022) (fused gather–GEMM–scatter CUDA kernels), TorchSparse/++ (Tang et al. 2022; Tang et al. 2023) (locality-driven reordering and auto-tuned kernel selection), and MinkowskiEngine (Choy et al. 2019) (generalized sparse convolution). However, all three are CUDA-only and PyTorch-coupled, cannot be composed with a runtime’s graph optimizer, memory planner, or INT8 pipeline, and lock deployment to NVIDIA hardware. For production on Intel, AMD, or Qualcomm GPUs, they are unavailable. Inference Runtimes and Compilers OpenVINO (Intel Corporation 2024), TensorRT (NVIDIA Corporation 2024), ONNX Runtime (Microsoft 2021), and TVM (Chen et al. 2018) optimize dense models via fusion, memory planning, and hardware-targeted codegen, and all expose custom-operator extension APIs. However, the runtime treats custom operators as opaque black boxes (Chen et al. 2018; Zheng et al. 2020): graph fusion stops at extension boundaries and memory planning cannot span runtime- and extension-managed buffers. Thus how and at what granularity we partition has first-order impact, a trade-off prior works have not analyzed. Compiler stacks (Ansor (Zheng et al. 2020), Triton (Tillet et al. 2019)) can generate novel-operator code but do not integrate the INT8 calibration, memory pools, and accelerated dense kernels that vendor runtimes provide, forcing reimplementation of existing functionality. BEV Model Deployment The closest prior work is BEVFusion’s deployments (Liu et al. 2023; Liang et al. 2022), using custom CUDA plugins for sparse convolution and BEV projection. MMDeploy offers a similar NVIDIA-stack pipeline (MMDeploy Contributors 2021). These show BEV deployment can work on one runtime and one hardware family, but do not analyze the methodology such as partitioning criteria, boundary cost trade-offs, operator-heterogeneity/scheduling interaction, nor the portability question of whether the approach holds when CUDA is unavailable? Geometric Operations Lift-Splat-Shoot (Philion and Fidler 2020) established depth-based camera-to-BEV projection, with efficient CUDA kernels (Liu et al. 2023), geometry caching (Huang and Huang 2022), and attention-based alternatives (Li et al. 2022); voxelization uses GPU hash tables with atomic accumulation (Yan et al. 2018; Choy et al. 2019). Both involve irregular access and atomic contention, poor fits for dense runtimes regardless of vendor. What a Solution Needs Surveying these state-of-the-art approaches, we identify three properties that a successful deployment approach must have: 1. Portability: no single platform dependency; must work through portable GPU compute APIs (OpenCL, SYCL) that target multiple hardware vendors. 2. Runtime integration: sparse and geometric kernels must compose with the runtime’s dense optimization pipeline, not bypass it entirely. 3. Partition-aware design: the boundary between runtime-native and custom operators must be explicitly managed to minimize synchronization and format conversion overhead. Next, we provide an overview of BEVPIPE, which is designed to satisfy all three. System Overview In this section, we describe the BEV perception pipeline, what a deployment system must handle, and a broad overview of the BEVPIPE architecture. Figure 1: System architecture of BEVPIPE. BEV Perception Model A BEV perception model’s computational graph G=(V,E)G=(V,E) contains operators viv_i from three distinct classes: Dense (D): Regular tensor operations with static shapes and predictable memory access, such as convolutions, batch normalization, activation functions, and linear projections. These are the native domains of every inference runtime. Sparse (S): Operations on irregularly-indexed data structures with dynamic shapes determined at runtime. The canonical example is submanifold sparse convolution (Graham and van der Maaten 2018), where the set of active voxels N and their neighbor connectivity change with every point cloud. Geometric (G): Coordinate-dependent transformations involving hash table construction, spatial indexing, and atomic scatter-gather. Voxelization (Yan et al. 2018) and BEV pooling (Philion and Fidler 2020) are representative examples. A critical observation is that operator classes cluster by pipeline branch: the camera branch is predominantly dense, the LiDAR branch is predominantly sparse (after an initial geometric voxelization), and the fusion stage mixes a geometric BEV projection with dense processing. This clustering is consistent across BEV architectures. The Partition Problem The deployment problem is to split G into subgraphs G1R,G2X,G3R,…\G_1^R,G_2^X,G_3^R,…\, where GiRG_i^R is managed by the runtime and GjXG_j^X is managed by external custom kernels, such that: 1. All operators in GiRG_i^R are dense (or have runtime-native support). 2. Operators in GjXG_j^X are sparse or geometric. 3. Total execution time, including partition boundary overhead, is minimized. The last constraint is the minimization of total execution time, including partition boundary overhead. Each boundary between runtime and extension incurs: Cboundary=Csync+Cformat+Cfusion_lossC_boundary=C_sync+C_format+C_fusion\_loss (1) where CsyncC_sync is the command queue synchronization cost, CformatC_format is the memory layout conversion cost, and Cfusion_lossC_fusion\_loss is the performance loss because the runtime’s graph optimizer cannot fuse across the boundary. For a sub-pipeline of L layers implemented externally, fine-grained partitioning (one extension per layer) incurs (L⋅Cboundary)(L· C_boundary). Coarse partitioning (one extension for the whole sub-pipeline) incurs just CboundaryC_boundary. With L=21L=21 for a typical sparse encoder, this difference is substantial. We identify four criteria that determine where to place boundaries: (i) Can the operator be expressed in the runtime’s Intermediate Representation (Intel Corporation 2024)? (i) Even if representable, does the runtime generate well optimized code? (i) The cost must be amortized over the computation saved by external execution. (iv) Related non-dense operators that would form separate extensions can often be fused into one. Architecture of BEVPIPE BEVPIPE implements the end-to-end BEV Perception pipeline as shown in Figure 1. The pipeline is partitioned into runtime-managed dense subgraphs (green) and external custom extensions (orange). Dashed red lines mark partition boundaries B1B_1–B4B_4. All operators execute within a shared GPU context, enabling zero-copy tensor passing at boundaries. The LiDAR branch contains two external extensions (Voxelizer, Sparse Encoder); the camera branch contains one (BEV Projector). The fusion stage and detection head are entirely runtime-managed. BEVPIPE: Design and Implementation This section presents the design and implementation of BEVPIPE. The architectural principles generalize to any runtime supporting custom operator extensions (e.g., TensorRT plugins (NVIDIA Corporation 2024), ONNX Runtime custom ops (Microsoft 2021)) and any GPU compute API (CUDA, SYCL, Vulkan Compute). We organize the section around four concerns: how the model is partitioned between the runtime and custom extensions, how sparse 3D convolution is implemented portably, how geometric operators handle irregular scatter patterns, and what system-level trade-offs govern the composition of these components. Partition Granularity The choice of where to place boundaries between runtime-managed and extension-managed execution has first-order performance impact. Each boundary incurs a cost CboundaryC_boundary comprising synchronization, format conversion, and lost fusion opportunities (Equation 1). We evaluate three granularity strategies, Fine-grained partitioning registers each non-dense layer as an individual custom operator. For the SECOND (Yan et al. 2018) sparse encoder with L=21L=21 convolutional layers, this produces 21 partition boundaries, each requiring a command-queue barrier and potentially a memory format conversion between the extension’s internal representation and the runtime’s expected layout. Medium-grained partitioning groups operators by their computational class. All sparse convolutions form one extension, all geometric operations form another. This reduces the boundary count to O(1)O(1) per class (typically 3 boundaries for the SECOND sparse encoder) but prevents optimizations that span operator types. For example, the sparse encoder cannot fuse its final layer with the sparse-to-dense scatter because they reside in separate extensions. Coarse-grained partitioning assigns one extension per pipeline branch. The sparse encoder becomes a single monolithic extension encompassing all convolutional layers, batch normalization, residual connections, and the sparse-to-dense BEV conversion. This introduces only one entry boundary and one exit boundary per branch. We adopt coarse-grained partitioning because it enables four internal optimizations that are impossible when the runtime controls intermediate buffers between layers: 1. Feature tensors remain in GPU memory across all layers in the boundary without host-side materialization or runtime-driven memory reallocation. 2. Hash tables and neighbor index maps, which are expensive to compute, are built once at the start of the extension invocation and reused across all layers sharing the same voxel topology. 3. Two persistent GPU buffers alternate as source and destination across consecutive layers, eliminating per-layer allocation and deallocation overhead. 4. Weights for all layers within a boundary are pre-converted to device-optimal format during the first invocation and retained in GPU memory, avoiding per-invocation format conversion that fine-grained extensions would require. Sparse 3D Convolution Engine The sparse convolution engine is responsible for processing the LiDAR voxel features through multiple layers of 3D convolution on an irregular grid. We describe the mathematical formulation and the engine design next. Fused Sparse Convolution Standard sparse 3D convolution (Graham 2015; Graham and van der Maaten 2018) computes output features only at locations where input voxels exist (submanifold-preserving property). For each active voxel i with input feature vector i∈ℝinf_i ^C_in, the fused sparse convolution with batch normalization and ReLU activation produces an output feature i∈ℝoutg_i ^C_out: i=ReLU(γ⋅∑k=1K3k⋅nk(i)−μσ2+ϵ+β)g_i=ReLU\! (γ· _k=1^K^3W_k·f_n_k(i)-μ σ^2+ε+β ) (2) Here, ℝnR^n depicts an n-dimensional vector of real numbers and inC_in and outC_out are the number of input and output channels. K=3K=3 is the kernel size (yielding K3=27K^3=27 positions in the 3×3×33× 3× 3 neighborhood), k∈ℝout×inW_k ^C_out×C_in is the weight matrix for the k-th kernel position, and nk(i)n_k(i) is the neighbor mapping function that returns the index of voxel i’s k-th spatial neighbor (or −1-1 if that neighbor is unoccupied). The batch normalization parameters γ (scale), β (shift), μ (running mean), and σ2σ^2 (running variance) are frozen at inference time and fused into the convolution output, eliminating the need for a separate normalization pass. The constant ϵε (typically 10−510^-5) prevents division by zero. Fusing batch normalization and ReLU activation into the convolution kernel eliminates two intermediate tensor materializations per layer, reducing memory traffic by 2⋅N⋅out2· N·C_out elements per layer (where N denotes the number of active voxels at that layer). Locality-Aware Voxel Ordering Sparse convolution processes voxels in the order they appear in the compact feature array. When voxels are stored in arbitrary order (e.g., insertion order during voxelization), spatially adjacent voxels may reside at distant memory addresses, causing poor cache utilization when their neighborhoods are accessed. We reorder voxels using a Morton Z-curve (space-filling curve) (Morton 1966) prior to convolution. Given a voxel’s integer grid coordinates (x,y,z)(x,y,z), its Morton code m is computed by bit-interleaving: m(x,y,z)=spread(x)|(spread(y)≪1)|(spread(z)≪2)m(x,y,z)=spread(x)\,|\,(spread(y) 1)\,|\,(spread(z) 2) (3) where spread(⋅)spread(·) inserts two zero bits between each bit of the input coordinate, and || denotes bitwise OR. Sorting voxels by ascending Morton code ensures that spatially proximate voxels occupy nearby positions in the linear array, improving spatial locality for neighbor feature loads. Reordering is applied after each strided convolution, since downsampling produces a new set of output coordinates with a different spatial distribution. Strided Sparse Convolution While submanifold convolutions preserve the input voxel topology, strided convolutions downsample the grid such that the output voxel set occupies a coarser resolution and generally contains fewer active positions. This capability is essential for the SECOND encoder’s multi-scale architecture, where each stage halves the spatial resolution while doubling the channel count (Yan et al. 2018). Implementing strided convolution requires two additional steps beyond the submanifold case: 1. Output coordinate computation. For each input voxel at position (x,y,z)(x,y,z), the corresponding output position is (⌊x/s⌋,⌊y/s⌋,⌊z/s⌋)( x/s , y/s , z/s ) where s is the stride. Duplicate output positions must be deduplicated to produce a compact output set. 2. Strided neighbor map. A new neighbor map must be precomputed that encodes, for each output voxel, which input voxels fall within its K3K^3 receptive field at the original (fine) resolution. We implement a gather-based approach where a preprocessing kernel executed on the GPU builds the complete strided neighbor map in a single pass, avoiding per-voxel hash table lookups during the subsequent convolution loop. This map is stored in a persistent buffer and reused for all samples. Geometric Operators Both geometric operators, point cloud voxelization and camera-to-BEV projection, perform irregular scatter operations where multiple source elements contribute to the same destination cell, requiring atomic accumulation. Both operators have data-dependent access patterns that prevent the inference runtime from generating efficient code, necessitating custom GPU kernel implementations. Point Cloud Voxelization Voxelization transforms an unstructured LiDAR point cloud of NptsN_pts points into a compact tensor of M voxels (M≪NptsM N_pts), where each voxel stores the mean of the feature vectors of all points falling within its spatial extent. We implement voxelization as a two-phase GPU pipeline: Phase 1: Atomic scatter. Each GPU work-item processes one point, computes its voxel bin index via a hash table, and atomically accumulates the point’s features into the corresponding bin. Direct floating-point atomic addition is not efficiently supported on all target hardware. Hence, we employ fixed-point integer atomics where each floating-point feature value f is multiplied by a scale factor S (typically S=216S=2^16) and rounded to a 32-bit integer before accumulation using hardware-native atomic add. A per-bin atomic counter simultaneously tracks the number of contributing points. This approach achieves significantly higher throughput than compare-and-swap (CAS) loops required for floating-point atomics, at the cost of introducing a quantization error bounded by 1/S1/S per accumulation. Phase 2: Compaction and normalization. A second kernel sequentially scans the hash table, identifies non-empty bins (M out of the table’s fixed capacity), divides each bin’s accumulated integer sum by its point count and the scale factor S to recover the mean feature in floating-point, and writes the result into a compact output tensor indexed from 00 to M−1M-1. Camera-to-BEV Projection The camera-to-BEV projection implements the Lift-Splat-Shoot (LSS) (Philion and Fidler 2020) geometric transformation where for each pixel in each of the 6 surround-view cameras, the depth estimation network produces a discrete depth distribution. We observe that the softmax-normalized depth distribution is typically highly peaked, i.e., for most pixels, the vast majority of probability mass concentrates in 2–5 bins (out of 59). We exploit this sparsity by introducing a threshold parameter τdepth _depth where for each pixel, depth bin d is processed only if softmax()d≥τdepthsoftmax(d)_d≥ _depth. Bins falling below the threshold contribute negligible feature weight and are skipped entirely. System-Level Design Decisions Next we address two cross-cutting system-level concerns that arise from composing heterogeneous operator types into a unified pipeline. Command Queue Scheduling The inference runtime and our custom extensions submit GPU work to the same physical device. To avoid serialization while maintaining correctness, we adopt a separate command queue architecture where each extension maintains its own command queue with independent dispatch cadence. Synchronization between the runtime’s queue and the extension queues occurs only at partition boundaries, through explicit barrier events. This design ensures that internal kernel sequences within an extension execute without external interference, while cross-boundary ordering is guaranteed. CPU–GPU Computation Overlap The sparse encoder’s execution naturally decomposes into CPU-bound phases (hash table construction, coordinate generation) and GPU-bound phases (convolution kernel execution). These phases can be overlapped. After submitting the GPU convolution kernels for stage s, the CPU immediately begins constructing the neighbor data structures for stage s+1s+1, synchronizing only when the GPU results from stage s are needed as input to the stage s+1s+1 convolution. Evaluation This section evaluates BEVPIPE along four dimensions: (1) operator-level profiling to identify where execution time is spent, (2) end-to-end pipeline latency along with the effectiveness of the partition granularity, (3) portability across different GPU backends, and (4) ablation studies to assess the impact of design decisions. Experimental Setup We describe the model configuration, hardware platform, and software environment, dataset, and evaluation protocol used throughout this section. Model Configuration We evaluate on BEVFusion (Liu et al. 2023), a representative multi-modal 3D object detection model. The LiDAR branch uses the SECOND (Yan et al. 2018) sparse encoder comprising 21 convolutional layers The voxel size is 0.075×0.075×0.20.075× 0.075× 0.2 m (width × length × height), and the detection range spans [−54m,54m][-54m,54m] in both the x- and y-axes. All experiments run on an Intel processor with an integrated GPU with 96 execution units, shared DDR5 memory subsystem. The inference runtime is OpenVINO 2026.x with its GPU plugin. The camera backbone uses INT8 quantization, while the fusion network and detection head use FP16 precision. Custom operator extensions (voxelizer, sparse encoder, BEV projector) are implemented in OpenCL 2.0 and registered as monolithic runtime extensions. Dataset and Evaluation Protocol We evaluate on an 81-sample subset of the nuScenes (Caesar et al. 2020) validation split. Each sample consists of a synchronized set of 6 surround-view camera images and one LiDAR point cloud sweep (∼70,000 70,000 points, yielding ∼20,000 20,000–26,00026,000 voxels after voxelization). Latency measurement. We execute 84 consecutive inference passes. The first 3 are discarded as warm-up to allow GPU resource initialization and JIT compilation to stabilize, and latency is reported as the mean wall-clock time over the remaining 81 steady-state samples. Accuracy measurement. We compute the official nuScenes mean Average Precision (mAP), defined as the mean of per-class Average Precision values computed at bird’s-eye-view center-distance thresholds of 0.5, 1.0, 2.0, and 4.0 meters. Competing Approaches We compare BEVPIPE against a conventional deployment baseline and against alternative partitioning strategies. The conventional deployment baseline (Conventional henceforth) reflects the current state of practice for non-CUDA BEV inference where dense operators execute on the GPU via the runtime’s native kernels, while sparse and geometric operators fall back to host-side CPU execution. This deployment strategy is followed by available non-CUDA tools, for e.g., ONNX Runtime’s CPU fallback for unregistered ops (Microsoft 2021), TVM’s OpenCL backend (Chen et al. 2018), and off-the-shelf OpenVINO (Intel Corporation 2024) without extension registration. The remaining three configurations all execute the same GPU kernels and differ only in the number of partitions. Fine granularity leads to 29 boundaries in the complete pipeline, medium granularity leads to 7 boundaries, and coarse granularity leads to just 4 boundaries. The coarse-grained strategy is the one adopted by BEVPIPE. Operator-Level Profiling To understand where computation time is spent, we profile each operator class independently. We define three classes following the earlier taxonomy, dense operators handled natively by the runtime, sparse operators in the custom sparse encoder extension, and geometric operators (voxelization, BEV projection) in the remaining extensions. Dense Operators Dense operators comprise the camera backbone, depth estimation network, feature pyramid neck, BEV fusion layer, and detection head. These are compiled and scheduled entirely by the OpenVINO GPU plugin. Table 1 reports aggregated timings. The camera branch averages 19.87%19.87\% of the end-to-end pipeline, comprising the INT8 backbone and BEV projection. The fusion and detection stages together average 23.65%23.65\%. In total, runtime-managed dense operators account for 43.52%43.52\% of the end-to-end pipeline latency. The LiDAR branch accounts for the remaining 56.48%56.48\%, driven by CPU/GPU synchronisation overhead in the sparse encoder. Table 1: Dense operator latency breakdown. Values are means over 81 steady-state samples on the 96-EU integrated GPU. “% Pipeline” is the fraction of end-to-end latency. Stage % Pipeline Camera branch (INT8 backbone + BEV proj.) 19.87% Fusion + detection (FP16) 23.65% Total runtime-managed 43.52% Sparse Operators The sparse encoder is the most complex of our three custom extensions. To understand its internal time allocation, we instrument it with per-phase timing. We decompose execution into four phases: precomputation, which builds hash tables and neighbor maps on the CPU; submanifold convolution, which applies same-topology 3D convolutions on the GPU; strided convolution, which downsamples the voxel grid; and sparse-to-BEV scatter, which converts the final sparse representation to a dense BEV feature map for downstream fusion. Table 2: Sparse encoder internal phase breakdown. Values are means over 81 steady-state samples on the 96-EU integrated GPU. “% Encoder” is the fraction of total sparse encoder latency. Phase % Encoder Precomputation (CPU hash + neighbor map) 27.29% Submanifold convolution (GPU kernel) 4.61% Strided convolution (GPU kernel) 4.04% Sparse-to-BEV scatter 11.15% GPU dispatch + sync overhead 52.91% Total 100% Table 2 presents the breakdown. GPU convolution kernels account for only 8.65%8.65\% of total encoder time (4.61%4.61\% submanifold and 4.04%4.04\% strided, measured via OpenCL event timers). The dominant cost is GPU dispatch and synchronisation overhead (52.91%52.91\%), arising from host–device round-trips at each encoder stage boundary. Fine-grained wall-clock profiling with per-stage clFinishclFinish barriers reveals that this timeframe arises from per-kernel dispatch latency accumulated across the 21 submanifold layers + 4 strided layers + 4 hash builds + 4 neighbour map builds and upload() calls for coords at each level, constituting of 30+ kernel submissions. The sparse-to-BEV scatter (at 11.15%11.15\%) reads 1.8MB1.8\,MB of sparse output features (∼ 7,000 voxels × 128 channels × 2 bytes in FP16) and writes them into a 33MB33\,MB dense BEV tensor (1440×1440×128×FP16/81440× 1440× 128×FP16/8). On a shared-memory architecture where CPU and GPU share the same physical DRAM, the bottleneck shifts from arithmetic throughput to scheduling and synchronisation latency. Geometric Operators The geometric operators, voxelization in the LiDAR branch and depth-based BEV projection in the camera branch, are subsumed within the branch-level timings. Isolating their individual contributions requires OpenCL event-level profiling of each kernel dispatch which is reported in Table 3. Table 3: Geometric operator kernel latencies as % of total pipeline time. Operator % Pipeline Voxelization (scatter) 0.934 Voxelization (compact) 0.198 BEV projection (scatter) 1.824 BEV projection (convert) 1.474 End-to-End Pipeline Performance Having profiled individual operators, we now report the end-to-end pipeline speedup of each competing approach relative to Conventional. The full inference pipeline executes three stages: (1) the LiDAR branch (voxelization followed by sparse encoding), (2) the camera branch (backbone, depth estimation, and BEV projection), and (3) fusion with detection. Table 4 reports per-stage speedup factors for each competing approach relative to Conventional. Table 4: End-to-end pipeline performance of all competing approaches relative to Conventional. All values show the latency speedup compared to Conventional. Stage Fine Medium BEVPIPE (Coarse) LiDAR branch (voxelize + sparse enc.) 19.2×19.2× 24.7×24.7× 29.5×29.5× Camera branch (backbone + depth + proj.) 2.6×2.6× 5.2×5.2× 7.9×7.9× Fusion + detection 2.8×2.8× 3.9×3.9× 5.6×5.6× Total 10.1×10.1× 14.9×14.9× 19.5×19.5× Moving custom operators from the CPU fallback to GPU-resident OpenCL kernels yields the largest per-stage gain, reaching 29.5×29.5× under the coarse configuration for the LiDAR branch. Even at fine granularity, where 21 boundary crossings impose substantial synchronisation overhead, the LiDAR branch still achieves a 19.2×19.2× speedup. Overall, the coarse-grained strategy adopted by BEVPIPE achieves a 19.5×19.5× end-to-end speedup over Conventional, compared to 14.9×14.9× for medium and 10.1×10.1× for fine granularity. The gap between fine and coarse, nearly 2×2× in end-to-end speedup, is attributable entirely to boundary-crossing overhead, since all three configurations execute identical GPU kernels. Portability: SYCL Backend A central design requirement of BEVPIPE is that custom kernels are implemented using portable GPU compute APIs rather than CUDA. To validate this, we re-implement the pipeline in SYCL, compiled with Intel oneAPI DPC++. The resulting binary is run on Intel and NVIDIA GPU without modification to the inference pipeline, and produces detection outputs numerically identical to the OpenCL baseline. Table 5: Functional portability across GPU backends with detection accuracy. mAP is computed on the same 81-sample subset. Max deviation is the maximum absolute difference in predicted output tensors relative to the OpenCL reference. Backend Device mAP Max deviation PyTorch + CUDA NVIDIA GPU 0.585 — OpenCL Intel iGPU 0.576 — SYCL Intel iGPU 0.578 <10−4<10^-4 SYCL NVIDIA GPU 0.578 <10−4<10^-4 Table 5 confirms the functional portability of BEVPIPE. The SYCL binary compiled for Intel and recompiled for NVIDIA with no kernel or host code modifications produces similar mAP on both devices, identical to the OpenCL baseline, with a maximum per-tensor output deviation below 10−410^-4. This confirms that the custom kernels contain no architecture-specific assumptions that produce incorrect results on a different device microarchitecture. Detection Accuracy Table 5 compares the detection accuracy of BEVPIPE against the reference PyTorch implementation. The reference achieves 0.585 mAP with CUDA. BEVPIPE achieves 0.576 mAP using mixed precision (INT8 camera backbone, FP16 fusion and detection) with OpenCL and similar mAP with SYCL on Intel and NVIDIA GPUs. This demonstrates that BEVPIPE maintains competitive detection accuracy while achieving efficient execution on a non-CUDA platform. Ablation Studies We conduct ablation experiments to quantify the contribution of individual design decisions. Each experiment modifies one component while holding all others fixed, and reports the effect on latency. We ablate four design decisions from Design and Implementation section, each measured relative to its disabled counterpart on the same hardware (Table 6). Table 6: Ablation of system-level design decisions. Each row disables one optimization and reports the relative regression (>×>1\!× means the enabled design is faster). Design Decision Relative Effect Depth-threshold sparsity 3.0×3.0× faster Int32 atomic accumulation 12.7×12.7× faster Async queue dispatch 1.35×1.35× faster CPU–GPU overlap 1.18×1.18× faster Depth-threshold sparsity. Skipping depth bins below τdepth _depth in the BEV scatter kernel reduces the number of atomic scatter-add operations by exploiting the peaked nature of the depth distribution. Disabling the threshold (τdepth=0 _depth=0) incurs a 3.0×3.0× latency increase in the scatter kernel, confirming the sparsity claim. Integer atomic strategy. Replacing the fixed-point int32 atomic accumulation with a floating-point compare-and-swap (CAS) loop degrades BEV projection throughput by 12.7×12.7×. The CAS loop requires multiple memory transactions per update under contention, whereas native atomic add on 32-bit integers retires in a single hardware operation. Command queue scheduling. Inserting clFinish barriers after every convolution block, serializing the GPU command stream, increases latency by 1.35×1.35×. The async dispatch design allows the driver to pipeline kernel launches, hiding scheduling overhead behind execution. CPU–GPU computation overlap. Forcing CPU-side coordinate precomputation to wait for the preceding GPU kernel increases the latency by 1.18×1.18×. The modest magnitude reflects that GPU convolution dominates the critical path; the overlap hides the shorter CPU preprocessing behind the longer GPU phase. Conclusion In this paper, we presented BEVPIPE, a deployment framework that solves the operator mismatch problem in BEV perception, namely the inability of production inference runtimes to handle the sparse and geometric operators that these models require. Our solution is systematic coarse-grained partitioning into runtime-managed dense subgraphs and monolithic custom extensions, which reduces the number of costly runtime/extension boundaries yielding a 19.5×19.5× end-to-end speedup over the conventional CPU-fallback baselines. We also showcased that the framework is portable to multiple GPU backends. Future work would focus on automating model partitioning by selecting optimal operator boundaries and granularity to minimize latency. We also plan to look into developing a portable abstraction layer for deployment across multiple inference runtimes, and enabling efficient CPU-GPU multi-device scheduling. References Caesar et al. (2020) H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom nuScenes: a multimodal dataset for autonomous driving. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Dataset and Evaluation Protocol. Chen et al. (2018) T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, M. Cowan, H. Shen, L. Wang, Y. Hu, L. Ceze, C. Guestrin, and A. Krishnamurthy TVM: an automated end-to-end optimizing compiler for deep learning. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), Cited by: Inference Runtimes and Compilers, Competing Approaches. Choy et al. (2019) C. Choy, J. Gwak, and S. Savarese 4D spatio-temporal ConvNets: Minkowski convolutional neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Introduction, Introduction, Sparse Convolution Libraries, Geometric Operations. Contributors (2022) S. Contributors SpConv: spatially sparse convolution library. https://github.com/traveller59/spconv. Cited by: Introduction, Introduction, Sparse Convolution Libraries. Graham (2015) B. Graham Sparse 3d convolutional neural networks. arXiv preprint arXiv:1505.02890. Cited by: Sparse Convolution Libraries, Fused Sparse Convolution. Graham and van der Maaten (2018) B. Graham and L. van der Maaten 3D semantic segmentation with submanifold sparse convolutional networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: Sparse Convolution Libraries, item Sparse ( S ):, Fused Sparse Convolution. Huang and Huang (2022) J. Huang and G. Huang BEVDet4D: exploit temporal cues in multi-camera 3D object detection. In arXiv preprint arXiv:2203.17054, Cited by: Geometric Operations. Intel Corporation (2024) Intel Corporation OpenVINO toolkit. https://docs.openvino.ai/. Cited by: Introduction, Inference Runtimes and Compilers, The Partition Problem, Competing Approaches. Li et al. (2022) Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Y. Qiao, and J. Dai BEVFormer: learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In European Conference on Computer Vision (ECCV), Cited by: Introduction, Geometric Operations. Liang et al. (2022) T. Liang, H. Xie, K. Yu, Z. Xia, Z. Lin, Y. Wang, T. Tang, B. Wang, and Z. Tang BEVFusion: a simple and robust LiDAR-camera fusion framework. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: Introduction, BEV Model Deployment. Liu et al. (2023) Z. Liu, H. Tang, A. Amini, X. Yang, H. Mao, D. Rus, and S. Han BEVFusion: multi-task multi-sensor fusion with unified bird’s-eye view representation. In IEEE International Conference on Robotics and Automation (ICRA), Cited by: Introduction, Introduction, BEV Model Deployment, Geometric Operations, Model Configuration. Microsoft (2021) Microsoft ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator. https://onnxruntime.ai/. Cited by: Inference Runtimes and Compilers, BEVPIPE: Design and Implementation, Competing Approaches. MMDeploy Contributors (2021) MMDeploy Contributors MMDeploy: OpenMMLab’s model deployment toolbox. Note: https://github.com/open-mmlab/mmdeploy Cited by: BEV Model Deployment. Morton (1966) G. M. Morton A computer oriented geodetic data base and a new technique in file sequencing. International Business Machines Company. Cited by: Locality-Aware Voxel Ordering. NVIDIA Corporation (2024) NVIDIA Corporation TensorRT: programmable inference accelerator. https://developer.nvidia.com/tensorrt. Cited by: Introduction, Inference Runtimes and Compilers, BEVPIPE: Design and Implementation. Philion and Fidler (2020) J. Philion and S. Fidler Lift, splat, shoot: encoding images from arbitrary camera rigs by implicitly unprojecting to 3D. In European Conference on Computer Vision (ECCV), Cited by: Geometric Operations, item Geometric ( G ):, Camera-to-BEV Projection. Tang et al. (2022) H. Tang, Z. Liu, S. Zhao, Y. Lin, J. Lin, H. Wang, and S. Han TorchSparse: efficient point cloud inference engine. In Conference on Machine Learning and Systems (MLSys), Cited by: Introduction, Introduction, Sparse Convolution Libraries. Tang et al. (2023) H. Tang, S. Yang, Z. Liu, K. Hong, Z. Yu, X. Li, G. Dai, Y. Wang, and S. Han TorchSparse++: efficient training and inference framework for sparse convolution on GPUs. In IEEE/ACM International Symposium on Microarchitecture (MICRO), Cited by: Introduction, Sparse Convolution Libraries. Tillet et al. (2019) P. Tillet, H. Kung, and D. Cox Triton: an intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, p. 10–19. Cited by: Inference Runtimes and Compilers. Yan et al. (2018) Y. Yan, Y. Mao, and B. Li SECOND: sparsely embedded convolutional detection. In Sensors, Vol. 18, p. 3337. Cited by: Geometric Operations, item Geometric ( G ):, Partition Granularity, Strided Sparse Convolution, Model Configuration. Zheng et al. (2020) L. Zheng, C. Jia, M. Sun, Z. Zhao, C. H. Yu, A. Haj-Ali, Y. Wang, J. Yang, D. Zhuo, K. Sen, J. E. Gonzalez, and I. Stoica Ansor: generating high-performance tensor programs for deep learning. In USENIX Symposium on Operating Systems Design and Implementation (OSDI), Cited by: Inference Runtimes and Compilers.