Paper deep dive
InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset
Felix Stillger, Lukas Hahn, Frederik Hasecke, Tobias Meisen
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/10/2026, 2:28:46 AM
Summary
InCaRPose is a Transformer-based architecture for robust relative camera pose estimation in constrained, highly distorted in-cabin automotive environments. By leveraging frozen DINOv3 backbone features and a Transformer decoder, the model achieves metric-scale translation prediction trained exclusively on synthetic data, demonstrating strong generalization to real-world cabin imagery and competitive performance on public datasets like 7-Scenes.
Entities (5)
Relation Signals (3)
InCaRPose → evaluatedon → In-Cabin-Pose
confidence 98% · We release our real-world In-Cabin-Pose test dataset
InCaRPose → evaluatedon → 7-Scenes
confidence 95% · additionally achieves competitive performance on the public 7-Scenes dataset
InCaRPose → uses → DINOv3
confidence 95% · By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In-Cabin-Pose test dataset consisting of highly distorted vehicle-interior images and our code at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2604.03814v1
- Canonical: https://arxiv.org/abs/2604.03814v1
Trouble viewing inline? Open PDF directly →
Full Text
70,466 characters extracted from source content.
Expand or collapse full text
inkscapelatex=false, InCaRPose: In-Cabin Relative Camera Pose Estimation Model and Dataset Felix Stillger1,2 Lukas Hahn2 Frederik Hasecke2 Tobias Meisen1 1University of Wuppertal, <lastname>@uni-wuppertal.de 2Aptiv, <firstname>.<lastname>@aptiv.de Abstract Camera extrinsic calibration is a fundamental task in computer vision. However, precise relative pose estimation in constrained, highly distorted environments, such as in-cabin automotive monitoring (ICAM), remains challenging. We present InCaRPose, a Transformer-based architecture designed for robust relative pose prediction between image pairs, which can be used for camera extrinsic calibration. By leveraging frozen backbone features such as DINOv3 and a Transformer-based decoder, our model effectively captures the geometric relationship between a reference and a target view. Unlike traditional methods, our approach achieves absolute metric-scale translation within the physically plausible adjustment range of in-cabin camera mounts in a single inference step, which is critical for ICAM, where accurate real-world distances are required for safety-relevant perception. We specifically address the challenges of highly distorted fisheye cameras in automotive interiors by training exclusively on synthetic data. Our model is capable of generalization to real-world cabin environments without relying on the exact same camera intrinsics and additionally achieves competitive performance on the public 7-Scenes dataset. Despite having limited training data, InCaRPose maintains high precision in both rotation and translation, even with a ViT-Small backbone. This enables real-time performance for time-critical inference, such as driver monitoring in supervised autonomous driving. We release our real-world In‑Cabin‑Pose test dataset consisting of highly distorted vehicle‑interior images and our code at https://github.com/felixstillger/InCaRPose. 1 Introduction Accurately estimating the relative camera pose between two views is essential for applications such as SLAM, AR, and 3D reconstruction. Classical pipelines rely on geometric feature matching and epipolar geometry, which can degrade under occlusions or strong lens distortions. Convolutional and Transformer-based architectures represent a significant advance in deep learning, providing robust global representations. However, state-of-the-art models that generalize well typically require large training sets for convergence and only work with specific camera intrinsics. Moreover, these models are unsuitable for edge deployment due to their prohibitive computational requirements and excessive parameter counts. Figure 1: Our InCaRPose predicts the relative camera pose between a reference and a target view (shown as camera frustums). Trained exclusively on synthetic images, the model generalizes to real-world cabin environments and enables camera extrinsic calibration. The integration of in-cabin sensing cameras is accelerating, driven by EU GSR and Euro NCAP mandates, as well as rapid growth in driver and occupant monitoring [30, 64, 49]. These cameras are commonly mounted at multiple possible positions within the cabin, with the rear-view mirror being a preferred position due to its optimal view inside the cabin. When equipped with wide-angle or fisheye lenses, this configuration enables effective monitoring of both the driver and vehicle occupants in the front and back rows. However, rear-view mirrors are dynamic components that are manually or automatically adjusted by drivers, resulting in frequent changes to the camera’s extrinsic parameters. Maintaining consistent camera-to-vehicle calibration is a key challenge for reliable in-cabin perception. Interior cabin cameras typically operate in the near-infrared (NIR) spectrum rather than the more common visible RGB spectrum [47]. Active NIR illumination (750–1400 nm) ensures robust sensor visibility while remaining imperceptible to the human eye [44, 58] and therefore does not distract vehicle occupants. This mitigates the effects of cast shadows, surface texture variations, and ambient lighting changes, and enables night-time operation. Accurate spatial awareness of in-cabin sensors is critical for driver monitoring, such as gaze estimation in autonomous driving to ensure drivers can return their hands to the wheel within required timeframes, as well as for other safety-relevant applications like occupant-position-aware airbag deployment. In the event of a collision, the system must infer occupant locations on the order of a few milliseconds to optimize restraint behavior because airbag control algorithms typically decide within 15–50 ms (encompassing sensing-to-actuation pipeline) after detection [15, 6, 59]. We address camera extrinsic calibration in interior cabin environments and release our real-world in-cabin test dataset and code. Our model estimates the relative transformation between a potentially shifted cabin view and a calibrated reference view (Fig. 1). It combines frozen self-supervised ViT features with a Transformer decoder and a lightweight prediction head, enabling data-efficient training and real-time inference for time-critical automotive and industrial applications. Training exclusively on a small synthetic cabin set is sufficient to recover large relative pose changes and generalize to real cabin imagery. We further evaluate the model’s generalization capabilities on the public 7-Scenes and Cambridge Landmarks datasets, achieving competitive results without iterative solvers or dense 3D reconstructions. In summary, our main contributions are: • A Novel In-Cabin-Pose Test Dataset. We publish highly distorted, wide-FoV NIR fisheye imagery with metric ground truth as a real-world in-cabin test set. • Vehicle-Agnostic Reference Formulation. We reformulate the in-cabin pose estimation problem as reference-relative pose prediction to avoid vehicle-specific coordinate frames and individual retraining. • Robust Fisheye Handling Without Undistortion. InCaRPose processes distorted fisheye images end-to-end and predicts translation in absolute metric units. • Synthetic-to-Real Capable Model. Training on limited synthetic renderings yields strong transfer to a real cabin and competitive performance on 7-Scenes. 2 Related Work 2.1 Absolute Pose Estimation Absolute pose methods localize a camera in a global scene by regressing the camera position in the world coordinate system from images. Typically, they use a 2D to 3D matching and Perspective-n-Point-algorithm (PnP) with RANSAC. End-to-end learning-based relocalizers encode the scene implicitly in neural network weights. This includes scene coordinate regression (SCR) methods [7, 8, 9] and absolute pose regression (APR), such as PoseNet [33, 32]. Some of them use image retrieval to estimate pose approximation [52]. While effective, these often require scene-specific training or dense reconstructions. Minimal solvers and robust estimators continue to improve sampling efficiency for these tasks [62]. More recently, several variants of absolute pose estimation aim to improve robustness and cross-scene generalization. DFNet [14] improves absolute pose regression through direct feature matching, while Neural Refinement [12] introduces a refinement stage that significantly improves pose accuracy. Map-relative pose regression [13] extends APR to settings where only sparse or lightweight maps are available. Furthermore, Yin et al. [68] propose a Laplace-inspired distribution to better model rotation uncertainty in regression-based localization. 2.2 Relative Pose Estimation Relative pose methods estimate the transformation between two views without explicit scene maps. Classical approaches rely on matching local features to recover the essential matrix [25], which is decomposed into relative rotation and a scaleless translation vector. This formulation has been enhanced by learned features [40, 18] and by improved matchers such as SuperGlue [51] and LoFTR [57]. Robust estimators such as MAGSAC++ [5] further improve reliability. Deep relative pose regression (RPR) bypasses explicit correspondences to directly predict pose [41, 35]. Although some RPR methods use image retrieval [1, 27] to look up posed database images, they often implicitly rely on prior SfM or SLAM reconstructions. To avoid reliance on fixed databases, Turkoglu et al. [60] proposed using graph neural networks to aggregate information across multiple views, utilizing relative pose supervision to improve re-localization accuracy. In contrast, map-free relocalization targets scenarios where only a single reference image is available [3, 66, 2]. ExReNet [66] shows that extremely sparse reference sets (as few as four images) can cover indoor rooms and LENS [42] uses NeRF-based view synthesis to enhance relative localization. Lin et al. [38] introduce neural volumetric pose features that fuse appearance and geometry for improved pairwise pose estimation. More challenging is the objective toward unseen scene generalization, where models are expected to localize in environments entirely absent from the training set. Idan et al. [28] demonstrated that RPRs can effectively generalize to these unseen scenes, a concept that was further expanded to completely new landscapes in follow-up work [29]. Frameworks such as DirectionNet [11] and, more recently, Reloc3r [17] further advance generalization. 2.3 Pretrained Foundation Models A core challenge in pairwise pose estimation is the scale ambiguity of the translation. Multi-view pipelines can triangulate metric structure [69], while pairwise map-free settings must rely on additional priors such as depth [19]. Recent vision foundation models provide strong geometric cues via self-supervised pretraining on large-scale data. CroCo [65] learns multiview consistency through cross-view completion. MAST3R and DUST3R [63, 36] explicitly ground correspondence by predicting dense 3D structure from image pairs. Distillation-based ViTs such as DINOv2 and DINOv3 [46, 55] produce transferable dense features. Notably, DINOv2 variants can include learned register tokens, while DINOv3 adopts 2D rotary positional embeddings (RoPE) [56] to improve flexibility across input resolutions. Positional encodings also explore explicit spatial tokens (SpatialFormer [67]) and refined rotary embeddings for multimodal models (MHRoPE [26]). DUNE [50] introduces a universal encoder that further distills from the teacher models DINOv2, MAST3R and Multi-HMR [46, 36, 4]. 3 Background 3.1 Problem Statement Modern vehicles increasingly deploy multiple interior cabin cameras at different locations. To fuse their outputs reliably, these sensors must operate in a shared reference frame, which requires accurate extrinsic calibration both per camera and relative to one another. Unlike fixed exterior sensors, some interior mounting points exhibit mechanical variability. A prominent example is a camera mounted on or near the rear-view mirror. This position offers a favorable cabin view, often with wide-angle or fisheye optics, but the mirror is adjustable and can change the camera extrinsics over time. Figure 2: Camera coordinate system of the standard view compared to the vehicle’s coordinate system. As illustrated in Fig. 2, a standard calibration is commonly defined in the vehicle coordinate system (e.g., ISO 8855 [31]). However, relying on such a global vehicle frame hinders generalization across vehicle models, since the geometric relationship between the cabin and the vehicle reference differs across platforms. To avoid vehicle-specific training, we reformulate the task as reference-relative pose estimation. Given a calibrated reference pose v1T_v1 (e.g., a standard view in the cabin) and a second view with pose v2T_v2, we estimate the relative transformation relT_rel such that v2=v1⋅rel.T_v2=T_v1·T_rel. (1) During training, relT_rel is derived from ground-truth poses via rel=v1−1v2T_rel=T_v1^-1T_v2. At inference time, a single calibrated reference frame v1T_v1 suffices, and the problem reduces to estimating the relative pose (rotation and translation) between a known calibrated state and a potentially shifted camera state from an image pair. This reference-relative formulation is vehicle-agnostic and enables deployment without retraining across different cabin configurations while supporting safety-relevant in-cabin perception. 3.2 Data Our experiments rely on two data sources that consist of synthetic and real-world data. The synthetic data is used for training and validation, while the real-world data is exclusively for testing to approximate actual deployment conditions. To collect the real-world dataset, the vehicle’s front windshield was removed, allowing free camera movement and preventing image disturbances caused by the confined cabin space. Controlling the camera from inside the vehicle would have influenced the data through operator-induced occlusions or motion artifacts. Because this process is invasive and difficult to repeat across multiple vehicles, collecting clean data in different car interiors is not straightforward. For this reason, our real-world test dataset is limited to a single vehicle cabin. The real-world data includes experimental components, such as ArUco markers [22], which are not present in the synthetic data. A key advantage of the synthetic data is the availability of ideal ground truth without modifying the images, whereas ground truth generation for real-world data is inherently error-prone due to measurement inaccuracies and estimation uncertainty. Figure 3: Standard view comparison: (left) real-world image and (right) synthetic image from the simulation environment. Fig. 3 shows a representative standard view for both real and synthetic data. Synthetic Data To cover a broad range of cabin geometries while retaining control over occlusions and distortions, we constructed synthetic scenes in Blender [16]. We allocate eight vehicles for training and three for validation to avoid overlap between training and validation. We randomly place objects and occupants inside the cabin. Per scene, we uniformly sample rotations (±80∘± 80 about x and y, ±50∘± 50 about z) and translations (±20± 20 cm along each camera axis) around the per-vehicle standard view (Fig. 3). Although individual images are reused, we form unique image pairs, yielding approximately ∼ 5000 rotation-only, ∼ 1500 rotation+translation training pairs and ∼ 2000 validation pairs. Real-World Data Obtaining reliable ground truth in automotive interiors is challenging due to the confined space and the resulting low depth-to-baseline ratio. While Structure-from-Motion (SfM) frameworks like COLMAP [53] are standard for large-scale scenes and can operate without visual cues such as ArUco markers, they suffer from scale ambiguity and geometric drift in environments. The lack of discriminative texture on monochrome surfaces often leads to ill-conditioned bundle adjustment leading to a non-optimal pose estimation in our particular use case. To address this issue, we prioritize marker-based estimation. However, we provide an evaluation of both methods (see Supplementary Material 7.5). To obtain the camera pose, we detect ArUco markers of known size (0.07m0.07\,m) [23] and estimate per-image marker-to-camera extrinsics using OpenCV’s ArUco library [10]. In contrast to SfM, this yields metric scale and remains robust under the low-parallax conditions typical of confined interiors [10]. For each marker mkm_k visible in both the reference image r and the query image j, we obtain c←mk(r,k)T^(r,k)_c← m_k and c←mk(j,k)T^(j,k)_c← m_k and compute the relative camera pose as cj←cr(k)=c←mk(j,k)(c←mk(r,k))−1.T^(k)_c_j← c_r=T^(j,k)_c← m_k (T^(r,k)_c← m_k )^-1. (2) Real-World Test Dataset To evaluate the robustness of our approach, we curated a real-world test dataset that systematically covers the camera’s six degrees of freedom (6-DoF). This dataset is organized into specific sequences designed to isolate transformations and identify edge cases or model weaknesses. The sequences cover translations and rotations along each axis. These axes are defined relative to the camera coordinate system. To ensure consistency across the dataset, we established a common reference frame based on a standard view within the cabin. This view was selected to maximize feature overlap across all subsequent transformations. All camera poses are computed as relative transformations from the origin of the standard view. We provide both ArUco- and COLMAP-generated ground truth. This results in 550 images and labels for our final released real-world test dataset called In-Cabin-Pose. 4 Method 4.1 Architecture We propose a two-view camera calibration network (see Fig. 4) that leverages a frozen vision Transformer backbone (e.g., DINOv3 [55]) to extract per-patch latent features from a reference image and a second-view image. We utilize a frozen backbone because our task-specific dataset is limited in size. Through intensive pre-training, such models learn robust, cross-domain feature representations that can be effectively leveraged for a wide range of downstream tasks. These features are further processed by a Transformer-based cross-attention decoder. This decoder is essential for refining the general-purpose feature relationships that are necessary for a relative pose estimation. To predict the final output, we utilize a compact multi-layer perceptron (MLP) head that regresses the representation-specific camera pose. While the backbone remains frozen, only the decoder and prediction head are trained end-to-end. Maintaining a frozen backbone ensures that early training iterations do not degrade the fine-grained feature representations and preserves the model’s ability to generalize across domains (e.g., from synthetic to real-world data). Figure 4: InCaRPose’s architecture overview. Two images are encoded by a frozen ViT backbone and fused by a cross-attention Transformer decoder. A prediction head outputs the relative camera pose between the views, optionally in both directions. Details on Decoder and Prediction Head The model employs a Transformer decoder with variable depth to combine spatial features from the reference and second-view images. The tokens output by the backbone are linearly projected into decoder embeddings and processed by a stack of decoder blocks. To capture the spatial relationship between the two views, we designed each decoder block around self-attention to refine features and cross-attention to attend features to features across views. We add LayerNorm [61] and residual paths to stabilize training. To incorporate spatial sensitivity without learned positional tokens, especially under limited training data, we introduce two-dimensional RoPE [56] on queries and keys. By default, the decoder uses twelve attention heads and an MLP expansion ratio of four, adopting a standard Transformer configuration that provides a balanced trade-off between computational efficiency and the capacity to model complex spatial dependencies. To reduce the feature-map dimensionality and connect the decoder with the prediction head, we use a residual convolutional bottleneck. This block performs channel-wise feature fusion to distill the high-dimensional Transformer embeddings, while the residual path maintains gradient flow and prevents information loss. These reduced features are then aggregated by global average pooling into a one-dimensional feature tensor. The final prediction head maps this low-dimensional tensor through a LayerNorm and a linear layer with a GELU activation function. We employ active dropout during training to improve generalization by preventing the model from overfitting to specific domain patterns. The final linear regression layer outputs a vector corresponding to the target output representation. To obtain a stronger supervisory signal and eliminate the need for image-ordering augmentation during training, we enable the model to predict the inverse relative pose. The final output is the concatenation of both outputs into a single tensor. This enforces geometric consistency by requiring the network to learn the bidirectional camera transformation relationship. During inference, double prediction can be disabled to maximize computational speed. 4.2 Output Representations We also examine how the model adapts to different output styles and investigate various representations of rotations. The common representation is R∈SO(3)R (3) which denotes a 3D rotation matrix, i.e., an orthogonal matrix with det(R)=1 (R)=1 that preserves lengths and angles while maintaining the orientation of the coordinate system (i.e., it performs a proper rotation without reflection or flipping). Furthermore, =[tx,ty,tz]⊤∈ℝ3t=[t_x,t_y,t_z] ^3 denotes a translation vector. The combination of rotation and translation gives us a complete transformation that can be used to describe a relative pose. We support five distinct parameterizations for the camera pose. In all cases, the translation is consistently appended as the final three components of the output vector y. We consider multiple camera pose output representations, including rotation vectors and both intrinsic and extrinsic Euler angles with 3D rotation (6D total including translation), quaternions with 4D rotation (7D total), and rotation matrices with 9D rotation (12D total). Details and loss formulations are provided in Supplementary Material 7.2 and 7.3.2. To ensure valid rotations, we apply representation-specific post-processing. Depending on the target representation, we either normalize quaternions to unit quaternions with a magnitude of one, or we orthogonalize the rotation matrices via Singular Value Decomposition (SVD) with a det =+1=+1 correction. These operations yield elements in SO(3)SO(3), ensuring no scaling or shearing [37, 24, 34]. 4.3 Labels and Preprocessing Datasets usually provide camera poses in dataset-specific global frames, e.g., the vehicle rear-axle frame (ISO 8855 [31]) or an arbitrary world frame as in 7-Scenes [54]. To make supervision independent of these conventions, we convert absolute poses into a canonical reference-relative target. Given absolute poses for a reference view Tv1T_v1 and a second view Tv2T_v2, we define Trel=Tv1−1Tv2,Tv1≡Tref,T_rel\;=\;T_v1^-1T_v2, T_v1≡ T_ref, which expresses the second view in the coordinate system of the reference view. Because fisheye edges might contain critical geometric cues, we avoided center cropping. Instead, we rescale images and zero-pad them to maintain the full field of view even with varying input sizes (see Supplementary Fig. 6). To increase the robustness against environmental changes, we use ColorJitter to manipulate the brightness, contrast, saturation, and hue of images. This prevents overfitting of the model on the clean synthetic data. 5 Experiments We evaluate the synthetic-to-real generalization of InCaRPose and report quantitative performance across our proprietary In-Cabin-Pose dataset, the public indoor 7-Scenes [54], and public outdoor Cambridge Landmarks [33] dataset. To estimate a metric translation we use direct Euclidean distance error which minimizes the distance of predicted and ground truth translation. For rotational representation we utilize unit quaternion representation and its specific rotational loss (see Table 4) since this yields the best performance. We train with a batch size of eight to enable parallel training on a single GPU and use the AdamW [39] optimizer with a conservative learning rate of 1×10−61× 10^-6 and weight decay of 1×10−51× 10^-5 to prevent overfitting. The backbone is pretrained and frozen during training to preserve its robust, domain-invariant features [46, 55, 50] and to prevent their degradation by the noisy gradients of the randomly initialized components during the early training stages. Consequently, only the decoder and the head are subject to optimization. 5.1 Keypoint-Based Detection We include a classical keypoint-based pipeline as a training-free geometric baseline and sanity check. Such methods are widely used for two-view pose estimation but typically recover translation only up to an unknown scale, which directly contrasts with our goal of metric translation. SIFT keypoints [40] are detected and matched using a Fast Library for Approximate Nearest Neighbors (FLANN) KD-tree [43], followed by filtering via Lowe’s ratio test with a threshold of 0.750.75. Pairs with fewer than 2020 correspondences are discarded to ensure geometric reliability. For undistorted images, we estimate the Essential matrix [45] using the RANdom SAmple Consensus (RANSAC) [21] algorithm to recover relative rotation (R) and translation (t). As SIFT is scale-ambiguous, translation is reported in terms of unit length only. 5.2 Results on In-Cabin Data We first evaluate performance on our primary In-Cabin-Pose dataset. Table 1 reports results comparing InCaRPose with different frozen backbones (ViT-Small, -Base, -Large) against a classical SIFT matching pipeline and large-scale trained baselines (Reloc3r224 and Reloc3r512). We note that Reloc3r predicts translation direction only and does not recover metric scale, and is trained on orders of magnitude more data than InCaRPose. When provided with undistorted images, Reloc3r achieves competitive directional translation performance, indicating that its strength lies in large-scale training under known camera intrinsics. In contrast, InCaRPose is designed to operate directly on highly distorted fisheye imagery and to recover metric-scale translation within the adjustment ranges encountered in in-cabin camera setups by a single forward pass, without requiring undistortion or scene-specific training. The InCaRPoseLarge224 configuration achieves a median translation error of just 0.07m0.07\,m and a median rotation error of 2.75∘. The low error rates indicate that the Transformer backbone effectively encodes global spatial relationships within the cabin. This robustness to real-world sensor noise, despite being trained exclusively on synthetic data, validates our approach to domain-invariant feature extraction. SIFT-matching has competitive rotation performance but cannot determine absolute distance, which is a critical requirement for safety-critical automotive applications. Our InCaRPoseLarge224 model achieves the lowest overall rotation error, though it slightly underperforms Reloc3r in directional translation. The DUNE-Base encoder variant [50], with an input resolution of 504 due to its patch size, yields better results than DINOv3-Base, but further gains are constrained by the absence of a publicly released ViT-Large counterpart. Table 1: Model Performance Comparison: Rotation and Translation Errors (Trans. = translation error in m, Dir. = translation direction error in °) on our In-Cabin-Pose real-world test set. Model Error Rot. (∘) Trans. (m) Dir. (∘) InCaRPoseSmall224 Mean 6.11 0.11 42.55 Median 4.43 0.08 37.74 InCaRPoseBase224 Mean 4.91 0.12 53.67 Median 3.55 0.09 42.45 InCaRPoseLarge224 Mean 4.15 0.10 36.17 Median 2.75 0.07 23.46 InCaRPoseDuneBase504 Mean 3.87 0.12 58.57 Median 3.05 0.09 39.71 Reloc3r224 [17] Mean 14.84 – 83.43 Median 12.73 – 76.79 Reloc3r512 [17] Mean 13.65 – 69.56 Median 11.48 – 61.55 Reloc3r224 [17] (undistort) Mean 4.74 – 16.21 Median 3.66 – 11.63 Reloc3r512 [17] (undistort) Mean 4.32 – 17.74 Median 3.23 – 13.05 SIFT Matching [40] (undistort) Mean 7.15 – 37.16 Median 4.83 – 28.30 Dust3r-Large [63] (undistort) Mean 11.61 – 19.60 Median 9.73 – 12.13 MAST3R-Large [36] (undistort) Mean 14.24 – 41.66 Median 11.27 – 35.90 In addition to accuracy, the 224-resolution configurations are suitable for low-latency in-cabin perception under our evaluation setup. On a consumer-grade NVIDIA RTX 4090 GPU (see Supplementary Table 6), all InCaRPose variants run in real-time, consistently exceeding 45 FPS, with the Small and Base backbones approaching 70 FPS. Here, we use real-time to denote batch-size-1 throughput sufficient to keep pace with a typical in-cabin camera stream (see Supplementary Table 6 for more details). InCaRPoseLarge224 achieves the best pose accuracy while remaining within real-time constraints despite its increased representational capacity. This illustrates the expected accuracy-latency trade-off. At ≥ 67 FPS, the per frame model inference latency is approximately 15 ms. However, 15–50 ms post-impact window for airbag deployment refers to the entire sensing-to-deployment pipeline, rather than an isolated perception module [59]. In practice, an airbag control system should therefore rely on the most recent pre-impact (or immediately pre-trigger) occupant-state estimates maintained in a rolling buffer by the in-cabin perception stack, instead of waiting to execute a fresh pose inference after impact to satisfy tight response constraints. For completeness, we evaluate inference on images without visible ArUco markers (see Supplementary Material chapter 7.6). Performance remains comparable, as expected, since markers are absent from the training data and the model does not depend on them. The model also remains robust to small distractors (a checkerboard is visible in one image only). 5.3 Qualitative Results Fig. 5 demonstrates InCaRPose on real-world examples. The predicted poses closely match the ground truth, indicating that the model captures spatial relationships even under challenging viewpoint changes and transfers effectively from synthetic training to real cabin scenes. We also visualize predictions from Reloc3r [17] and SIFT matching [40], which show noticeably larger deviations in these cases. For visualization, all translation vectors are normalized to a common scale so that trajectories can be compared in a consistent coordinate frame. The last example highlights a remaining failure mode: although the model estimates rotation accurately, translation, particularly along the z-axis, remains challenging under extreme relative motion. Figure 5: Qualitative results on real-world inference. All translation vectors are normalized to a common scale for visualization. 5.4 Results on Public Data To evaluate how well our architecture generalizes to general indoor environments, we trained InCaRPose on the 7-Scenes dataset using the training split defined by [54] in the original paper. As shown in Table 2, InCaRPose achieves substantially lower mean rotation error than Reloc3r and competitive or better median rotation error. Specifically, InCaRPoseLarge512 achieves a mean rotation error of 2.21∘, representing a 65% improvement over the Reloc3r512 baseline (6.37∘). Furthermore, unlike Reloc3r, which is restricted to direction vectors, our model recovers the absolute metric translation. InCaRPose with ViT-Large backbones reaches the best median translation performance with 0.13m for the 224 input resolution. In contrast, the 512 version has slightly worse median translation performance but a lower rotational error. Our model also outperforms Reloc3r in directional translation error. The mean performance of our models is much closer to the median, suggesting that our model does not predict many drastic outliers. This also supports our use case. In contrast to the results on the In-Cabin-Pose dataset, the discrepancy between different backbones is not as large. This suggests that larger backbones can handle distortions more effectively, or that there is sufficient training data available to enable generalization of this dataset-specific problem, even for smaller backbones. Additionally, we observe that increasing the input resolution only improves rotational performance, whereas translation performance does not improve with an increased input resolution. Table 2: Model performance: rotation and translation errors on 7-Scenes. Not available values are indicated with “-”. Bold indicates best performance. Model Error Rot. (∘) Trans. (m) Dir. (∘) Reloc3r224 [17]222Trained on additional data Mean 7.96 – 39.43 Median 2.45 – 33.51 RelPoseNet[35]111Values are taken from the authors’ paper Mean – – – Median 9.30 0.21 – Relformer[28]111Values are taken from the authors’ paper Mean – – – Median 6.27 0.18 – RelPoseGNN [60]111Values are taken from the authors’ paper Mean – – – Median 5.20 0.17 – InCaRPoseSmall224 Mean 2.69 0.24 14.19 Median 2.39 0.14 9.32 InCaRPoseBase224 Mean 2.56 0.24 14.36 Median 2.27 0.14 9.50 InCaRPoseLarge224 Mean 2.55 0.22 13.48 Median 2.25 0.13 9.08 Reloc3r512 [17]222Trained on additional data Mean 6.37 – 38.80 Median 2.17 – 33.14 InCaRPoseLarge512 Mean 2.21 0.22 13.86 Median 1.91 0.15 10.35 To evaluate our model’s performance on outdoor datasets, we present the results on the Cambridge Landmarks [33] dataset in Table 3. We utilize the specific training and validation image pairs provided by RPNet [20] for relative pose estimation, which are only available for the Kings College, Old Hospital, Shop Facade, and St Mary’s Church subsets. The models were trained on all training data from all available scenes. Table 3: Validation results on Cambridge Landmarks. Scope Scene Model Rot. Err (∘) Trans. Err (m) Mean Median Mean Median Overall All InCaRPoseSmall224 6.99 3.44 2.31 1.45 InCaRPoseBase224 6.53 3.01 2.29 1.35 InCaRPoseLarge224 6.49 2.66 2.19 1.21 Reloc3r224 [17]22footnotemark: 2 9.95 1.13 – – Scene KingsCollege InCaRPoseSmall224 2.74 2.67 1.20 1.05 InCaRPoseBase224 2.53 2.29 1.15 1.09 InCaRPoseLarge224 2.19 2.03 1.04 0.93 Reloc3r224 [17]22footnotemark: 2 0.86 0.73 – – RPNet [20]11footnotemark: 1 – 5.40 – 1.92 OldHospital InCaRPoseSmall224 3.62 2.96 1.55 1.45 InCaRPoseBase224 3.16 2.54 1.40 1.32 InCaRPoseLarge224 2.64 2.09 1.25 1.20 Reloc3r224 [17]22footnotemark: 2 1.44 1.20 – – RPNet [20]11footnotemark: 1 – 5.40 – 2.31 ShopFacade InCaRPoseSmall224 19.97 8.75 5.34 4.04 InCaRPoseBase224 18.62 5.81 5.51 2.64 InCaRPoseLarge224 20.31 9.31 5.52 3.76 Reloc3r224 [17]22footnotemark: 2 42.70 11.78 – – RPNet [20]11footnotemark: 1 – 8.00 – 1.46 StMarysChurch InCaRPoseSmall224 5.34 4.61 2.35 1.99 InCaRPoseBase224 5.99 5.47 2.43 2.38 InCaRPoseLarge224 4.73 4.66 2.32 2.22 Reloc3r224 [17]22footnotemark: 2 11.69 2.27 – – RPNet [20]11footnotemark: 1 – 8.48 – 2.65 A performance disparity is observed when comparing these results to the 7-Scenes benchmark, with the Cambridge dataset proving more challenging. This degradation in precision can be attributed to the transition from indoor to outdoor environments. Indoor scenes typically offer higher feature density and consistency, whereas outdoor landmarks introduce significant scale variations and complex scenes. The most notable loss of precision is on the Shop Facade subset. These findings suggest that the current architecture is better suited for environments characterized by high feature recurrence, which is more prevalent in structured indoor settings than in more variable outdoor scenes. This may stem from the RPNet [20] split, which yields larger viewpoint and appearance gaps. We observe a similar degradation for Reloc3r [17] on ShopFacade, which influences the overall mean performance drastically. Overall, Reloc3r attains lower rotation errors on several individual scenes, while InCaRPose is more competitive on ShopFacade and achieves stronger overall aggregated mean rotation. 5.5 Ablations We have already demonstrated the effectiveness of different backbone sizes in Tables 1,2 and 3, where InCaRPoseSmall has substantially fewer parameters than InCaRPoseLarge (see Supplementary Fig. 8 for number of parameters against rotational error). For our In-Cabin-Pose data, the larger backbone improves quality, and the scope of the features helps learning with fewer examples. By contrast, the effect on the 7-Scenes and Cambridge Landmarks dataset is less significant. As shown in Table 2, increasing the resolution from 224 to 512 has a noticeable impact on the overall rotational performance, whereas the larger backbone in 224-resolution only has a marginal effect. The translation direction error and absolute metric error vary less than rotation across the backbones. Output Representation We investigated different output mappings of the rotation (see Table 4). Table 4: Output rotation representations ablation on In-Cabin-Pose with InCaRPoseBase224. Representation Error Rot. (∘) Trans. (m) Dir. (∘) Rotation Vector Mean 6.35 0.14 81.24 Median 4.80 0.11 70.62 Euler Intrinsic Mean 7.61 0.14 99.14 Median 5.77 0.10 106.76 Euler Extrinsic Mean 6.39 0.14 108.73 Median 4.67 0.11 110.92 Quaternion Mean 6.43 0.11 41.95 Median 4.89 0.07 36.23 Rotation Matrix Mean 5.83 0.16 71.97 Median 5.03 0.13 73.67 Quaternion + Loss Mean 4.91 0.12 53.67 Median 3.55 0.09 42.45 Because prior work [17, 37] suggested benefits of 9D parameterizations, we compare multiple rotational representations. Our goal was to identify the option that yields the best performance. Our ablation study was conducted on the ViT-Base size of DINOv3 [55]. In our architecture, the configuration utilizing quaternions and the Quaternion-based Pose Loss achieved the lowest rotation error (3.55∘3.55 median) compared to different rotation representations using the Universal Transformation Loss (see Supplementary Material chapter 7.2). Backbone Type Table 5 compares different backbone architectures. Table 5: Backbone ablation on In-Cabin-Pose with InCaRPoseBase224. Backbone Error Rot. (∘) Trans. (m) Dir. (∘) Dune-Base Mean 4.52 0.13 55.35 Median 3.00 0.09 53.40 DUST3R-Large Mean 5.23 0.14 81.18 Median 3.99 0.10 77.63 DINOv2-Base Mean 5.98 0.12 45.78 Median 3.95 0.09 40.43 DINOv3-Base Mean 4.91 0.12 53.67 Median 3.55 0.09 42.45 As expected, we have noticed that the DINOv3 performs better than the DINOv2 as backbone. Like previously noticed, the Dune-Base encoder variant also shows strong performance in direct comparison, which is based on multiple teacher models, such as DINOv2, and outperforms our model with the DINOv3 backbone. 6 Conclusion We presented InCaRPose, an effective relative pose estimator for highly distorted input images, primarily designed for automotive interiors but applicable beyond this setting. Relative pose estimation can be used to achieve absolute pose estimation with only a single calibrated frame to the vehicle coordinate system. Our architecture achieves competitive performance on the public 7‑Scenes and can generalize to an unseen real‑world car interior and an unseen camera, even when it is trained exclusively on synthetic data. It directly processes highly distorted fisheye images end-to-end, without any need for undistortion, preserving the original corner information. A key outcome of our research is that the model is data‑efficient, achieving over 70 FPS on a consumer GPU and successfully bridges the synthetic‑to‑real gap for real‑world inference, despite being trained on only 6,500 synthetic image pairs. This is achieved by leveraging powerful pre-trained backbones (such as the DINO-family) that remain completely frozen during training, enabling an efficient use of the features for other downstream tasks. Our model predicts translation in absolute metric units (meters) and performs competitively under challenging conditions. Looking forward, we aim to enhance the model’s robustness in partially occluded environments. While the current architecture has been trained on occlusion and obstacle data, future research will focus on developing a dedicated test set to quantify performance in complex, dynamic cabin conditions involving passengers and cargo. References [1] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic (2016) NetVLAD: cnn architecture for weakly supervised place recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 5297–5307. Cited by: §2.2. [2] E. Arnold, J. Wynn, S. Vicente, G. Garcia-Hernando, A. Monszpart, V. Prisacariu, D. Turmukhambetov, and E. Brachmann (2022) Map-free visual relocalization: metric pose relative to a single image. In European Conference on Computer Vision, p. 690–708. Cited by: §2.2. [3] V. Balntas, S. Li, and V. Prisacariu (2018) Relocnet: continuous metric learning relocalisation using neural nets. In Proceedings of the European conference on computer vision (ECCV), p. 751–767. Cited by: §2.2. [4] F. Baradel, M. Armando, S. Galaaoui, R. Brégier, P. Weinzaepfel, G. Rogez, and T. Lucas (2024) Multi-hmr: multi-person whole-body human mesh recovery in a single shot. In European Conference on Computer Vision, p. 202–218. Cited by: §2.3. [5] D. Barath, J. Noskova, M. Ivashechkin, and J. Matas (2020) MAGSAC++, a fast, reliable and accurate robust estimator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 1304–1312. Cited by: §2.2. [6] W. Bortles and R. Hostetler (2019) Performance of event data recorders found in toyota airbag control modules in high severity frontal oblique offset crash tests. Technical report SAE Technical Paper. Cited by: §1. [7] E. Brachmann, A. Krull, S. Nowozin, J. Shotton, F. Michel, S. Gumhold, and C. Rother (2017) Dsac-differentiable ransac for camera localization. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 6684–6692. Cited by: §2.1. [8] E. Brachmann and C. Rother (2018) Learning less is more-6d camera localization via 3d surface regression. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 4654–4662. Cited by: §2.1. [9] E. Brachmann and C. Rother (2021) Visual camera re-localization from rgb and rgb-d images using dsac. IEEE transactions on pattern analysis and machine intelligence 44 (9), p. 5847–5865. Cited by: §2.1. [10] G. Bradski (2000) The OpenCV Library. Dr. Dobb’s Journal of Software Tools. Cited by: §3.2. [11] K. Chen, N. Snavely, and A. Makadia (2021) Wide-baseline relative camera pose estimation with directional learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 3258–3268. Cited by: §2.2. [12] S. Chen, Y. Bhalgat, X. Li, J. Bian, K. Li, Z. Wang, and V. A. Prisacariu (2024) Neural refinement for absolute pose regression with feature synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 20987–20996. Cited by: §2.1. [13] S. Chen, T. Cavallari, V. A. Prisacariu, and E. Brachmann (2024) Map-relative pose regression for visual re-localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 20665–20674. Cited by: §2.1. [14] S. Chen, X. Li, Z. Wang, and V. A. Prisacariu (2022) Dfnet: enhance absolute pose regression with direct feature matching. In European Conference on Computer Vision, p. 1–17. Cited by: §2.1. [15] A. Chidester, J. Hinch, T. C. Mercer, and K. S. Schultz (1999) Recording automotive crash event data. In Transportation Recording: 2000 and Beyond. International Symposium on Transportation RecordersNational Transportation Safety BoardInternational Transportation Safety Association, Cited by: §1. [16] B. O. Community (2018) Blender - a 3d modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam. External Links: Link Cited by: §3.2. [17] S. Dong, S. Wang, S. Liu, L. Cai, Q. Fan, J. Kannala, and Y. Yang (2025) Reloc3r: large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 16739–16752. Cited by: §2.2, §5.3, §5.4, §5.5, Table 1, Table 1, Table 1, Table 1, Table 2, Table 2, Table 3, Table 3, Table 3, Table 3, Table 3, §7.3.2. [18] M. Dusmanu, I. Rocco, T. Pajdla, M. Pollefeys, J. Sivic, A. Torii, and T. Sattler (2019) D2-net: a trainable cnn for joint detection and description of local features. In CVPR 2019-IEEE Conference on Computer Vision and Pattern Recognition, Cited by: §2.2. [19] D. Eigen, C. Puhrsch, and R. Fergus (2014) Depth map prediction from a single image using a multi-scale deep network. Advances in neural information processing systems 27. Cited by: §2.3. [20] S. En, A. Lechervy, and F. Jurie (2018) Rpnet: an end-to-end network for relative camera pose estimation. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops, p. 0–0. Cited by: §5.4, §5.4, Table 3, Table 3, Table 3, Table 3. [21] M. A. Fischler and R. C. Bolles (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24 (6), p. 381–395. Cited by: §5.1. [22] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47 (6), p. 2280–2292. Cited by: §3.2. [23] S. Garrido-Jurado, R. Munoz-Salinas, F. J. Madrid-Cuevas, and R. Medina-Carnicer (2016) Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern recognition 51, p. 481–491. Cited by: §3.2. [24] G. H. Golub and C. F. Van Loan (2013) Matrix computations. JHU press. Cited by: §4.2. [25] R. Hartley and A. Zisserman (2003) Multiple view geometry in computer vision. Cambridge university press. Cited by: §2.2. [26] J. Huang, X. Liu, S. Song, R. Hou, H. Chang, J. Lin, and S. Bai (2025) Revisiting multimodal positional encoding in vision-language models. arXiv preprint arXiv:2510.23095. Cited by: §2.3. [27] M. Humenberger, Y. Cabon, N. Guerin, J. Morat, V. Leroy, J. Revaud, P. Rerole, N. Pion, C. De Souza, and G. Csurka (2020) Robust image retrieval-based visual localization using kapture. arXiv preprint arXiv:2007.13867. Cited by: §2.2. [28] O. Idan, Y. Shavit, and Y. Keller (2023) Learning to localize in unseen scenes with relative pose regressors. arXiv preprint arXiv:2303.02717. Cited by: §2.2, Table 2. [29] O. Idan, Y. Shavit, and Y. Keller (2026) Beyond familiar landscapes: exploring the limits of relative pose regressors in new environments. Computer Vision and Image Understanding, p. 104629. Cited by: §2.2. [30] IDTechEx (2024) In-cabin sensing 2024–2034: technologies, opportunities and markets. Technical report IDTechEx Research. Note: Market analysis report projecting rapid growth of in-cabin sensing driven by regulation and OEM adoption External Links: Link Cited by: §1. [31] International Organization for Standardization (2011) Road vehicles — vehicle dynamics and road-holding ability — vocabulary. International Organization for Standardization, Geneva, Switzerland. Cited by: §3.1, §4.3. [32] A. Kendall and R. Cipolla (2017) Geometric loss functions for camera pose regression with deep learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 5974–5983. Cited by: §2.1. [33] A. Kendall, M. Grimes, and R. Cipolla (2015) Posenet: a convolutional network for real-time 6-dof camera relocalization. In Proceedings of the IEEE international conference on computer vision, p. 2938–2946. Cited by: §2.1, §5.4, §5. [34] J. B. Kuipers (1999) Quaternions and rotation sequences: a primer with applications to orbits, aerospace, and virtual reality. Princeton university press. Cited by: §4.2. [35] Z. Laskar, I. Melekhov, S. Kalia, and J. Kannala (2017) Camera relocalization by computing pairwise relative poses using convolutional neural network. In Proceedings of the IEEE international conference on computer vision workshops, p. 929–938. Cited by: §2.2, Table 2. [36] V. Leroy, Y. Cabon, and J. Revaud (2024) Grounding image matching in 3d with mast3r. In European conference on computer vision, p. 71–91. Cited by: §2.3, Table 1. [37] J. Levinson, C. Esteves, K. Chen, N. Snavely, A. Kanazawa, A. Rostamizadeh, and A. Makadia (2020) An analysis of svd for deep rotation estimation. Advances in Neural Information Processing Systems 33, p. 22554–22565. Cited by: §4.2, §5.5. [38] J. Lin, J. Gu, B. Wu, L. Fan, R. Chen, L. Liu, and J. Ye (2024) Learning neural volumetric pose features for camera localization. In European Conference on Computer Vision, p. 198–214. Cited by: §2.2. [39] I. Loshchilov and F. Hutter (2017) Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. Cited by: §5. [40] D. G. Lowe (2004) Distinctive image features from scale-invariant keypoints. International journal of computer vision 60 (2), p. 91–110. Cited by: §2.2, §5.1, §5.3, Table 1. [41] I. Melekhov, J. Ylioinas, J. Kannala, and E. Rahtu (2017) Relative camera pose estimation using convolutional neural networks. In International Conference on Advanced Concepts for Intelligent Vision Systems, p. 675–687. Cited by: §2.2. [42] A. Moreau, N. Piasco, D. Tsishkou, B. Stanciulescu, and A. de La Fortelle (2022) Lens: localization enhanced by nerf synthesis. In Conference on robot learning, p. 1347–1356. Cited by: §2.2. [43] M. Muja and D. G. Lowe (2009) Fast approximate nearest neighbors with automatic algorithm configuration. In International conference on computer vision theory and applications, Vol. 1, p. 331–340. Cited by: §5.1. [44] R. A. Naqvi, M. Arsalan, G. Batchuluun, H. S. Yoon, and K. R. Park (2018) Deep learning-based gaze detection system for automobile drivers using a nir camera sensor. Sensors 18 (2), p. 456. Cited by: §1. [45] D. Nistér (2004) An efficient solution to the five-point relative pose problem. IEEE transactions on pattern analysis and machine intelligence 26 (6), p. 756–770. Cited by: §5.1. [46] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. (2023) Dinov2: learning robust visual features without supervision. arXiv preprint arXiv:2304.07193. Cited by: §2.3, §5. [47] M. Piotrowski, L. Dziuda, and P. Baran (2025-03) AUTOMOTIVE interior monitoring systems: a review of selected technical solutions for the recognition of fatigue symptoms in motor vehicle drivers. The Polish Journal of Aviation Medicine, Bioengineering and Psychology 28, p. 31–41. External Links: Document Cited by: §1. [48] O. Rodrigues (1840) Des lois géométriques qui régissent les déplacements d’un système solide dans l’espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire. Journal de mathématiques pures et appliquées 5, p. 380–440. Cited by: item 1. [49] S. Salamone (2025) In-cabin sensing has automakers looking inward. RTInsights. Note: Industry analysis of OEM adoption of in-cabin sensing driven by safety ratings and semi-autonomous driving External Links: Link Cited by: §1. [50] M. B. Sarıyıldız, P. Weinzaepfel, T. Lucas, P. De Jorge, D. Larlus, and Y. Kalantidis (2025) Dune: distilling a universal encoder from heterogeneous 2d and 3d teachers. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 30084–30094. Cited by: §2.3, §5.2, §5. [51] P. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich (2020) Superglue: learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 4938–4947. Cited by: §2.2. [52] T. Sattler, Q. Zhou, M. Pollefeys, and L. Leal-Taixe (2019) Understanding the limitations of cnn-based absolute camera pose regression. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 3302–3312. Cited by: §2.1. [53] J. L. Schonberger and J. Frahm (2016) Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 4104–4113. Cited by: §3.2. [54] J. Shotton, B. Glocker, C. Zach, S. Izadi, A. Criminisi, and A. Fitzgibbon (2013) Scene coordinate regression forests for camera relocalization in rgb-d images. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 2930–2937. Cited by: §4.3, §5.4, §5. [55] O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, et al. (2025) Dinov3. arXiv preprint arXiv:2508.10104. Cited by: §2.3, §4.1, §5.5, §5. [56] J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu (2024) Roformer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. Cited by: §2.3, §4.1. [57] J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou (2021) LoFTR: detector-free local feature matching with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 8922–8931. Cited by: §2.2. [58] M. C. Sundermeier, H. Dierend, P. Ley, A. Wolf, and R. Lachmayer (2022) Active nir illumination for improved camera view in automated driving application. In Light-Emitting Devices, Materials, and Applications XXVI, Vol. 12022, p. 54–62. Cited by: §1. [59] A. Tsoi, J. Hinch, R. Ruth, and H. Gabler (2013) Validation of event data recorders in high severity full-frontal crash tests. SAE International Journal of Transportation Safety 1 (2013-01-1265), p. 76–99. Cited by: §1, §5.2. [60] M. O. Turkoglu, E. Brachmann, K. Schindler, G. J. Brostow, and A. Monszpart (2021) Visual camera re-localization using graph neural networks and relative pose supervision. In 2021 International Conference on 3D Vision (3DV), p. 145–155. Cited by: §2.2, Table 2. [61] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: §4.1. [62] J. Ventura, Z. Kukelova, T. Sattler, and D. Baráth (2024) Absolute pose from one or two scaled and oriented features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 20870–20880. Cited by: §2.1. [63] S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud (2024) Dust3r: geometric 3d vision made easy. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 20697–20709. Cited by: §2.3, Table 1. [64] Y. Wang (2023) Summarizing regional regulations for mandating driver monitoring systems. Automation.com. Note: Overview of global driver monitoring regulations including EU GSR and ADDW requirements External Links: Link Cited by: §1. [65] P. Weinzaepfel, V. Leroy, T. Lucas, R. Brégier, Y. Cabon, V. Arora, L. Antsfeld, B. Chidlovskii, G. Csurka, and J. Revaud (2022) Croco: self-supervised pre-training for 3d vision tasks by cross-view completion. Advances in Neural Information Processing Systems 35, p. 3502–3516. Cited by: §2.3. [66] D. Winkelbauer, M. Denninger, and R. Triebel (2021) Learning to localize in new environments from synthetic training data. In 2021 IEEE International Conference on Robotics and Automation (ICRA), p. 5840–5846. Cited by: §2.2. [67] H. Xiao, W. Zheng, S. Zuo, P. Gao, J. Zhou, and J. Lu (2024) SpatialFormer: towards generalizable vision transformers with explicit spatial understanding. In European Conference on Computer Vision, p. 37–54. Cited by: §2.3. [68] Y. Yin, J. Lyu, Y. Wang, H. Liu, H. Wang, and B. Chen (2025) Towards robust probabilistic modeling on so (3) via rotation laplace distribution. IEEE Transactions on Pattern Analysis and Machine Intelligence 47 (5), p. 3469–3486. Cited by: §2.1. [69] W. Zhang and J. Kosecka (2006) Image based localization in urban environments. In Third international symposium on 3D data processing, visualization, and transmission (3DPVT’06), p. 33–40. Cited by: §2.3. 7 Supplementary Material: InCaRPose 7.1 Supplementary Tables Table 6: Detailed inference runtime measurements on a single NVIDIA RTX 4090 GPU. We report average per-frame latency (ms), frames per second (FPS), and relative speedup with respect to the FP32 baseline at the corresponding backbone and resolution. Backbone Res. Config Latency (ms) FPS Speedup Small 224 Baseline (FP32) 14.6514.65 68.2868.28 1.001.00 FP16 14.5714.57 68.6468.64 1.011.01 torch.compile 14.0014.00 71.4171.41 1.051.05 FP16 + torch.compile 14.4814.48 69.0669.06 1.011.01 Small 512 Baseline (FP32) 24.7524.75 40.4040.40 1.001.00 FP16 16.4916.49 60.6560.65 1.501.50 torch.compile 24.3824.38 41.0341.03 1.021.02 FP16 + torch.compile 14.7614.76 67.7567.75 1.681.68 Base 224 Baseline (FP32) 14.9114.91 67.0667.06 1.001.00 FP16 14.9314.93 66.9766.97 1.001.00 torch.compile 15.3615.36 65.1265.12 0.970.97 FP16 + torch.compile 14.4614.46 69.1669.16 1.031.03 Base 512 Baseline (FP32) 31.1831.18 32.0732.07 1.001.00 FP16 17.6217.62 56.7556.75 1.771.77 torch.compile 30.7930.79 32.4832.48 1.011.01 FP16 + torch.compile 17.6217.62 56.7456.74 1.771.77 Large 224 Baseline (FP32) 21.5121.51 46.4846.48 1.001.00 FP16 21.7321.73 46.0246.02 0.990.99 torch.compile 17.0417.04 58.6958.69 1.261.26 FP16 + torch.compile 14.6214.62 68.3868.38 1.471.47 Large 512 Baseline (FP32) 62.4962.49 16.0016.00 1.001.00 FP16 30.4430.44 32.8532.85 2.052.05 torch.compile 60.9760.97 16.4016.40 1.021.02 FP16 + torch.compile 24.7024.70 40.4940.49 2.532.53 Table 7: Backbone ablation on COLMAP ground truth (distorted images and 224 resolution). Backbone Metric Rot. Err (∘) Dir. Err (∘) Dune-Base Mean 6.26 52.11 Median 3.88 50.90 DUST3R-Large Mean 6.98 81.52 Median 5.29 73.93 DINOv2-Base Mean 8.11 45.60 Median 5.30 39.58 DINOv3-Base Mean 8.36 48.74 Median 6.14 41.23 7.2 Rotation Representation We investigate different rotation representations and show how each can be mapped back to a uniform rotation matrix for the Universal Loss as described below: 1. Rotation Vector (ℝ3R^3): The rotation is represented by a compact axis-angle vector ω. The vector’s direction specifies the rotation axis u, and its magnitude represents the rotation angle θ=‖ω‖2θ=\|ω\|_2 in radians. The mappingto a rotation matrix R is given by Rodrigues’ [48] formula: R=I+sinθ[ω]×+1−cosθ2[ω]×2R=I+ θ[ω]_×+ 1- θ^2[ω]_×^2 (3) where [ω]×[ω]_× is the skew-symmetric matrix of ω. The final output is =[ω⊤,⊤]⊤y=[ω ,t ] . 2. Euler Angles: Intrinsic Rotation (ℝ3R^3): We support intrinsic rotations (moving axes) using the standard ZYXZYX convention. Given angles (α,β,γ)(α,β,γ), the final rotation matrix is computed by successive rotations around the transformed axes: Rint=Rz(α)Ry′(β)Rx′(γ)R_int=R_z(α)R_y (β)R_x (γ) (4) The final output is =[α,β,γ,tx,ty,tz]⊤y=[α,β,γ,t_x,t_y,t_z] . 3. Euler Angles: Extrinsic Rotation (ℝ3R^3): Extrinsic rotations are performed around the fixed, global axes (X,Y,Z)(X,Y,Z). For a sequence (γ,β,α)(γ,β,α), the resulting matrix is: Rext=Rz(α)Ry(β)Rx(γ)R_ext=R_z(α)R_y(β)R_x(γ) (5) The final output is =[γ,β,α,tx,ty,tz]⊤y=[γ,β,α,t_x,t_y,t_z] . 4. Quaternions (ℝ4R^4): The rotation is represented by a unit quaternion =[w,x,y,z]⊤q=[w,x,y,z] , where ‖2=1\|q\|_2=1. The mapping to R is defined as: R=[1−2(y2+z2)2(xy−wz)2(xz+wy)2(xy+wz)1−2(x2+z2)2(yz−wx)2(xz−wy)2(yz+wx)1−2(x2+y2)]R= bmatrix1-2(y^2+z^2)&2(xy-wz)&2(xz+wy)\\ 2(xy+wz)&1-2(x^2+z^2)&2(yz-wx)\\ 2(xz-wy)&2(yz+wx)&1-2(x^2+y^2) bmatrix (6) The final output is =[⊤,⊤]⊤y=[q ,t ] . 5. Rotation Matrix (ℝ9R^9): The rotation is represented directly by the flattened elements of R∈ℝ3×3R ^3× 3. The matrix must satisfy the constraints of the Special Orthogonal group: SO(3)=R∈ℝ3×3:R⊤R=I,det(R)=+1SO(3)=\R ^3× 3:R R=I, (R)=+1\ (7) The final output is the flattened nine elements of R followed by t, resulting in =[r11,r12,…,r33,tx,ty,tz]⊤y=[r_11,r_12,…,r_33,t_x,t_y,t_z] . If the rotation is described as rotation matrix we can describe the full relative transformation as follows: Trel=Tview1−1Tview2,Tview1≡TrefT_rel\;=\;T_view1^-1\,T_view2, T_view1≡ T_ref Tview=(Rviewtview01),Trel=(Rreltrel01)T_view= pmatrixR_view&t_view\\[4.0pt] 0&1 pmatrix, T_rel= pmatrixR_rel&t_rel\\[4.0pt] 0&1 pmatrix Tview1−1=(Rview1⊤−Rview1⊤tview101)T_view1^-1= pmatrixR_view1 &-R_view1 t_view1\\[4.0pt] 0&1 pmatrix Rrel R_rel =Rview1⊤Rview2, =R_view1 R_view2, (8) trel t_rel =Rview1⊤(tview2−tview1). =R_view1 (t_view2-t_view1 ). (9) 7.3 Error Metrics and Loss Functions The introduction of various transformation representations necessitates a systematic investigation into the loss functions and error metrics tailored to each output format. We observed that the choice of representation significantly impacts optimization behavior and that final model performance is highly sensitive to the specific objective function employed. 7.3.1 Individual Error Metrics Geodesic Distance: Measures the minimum rotation angle required to align the estimated rotation matrix estR_est with the ground truth gtR_gt: erot_geo=arccos(Tr(est⊤gt)−12)e_rot\_geo= ( Tr(R_est R_gt)-12 ) (10) Quaternion Error: Minimizes the angular distance on the hypersphere between unit quaternions estq_est and gtq_gt, accounting for the double-cover property of SO(3)SO(3): erot_quat=2arccos(|est⊤gt|)e_rot\_quat=2 ( |q_est q_gt | ) (11) Euclidean Distance: Measures the absolute metric distance between translation vectors in meters: etrans_eucl=‖est−gt‖2e_trans\_eucl= \|t_est-t_gt \|_2 (12) Translation Direction Error: Measures the angular difference between predicted and ground truth translation vectors, providing a scale-invariant metric: etrans_dir=arccos(est⊤gt‖est‖⋅‖gt‖)e_trans\_dir= ( t_est t_gt\|t_est\|·\|t_gt\| ) (13) 7.3.2 Composite Loss Functions During training, these metrics are combined into the following loss formulations: Universal Transformation Loss This loss handles full transformation matrices ∈SE(3)T∈ SE(3) (rotation and translation). We decompose the matrices back into rotation and translation components and apply a weighted sum of geodesic and Euclidean errors: ℒuniversal=[erot_geo]+α⋅[etrans_eucl]L_universal=E [e_rot\_geo ]+α·E [e_trans\_eucl ] (14) where α is a weighting factor to balance the different units. Reloc3r Loss Inspired by [17], this loss is designed for the estimation of relative poses, where translation is predicted as a unit vector. It combines geodesic rotation error with translation direction error: ℒReloc3r=[erot_geo+α⋅etrans_dir]L_Reloc3r=E [e_rot\_geo+α· e_trans\_dir ] (15) Mean Squared Error (MSE) Loss A standard baseline applied to raw output vectors ∈ℝdp ^d. This is utilized for different output representations, such as Euler angles (d=6d=6) or Quaternions (d=7d=7): ℒMSE=1d‖est−gt‖22L_MSE= 1d \|p_est-p_gt \|_2^2 (16) Quaternion-based Pose Loss: This is our primary loss for the metric estimation. It utilizes the quaternion error for orientation and either Euclidean distance (for metric pose) or direction error (for scale-invariant pose) for translation: ℒquat=[erot_quat]+α⋅[etrans]L_quat=E [e_rot\_quat ]+α·E [e_trans ] (17) The choice of etranse_trans (etrans_eucle_trans\_eucl or etrans_dire_trans\_dir) allows the model to trade off between absolute metric accuracy and directional consistency. 7.4 Datasets We show two different datasets in Fig. 6. The center-crop dataset crops the image, resulting in data loss, while the zero-padded dataset does not cut the image. Instead, it pads the image to make it square. In both pipelines, images are converted to 8-bit RGB and, optionally, undistorted when intrinsics are available. The center-crop dataset resizes and center-crops to the target resolution, then applies standard ImageNet normalization. The zero-padded dataset rescales while preserving the aspect ratio, pads to a square canvas, and applies the same per-channel normalization. The two datasets also demonstrate the trade-off between higher detail per resolution (center-crop dataset) and more border information (zero-padded dataset) while producing a predefined, fixed image size. (a) Center-crop: distorted (b) Center-crop: undistorted (c) Zero-padded: distorted (d) Zero-padded: undistorted Figure 6: Comparison of preprocessing methods. (a) and (b): images are cropped to the center. (c) and (d): images are zero-padded to a square aspect ratio. This adjustment is necessary to handle varying input resolutions within the dataset while maintaining a consistent model input. 7.5 Ground Truth Discussion To assess the reliability of ground truth pose estimation in confined automotive interiors, we compare ArUco-based tracking with trajectories reconstructed using COLMAP. Both translations are normalized to a uniform scale for a consistent comparison. All sequences are expressed in a common local coordinate system defined by a single reference view within the vehicle cabin. This reference view is selected to maximize feature overlap across all other views. This is important to maximize the likelihood that matching ArUco markers are visible across all scenes. All camera poses are represented as relative transformations with respect to this origin. Table 8 summarizes the observed discrepancies between COLMAP and ArUco-based estimates. Rotation error is measured as the angular deviation between the COLMAP-estimated orientation and the ArUco-based ground truth. Since COLMAP recovers translation only in unknown scale, translation error is evaluated exclusively in terms of direction, computed as the angular difference between the estimated and ground-truth translation vectors. To avoid degenerate cases, translation direction error is reported only for frames where the relative Euclidean displacement of the ArUco ground truth exceeds 0.1 m. We apply this threshold to ensure that only frames with meaningful translation contribute to this metric. Table 8: Rotation and translation direction errors. Translation direction error is only evaluated when the relative ArUco ground truth translation exceeds 0.1 m. Metric Value Max rotation error 19.86∘ Mean rotation error 3.08∘ Median rotation error 2.27∘ Max translation direction error 55.83∘ Mean translation direction error 5.75∘ Median translation direction error 4.60∘ As illustrated in Fig. 7 and Fig. 9, discrepancies between the two methods arise in corner views. In particular, COLMAP occasionally fails to recover vertical displacement (translation along the y-axis) or rotation around the z-axis, resulting in poses that are inconsistent with the physical camera placement. Such failure cases are characteristic of confined interior environments, where limited baseline, weak texture, and reflective surfaces can lead to ill-conditioned structure-from-motion reconstructions. Figure 7: COLMAP fails to estimate translation along the y-axis. In the first view the camera is to the side of the steering wheel. In the second view the camera moved upwards (y) above the steering wheel. Translation is normalized. Given that ArUco markers provide metric-scale translation and yield more physically plausible poses in these edge cases, we adopt the ArUco-based estimates as the primary ground truth for all quantitative evaluations in this work. For completeness and reproducibility, we additionally release the corresponding COLMAP-based trajectories. Figure 8: Rotation error in degrees versus the number of parameters. Evaluated on image resolution of 224 on the In-Cabin-Pose dataset. (a) X-Translation (b) Y-Translation (c) Z-Translation (d) Z-Rotation (e) Y-Rotation (f) X-Rotation Figure 9: ArUco (orange) vs. COLMAP (blue) camera trajectories for intervals focused on specific transformations. Each sequence predominantly captures motion along the target axis, though residual degrees-of-freedom are also present. 7.6 ArUco-free Inference We also provide several samples with occluded ArUco markers, shown in Fig. 10. These examples demonstrate that the model does not rely on the presence of ArUco markers in the real‑world test data to make its predictions. For this data collection, we first captured frames in which the ArUco markers were fully visible. We then physically occluded the markers and recorded additional frames. Finally, we assigned the ground‑truth poses from the visible‑marker frames to the corresponding occluded‑marker frames. (a) (b) Y (c) Z (d) Y Figure 10: Inference on frames with physically occluded ArUco markers. We also changed the reference image, in which a different object (a high-contrast checkerboard on the left seat) is now visible in only one image to additionally challenge the model.