Paper deep dive
Do LiDAR Language Models Really Understand Spatio-temporal Relationships?
Runyi Yang, Murat Akkoyun, Di Wen, Ruiping Liu, Yufan Chen, Junwei Zheng, Xiaoye Wang, Kailun Yang, Danda Pani Paudel, Luc Van Gool, Kunyu Peng
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the multiple-choice accuracy of two B4DL-derived configurations. We introduce LiDAR-Hallu, a geometry-referenced benchmark and diagnostic protocol with 10,000 questions across 150 nuScenes scenes. It covers object existence, ego-relative position, distance ordering, relative motion, and temporal localization, with explicit rules for selecting objects, comparing times, and determining reference answers. Our protocol combines fixed-answer and candidate-content controls, cross-scene pairs with identical prompts but opposite reference answers, and relation-specific recall. Analysis of 100,000 recorded responses reveals failures hidden by aggregate accuracy. Candidate duration alone makes temporal answers predictable without observing LiDAR. On paired questions, the models frequently give the same answer to scenes requiring opposite answers. Relation-specific analysis further shows that both configurations miss every positive lateral-motion case across all tested conditions. Temporal-shuffle contrastive decoding provides little net improvement, as repairs are largely offset by new errors and the main failures persist. These results show that evaluating spatio-temporal reasoning requires testing whether models distinguish the queried physical relationships, rather than relying on individual-answer accuracy alone. The source code, checkpoints, and data are released at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2609.24452v1
- Canonical: https://arxiv.org/abs/2609.24452v1
Trouble viewing inline? Open PDF directly →
Full Text
39,222 characters extracted from source content.
Expand or collapse full text
Do LiDAR Language Models Really Understand Spatio-temporal Relationships? Runyi Yang 1,∗ Murat Akkoyun 2,∗ Di Wen 2 Ruiping Liu 2 Yufan Chen 2 Junwei Zheng 2 Xiaoye Wang 1 Kailun Yang 3 Danda Pani Paudel 1 Luc Van Gool 1 Kunyu Peng 2,† During frames 025–033, is the nearest car moving back relative to the ego vehicle? Ground truth: Model answer: Yes No During frames 025–033, is the nearest car moving forward relative to the ego vehicle? Ground truth: Model answer: No No (a) Same Reference with Different Prompt: Same Prediction During frames 025–033, is the busmoving closerrelative to the ego vehicle? Ground truth No Yes Modelanswer No No (b) Same Prompt with Different Reference: Same Prediction Percentage of answers choosing option A (B4DL, default decoding) Spatial position 95.60% Distance ordering 99.30% Temporal localization100.00% When does the pedestrian enter the crosswalk? (c) MCQ Collapse to a fixed answer Fig. 1.Response collapse in LiDAR QA. (a) Different motion queries about the same car receive “No” despite opposite reference answers. (b) The same query receives “No” in two scenes requiring opposite answers. (c) Option-A selection rates for B4DL (MCQ stands for Multi-choice Question). Abstract— Recent 4D LiDAR language models aim to reason about objects and their evolving spatial relationships. Yet, in our evaluation, always selecting the same option nearly matches the multiple-choice accuracy of two B4DL-derived configurations. We introduce LiDAR-Hallu, a geometry-referenced benchmark and diagnostic protocol with 10,000 questions across 150 nuScenes scenes. It covers object existence, ego-relative position, distance ordering, relative motion, and temporal localization, with explicit rules for selecting objects, comparing times, and determining reference answers. Our protocol combines fixed- answer and candidate-content controls, cross-scene pairs with identical prompts but opposite reference answers, and relation- specific recall. Analysis of 100,000 recorded responses reveals failures hidden by aggregate accuracy. Candidate duration alone makes temporal answers predictable without observing LiDAR. On paired questions, the models frequently give the same answer to scenes requiring opposite answers. Relation- specific analysis further shows that both configurations miss every positive lateral-motion case across all tested conditions. Temporal-shuffle contrastive decoding provides little net im- provement, as repairs are largely offset by new errors and the main failures persist. These results show that evaluating spatio-temporal reasoning requires testing whether models distinguish the queried physical relationships, rather than relying on individual-answer accuracy alone. The source code, checkpoints, and data are released at this repository. 1 INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria 2 Karlsruhe Institute of Technology, Germany 3 Hunan University, China *Equal contribution †Corresponding author: kunyu.peng@kit.edu I. INTRODUCTION Autonomous driving requires understanding where sur- rounding objects are and how they move relative to the ego vehicle [1]. Decisions about yielding, passing, and maintain- ing separation depend on which obstacle is closest, whether another vehicle is approaching, and when an event occurs [2]. Recent methods connect LiDAR observations with language to support scene understanding [3], [4], while question- answering benchmarks assess these capabilities. Such judg- ments require more than recognizing objects. Determining whether a car is approaching requires following the same car and comparing its distance to the ego vehicle over the specified interval. B4DL [5] extends LiDAR-language modeling to point- cloud sequences, providing a benchmark and an accompa- nying model for spatial and temporal question answering. However, average accuracy can conceal systematic errors. Figure 1(a) shows two questions about the same car over the same interval, asking whether it is moving back or forward relative to the ego vehicle. The reference answers are “Yes” and “No”, but the model answers “No” to both. It answers one question correctly while missing the motion identified by the reference. Panel (b) holds the question fixed and changes the scene: the reference answer changes, but the model again answers “No” to both. In multiple-choice questions (MCQ), always selecting “A” nearly matches the accuracy of the two arXiv:2609.24452v1 [cs.CV] 21 Sep 2026 B4DL-derived configurations evaluated here. Panel (c) shows this fixed-option behavior across spatial and temporal tasks. Evaluation must therefore examine both answers that are predictable without the scene and relations that a model fails to distinguish. Existing driving benchmarks cover object- centric questions [6], perception and planning [2], spatial measurements [7], and cross-view temporal reasoning [8]. In static 3D question answering, text-only models can match or exceed 3D models on SQA3D [9]. Testing for answer biases is therefore only a first step, as it reveals whether predictions can be driven by textual or distributional cues without observing the scene. We further use paired questions and relation-specific recall to test whether the model actually distinguishes the physical relations that determine the correct answer. We introduce LiDAR-Hallu, a benchmark and diagnos- tic protocol with 10,000 questions across 150 nuScenes scenes [1]. Its five task families cover existence, position, distance, motion, and event timing. Each family specifies the queried entities, relevant times, coordinate frame, and answer rule. The first four derive answers from nuScenes annota- tions. Temporal localization retains B4DL’s event intervals and adds three distractors (Figure 2). Table I compares this combination of sequence question answering and diagnostic evaluation with prior benchmarks. We evaluate two B4DL-derived configurations under five inference settings, obtaining 100,000 responses to the same 10,000 questions. Three questions guide the study: RQ1. What can be answered without observing LiDAR? We measure fixed-answer performance and predict answers from questions and candidate text using statistics from other scenes. RQ2. Can models distinguish opposite physical relations? We measure joint correctness on exact-prompt cross- scene pairs and positive recall for individual relations. RQ3. Can inference-time interventions correct these errors? We evaluate anti-position prompting and Temporal- Shuffle Contrastive Decoding (TSCD), counting re- paired and newly introduced errors separately. Our contributions are threefold: • A 10,000-question LiDAR benchmark with five task families, explicit reference-answer rules, and executable integrity checks. • A diagnostic protocol combining answer-bias controls, exact-prompt pairs, relation-specific recall, and paired correction analysis. • An analysis of 100,000 responses that identifies fixed- answer behavior, systematic relation omissions, and the limited benefit of the tested inference-time interven- tions. I. RELATED WORK Driving-scene question answering. NuScenes-QA gener- ates questions about object existence, counts, states, and spatial relations from annotated scene graphs [6]. Driv- eLM links perception, prediction, and planning through Frame 027 (t) +x +y ego same bus y = −5.74 m Frame 035 (t + 8) +x +y ego same bus y = −1.03 m Δy = y ₃₅ − y ₂₇ = +4.71 m ≥ 1 m During frames 027–035, does the nearest bus shift left relative to the ego vehicle? Reference Yes (a) Annotation-derived geometric reference Frame 010 x = +8.14 m Frame 012 x = +6.59 m Frame 017 x = −1.96 m Frame 022 x = −14.80 m ↑ forward · blue: ego frame · orange: GT footprint When does the truck begin to move towards the rear view from the front perspective? Inherited source interval: [012, 020] Reference C A[019, 029] B[024, 031] C[012, 020]source D[020, 032] 000012020030039Frame index (b) Inherited temporal reference Fig. 2. Question and reference construction. (a) Tracking the same bus gives its ego-relative lateral change. (b) A source event interval and three distractors form a temporal-localization question. graph-structured questions [2]. LingoQA evaluates scene descriptions, anticipation, and action justification in driving videos [10], while DriveGPT4 combines video question an- swering and action explanations with vehicle-control predic- tion [11]. NuScenes-SpatialQA focuses on qualitative spatial relations and quantitative measurements [7], and DriveSpatial examines cross-view and temporal reasoning [8]. LiDAR-language modeling. LidarCLIP aligns automotive point clouds with CLIP embeddings for retrieval and cross- modal applications [3]. LiDAR-LLM connects point clouds to captioning, question answering, and planning through a view-aware transformer and staged training [4]. B4DL extends this setting to sequences, with spatial reasoning, temporal localization, and scene-dynamics tasks [5]. LiDAR- Hallu builds on this setting with explicit geometric reference rules and tests whether answers distinguish the corresponding relations, as shown in Table I. Answer biases and relational evaluation. VQA v2 pairs TABLE I DRIVING-SCENE LANGUAGE BENCHMARKS: SENSOR INPUT, TEMPORAL COVERAGE, AND DIAGNOSTIC EVALUATION. DATASET CHARACTERISTICS FOLLOW TABLE 1 OF B4DL [5]. THE DIAGNOSTIC COLUMNS INDICATE REPORTED EVALUATIONS. Dataset characteristicsDiagnostic evaluation DatasetInput LiDAR- specific 360 ◦ coverage Multi- frameSequence Sequence annotation Answer-bias controls Exact-prompt pairs Relation recall DriveLM [2]Camera ×✓× LingoQA [10]Camera ×✓× DriveGPT4 [11]Camera ×✓× nuScenes-QA [6] Camera + LiDAR✓×✓× LiDARLLM [4]LiDAR✓× B4DL [5]LiDAR✓× LiDAR-Hallu (ours)LiDAR✓ identical questions with complementary images requiring different answers [12], while VQA-CP changes question- conditioned answer distributions between training and test- ing [13]. NuScenes-QA and LingoQA include question-only or text-only baselines [6], [10]. Real-3DQA studies textual shortcuts in 3D question answering and filters questions readily answered without 3D input [9]. Multiple-choice stud- ies also identify option-order sensitivity and weaknesses in binding answer symbols to candidates [14], [15]. Our pro- tocol measures fixed-answer behavior and candidate-content predictability separately, then tests joint correctness under identical complete prompts and opposite reference answers. Diagnostic benchmarks and mitigation. Winoground and ARO test compositional, attribute, and relation distinc- tions [16], [17]. Beacon3D examines object-centric de- scriptions and agreement between grounding and question answering [18]. 4D-Bench and Spatial4D-Bench address dynamic objects and spatio-temporal reasoning [19], [20]. CHAIR, POPE, and AMBER assess unsupported object and relational claims [21]–[23], while VidHalluc and EventHal- lusion study temporal and event-level errors [24], [25]. We measure both false assertions and missed true relations in LiDAR sequences. For correction, visual and temporal con- trastive decoding methods compare predictions from original and perturbed inputs [25], [26]. We adapt temporal shuffling to LiDAR features and measure whether its repairs outweigh the errors it introduces. I. THE LIDAR-HALLU BENCHMARK LiDAR-Hallu contains 10,000 unique questions across 150 nuScenes scenes [1], with 2,000 per family (Table I). Object existence, ego-relative position, distance ordering, and relative motion derive answers from object annotations and ego poses. Temporal localization retains B4DL’s event questions and reference intervals [5]. Figure 3 summarizes their construction. (A) Spatial-oriented QA (a) Object existence egoobject (b) Ego-relative position egoobject (c) Distance ordering egoobjects (d) Relative motion t + 8 egoobject (B) Temporal-oriented QA 1 Read GT interval 2 Generate distractors 3 Shuffle into MCQ 012020 [012, 020] A[019, 029] B[024, 031] D[020, 032] A[019, 029] B[024, 031] C[012, 020] D[020, 032] (C) Summary 10,000 unique questions 150 scenes · 5 families 4,000 binary · 6,000 MCQ Fig. 3.Benchmark overview. Four task families use object annotations and ego poses. Temporal questions use source event intervals and sampled distractors. A. Reference answers and coordinates Each geometry-derived question specifies entities E q , frames T q , a coordinate frame F q , and an answer rule φ q : C q = (E q ,T q ,F q ,φ q ), y q = φ q (A;E q ,T q ,F q ), (1) where A denotes the nuScenes validation annotations. The rule evaluates class membership, a spatial or motion relation, or distance ordering. For global object center p o,t and ego pose (R t ,t t ), we compute p e o,t = R ⊤ t (p o,t − t t ), d o,t = q (x e o,t ) 2 + (y e o,t ) 2 . (2) The ego axes point forward (x), left (y), and up (z). Distance is measured from the ego origin to the object center in the ground plane. We use ten classes: car, truck, bus, pedestrian, bicycle, motorcycle, trailer, construction vehicle, barrier, and traffic cone. Annotations reporting zero LiDAR returns are excluded. The index t follows the annotated keyframes at 2 Hz [1]. TABLE I BENCHMARK COMPOSITION AND REFERENCE ANSWERS. MCQ DENOTES A FOUR-CHOICE QUESTION. FamilyBinary MCQ Reference Existence1,000 1,000 Class membership Position1,000 1,000 Ego-frame sector Distance0 2,000 Nearest class Motion2,0000 Same-instance deltas Temporal0 2,000 Source interval Total4,000 6,000 150 scenes B. Question construction Object existence. Binary questions ask whether a class is present at a specified keyframe, with 500 positive and 500 negative examples. Each of the 1,000 multiple-choice questions contains one present class and three absent classes. Ego-relative position. We assign the nearest instance of the queried class to one of eight 45 ◦ bearing sectors: front, front- left, left, back-left, back, back-right, right, and front-right. Binary questions balance correct and incorrect proposed sec- tors. Multiple-choice questions contain the reference sector and three alternatives. Distance ordering. Classes are ranked by the distance of their nearest instance. The closest class is the reference, with three other present classes as distractors. Near-tied distances are retained. Relative motion. We select the nearest instance of the queried class at keyframe t and follow it to t + 8, approximately 4 s later, requiring annotations at both endpoints (Figure 2(a)). Using each endpoint’s ego coordinates, we compute ∆d = d o,t+8 − d o,t and ∆y = y e o,t+8 − y e o,t . The prompts use five qualitative relations, with positive labels defined by closer/farther : ∆d≤−2 m / ∆d≥ 2 m, left/right : ∆y ≥ 1 m / ∆y ≤−1 m, similar : |∆d| < 1 m ∧ |∆y| < 0.5 m.(3) Each predicate is evaluated independently, receiving “Yes” when its condition holds and “No” otherwise. Sub-threshold directional changes remain negative candidates. We sam- ple 1,000 positive and 1,000 negative questions across the five predicates. Temporal localization. Each source event interval is paired with three distinct, in-bounds distractors (Figure 2(b)). A distractor [s,e] has frame-index span ℓ = e− s ∈ 5,..., 19, covering 6–20 frames including both endpoints. Its intersection with the reference, divided by their enclosing span, must be at most 0.2. All spans use endpoint differences. The four candidates are randomly ordered. C. Answer distribution and records The 6,000 multiple-choice questions have A/B/C/D refer- ence counts of 1,474/1,572/1,453/1,501. Among the temporal references, 242 spans fall outside the distractor range of 5– 19. We measure the resulting candidate-content predictability in Sec. IV. Records store prompts, candidates, references, scene/frame identifiers, and motion deltas. All prompt variants share the same questions and labels. Integrity checks verify record correspondence, candidate uniqueness, motion labels, in- terval bounds, and temporal-shuffle permutations using the recorded feature-sequence lengths of 39–41 steps. D. Exact-prompt contrast pairs Within each binary family, we group questions by com- plete prompt and feature-sequence length. Deterministic maximum bipartite matching pairs opposite reference an- swers from different scenes, using each question at most once. Selection follows a fixed scene and question-index order without using model predictions. B4DL has 348 pairs: 104 existence, 41 position, and 203 motion, covering 696 questions and 140 scenes. B4DL- Ego additionally requires identical ego-motion text, pro- ducing a separate set of 38 pairs across 56 scenes. These configuration-specific sets remain fixed across inference set- tings and are evaluated using joint correctness in Sec. IV. IV. A DIAGNOSTIC EVALUATION PROTOCOL We test answer biases, discrimination between physical relations, and the net benefit of inference-time correction. Accuracies and rates in the equations are proportions. A. Answer parsing and fixed-answer controls We map each response r i to a label ˆy i = P(r i ). The parser accepts explicit Yes/No answers, A–D option labels or answer declarations, and unique exact matches to candidate text. A leading explicit label takes precedence over the explanation. Invalid responses count as incorrect, and valid- response rates are reported separately. The fixed-answer controls select A for multiple-choice questions and No for binary questions. On N multiple-choice questions, let A denote model accuracy, π A the proportion of reference answers labeled A, and s A the model’s A-selection rate. If R A counts correct non-A predictions and D A counts reference-A questions answered otherwise, then A− π A = R A − D A N , |A− π A |≤ 1− s A .(4) Thus only departures from A can change the score relative to that fixed answer. B. Candidate-content predictability For each scored scene s, we estimate answer frequencies from all other scenes within the same task. The controls use reference labels to estimate these frequencies and only ques- tion and candidate text to predict answers. Multiple-choice features are class names for existence and distance, sectors conditioned on the queried class for position, and frame- index spans for temporal localization. The temporal control uses neither event descriptions nor absolute endpoints. Let n + −s,c (v) and n − −s,c (v) count reference and distractor occurrences of feature v, excluding scene s. Context c is the queried class for position and the task alone for existence, distance, and temporal localization. For candidate feature v k , we use ρ −s,c (v) = n + −s,c (v) + 1 n − −s,c (v) + 1 ,ˆy = arg max k ρ −s,c (v k ). (5) Here k ranges over option labels A–D, with ties resolved by candidate order. Binary controls predict the majority label conditioned on class, class–sector, or class–action, break- ing ties as No. These leave-one-scene-out controls measure predictability within the benchmark. We also evaluate the models on questions the controls answer incorrectly. C. Joint correctness under identical prompts For the M pairs in Sec. I-D, let (ˆy i , ˆy ′ i ) denote predic- tions and (y i ,y ′ i ) their opposite binary references. A pair is correct only when both answers are correct: J = 1 M M X i=1 1[ˆy i = y i ∧ ˆy ′ i = y ′ i ].(6) We report J , individual-question accuracy, and the same- answer rate. A deterministic Yes/No predictor using only the shared prompt achieves 50% individual accuracy and zero joint correctness. Under independent prompt-only sampling, expected joint success for a pair is p(1−p)≤ 1/4, where p is the probability of answering Yes. These give deterministic and sampling references for interpreting the paired results. D. Relation recall and correction accounting For each binary relation, we report positive and negative counts, true positive recall (TPR), and false-positive rate (FPR). On a balanced subset with all responses valid, A = 1 2 (TPR + 1− FPR).(7) Reporting both rates shows whether higher accuracy comes from recognizing more true relations or rejecting more false ones. We compare inference settings on the same N questions. If R counts wrong-to-correct repairs and D counts correct- to-wrong regressions, then ∆A = (R− D)/N.(8) Wrong-to-wrong changes are recorded separately. For item accuracy and accuracy differences, 95% confidence inter- vals use 10,000 whole-scene bootstrap samples with seed 20260904. Each draw recomputes question-weighted scores and keeps predictions from compared settings paired. Predic- tions and fitted content-control outputs remain fixed during resampling. V. EXPERIMENTS A. Models, training, and inference Architecture and inputs. We evaluate B4DL and B4DL- Ego, two B4DL-derived checkpoints instruction-tuned sep- arately on B4DL’s nuScenes-based data [5]. Both use a learned projector to map cached LiDAR-CLIP features [3] into Vicuna-1.5-7B token embeddings, as shown in Fig- ure 4. B4DL takes LiDAR features and question text as 4D LiDAR scene nuScenes / B4DL sequence t₁t₂t₃ · tₜ t LiDAR-CLIP encoder Ordered LiDAR features f₁, f₂, ..., fₜ · Cached scene features (about 40 time steps) Full sequence supplied M projector LiDAR features to language-token embeddings LiDAR tokens · Question prompt “Is there a vehicle in front of the ego car?” Text tokenizer Text tokens · + Vicuna-1.5-7B LLM Answer Ye s / N o A / B / C / D concat Fig. 4.Evaluated model architecture. Projected LiDAR-CLIP features and question tokens form the input to Vicuna-1.5-7B. B4DL-Ego also includes ego-motion text. input, while B4DL-Ego also receives ego-motion text. All questions, including single-frame queries, use the full scene sequence of 39–41 feature steps in temporal order. Ego- motion text summarizes ego speed, displacement direction, heading change, and acceleration from local pose changes at two selected endpoints. The enclosing span is determined by all frame intervals in the prompt, including those in the answer choices. Single-frame queries use a five-frame window adjusted at scene boundaries, while prompts with- out frame indices use the scene endpoints. Neither model receives object annotations or reference answers at inference. Training configuration. The archived training configuration specifies one epoch of projector alignment at a learning rate of 10 −3 , followed by two epochs of Vicuna instruction tuning at 10 −4 with the projector frozen. Instruction tuning uses LoRA with rank 64 and α LoRA = 128, combining B4DL’s stage-2 and stage-3 records with ego-motion text added for B4DL-Ego. Alignment utilizes a per-GPU batch size of 16 with eight gradient-accumulation steps, while instruction tuning applies a per-GPU batch size of eight with 16 accumulation steps. Both stages use fixed cached features, FP16, cosine learning-rate decay, 3% warmup, zero weight decay, and a 2,048-token context limit. Inference. We evaluate each model with default decoding, anti-position prompting, and TSCD at α ∈ 0.5, 1, 1.5. Standard prompts request Yes/No or an A–D label. Anti- position prompting adds “Do not favor any option based on its position; evaluate all options equally before selecting your answer.” to MCQ instructions only, leaving candidate order and binary prompts unchanged. All settings sample at temperature 0.05, one beam, and at most 32 new tokens. We generate one response per question and setting, yielding 100,000 responses from the two models on the same 10,000 questions. Invalid responses are counted as incorrect under the scoring protocol in Sec. IV. B. Temporal-shuffle contrastive decoding Following visual and temporal contrastive decoding [25], [26], TSCD contrasts the ordered feature sequence H with a temporal permutation e H , drawn once per question. Both branches use the same checkpoint, question, and ego-motion text when present. At decoding step j, they share the gen- erated answer prefix but maintain separate key/value caches. The combined logits are z α j = z c j + α(z c j − z s j ),(9) TABLE I ACCURACY (%) BY TASK AND ANSWER FORMAT. B: BINARY; M: FOUR-CHOICE. FIXED ANSWERS ARE NO FOR B AND A FOR M. CONTENT CONTROLS USE STATISTICS FROM OTHER SCENES. TaskFixed B4DL B4DL-Ego Content Existence (B)50.0066.3067.2072.10 Existence (M) 25.0026.7026.3066.10 Position (B)50.0045.7044.5054.90 Position (M)25.5026.1027.0029.10 Motion (B)50.0052.4050.0059.05 Distance (M)24.3024.3024.4052.70 Temporal (M) 24.1524.1524.1569.30 Overall34.7436.6536.2158.43 During which frames do pedestrians appear on the sidewalks? frame 010203040 A [13,18]: model B [7,11]: reference C D Reference span: 4. Distractor range: 5–19. Fig. 5. Temporal candidate spans. The span-only control selects B, while B4DL selects A under default decoding (record 5, scene 006599465). where c and s denote the clean and shuffled branches, respectively. For α > 0, the contrastive term increases the logits of tokens whose clean logits exceed shuffled logits. The permutation and sampling seeds are 42+i and 1234+ i, respectively, where i is the zero-based question index within each task family. TSCD uses a token-wise decoding loop without an adaptive token-plausibility mask. Default and anti-position runs use model.generate without an explicit sampling seed. C. RQ1: Fixed answers and candidate cues Multiple-choice accuracy largely follows a fixed answer. Under default decoding, both models remain within 1.70 percentage points of constant A across the four MCQ tasks as in Table I. Both select A in more than 96% of distance- ordering and temporal-localization responses, constraining their accuracy differences from constant A, per Eq. (4). The margins are larger on binary existence questions, i.e., B4DL and B4DL-Ego achieve 66.30% and 67.20% accu- racy, respectively, compared with 50.00% for constant No. B4DL-Ego produces more varied spatial-position answers, but achieves 27.00% , compared with 26.10% for B4DL. Candidate contents remain informative. The scene-held- out controls, in Table I, are fitted using reference labels from the other benchmark scenes as in Sec. IV. Candidate class names yield 66.10% accuracy on existence MCQs and 52.70% on distance ordering. Although every distance can- didate is present in the scene, its class remains informative about whether it is the nearest object. TABLE IV B4DL ON 348 EXACT-PROMPT PAIRS WITH OPPOSITE REFERENCE ANSWERS. THE PROMPT-ONLY ROW GIVES THE DETERMINISTIC REFERENCE. ConditionItem acc. (%) Both correct Joint (%) Prompt-only null50.000/3480.00 Baseline53.3043/34812.36 Anti-position51.8739/34811.21 TSCD α = 0.552.1642/34812.07 TSCD α = 152.3042/34812.07 TSCD α = 1.552.0140/34811.49 TABLE V MOTION LABELS, POSITIVE RECALL, AND ACCURACY (%) UNDER DEFAULT DECODING. Positive recallAccuracy Relation Yes / NoB4DL B4DL-Ego B4DL B4DL-Ego Closer206 / 2094.377.7745.7844.58 Farther347 / 14130.267.2043.4433.81 Left145 / 1990.000.0057.8557.85 Right157 / 2210.000.0058.4758.47 Similar145 / 23055.8661.3860.2761.33 Temporal localization shows the clearest candidate cue. The span-only control achieves 69.30% accuracy without event descriptions or LiDAR input, whereas both models always select A and score 24.15%. In Figure 5, the reference intervel [7, 11] has span 4, below the permitted distractor range of 5–19, which length reveals the answer. The control selects it, whereas B4DL stays with A. D. RQ2: Distinguishing physical relations Individual accuracy hides failed contrasts. On 348 exact-prompt pairs with opposite reference answers, B4DL achieves 53.30% accuracy on individual questions but an- swers both questions correctly in only 43 pairs as in Table IV. Most pairs receive identical answers, and only one of the 41 spatial-postion pairs is jointly correct. A deterministic prompt-only predictor has zero joint correctness on these pairs, although sampling can produce nonzero joint correct- ness without scene information. Anti-position prompting and TSCD improves yield 39 and 40–42 jointly correct B4DL pairs, respectively, compared with 43 under default decoding. On the separate 38-pair B4DL-Ego set, four pairs are jointly correct with TSCD, compared with two under default decoding, while most pairs remain unresolved. Each configuration is compared across inference settings on its own fixed pair set. Errors concentrate in particular relations.Table V explains why motion accuracy alone gives an incomplete picture. Both models answer No to every left- and right- motion question under all evaluated settings. All 302 positive lateral-motion cases are missed. The corresponding accura- cies of 57.85% and 58.47% equal the proportions of negative examples, with zero positive recall for both predicates. Q:In frame 024, which object class is closest to the ego vehicle? A. carB. barrier C. pedestrianD. truck GT: A. car B4DL: A. car B4DL-Ego: A. car B4DL + TSCD: A. car Q: In frame 032, at which position is the nearest car relative to the ego vehicle? A. back-leftB. left C. rightD. front GT: B. left B4DL: A. back-left B4DL-Ego: D. front B4DL + TSCD: D. front Q:Which of the following object classes is visible in frame 034? A. bicycleB. barrier C. carD. construction vehicle GT: C. car B4DL: C. car B4DL-Ego: A. bicycle B4DL + TSCD: C. car Q: At what frames should the driver be aware of the large truck transitioning from right to rear view? A. from frame 023 to frame 036B. from frame 005 to frame 015 C. from frame 006 to frame 013D. from frame 012 to frame 020 GT: D. [012, 020] B4DL: A. [023, 036] B4DL-Ego: A. [023, 036] B4DL + TSCD: A. [023, 036] Q: During frames 001-009, does the nearest car move farther away from the ego vehicle? GT: Yes B4DL: No B4DL-Ego: No B4DL + TSCD: No Q: During frames 001-009, does the nearest car move farther away from the ego vehicle? GT: No B4DL: Yes B4DL-Ego: No B4DL + TSCD: Yes (A) (B) (C) (D) (E) (F) Fig. 6. Qualitative results on LiDAR-Hallu. Examples cover distance, position, existence, temporal localization, and paired motion questions. Predictions compare default decoding with TSCD. TABLE VI POSITIVE RECALL (TPR) AND FALSE-POSITIVE RATE (FPR), IN %, UNDER DEFAULT DECODING. TPR↑FPR↓ FamilyB4DL B4DL-Ego B4DL B4DL-Ego Existence49.8040.6017.206.20 Position69.8063.8078.4074.80 Motion19.5013.0014.7013.00 Recall also varies between distance-change predicates, i.e., closer recall is below 8% for both models, whereas farther recall is 30.26% for B4DL and 7.20% for B4DL-Ego. Recall for the predefined Similar predicate is higher, at 55.86% and 61.38%, respectively, indicating relation-specific errors. Table VI reveals that B4DL-Ego has a lower false-positive rate than B4DL on binary existence questions, but also lower positive recall. The reduction in false assertions outweighs the increase in missed positive claims, yielding accuracies of 67.20% and 66.30%, respectively. False assertions are more frequent on position questions, where false-positive rates are 78.40% for B4DL and 74.80% for B4DL-Ego. E. RQ3: Do inference-time changes repair errors? Relative to default decoding, TSCD accuracy gains are 0.03–0.04 percentage points for B4DL and 0.15–0.19 for TABLE VII PAIRED INTERVENTION RESULTS. R/D: REPAIRS/REGRESSIONS. ACCURACY CHANGES AND 95% SCENE-BOOTSTRAP INTERVALS ARE IN PERCENTAGE POINTS. PROMPT DENOTES ANTI-POSITION PROMPTING. ModelIntervention R D∆A 95% interval B4DLPrompt5865-0.07 [-0.27, +0.13] B4DLα = 0.5117 114 +0.03 [-0.25, +0.31] B4DLα = 1119 115 +0.04 [-0.25, +0.34] B4DLα = 1.5119 116 +0.03 [-0.27, +0.33] B4DL-Ego Prompt50 117-0.67 [-0.93, -0.41] B4DL-Ego α = 0.5161 146 +0.15 [-0.22, +0.52] B4DL-Ego α = 1161 143 +0.18 [-0.19, +0.54] B4DL-Ego α = 1.5161 142 +0.19 [-0.19, +0.55] B4DL-Ego (Table VII). All six 95% scene-bootstrap con- fidence intervals include zero. For B4DL-Ego at α = 1.5, 161 errors are corrected but 142 previously correct answers become incorrect, leaving a net gain of 19 out of 10,000 questions. The dominant failures are unchanged: temporal answers remain A and lateral-motion answers remain No. Anti-position prompting also fails to improve aggregate accuracy and reduces response validity for B4DL-Ego. An instruction to avoid positional preference does not supply the missing distinction between candidates. Since this instruction changes only multiple-choice prompts, variation in binary answers reflects the separately sampled runs rather than a change to those prompts. In Figure 6, TSCD changes an incorrect spatial answer to another incorrect answer in (B), preserves a correct non- A answer in (C), and leaves the incorrect temporal choice unchanged in (D). The motion pair in (E,F) illustrate the distinction between rejecting a false relation. F. Scoring and indexing checks Strict parsing changes fewer than 0.1% of responses and leaves the dominant answer patterns unchanged. Saved reference indices and temporal permutations are valid for the recorded feature lengths. The dominant response pat- terns therefore persist after correcting the identified scoring discrepancies. VI. DISCUSSION AND LIMITATIONS What the diagnostics add. The observed errors do not follow a single preference for “No” or “A”. Multiple-choice responses favor A, lateral-motion questions receive No, and position questions often elicit false Yes answers. This task dependence matters when comparing checkpoints: a higher average can result from rejecting more false claims while detecting fewer true relations. The protocol identifies these trade-offs and the relations on which they occur. For driving- related questions, separate measures of missed objects, false assertions, and paired discrimination indicate what needs to improve, rather than treating all accuracy gains as equivalent. Reference quality and evaluation scope. The benchmark evaluates two configurations from one model family and inherits its temporal intervals from B4DL. Sparse returns, near-tied distances, and boundary cases warrant an inde- pendent annotation and observability audit. The motion prompts use qualitative words with threshold-based labels, so small directional changes can receive negative references. A threshold-explicit prompt evaluation would help quantify this ambiguity. Broader model coverage, option permutations, and same-checkpoint tests with and without LiDAR would further separate answer-interface effects from failures to use geometric evidence. The current experiments concern ques- tion answering, rather than closed-loop driving performance. Temporal contrast and correction. TSCD rewards clean– shuffled disagreement, which helps only when that disagree- ment favors the correct answer. A logit component shared by both branches remains in Eq. (9), so temporal contrast alone does not remove a shared answer preference. Testing this explanation requires branch logits and matched decoding runs. In particular, α = 0, identity-permutation controls, and repeated sampling seeds would isolate the contrastive term from differences in the decoding loop and sampling. VII. CONCLUSION LiDAR-Hallu combines explicit reference-answer rules with diagnostic evaluation of LiDAR language models. Across the recorded responses, fixed-option predictions ob- scure weak multiple-choice performance, identical questions often fail to distinguish opposite scene relations, and true lateral-motion relations are consistently missed. The tested inference-time interventions leave these failures largely un- changed. Fixed-answer controls, paired questions, relation- specific recall, and repair counts provide a more informative account of spatio-temporal reasoning than aggregate accu- racy alone. Generative AI use disclosure. OpenAI Codex was used to assist with drafting and debugging portions of the ex- perimental code and with language editing throughout the manuscript. The authors reviewed and validated the resulting code and text. All reported results were obtained from actual experiment runs and verified by the authors; no empirical result values were invented, altered, or synthesized by AI. REFERENCES [1] H. Caesar et al., “nuScenes: A multimodal dataset for autonomous driving,” in CVPR, 2020. [2] C. Sima et al., “DriveLM: Driving with graph visual question answer- ing,” in ECCV, 2024. [3] G. Hess et al., “LidarCLIP or: How I learned to talk to point clouds,” arXiv preprint arXiv:2212.06858, 2022. [4] S. Yang et al., “LiDAR-LLM: Exploring the potential of large lan- guage models for 3D LiDAR understanding,” in AAAI, 2025. [5] C. Choi et al., “B4DL: A benchmark for 4D LiDAR LLM in spatio- temporal understanding,” in M, 2025. [6] T. Qian et al., “NuScenes-QA: A multi-modal visual question answer- ing benchmark for autonomous driving scenario,” in AAAI, 2024. [7] K. Tian et al., “NuScenes-SpatialQA: A spatial understanding and rea- soning benchmark for vision-language models in autonomous driving,” in ICCVW, 2025. [8] H. Vo et al., “DriveSpatial: A benchmark for spatiotemporal intelligence in VLMs for autonomous driving,” arXiv preprint arXiv:2605.23176, 2026. [9] X. Ma et al., “Do 3D large language models really understand 3D spatial relationships?” arXiv preprint arXiv:2603.23523, 2026. [10] A.-M. Marcu et al., “LingoQA: Visual question answering for au- tonomous driving,” in ECCV, 2024. [11] Z. Xu et al., “DriveGPT4: Interpretable end-to-end autonomous driv- ing via large language model,” RAL, 2024. [12] Y. Goyal et al., “Making the V in VQA matter: Elevating the role of image understanding in visual question answering,” in CVPR, 2017. [13] A. Agrawal et al., “Don’t just assume; look and answer: Overcoming priors for visual question answering,” in CVPR, 2018. [14] P. Pezeshkpour and E. Hruschka, “Large language models sensitivity to the order of options in multiple-choice questions,” in NAACL, 2024. [15] M. Xue et al., “Strengthened symbol binding makes large language models reliable multiple-choice selectors,” in ACL, 2024. [16] T. Thrush et al., “Winoground: Probing vision and language models for visio-linguistic compositionality,” in CVPR, 2022. [17] M. Yuksekgonul et al., “When and why vision-language models behave like bags-of-words, and what to do about it?” in ICLR, 2023. [18] J. Huang et al., “Unveiling the mist over 3D vision-language under- standing: Object-centric evaluation with chain-of-analysis,” in CVPR, 2025. [19] W. Zhu et al., “4D-Bench: Benchmarking multi-modal large language models for 4D object understanding,” in ICCV, 2025. [20] P. Wang et al., “Spatial4D-Bench: A versatile 4D spatial intelligence benchmark,” arXiv preprint arXiv:2601.00092, 2026. [21] A. Rohrbach et al., “Object hallucination in image captioning,” in EMNLP, 2018. [22] Y. Li et al., “Evaluating object hallucination in large vision-language models,” in EMNLP, 2023. [23] J. Wang et al., “AMBER: An LLM-free multi-dimensional benchmark for MLLMs hallucination evaluation,” arXiv preprint arXiv:2311.07397, 2023. [24] C. Li et al., “VidHalluc: Evaluating temporal hallucinations in mul- timodal large language models for video understanding,” in CVPR, 2025. [25] J. Zhang et al., “EventHallusion: Diagnosing event hallucinations in video LLMs,” arXiv preprint arXiv:2409.16597, 2024. [26] S. Leng et al., “Mitigating object hallucinations in large vision- language models through visual contrastive decoding,” in CVPR, 2024.