Paper deep dive
Auditing Quality Filters for Long-Tail Human Data Curation
Rishav Agarwal, Nirshal Chandra Sekar, Anirudh Vemula
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/29/2026, 12:21:10 PM
Summary
This paper investigates the scarcity of 'low poses' (kneeling, bending) in construction robotics datasets, attributing it to quality filters that reject uncertain joint estimates and bounding box shape anomalies. Using alternative pose identification methods and simulation, the authors demonstrate that low poses are disproportionately filtered out and harder for person detectors to identify, even when bounding box height is normalized. The study concludes that relying on pose quality filters for data curation leads to selection bias and recommends retaining valid person boxes regardless of pose estimate reliability.
Entities (11)
Relation Signals (9)
YOLO11s → finetunedon → rbyz/ppe-6-classes-ntld9
confidence 95% · fine-tuned on the public rbyz/ppe-6-classes-ntld9 dataset
YOLO11s → pretrainedon → COCO
confidence 95% · The detector is YOLO11s, pretrained on COCO
quality filter → rejects → low poses
confidence 90% · All four suggest that low poses are rejected by the filter more often
DinoV2 → usedfor → pose classification
confidence 90% · A classifier using DINOv2 features identifies low poses
OwlV2 → usedfor → Object Detection
confidence 90% · A zero-shot open-vocabulary detector, OWLv2... points the same way
SMPL → usedfor → body joint estimation
confidence 90% · estimates body-joint positions and their uncertainty using the SMPL representation
GeoCalib → usedfor → vertical direction estimation
confidence 90% · GeoCalib [4] estimates vertical from the surrounding scene
NLF → usedfor → body joint estimation
confidence 90% · estimates their body joints using NLF
bounding box height → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF, and groups similar poses. Low poses account for only about 2 percent of the retained examples. This low share may partly reflect the pipeline's quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene's vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation.
Tags
Links
- Source: https://arxiv.org/abs/2609.31896v1
- Canonical: https://arxiv.org/abs/2609.31896v1
Trouble viewing inline? Open PDF directly →
Full Text
27,715 characters extracted from source content.
Expand or collapse full text
Auditing Quality Filters for Long-Tail Human Data Curation Rishav Agarwal Gritt Robotics rishav@gritt.ai Nirshal Chandra Sekar Gritt Robotics nsekar@gritt.ai Anirudh Vemula Gritt Robotics anirudh@gritt.ai Abstract—Robots on construction sites must detect workers who are kneeling or bending, which we call low poses. These workers can be lost from training datasets during automatic labeling. We study a pipeline that detects people, estimates their body joints using NLF [1], and groups similar poses. Low poses account for only about 2% of the retained examples. This low share may partly reflect the pipeline’s quality filter, which rejects examples with low detection confidence or uncertain joint estimates. We examine this filtering using four alternative pose clues: bounding-box shape, vertical body span, pose grouping aligned to the scene’s vertical direction, and image appearance. All four suggest that low poses are rejected by the filter more often. Separately, controlled simulated scenes show that a person detector fine-tuned on a public construction dataset misses more workers in these poses even when we correct their bounding box height is matched to that of standing workers. These findings suggest that low poses are scarce and hard to find, and we cannot rely on bounding boxes or poses for long-tail human data curation. Index Terms—long-term perception, human pose, construction robotics, dataset shift, rare classes I. INTRODUCTION Gritt robots operate on active solar-construction sites, mov- ing among crews who guide panels and work beneath the solar array. Reliable person detection must cover difficult cases, including workers kneeling behind material piles or partially hidden by structural supports. We use low poses to describe kneeling, squatting, and deeply bent-over body configurations (Fig. 1). The term describes posture, not visibility or a mea- sured distance above the ground: a worker in a low pose may be fully visible or partially occluded. These poses form part of the dataset’s long tail, with far fewer examples than common standing poses. These poses are rare in the curated dataset, so a natural response is to collect and annotate more examples. However, collecting more footage may not address how examples are selected. Manually labeling body joints across months of site video is expensive, so automated pipelines estimate them instead [1], [2] and often discard uncertain estimates. Low poses can be difficult to reconstruct, especially when they appear small or are partly hidden. A filter intended to improve pose-label reliability may therefore discard examples with usable person boxes. The resulting dataset mixes two sources of scarcity: how often a pose appears in the footage and how often it survives filtering. We call this the selection–scarcity confound. kneeling, w/h = 0.60bent over, w/h = 1.03standing, w/h = 0.37 Fig. 1. Low poses in the mined library, with the box the detector produced. A worker kneeling at a tracker motor and a worker bent over a panel (orange), next to a standing worker for comparison (grey). The width-to-height ratio above each panel is the quantity the box-shape check in Section I uses. We investigate this problem through two analyses. First, we examine people already found by the detector and ask whether the pose quality filter rejects low poses more often than other poses. Second, we render the same scenes with people in different poses and measure how often a detector fine-tuned on a public construction dataset finds them. The first analysis examines which examples survive annotation; the second examines which poses are better detected. Finally, since the pipeline’s own pose labels may be unreli- able for checking what the filter discarded. We therefore com- pare four alternative visual and geometric clues for identifying low poses, and examine how the pipeline’s body-alignment step can also fail. In simulation, we separately test changes in character appearance, pose, and the height of a person’s bounding box in pixels. Together, these analyses motivate retaining valid person boxes for detection training even when the corresponding pose estimates are uncertain. I. POSE MINING AND OBSERVED SCARCITY We process the images with a person detector and a pose estimator. Candidate person boxes come from the person detector bundled with the Neural Localizer Fields multi-person model (nlf_l_multi 0.3.2) [1], which also estimates body- joint positions and their uncertainty using the SMPL repre- sentation [3]. The detector proposes 7K candidate people in a 4.3K subset of a 17K image dataset. We retain estimates with detector confidence of at least 0.5 and mean joint uncertainty of at most 150 m, which keeps 3.9K images. The estimated joints are expressed in camera coordinates, so camera tilt can make similar poses appear differently oriented. arXiv:2609.31896v1 [cs.CV] 25 Sep 2026 TABLE I LOW-POSE SHARE BY CAPTURE SOURCE. Capture sourceInst. Low Share [95% CI] Gritt dataset3,258310.95% [0.54, 1.56] External construction dataset50453 10.52% [7.24, 14.20] Full retained library3,762842.23% [1.51, 3.23] The apparent frequency of the tail depends on where the images were collected. Intervals are 95% bootstrap intervals clustered by capture recording. kneeling or squatting bent double, hands down stooped, arms down both arms overhead standing, reaching forward standing, right hand to head standing, left hand to head walking, left leg leading standing, arms folded standing, hands at chest walking, right leg leading standing, weight on one leg standing, hands at waist standing, arms down 41 43 88 110 150 177 203 225 232 239 254 348 660 992 Fig. 2. The fourteen pose groups mined from 3,762 detections. The two low-pose groups contain 41 and 43 instances. To correct for this, we estimate the vertical direction in each image from the median ankle-to-pelvis axis of detected people and align the bodies to it. We then represent each pose by the coordinates of 20 joints relative to the body root, producing a 60-dimensional vector. We normalize this vector by body extent and rotate it to a common hip orientation so that grouping depends on body arrangement rather than position, scale, or facing direction. Finally, we remove near- repeated poses in neighboring frames, dropping a body whose vector L2 norm lies within 0.08 of one already kept in the preceding 90 frames, which gives us 3.7K images. We cluster the remaining vectors into fourteen groups, each named after a representative example. The kneeling/squatting and bent- double groups define our low poses and together account for about 2% of the retained library (Fig. 2). Where images come from affects pose coverage: Low poses account for a larger share of retained examples in external construction images than in camera footage from our deployments (Table I). These sources differ in camera placement, the work being performed, and which images were selected. The comparison shows that pose coverage varies with the image source, but does not identify the cause or establish how common low poses are across construction sites. As neighboring video frames often show the same workers under similar conditions, we compute confidence intervals by resampling whole recordings rather than individual detections. External still images are treated as separate units. The filter audit includes only people already found by the detector and it cannot account for people the detector missed. I. AUDITING POSE-DEPENDENT FILTERING Most quality-filter rejections are due to uncertain joint estimates. To check whether low poses are rejected more often, we first assign each rejected pose estimate to its nearest pose group. This suggests that rejected examples contain a larger share of low poses. However, the same uncertain joints determine both rejection and group assignment. Errors in those joints could make other poses appear to be low poses. Adding comparable noise to trusted poses produced a similar share of low-pose labels, so the observed grouping does not establish that the filter preferentially rejects low poses. Alternative measurements: We check the filter using four alternative clues for identifying low poses (Table I, rows 2–5; row 1 is the original pose labels). Each avoids some, but not all, of the weaknesses of the original pose labels: • Box shape: The width-to-height ratio of the person’s box. • Vertical body span: The estimated skeleton’s vertical span, normalized by body extent, identifies folded bodies without assigning them to pose clusters. • Scene-based alignment: GeoCalib [4] estimates vertical from the surrounding scene rather than from people’s bodies. We use this direction to align the estimated joints before assigning pose groups. • Image appearance: A classifier using DINOv2 fea- tures [5] identifies low poses from cropped person images rather than estimated joints. All four checks suggest that the filter rejects low poses more often than other poses. However, these checks can also misidentify poses, especially when people are difficult to see. Although the image classifier performs well on held- out recordings, we have not verified its accuracy on rejected examples. Manually checking poses in both accepted and rejected examples would help confirm the finding. Factors associated with filtering: The filter rejects people whose boxes are wide relative to their height more often. This remains true after accounting for box height, contact with the image boundary, and image source. However, box shape can reflect either a low pose or partial occlusion, so pose alone may not explain the difference. Allowing less certain joint estimates reduces this rejection gap. Changing the required detection confidence has much less effect. Among people already detected, the joint-uncertainty requirement therefore appears to drive much of the unequal filtering. Together, these findings suggest that filtering makes low poses look rarer in the retained dataset. Their frequency before filtering remains uncertain because the pose checks can mislabel examples, and removing repeated poses also changes the counts. Estimating vertical from people can fail: Before grouping poses, the pipeline corrects for camera tilt by assuming that the direction from a person’s ankles to their pelvis points upward. This works best for standing people and can fail when someone kneels or bends over. We compare this direction with GeoCalib’s estimate, which uses the surrounding scene. The two estimates disagree much more in frames containing low poses, sometimes pointing in nearly opposite directions (Fig. 3). This also happens in frames with only one person. TABLE I FILTER PASS RATE FOR low poses, MEASURED FIVE WAYS. CheckEstimated joints usedRatio Original pose labelspose vector and alignment(0.32) Box shapenone0.26 Vertical body spanspan only0.43–0.47 Scene-based alignment pose vector only0.66 Image appearancenone0.50 Filter pass rate for examples identified as low poses, divided by the pass rate for the remaining examples. A ratio of 1 indicates equal pass rates. The first row is excluded from our evidence because it is sensitive to joint-estimation errors; row 2 uses untruncated boxes. all framessingle-person frames 0 30 60 90 120 median disagreement (°) inverted above 90° 12°, 6% inv. 68°, 37% inv. 14°, 10% inv. 93°, 54% inv. frames with other posesframes with low poses Fig. 3. The body-based estimate of vertical compared with a scene-based gravity estimate [4]. Bootstrap 95% intervals. An incorrect estimate of vertical can rotate a body incor- rectly and place it in the wrong pose group. However, neither GeoCalib nor our pose labels provide ground truth, so this comparison reveals disagreement rather than measuring the exact alignment error. Using depth sensors to estimate vertical from the ground plane is an alternative we have not yet tested. IV. TESTING THE DETECTOR IN SIMULATION Experimental design: We now ask a separate question: how well does a person detector find people in the mined poses? We render 200 construction scenes three times: first with the original human model, then with four new models standing upright, and finally with the new models in poses from the mined library (Fig. 4). Person locations, cameras, lighting, and the surrounding scene remain fixed. Comparing the first two versions tests whether changing the human models affects detection. Comparing the last two tests the effect of changing pose, which can also change bounding-box height and how much of the person is visible. ThedetectorisYOLO11s[6],pretrainedon COCO [7] and fine-tuned for this paper on the public rbyz/ppe-6-classes-ntld9 dataset [8], using that export’s own split and its person class only: 200 training images with 536 person boxes, and 50 validation images with 128. The validation images are used only to select the operating confidence threshold, as the value maximizing F 1 on them (0.45, at which recall on those images is 87.5%). It has seen no synthetic imagery and none of our own data, so these renders are entirely out of sample. We measure recall, the fraction of rendered people it finds, over 968 person instances matched across all five rendering conditions. A prediction counts as a detection when its box overlaps the Original models 37.5% recovered New models, standing 54.6% New models, mined poses 37.3% Fig. 4. Two scenes under three rendering conditions: (a) original human models, (b) new models standing upright, and (c) the same new models in mined poses. Ground-truth boxes are orange where the detector misses a person, and the percentages are computed over the 968 people present in all five conditions. Changing the models changes recall for this detector, so the pose comparisons hold the models fixed; changing their poses reduces recall and also changes their image size. ground-truth person box with intersection-over-union of at least 0.5. Pose changes also change bounding-box height: Re- placing the original human models with the new ones, still standing, raises recall by about 17 percentage points: the assets themselves matter to this detector, so the later comparisons hold them fixed. Applying the mined poses to those same models then reduces recall by about 17 points. However, a crouching person usually occupies fewer vertical pixels than a standing person at the same location. The initial comparison therefore measures the effect of changing pose together with changes in bounding-box height, body outline, and visibility. To test whether smaller bounding boxes explain the de- tector’s difficulty, we render two additional conditions: posed figures enlarged to match the box height of standing figures, and standing figures shrunk to match the box height of posed figures. We restrict the comparison to 130 people across 79 scenes. In all five rendering conditions, each selected person’s box must overlap every other person’s box by at most IoU 0.3 and remain at least 4 px from every image edge. This reduces the effects of crowding and image-boundary clipping. Within this subset, matching bounding-box height reduces but does not eliminate the recall gap (Table I). It falls from about 29 points at the original scale to about 18 points when standing figures are shrunk to the posed size, and to about 25 points when posed figures are enlarged to the standing size; both intervals exclude zero. We compute 95% bootstrap confidence intervals by resampling whole scenes, keeping people from the same scene together. Scoring the same detector at full render resolution rather than its training resolu- tion leaves the shrunk-standing comparison unchanged (−23.8 p, [−32.5,−15.5]). Matching box height does not match silhouette or visibility. For this detector, smaller box height alone therefore cannot explain the poorer detection of mined poses. A zero-shot open-vocabulary detector, OWLv2 [9] prompted with person and worker and thresholded on the same validation images, points the same way over all 968 people (−14.7 p for the pose change and−10.0 p at matched height), although on the restricted subset its intervals include TABLE I CHANGES IN RECALL BETWEEN RENDERING CONDITIONS. ComparisonChange (p)95% interval Original to new upright models+17.1 [+13.6, +20.6] Standing to mined poses-17.4[-20.3, -14.4] Restricted subset: original scale-29.2[-38.1, -20.8] Restricted subset: standing shrunk-17.7[-25.6, -10.2] Restricted subset: posed enlarged-24.6[-33.3, -16.4] Changes are in percentage points (p); negative values indicate lower recall. The last three rows use the restricted subset and should be compared with each other rather than with the full-scene result. The two matched-height rows scale in opposite directions, which changes mutual occlusion in opposite ways; the gap survives both. zero. Implications: Bounding-box height alone does not ex- plain either result. Matching it leaves most of the detector’s recall gap in simulation, and accounting for it leaves the field filter’s association with box shape largely unchanged. The detector and the pose-quality filter therefore require separate checks: the same postures are both harder to detect and less likely to survive annotation. Simulated scenes let us test specific detection failures and could provide additional training examples. We have not tested whether training on these examples improves detection in real construction footage. The paired pose comparisons keep the human models and surrounding scene fixed, but changing pose can also change body outline and visibility. The results show that this detector performs worse on mined poses, even when bounding-box height is matched. V. RELATED WORK Rare-case mining in autonomous driving: Mining diffi- cult cases and improving their coverage are established goals. CODA provides a road corner-case detection benchmark [10]. AIDE combines issue discovery, data curation, automatic labeling, model updating, and verification for autonomous- driving detection [11]. Our study examines whether a pose- annotation filter selectively removes a within-person pose group, and whether the labels used to audit that removal are themselves reliable. Selection bias and latent subgroups: Class-balanced self- training addresses dominance of large classes during pseudo- label generation [12], while the DataComp audit demonstrates that quality filtering can change group representation [13]. Our contribution is a construction-robotics case study of this broader problem, including a noise control that undermines a naive audit and alternative measurements that corroborate selective rejection. Hwang et al. study rare human poses using clustering and targeted learning strategies [14]. Hidden stratification explains how aggregate accuracy can conceal subgroup failures [15], and GEORGE uses learned features to discover latent sub- classes [16]. These motivate pose-level analysis, but recovering a pose group from model outputs does not by itself validate its frequency among rejected annotations. Synthetic evaluation and application context: Synthetic human datasets support perception training [17]–[19]. Coun- terfactual simulation testing [20], Virtual KITTI [21], and 3DB [22] establish rendering as a tool for controlled model analysis. We apply that approach to construction poses and bounding-box height. Off-road pedestrian detection [23] and construction datasets [24] provide the closest application con- text. Our focus is the reliability of pose labels used to audit rejected examples in a construction-robotics pipeline. VI. IMPLICATIONS AND LIMITATIONS Keep valid boxes even when pose estimates fail: A body can have a usable person box but an unreliable 3D recon- struction. Whether an example is kept for person-detection training should depend on the validity of its box, not on whether its pose estimate passes the quality filter. Pose labels can retain a stricter criterion. This does not mean accepting every detector prediction: false positives and localization errors still require quality control. The proposed separation remains to be evaluated end to end. Audit what is discarded: Report which population each frequency describes: raw images, detected candidates, quality- filtered examples, or deduplicated poses. Validate proxy labels on a sample from both sides of the filter and report pose group admission alongside label quality. Relaxing an uncertainty threshold trades coverage against fidelity; the present mea- surements do not identify a universally appropriate operating point. Evaluate important pose groups: Alongside recall, report how often the detector finds people in each important pose group, with the number of examples and confidence intervals. Breaking results down by bounding-box height, how much of the person is hidden, and image source can help explain when detection fails. Also, report precision at the chosen confidence threshold. Reporting precision by pose requires a rule for assigning false detections to pose groups, which we do not investigate here. These measures assess detector performance but do not establish the coverage of the robot [25], [26]. What remains unresolved: Our filter audit includes only people already found by the detector. It uses alternative pose checks whose accuracy on rejected examples has not been independently verified. We also test only one pose estimator, and GeoCalib’s estimate of vertical may be wrong. In simu- lation, resizing people can change how much they hide one another, which is why we match size in both directions and report both, and our restricted comparison may not represent crowded scenes. The next steps are to manually label accepted and rejected examples, repeat the audit with another pose estimator, and test on real images not used during training. The practical lesson is that keeping only reliable pose estimates can remove useful examples for person detection. We therefore need to check both which examples the filter discards and which people the detector misses, with results reported separately for important pose groups. REFERENCES [1] I. S ́ ar ́ andi and G. Pons-Moll, “Neural localizer fields for continuous 3D human pose and shape estimation,” in Advances in Neural Information Processing Systems (NeurIPS), 2024. [2] G. Moon, H. Choi, and K. M. Lee, “Neuralannot: Neural annotator for 3d human mesh training sets,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2022, p. 2298–2306. [3] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “SMPL: A skinned multi-person linear model,” ACM Transactions on Graphics, vol. 34, no. 6, p. 248:1–248:16, 2015. [4] A. Veicht, P.-E. Sarlin, P. Lindenberger, and M. Pollefeys, “GeoCalib: Learning single-image calibration with geometric optimization,” in Eu- ropean Conference on Computer Vision (ECCV), 2024. [5] M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec et al., “DINOv2: Learning robust visual features without supervision,” Trans- actions on Machine Learning Research, 2024, arXiv:2304.07193. [6] G. Jocher and J. Qiu, “Ultralytics YOLO11,” https://github.com/ ultralytics/ultralytics, 2024. [7] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ́ ar, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in European Conference on Computer Vision (ECCV), 2014. [8] rbyz, “PPE 6 classes dataset,” Roboflow Universe, https://universe. roboflow.com/rbyz/ppe-6-classes-ntld9, accessed 2026-09-06. [9] M. Minderer, A. Gritsenko, and N. Houlsby, “Scaling open-vocabulary object detection,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [10] K. Li, K. Chen, H. Wang, L. Hong, C. Ye, J. Han, Y. Chen, W. Zhang, C. Xu, D.-Y. Yeung, X. Liang, Z. Li, and H. Xu, “CODA: A real- world road corner case dataset for object detection in autonomous driving,” in European Conference on Computer Vision (ECCV), 2022, arXiv:2203.07724. [11] M. Liang, J.-C. Su, S. Schulter, S. Garg, S. Zhao, Y. Wu, and M. Chandraker, “AIDE: An automatic data engine for object detection in autonomous driving,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, arXiv:2403.17373. [12] Y. Zou, Z. Yu, B. V. K. V. Kumar, and J. Wang, “Unsupervised domain adaptation for semantic segmentation via class-balanced self- training,” in European Conference on Computer Vision (ECCV), 2018, arXiv:1810.07911. [13] R. Hong, W. Agnew, T. Kohno, and J. Morgenstern, “Who’s in and who’s out? a case study of multimodal CLIP-filtering in DataComp,” in ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO), 2024, arXiv:2405.08209. [14] J. Hwang, J. Yang, and N. Kwak, “Exploring rare pose in human pose estimation,” IEEE Access, vol. 8, p. 194 964–194 977, 2020. [15] L. Oakden-Rayner, J. Dunnmon, G. Carneiro, and C. R ́ e, “Hidden stratification causes clinically meaningful failures in machine learning for medical imaging,” in ACM Conference on Health, Inference, and Learning (CHIL), 2020. [16] N. S. Sohoni, J. A. Dunnmon, G. Angus, A. Gu, and C. R ́ e, “No subclass left behind: Fine-grained robustness in coarse-grained classifi- cation problems,” in Advances in Neural Information Processing Systems (NeurIPS), 2020. [17] G. Varol, J. Romero, X. Martin, N. Mahmood, M. J. Black, I. Laptev, and C. Schmid, “Learning from synthetic humans,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [18] M. Fabbri, G. Bra ˇ so, G. Maugeri, O. Cetintas, R. Gasparini, A. O ˇ sep, S. Calderara, L. Leal-Taix ́ e, and R. Cucchiara, “MOTSynth: How can synthetic data help pedestrian detection and tracking?” in IEEE/CVF International Conference on Computer Vision (ICCV), 2021. [19] M. J. Black, P. Patel, J. Tesch, and J. Yang, “BEDLAM: A synthetic dataset of bodies exhibiting detailed lifelike animated motion,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. [20] N. Ruiz, S. A. Bargal, C. Xie, K. Saenko, and S. Sclaroff, “Finding differences between transformers and ConvNets using counterfactual simulation testing,” in Advances in Neural Information Processing Systems (NeurIPS), 2022. [21] A. Gaidon, Q. Wang, Y. Cabon, and E. Vig, “Virtual worlds as proxy for multi-object tracking analysis,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. [22] G. Leclerc, H. Salman, A. Ilyas, S. Vemprala, L. Engstrom, V. Vineet, K. Xiao, P. Zhang, S. Santurkar, G. Yang, A. Kapoor, and A. Madry, “3DB: A framework for debugging computer vision models,” in Ad- vances in Neural Information Processing Systems (NeurIPS), 2022. [23] Z. Pezzementi, T. Tabor, P. Hu, J. K. Chang, D. Ramanan, C. Welling- ton, B. P. W. Babu, and H. Herman, “Comparing apples and or- anges: Off-road pedestrian detection on the NREC agricultural person- detection dataset,” Journal of Field Robotics, vol. 35, no. 4, 2018, arXiv:1707.07169. [24] R. Duan, H. Deng, M. Tian, Y. Deng, and J. Lin, “SODA: A large-scale open site object detection dataset for deep learning in construction,” Automation in Construction, vol. 142, p. 104499, 2022. [25] R. Salay, K. Czarnecki, H. Kuwajima, H. Yasuoka, T. Nakae, V. Ab- delzad, C. Huang, M. Kahn, and V. D. Nguyen, “The missing link: Developing a safety case for perception components in automated driving,” SAE International Journal of Advances and Current Practices in Mobility, vol. 5, no. 2, p. 567–579, 2022, arXiv:2108.13294. [26] E. Breck, S. Cai, E. Nielsen, M. Salib, and D. Sculley, “The ML test score: A rubric for ML production readiness and technical debt reduction,” in IEEE International Conference on Big Data, 2017.