Paper deep dive
NegROI: Click-Centric Uncertainty-Guided Refinement with Scene-Conditioned Negative Prompts for Robust Interactive 3D Segmentation
Shuheng Zhang, Feng Wu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/8/2026, 4:09:33 AM
Summary
The paper introduces NegROI, a novel transformer-based framework for robust interactive 3D segmentation. It addresses coarse voxel resolution and hard false positives by integrating click-centric multi-resolution refinement, uncertainty-driven selective refinement, scene-conditioned negative prompts, and boundary-aware hard negative mining. Evaluated on ScanNet, S3DIS, and KITTI-360, NegROI demonstrates improved click efficiency, reduced false positives, and superior cross-dataset robustness compared to state-of-the-art baselines.
Entities (12)
Relation Signals (12)
NegROI → evaluatedon → KITTI-360
confidence 95% · We evaluate interactive 3D instance segmentation on three widely-used benchmarks: ScanNet40, S3DIS, and KITTI-360.
NegROI → evaluatedon → ScanNet++
confidence 95% · Our experiments on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI)
NegROI → evaluatedon → S3DIS
confidence 95% · Our experiments on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI)
NegROI → solves → Interactive 3D Segmentation
confidence 95% · Interactive 3D segmentation aims to extract object masks in point clouds with minimal user clicks... we propose NegROI — a novel transformer-based interactive framework
NegROI → incorporates → Click-centric multi-resolution refinement
confidence 92% · we introduce click-centric multi-resolution refinement: after a coarse prediction, we refine only a local ROI around the current click on a finer grid
NegROI → incorporates → Scene-conditioned negative prompts
confidence 92% · we introduce scene-conditioned negative prompts that act as background prototypes learned by cross-attending to scene tokens
NegROI → incorporates → Uncertainty-driven selective refinement
confidence 90% · we add uncertainty-driven selective refinement to focus refinement computation on ambiguous regions
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Interactive 3D segmentation aims to extract object masks in point clouds with minimal user clicks. Despite recent progress, most existing approaches still struggle with (i) coarse voxel resolution that blurs fine boundaries under limited clicks and (ii) hard false positives caused by confusing background structures. These issues are exacerbated by density and scale shifts across datasets (e.g., dense RGB-D reconstructions vs. sparse LiDAR scans), where fixed refinement heuristics and purely click-driven decoding generalize poorly. To address them, we propose NegROI -- a novel transformer-based interactive framework that couples click-centric multi-resolution refinement with scene-conditioned negative prompts. Given a coarse voxel prediction, it refines only a local Region Of Interest (ROI) around the current click on a finer grid and fuses refined logits back to the coarse mask. To improve robustness and efficiency, we introduce uncertainty-driven selective refinement that prioritizes ambiguous regions. Meanwhile, we model hard background patterns via a set of scene-conditioned negative prompts obtained by cross-attention over scene tokens. We further stabilize these prompts with a diversity regularizer. Finally, we propose boundary-aware hard negative mining to supervise negative-prompt attention toward boundary-proximal, high-confidence false positives. Our experiments on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI) demonstrate improved click efficiency and reduced false positives, with stronger cross-dataset robustness than the state-of-the-art baselines.
Tags
Links
- Source: https://arxiv.org/abs/2607.05955v1
- Canonical: https://arxiv.org/abs/2607.05955v1
Trouble viewing inline? Open PDF directly →
Full Text
40,211 characters extracted from source content.
Expand or collapse full text
NegROI: Click-Centric Uncertainty-Guided Refinement with Scene-Conditioned Negative Prompts for Robust Interactive 3D Segmentation Shuheng Zhangand Feng Wu (B)⋆ School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China. zsh123456@mail.ustc.edu.cn, wufeng02@ustc.edu.cn Abstract. Interactive 3D segmentation aims to extract object masks in point clouds with minimal user clicks. Despite recent progress, most ex- isting approaches still struggle with (i) coarse voxel resolution that blurs fine boundaries under limited clicks and (i) hard false positives caused by confusing background structures. These issues are exacerbated by density and scale shifts across datasets (e.g., dense RGB-D reconstruc- tions vs. sparse LiDAR scans), where fixed refinement heuristics and purely click-driven decoding generalize poorly. To address them, we pro- pose NegROI — a novel transformer-based interactive framework that couples click-centric multi-resolution refinement with scene-conditioned negative prompts. Given a coarse voxel prediction, it refines only a lo- cal Region Of Interest (ROI) around the current click on a finer grid and fuses refined logits back to the coarse mask. To improve robustness and efficiency, we introduce uncertainty-driven selective refinement that prioritizes ambiguous regions. Meanwhile, we model hard background patterns via a set of scene-conditioned negative prompts obtained by cross-attention over scene tokens. We further stabilize these prompts with a diversity regularizer. Finally, we propose boundary-aware hard nega- tive mining to supervise negative-prompt attention toward boundary- proximal, high-confidence false positives. Our experiments on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI) demonstrate im- proved click efficiency and reduced false positives, with stronger cross- dataset robustness than the state-of-the-art baselines. Keywords: Interactive 3D segmentation· Point clouds· Transformers · Prompting· Domain robustness 1 Introduction Interactive segmentation reduces annotation cost by iteratively refining a mask with user feedback. In many 3D applications, interactive segmentation is partic- ularly valuable because dense point-wise labels are often expensive to acquire. ⋆ Feng Wu is the corresponding author. arXiv:2607.05955v1 [cs.CV] 7 Jul 2026 2S. Zhang and F. Wu However, point clouds exhibit large variability in density, noise, and scale. There- fore, interactive pipelines must respond reliably to sparse user clicks. Modern interactive 3D segmentation methods [12,25,32,34] commonly oper- ate on voxel grids for efficiency and use transformer decoders to integrate click tokens with scene features. While effective, we observe two persistent failure modes: (1) Boundary under-segmentation: A single-resolution voxel representa- tion tends to blur object boundaries, especially when the click budget is small. (2) Hard false positives (FPs): Even with clicks, confusing background struc- tures can trigger high-confidence FPs. Both issues are amplified under distri- bution shifts (e.g., ScanNet-trained models tested on S3DIS or KITTI), where background appearance and sampling patterns differ substantially. Against this background, we propose a novel method named NegROI, which targets both issues with a unified design. Firstly, we introduce click-centric multi- resolution refinement: after a coarse prediction, we refine only a local ROI around the current click on a finer grid and fuse the refined prediction back to the coarse mask. Secondly, we introduce scene-conditioned negative prompts that act as background prototypes learned by cross-attending to scene tokens, providing a structured way to suppress hard FPs. To improve robustness and efficiency, we add uncertainty-driven selective refinement, a diversity regularizer to prevent prompt collapse, and boundary-aware hard negative mining to directly supervise negative-prompt attention on boundary-adjacent confusing background. Our main contributions are summarized as follows: (1) We propose the click-centric multi-resolution refinement module that per- forms efficient fine-grid refinement inside a local ROI and fuses fine predic- tions back to coarse logits. (2) We propose the uncertainty-driven selective refinement to focus refinement computation on ambiguous regions, improving efficiency and robustness. (3) We propose the scene-conditioned negative prompts for explicit background modeling, augmented with a prompt diversity regularizer. (4) We propose the boundary-aware hard negative mining to supervise negative- prompt attention toward boundary-proximal, high-confidence false positives. All together, we advance the state-of-the-art for robust interactive 3D segmen- tation. The experimental results on common benchmark datasets (i.e., ScanNet, S3DIS, and KITTI) demonstrate im- proved click efficiency, reduced false posi- tives, and cross-dataset robustness of our approach. 2 Related Work In this section, we briefly review the related work on interactive 3D segmentation. Interactive image segmentation. Early interactive segmentation methods lever- age sparse user cues such as extreme points [17] and iterative refinement strate- gies [26, 27]. Recent work improves click efficiency by better modeling click lo- cality and hard errors (e.g., [2, 15]), and by adding lightweight high-resolution NegROI for Robust Interactive 3D Segmentation3 refinement heads for sharper boundaries [10]. Foundation models further broaden promptability in 2D, most notably Segment Anything (SAM) [9] and its video extension [23]. Our work adapts the “refine-where-needed” principle to 3D by performing click-centric local refinement on a fine voxel grid and explicitly sup- pressing hard false positives. 3D point cloud segmentation. Backbones for 3D understanding span point-based networks [20, 21, 30, 35] and efficient large-scale designs [6], as well as sparse convolutional approaches [3, 5, 29]. For 3D instance segmentation, grouping- based methods such as PointGroup [8] and SoftGroup [31] are widely used, while transformer-based mask prediction has shown strong performance in re- cent unified frameworks [11,24]. NegROI is complementary to these backbones and focuses on interactive prompting, local refinement, and robust background modeling. Interactive and promptable 3D segmentation. Interactive 3D segmentation has advanced from early click-based pipelines to transformer-driven promptable sys- tems. InterObject3D [12] and AGILE3D [34] demonstrate that attention-guided decoding can effectively integrate user clicks for object extraction in cluttered scenes. Easy3D [25] provides a strong and simple interactive baseline, while Point-SAM [37] extends the SAM-style prompting paradigm to point clouds. Compared to prior work, we explicitly model confusing background via scene- conditioned negative prompts and couple them with click-centric fine-grid re- finement to improve boundary fidelity and cross-dataset robustness. Open-vocabulary vision-language and 3D understanding. Open-vocabulary seg- mentation commonly builds on large-scale vision-language pretraining [7, 22]. Text-supervised grouping and segmentation emerge in models such as GroupViT [33], and open-vocabulary recognition can be bootstrapped from image-level supervision [36] or open-set detectors [16]. For segmentation, CLIP-adapted mask models provide a practical recipe for open-vocabulary transfer [13]. In 3D, OpenScene [19] studies open-vocabulary 3D scene understanding, while Open- Mask3D [28] and Open3DIS [18] address open-vocabulary 3D instance segmen- tation by leveraging 2D foundation models and mask guidance. Our work targets the interactive setting and is compatible with open-vocabulary querying, while focusing on improved error localization and suppression of hard distractors. Hard negatives and boundary-focused refinement. A recurring challenge in in- teractive segmentation is that errors are highly concentrated around ambiguous boundaries and confusing distractors. Prior 2D interactive methods improve click efficiency by explicitly focusing on hard regions [2,15] and by adding lightweight high-resolution heads to sharpen boundaries [10]. In 3D interactive segmenta- tion, clutter and object attachments further amplify boundary-proximal false positives [12, 34]. Our approach follows the same philosophy of refine-where- needed and focus-on-hard-negatives, but instantiates it with (i) boundary-aware 4S. Zhang and F. Wu Backbone & Interaction Scene-cond. NEG prompts + Prompt Supervision ROI Refinement Point Cloud (x i , c i ) Voxelize (coarse) Encoder E + proj. Two-Way Transformer D H + P Relative PE (1st click) Clicks (k t ,y t ) Click Tokens t t [t fg , t bg , t 1:c ] Scene-cond. NEG prompts scene tok. append L div Coarse Logits s (c) Boundary-aware hard negatives L hn p = σ(s) ROI Select r c + uncertainty Fine Voxelize Shared Enc E (ROI / fine) ROI Dec D (fine) MaxAgg + Fuse s ← (1− α)s + α ˆ s Mask M = σ(s) coarse coarse L seg BCE+Dice Fig. 1: NegROI overview (aligned with our implementation). We use a sparse voxel backbone and the Easy3D two-way transformer decoder to produce coarse logits. We augment decoding with scene-conditioned negative prompts supervised by L hn and L div . For boundary accuracy, we refine a click-centric ROI on a fine grid using the shared encoder and an ROI decoder, then max-aggregate and fuse back to coarse logits. hard negative mining that supervises negative-prompt attention, and (i) click- centric fine-grid ROI refinement that improves boundary fidelity without dense high-resolution decoding. 3 Main Method As aforementioned, we refer to our approach as NegROI, which combines scene- conditioned negative prompts with click-centric ROI refinement for robust in- teractive 3D segmentation. 3.1 Problem Statement Given a point cloudP =(x i ,c i ) P i=1 with coordinates x i ∈R 3 and color/features c i , we voxelize it into a coarse voxel set V = (v j ,f j ) V j=1 , where v j ∈Z 3 and f j is a voxel feature. In interactive segmentation, at step c we are given clicks K c =(k t ,y t ) c t=1 with y t ∈pos, neg. We predict voxel logits s (c) ∈R V and a binary mask by thresholding σ(s (c) ). Click coordinates and voxel indexing. We denote each click location k t in metric coordinates, i.e., k t ∈R 3 in the same coordinate system as x i . For operations on the coarse voxel grid (voxel size v), we quantize k t to a voxel coordinate ̃ k t = Π v (k t ) ∈Z 3 using the same voxelization rule as V. We further denote by j t ∈ 1,...,V the index of the coarse voxel whose coordinate matches (or is nearest to) ̃ k t . Unless otherwise specified, we use k t for metric coordinates and ( ̃ k t ,j t ) for coarse-voxel coordinates/indexing. NegROI for Robust Interactive 3D Segmentation5 3.2 Framework Overview As illustrated in Fig. 1, NegROI consists of three coupled parts: (i) Backbone & interaction, which encodes the coarse voxel grid and injects click tokens into a two-way transformer decoder; (i) Scene-conditioned negative prompts, which explicitly model confusing background via K negative prototypes and supervises their attention on boundary hard negatives; and (i) ROI refinement & output, which refines the prediction in a click-centered region on a fine grid and fuses it back to the coarse logits. 3.3 Backbone and Interaction Tokens Coarse voxel encoding. A voxel encoder E(·) maps voxelized inputs to scene tokens as below: H = E(V)∈R V×F .(1) Relative positional encoding. We use the quantized first click ̃ k 1 as the coordinate origin on the coarse voxel grid. For each voxel j, ∆v j = v j − ̃ k 1 , P j = W p φ(∆v j /S)∈R F ,(2) where φ(·) is harmonic encoding. We set S to the per-scene voxel-grid scale, e.g., S = max(v max − v min ) x , (v max − v min ) y , (v max − v min ) z ,(3) with v min ,v max ∈Z 3 being the min/max voxel coordinates of the scene. Click tokens. We construct interaction tokens consisting of two learnable mask tokens t fg ,t bg ∈R F and c click tokens. For click t, we map it to the coarse voxel index j t (defined in Sec. 3) and form t t = P j t + e(y t ),(4) where e(·) is a learned embedding for click type. The base token sequence is T (c) base = t fg ,t bg ,t 1 ,...,t c ∈R (2+c)×F .(5) 3.4 Two-Way Transformer Decoder We adopt the two-way transformer decoder from Easy3D [25], which exchanges information between scene tokens and interaction tokens: ( ̃ H, ̃ T) = D(H,P,T).(6) Coarse voxel logits are obtained via a foreground/background token difference: s (c) j =⟨ ̃ t fg , ̃ h j ⟩−⟨ ̃ t bg , ̃ h j ⟩.(7) 6S. Zhang and F. Wu 3.5 Scene-Conditioned Negative Prompts and Supervision Hard false positives often originate from confusing background structures. To explicitly model such distractors, we introduce K learnable negative prompt queries q k K k=1 and compute scene-conditioned prototypes by cross-attending to the scene tokens: p (c) k = Attn(q k , H + P, H + P)∈R F , ˆ p (c) k = MLP(p (c) k ) + e(neg). (8) We append ˆ p (c) k to the interaction tokens before decoding: T (c) = h T (c) base , ˆ p (c) 1 ,..., ˆ p (c) K i .(9) Prompt diversity regularization. To avoid prompt collapse, we encourage di- versity among negative prototypes at each click step by penalizing off-diagonal cosine similarity: L (c) div = 1 K(K− 1) X i̸=j cos(p (c) i ,p (c) j ) 2 .(10) wherep (c) k K k=1 are the scene-conditioned negative prototypes computed at click step c. Boundary-aware hard negative supervision. Let A (c) ∈R K×V denote the at- tention weights produced when generating negative prototypes (averaged across heads), and ̄ A (c) = 1 K P k A (c) k . We supervise ̄ A (c) to focus on hard background near object boundaries. Specifically, we define boundary-adjacent background candidates B =j | y j = 0, ∃j ′ ∈N (j) s.t. y j ′ = 1,(11) where N (j) is the 6-neighborhood in the voxel grid and y j denotes the ground- truth voxel label. Among candidates in B (falling back to all background if B is too small), we select the top-k voxels with highest predicted foreground probability p j = σ(s (c) j ) to form H and define a uniform target distribution t j = 1 |H| 1[j ∈H]. We minimize cross-entropy: L (c) hn =− V X j=1 t j log ̄ A (c) j + ε .(12) 3.6 Click-Centric ROI Refinement and Output Coarse voxels may blur thin structures and boundaries. We therefore refine lo- cally around the current click on a finer grid and fuse the refined logits back to the coarse prediction, as shown in Fig. 1. NegROI for Robust Interactive 3D Segmentation7 ROI selection with adaptive radius. Let the current click center be k c (metric coordinates). We select ROI points P roi =i|∥x i − k c ∥≤ r c ,(13) where r c is an adaptive radius predicted from click context and local density (Sec. 3.6). We voxelizeP roi at a finer resolution v f = v/η (fine scale η) to obtain fine voxels V f and fine scene tokens H f ∈R V f ×F . Shared encoder and ROI decoder. We reuse the same encoder E to embed the fine ROI voxels (weight sharing with the coarse branch). We then apply the same Easy3D-style two-way decoder D to the ROI tokens (again weight- shared), producing fine logits s (c) f . Inside the ROI branch, we optionally build ROI-conditioned negative prompts by cross-attending learnable NEG queries to the ROI tokens, and append them to the ROI interaction tokens (“ROI-cond. NEG” in Fig. 1). Fine-to-coarse max aggregation and fusion. Each fine voxel maps to a coarse voxel by ⌊u/η⌋. We aggregate by max: ˆs (c) j = max u∈M(j) s (c) f,u ,(14) and fuse with the coarse logits using a residual update: s (c) j ← (1− α)s (c) j + α ˆs (c) j .(15) Adaptive Radius Prediction. We begin with a decayed base radius r base c = r 0 γ c−1 . We estimate local density around the click using kNN distances and compute mean/median (μ,m). We predict a log-scale adjustment ∆ c = MLP [t c , logμ, logm, logr base c ,c/C] , r c = clip r base c exp(∆ c ) , (16) where clip clamps to [r min ,r max ]. If the ROI contains too few points, we enlarge r c (bounded) a few times to ensure stable refinement. Uncertainty-Driven Selective Refinement. We prioritize ambiguous re- gions inside the ROI using coarse uncertainty: u j = 1− 2 σ(s (c) j )− 0.5 ∈ [0, 1].(17) Each ROI point is mapped to its coarse voxel index v(i); we keep points with u v(i) ≥ τ (fallback to the unfiltered ROI if too few points remain), focusing fine decoding on boundary and hard regions. 8S. Zhang and F. Wu 3.7 Training Objective At click step c, we use the segmentation loss L (c) seg =L bce (s (c) ,y) +L dice (s (c) ,y),(18) and add scene-conditioned negative prompts regularizers: L = 1 C C X c=1 L (c) seg + λ hn L (c) hn + λ div L (c) div .(19) Click simulation. During training, we simulate interactions using an oracle policy that places the next click at informative error regions between prediction and ground truth. Implementation note (DDP stability). In our implementation, adaptive radii affect ROI selection via a hard threshold. To avoid unused-parameter issues in DDP, we optionally include a tiny regularizer on ∆ c with detached inputs: L rad = λ rad ∥∆ c ∥ 2 2 .(20) 4 Experiments In this section, we conducted extensive experiments on several common bench- marks to empirically evaluate the advantages of our method by comparing with the leading interactive and non-interactive baselines. We also did ablation studies to show the effectiveness of our every modules. 4.1 Benchmarks We evaluate interactive 3D instance segmentation on three widely-used bench- marks: ScanNet40 [4], S3DIS [1], and KITTI-360 [14]. ScanNet40 and S3DIS represent cluttered indoor environments with frequent object attachments and ambiguous boundaries, while KITTI-360 features large-scale outdoor scans with sparse geometry and long-range context. Unless otherwise stated, we follow the official dataset splits and evaluation protocols adopted in prior interactive 3D segmentation works (e.g., [12,34]). 4.2 Baselines We compare against representative interactive 3D segmentation baselines, in- cluding click-conditioned transformer decoders without ROI refinement, multi- resolution refinement methods without explicit background prompting, and prompt/query- based interactive approaches (e.g., [12,34,37]). When official implementations are available, we run them under the same evaluation protocol. Otherwise, we follow the paper descriptions and ensure identical click simulation, data preprocessing, and evaluation metrics. We also report Easy3D [25] as a strong promptable in- teractive baseline in our setting. NegROI for Robust Interactive 3D Segmentation9 4.3 Interactive Evaluation Protocol Click simulation. To ensure fair and repeatable evaluation, we adopt a standard simulated-click protocol. For each target instance, the first click is sampled inside the ground-truth region. At each subsequent step, we update the mask prediction and place the next click on the largest error region: a positive click on the largest false-negative component if under-segmentation dominates, otherwise a negative click on the largest false-positive component. This policy approximates an oracle user that always clicks the most informative mistake and is commonly used for benchmarking interactive methods. Target specification (prompts). Our primary setting is class-agnostic interactive instance segmentation, where the target instance is specified by clicks only. If a baseline requires an additional query signal (e.g., an open-vocabulary text prompt), we use simple class-name prompts with a consistent template across methods. Metrics. We report point-wise IoU (%) under click budgets k ∈ 1, 2, 3, 5, 10, denoted as IoU@k. All methods are evaluated with the same click simulator and the same preprocessing on each dataset to isolate algorithmic differences. 4.4 Implementation Details Backbone and voxelization. Our model uses a sparse voxel encoder and a two-way transformer decoder. We follow dataset-standard sparse voxel preprocessing and keep voxelization consistent between training and testing. To improve boundary accuracy without incurring dense high-resolution costs, our refinement branch re-voxelizes only within selected ROIs using a finer resolution than the coarse grid. Scene-conditioned negative prompts and click-centric ROI refinement. NegROI combines scene-conditioned negative prompts with click-centric ROI refinement (Fig. 1). At each click step, we first decode coarse mask logits using click tokens augmented by K scene-conditioned negative prototypes. We then refine bound- aries with a click-centric ROI branch: we select an ROI centered at the current click using an adaptive radius r c and a coarse uncertainty signal, re-voxelize the ROI on a finer grid, and run the shared encoder E together with an ROI decoder to produce fine logits. Finally, we max-aggregate fine logits to the coarse grid and fuse them back with weight α. For prompt learning, we supervise negative- prompt attention via a boundary-aware hard-negative loss L hn and regularize prompt collapse with the diversity loss L div . We tune these hyperparameters on a ScanNet40 validation split and keep them fixed across datasets. Hyperparameters. Unless otherwise stated, we use K=8 negative prompts, fine scale η=2 (i.e., v f =v/η), uncertainty threshold τ =0.20, and fusion weight α=0.7. For the click-centric ROI, we set the base radius r 0 =1.0m with decay γ=0.9, 10S. Zhang and F. Wu and clamp the adaptive radius to [r min ,r max ]=[0.35r 0 , 3.0r 0 ]. We estimate local density with kNN using k=32, and mine hard negatives with top-k background voxels using k=256. All these hyperparameters are tuned once on a ScanNet40 validation split and then fixed for all datasets. Training. We train end-to-end using the segmentation loss on the fused logits and auxiliary objectives: L seg = L bce +L dice , plus the prompt regularizers λ hn L hn and λ div L div . We simulate up to C clicks per query during training using the oracle policy described above. 4.5 Main Results Table 1 reports the main comparison under a unified train-on-ScanNet40 set- ting. We train all methods on ScanNet40 and evaluate on three test benchmarks: ScanNet40 (in-domain) and S3DIS/KITTI-360 (out-of-domain). This pro- tocol directly measures cross-dataset robustness under substantial domain shifts in scene scale, point density, and sensor characteristics. In-domain performance. On ScanNet40, NegROI consistently improves IoU across click budgets, with the largest gains under sparse interactions (IoU@1–IoU@3). These improvements align with our design goals: scene-conditioned negative prompts reduce hard distractors and attached clutter early, while the click- centric fine-grid ROI branch sharpens boundaries through re-voxelization and ROI decoding, followed by max-aggregation and fusion back to coarse logits. Out-of-domain generalization. When transferring to S3DIS and KITTI-360 with- out any target-domain training, NegROI maintains strong performance and out- performs prior baselines across most click budgets. The gains are particularly pronounced at low click counts, where boundary ambiguity and hard distractors dominate. This behavior is consistent with (i) boundary-aware hard-negative prompt supervision (L hn ), which suppresses boundary-proximal false positives, and (i) click-centric ROI refinement, which allocates fine-grained computation to the most informative region indicated by the click. 4.6 Comparison with Non-Interactive Methods Although NegROI is designed for interactive refinement, we further evaluate its single-click performance under the ScanNet20 protocol commonly used for as- sessing generalization to unseen categories. Specifically, we train models only on ScanNet20 and test in two settings: ScanNet20 → ScanNet20 (only seen ob- jects) and ScanNet20 → ScanNet40 (seen + unseen objects). Table 2 compares NegROI with representative non-interactive/open-vocabulary 3D instance seg- mentation baselines, including Mask3D [24], AGILE3D [34], and Easy3D [25]. NegROI improves mAP as well as AP 50 /AP 25 in both the in-domain (seen-only) and the seen + unseen transfer setting, indicating that our scene-conditioned negative prompting and click-centric refinement also benefit early (single-click) NegROI for Robust Interactive 3D Segmentation11 Table 1: Main results under a unified train-on-ScanNet40 protocol. All methods are trained on ScanNet40 and tested on ScanNet40 (in-domain), S3DIS and KITTI-360 (out-of-domain). Numbers denote point-wise IoU (%) under click budgets k ∈1, 2, 3, 5, 10 (IoU@k). Best results are in bold. Dataset MethodIoU@1 IoU@2 IoU@3 IoU@5 IoU@10 ScanNet40 InterObject3D [12] 40.855.963.967.677.6 AGILE3D [34]63.070.675.179.783.5 Easy3D [25]68.274.677.379.681.7 NegROI (Ours) 72.1 78.1 80.9 82.383.6 S3DIS InterObject3D [12] 38.554.062.572.479.9 AGILE3D [34]58.570.777.483.688.3 Point-SAM [37]38.8n/a67.172.280.6 Easy3D [25]65.776.080.884.987.8 NegROI (Ours) 66.9 77.8 82.2 85.487.8 KITTI-360 InterObject3D [12]2.05.18.572.483.6 AGILE3D [34]34.840.742.744.449.6 Point-SAM [37]44.0n/a67.172.280.8 Easy3D [25]46.358.766.776.283.6 NegROI (Ours) 47.5 62.6 71.4 80.187.7 predictions and enhance robustness beyond the ScanNet40-trained interactive evaluation in Table 1. 4.7 Ablation Studies We conduct ablations to quantify the contribution of each component and the proposed regularizers. Component ablation. We progressively enable (i) scene-conditioned negative prompts, (i) click-centric fine-grid ROI refinement, (i) uncertainty-driven ROI selection, (iv) boundary-aware hard negative supervision for prompt attention (L hn ), and (v) prompt diversity regularization (L div ). Table 3 summarizes the results on ScanNet40 and S3DIS. Scene-conditioned negative prompts boost performance under sparse clicks (e.g., IoU@1: 67.0→69.2 on ScanNet40 and 62.5→64.2 on S3DIS). Adding click- centric fine-grid ROI refinement further improves boundary quality (e.g., Scan- Net40 IoU@5: 80.4→81.6). Uncertainty-driven ROI selection brings additional 12S. Zhang and F. Wu Table 2: Single-click evaluation for models trained only on ScanNet20. We report results under the ScanNet20 → ScanNet20 setting (only seen objects) and the ScanNet20 → ScanNet40 setting (seen + unseen objects). SettingMethodmAPAP 50 AP 25 ScanNet20 → ScanNet20 Mask3D [24]51.577.090.2 AGILE3D [34]53.575.691.3 Easy3D [25]56.179.593.1 NegROI (Ours) 59.682.395.2 ScanNet20 → ScanNet40 Mask3D [24]5.313.124.7 AGILE3D [34]24.845.772.4 Easy3D [25]39.264.685.5 NegROI (Ours) 43.067.987.8 gains (e.g., ScanNet40 IoU@1: 70.9→71.8). Overall, combining the proposed reg- ularizers yields the best full model on both datasets (ScanNet40: 72.1/82.3/83.6; S3DIS: 66.9/85.4/87.8 for IoU@1/5/10). Loss ablation. We further examine the effect of the proposed prompt regularizers. As shown in Table 3, adding boundary-aware hard negatives (L hn ) and prompt diversity (L div ) improves the overall results, with the full model achieving the best performance across datasets and click budgets. In particular, L div provides consistent gains when combined with L hn , suggesting that preventing prompt collapse helps stabilize negative guidance during refinement. 4.8 Qualitative Results We visualize representative cases across datasets in Fig. 2 to illustrate how scene- conditioned negative prompts and click-centric ROI refinement improve interac- tive segmentation. Compared to Easy3D [25], NegROI better suppresses at- tached or visually similar distractors and recovers sharper object boundaries under sparse clicks (e.g., 1–3 clicks), consistently yielding higher IoU@k. Boundary-proximal false positives. Figure 3 visualizes boundary-band false pos- itives (Band-FP): Gray denotes a narrow ground-truth boundary band (darker is closer to the boundary) and red marks false-positive predictions inside the band. Across ScanNet40, S3DIS, and KITTI-360, NegROI consistently shows fewer boundary-proximal false positives than Easy3D [25], and the red points diminish faster from Click 1 to Click 3. This trend is consistent with our design: the boundary-aware hard negative loss explicitly guides the negative prompts NegROI for Robust Interactive 3D Segmentation13 Table 3: Component ablation on ScanNet40 and S3DIS. Numbers denote point- wise IoU (%) under click budgets k ∈1, 5, 10. Variant ScanNet40S3DIS IoU@1 IoU@5 IoU@10 IoU@1 IoU@5 IoU@10 Base (no ROI, no NEG prompts) 67.0 78.8 80.862.5 80.8 83.8 + Scene-cond. NEG prompts69.2 80.4 82.364.2 83.0 85.7 + ROI refinement (fine-grid)70.9 81.6 83.065.8 84.1 86.9 + Uncertainty-driven ROI selection 71.8 81.9 82.866.5 85.0 87.4 + Boundary hard negatives (L hn ) 71.2 82.1 83.165.6 85.2 87.1 + Diversity (L div ) (full)72.1 82.3 83.6 66.9 85.4 87.8 toward boundary-adjacent confusers (Sec. 3.5), while click-centric ROI refine- ment sharpens local decision boundaries on a finer grid and fuses them back to the coarse mask (Sec. 3.6). 5 Conclusions We presented a robust interactive 3D segmentation framework that couples click-centric multi-resolution ROI refinement with scene-conditioned negative prompts. NegROI refines boundaries efficiently near the current click, focuses computation on uncertain regions, and explicitly suppresses hard false positives via boundary-aware negative prompt supervision and diversity regularization. Experiments on ScanNet, S3DIS, and KITTI-style point clouds indicate im- proved click efficiency and stronger cross-dataset robustness. Future work in- cludes fully differentiable ROI selection and stronger test-time adaptation driven solely by user clicks. Acknowledgments We thank all reviewers and area chairs for their insightful comments and valuable suggestions. We also thank Yueyang Wen for the fruitful discussions and sharing source codes. This work was supported in part by Major Research Plan of the National Natural Science Foundation of China (92048301), Anhui Provincial Ma- jor Research and Development Plan (202004H07020008), and Anhui Provincial Natural Science Foundation (2208085MF172). 14S. Zhang and F. Wu TargetEasy3DNegROI (Ours) ScanNet40 IoU@1=68.8IoU@2=82.5IoU@3=85.9IoU@1=70.8IoU@2=86.0IoU@3=88.0 S3DIS IoU@1=5.1IoU@2=43.2IoU@3=69.5IoU@1=18.6IoU@2=70.5IoU@3=76.8 KITTI-360 IoU@1=25.2IoU@2=51.3IoU@3=67.6IoU@1=55.8IoU@2=60.7IoU@3=80.9 Fig. 2: Qualitative comparison across datasets. We compare Easy3D [25] and NegROI under the same simulated-click protocol on ScanNet40, S3DIS, and KITTI- 360. Each row shows the target instance (green) and predicted masks (red) after 1/2/3 clicks, with the corresponding IoU@1/2/3 reported below each prediction. Easy3D (Band-FP)NegROI (Band-FP) Click 1Click 2Click 3Click 1Click 2Click 3 ScanNet40 S3DIS KITTI-360 Fig. 3: Boundary-band false positive (Band-FP) visualization. Gray indicates the ground-truth boundary band (darker near the boundary); red highlights false- positive predictions within the band. Compared to Easy3D [25], NegROI produces fewer boundary-proximal false positives under the same simulated clicks across Scan- Net40, S3DIS, and KITTI-360. NegROI for Robust Interactive 3D Segmentation15 References 1. Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3d semantic parsing of large-scale indoor spaces. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 1534–1543 (2016) 2. Chen, X., Zhao, Z., Zhang, Y., Duan, M., Qi, D., Zhao, H.: Focalclick: Towards practical interactive image segmentation. In: Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition (CVPR). p. 1300–1309 (2022) 3. Choy, C., Gwak, J., Savarese, S.: 4d spatio-temporal convnets: Minkowski convolu- tional neural networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 3075–3084 (2019) 4. Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: Scan- net: Richly-annotated 3d reconstructions of indoor scenes. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 5828–5839 (2017) 5. Graham, B., Engelcke, M., Van Der Maaten, L.: 3d semantic segmentation with submanifold sparse convolutional networks. In: Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition (CVPR). p. 9224–9232 (2018) 6. Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., Markham, A.: Randla-net: Efficient semantic segmentation of large-scale point clouds. In: Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 11108–11117 (2020) 7. Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: Proceedings of the international conference on machine learning (ICML). p. 4904–4916 (2021) 8. Jiang, L., Zhao, H., Shi, S., Liu, S., Fu, C.W., Jia, J.: Pointgroup: Dual-set point grouping for 3d instance segmentation. In: Proceedings of the IEEE/CVF confer- ence on computer vision and Pattern recognition (CVPR). p. 4867–4876 (2020) 9. Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV). p. 4015– 4026 (2023) 10. Kirillov, A., Wu, Y., He, K., Girshick, R.: Pointrend: Image segmentation as render- ing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 9799–9808 (2020) 11. Kolodiazhnyi, M., Vorontsova, A., Konushin, A., Rukhovich, D.: Oneformer3d: One transformer for unified point cloud segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 20943–20953 (2024) 12. Kontogianni, T., Celikkan, E., Tang, S., Schindler, K.: Interactive object segmenta- tion in 3d point clouds. In: Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA). p. 2891–2897 (2023) 13. Liang, F., Wu, B., Dai, X., Li, K., Zhao, Y., Zhang, H., Zhang, P., Vajda, P., Mar- culescu, D.: Open-vocabulary semantic segmentation with mask-adapted clip. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition (CVPR). p. 7061–7070 (2023) 14. Liao, Y., Xie, J., Geiger, A.: Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(3), 3292–3310 (2022) 16S. Zhang and F. Wu 15. Liu, Q., Xu, Z., Bertasius, G., Niethammer, M.: Simpleclick: Interactive image segmentation with simple vision transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 22290–22300 (2023) 16. Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In: Proceedings of the European conference on computer vision (ECCV). p. 38–55 (2024) 17. Maninis, K.K., Caelles, S., Pont-Tuset, J., Van Gool, L.: Deep extreme cut: From extreme points to object segmentation. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition (CVPR). p. 616–625 (2018) 18. Nguyen, P., Ngo, T.D., Kalogerakis, E., Gan, C., Tran, A., Pham, C., Nguyen, K.: Open3dis: Open-vocabulary 3d instance segmentation with 2d mask guidance. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition (CVPR). p. 4018–4028 (2024) 19. Peng, S., Genova, K., Jiang, C., Tagliasacchi, A., Pollefeys, M., Funkhouser, T., et al.: Openscene: 3d scene understanding with open vocabularies. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 815–824 (2023) 20. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 652–660 (2017) 21. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learn- ing on point sets in a metric space. Advances in neural information processing systems (NeurIPS) 30 (2017) 22. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the international conference on machine learning (ICML). p. 8748–8763 (2021) 23. Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., et al.: Sam 2: Segment anything in images and videos. In: Proceedings of the International Conference on Learning Representa- tions (ICLR). vol. 2025, p. 28085–28128 (2025) 24. Schult, J., Engelmann, F., Hermans, A., Litany, O., Tang, S., Leibe, B.: Mask3d: Mask transformer for 3d semantic instance segmentation. In: Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA). p. 8216–8223. IEEE (2023) 25. Simonelli, A., Müller, N., Kontschieder, P.: Easy3d: A simple yet effective method for 3d interactive segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 24707–24716 (2025) 26. Sofiiuk, K., Petrov, I., Barinova, O., Konushin, A.: f-brs: Rethinking backpropa- gating refinement for interactive segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 8623–8632 (2020) 27. Sofiiuk, K., Petrov, I.A., Konushin, A.: Reviving iterative training with mask guid- ance for interactive segmentation. In: Proceedings of the 2022 IEEE international conference on image processing (ICIP). p. 3141–3145. IEEE (2022) 28. Takmaz, A., Fedele, E., Sumner, R.W., Pollefeys, M., Tombari, F., Engelmann, F.: Openmask3d: Open-vocabulary 3d instance segmentation. arXiv preprint arXiv:2306.13631 (2023) NegROI for Robust Interactive 3D Segmentation17 29. Tang, H., Liu, Z., Zhao, S., Lin, Y., Lin, J., Wang, H., Han, S.: Searching efficient 3d architectures with sparse point-voxel convolution. In: Proceedings of the European conference on computer vision (ECCV). p. 685–702 (2020) 30. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF international conference on computer vision (ICCV). p. 6411– 6420 (2019) 31. Vu, T., Kim, K., Luu, T.M., Nguyen, T., Yoo, C.D.: Softgroup for 3d instance segmentation on point clouds. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 2708–2717 (2022) 32. Wen, Y., Hou, Y., Zhang, S., Wu, F.: Clickenhance: Efficient 3d interactive seg- mentation with click-specific encoder and contrastive learning. IEEE Transactions on Multimedia (IEEE TMM) p. 1–12 (2026) 33. Xu, J., De Mello, S., Liu, S., Byeon, W., Breuel, T., Kautz, J., Wang, X.: Groupvit: Semantic segmentation emerges from text supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 18134–18144 (2022) 34. Yue, Y., Mahadevan, S., Schult, J., Engelmann, F., Leibe, B., Schindler, K., Kon- togianni, T.: Agile3d: Attention guided interactive multi-object 3d segmentation. In: Proceedings of the international conference on learning representations (ICLR). vol. 2024, p. 11391–11411 (2024) 35. Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V.: Point transformer. In: Proceed- ings of the IEEE/CVF international conference on computer vision (ICCV). p. 16259–16268 (2021) 36. Zhou, X., Girdhar, R., Joulin, A., Krähenbühl, P., Misra, I.: Detecting twenty- thousand classes using image-level supervision. In: Proceedings of the European conference on computer vision (ECCV). p. 350–368 (2022) 37. Zhou, Y., Gu, J., Chiang, T.Y., Xiang, F., Su, H.: Point-sam: Promptable 3d segmentation model for point clouds. arXiv preprint arXiv:2406.17741 (2024)