Paper deep dive
Hyperbolic Geometry for Open-World Object Detection in Remote Sensing Imagery
Wuzhou Li, Jiawei Zhou, Shenghang Wang, Xiang Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/10/2026, 3:04:36 AM
Summary
The paper introduces HyRS-OWOD, a framework for Open-World Object Detection (OWOD) in remote sensing imagery that utilizes hyperbolic geometry to address the limitations of Euclidean spaces in modeling hierarchical object relationships. The method features a two-step unknown-object discovery mechanism comprising Decoupled Objectness Learning (DOL) for foreground-background separation and Hyperbolic Uncertainty Learning (HUL) for known-unknown discrimination using hyperbolic radius. Additionally, it employs Hyperbolic Metric Learning (HML) to mitigate catastrophic forgetting during incremental learning by enhancing inter-class separability. Experiments on remote sensing benchmarks demonstrate improved unknown recall and incremental learning performance.
Entities (12)
Relation Signals (9)
HyRS-OWOD → containsmodule → Decoupled Objectness Learning
confidence 95% · we design a two-step unknown-object discovery mechanism: a Decoupled Objectness Learning (DOL) module
HyRS-OWOD → containsmodule → Hyperbolic Uncertainty Learning
confidence 95% · followed by a Hyperbolic Uncertainty Learning (HUL) component
HyRS-OWOD → containsstrategy → Hyperbolic Metric Learning
confidence 95% · we develop a Hyperbolic Metric Learning (HML) strategy
HyRS-OWOD → uses → Hyperbolic Geometry
confidence 95% · we investigate hyperbolic geometry for OWOD in remote sensing imagery and propose HyRS-OWOD.
Hyperbolic Metric Learning → mitigates → Catastrophic Forgetting
confidence 93% · HML strategy that enhances inter-class separability... mitigating catastrophic forgetting.
Hyperbolic Uncertainty Learning → usesfeature → Hyperbolic Radius
confidence 92% · HUL component that leverages the radius of hyperbolic embeddings as an uncertainty-aware cue
HyRS-OWOD → evaluatedon → NWPU-VHR-10
confidence 90% · Experiments on three remote sensing benchmarks... NWPU VHR-10
HyRS-OWOD → evaluatedon → DIOR
confidence 90% · Experiments on three remote sensing benchmarks... DIOR
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Open-world object detection (OWOD) extends closed-set detection by requiring models to identify unknown objects and incrementally learn them once annotations become available. In remote sensing imagery, object categories often exhibit latent hierarchical relationships that may be inadequately represented in the Euclidean spaces commonly adopted by existing methods, limiting unknown-object recall and incremental-learning performance. To address this issue, we investigate hyperbolic geometry for OWOD in remote sensing imagery and propose HyRS-OWOD. To improve unknown object recall, we design a two-step unknown-object discovery mechanism: a Decoupled Objectness Learning (DOL) module that disentangles foreground perception from semantic information to separate foreground proposals from background regions, followed by a Hyperbolic Uncertainty Learning (HUL) component that leverages the radius of hyperbolic embeddings as an uncertainty-aware cue for known-unknown discrimination. For incremental learning, we develop a Hyperbolic Metric Learning (HML) strategy that enhances inter-class separability, facilitating the incorporation of novel categories while mitigating catastrophic forgetting. Experiments on three remote sensing benchmarks demonstrate consistent improvements in unknown recall and incremental learning over state-of-the-art OWOD methods.
Tags
Links
- Source: https://arxiv.org/abs/2609.09626v1
- Canonical: https://arxiv.org/abs/2609.09626v1
Trouble viewing inline? Open PDF directly →
Full Text
72,711 characters extracted from source content.
Expand or collapse full text
Hyperbolic Geometry for Open-World Object Detection in Remote Sensing Imagery Wuzhou Li Jiawei Zhou Shenghang Wang and Xiang Li †thanks: (Corresponding author: .)†thanks: Wuzhou˜Li is with the School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan 430072, China (email: 2025054@wtu.edu.cn).†thanks: Jiawei˜Zhou is with the Electronic Information School, Wuhan University, Wuhan 430072, China (email: zhoujw@whu.edu.cn).†thanks: Shenghang˜Wang is with the Electrical and Computer Engineering, Ohio State University, Columbus, OH, USA (email: wang.19780@osu.edu).†thanks: Xiang˜Li is with the School of Artificial Intelligence, Wuhan University, Wuhan 430072, China (email: xiangli92@ieee.org). Abstract Open-world object detection (OWOD) extends closed-set detection by requiring models to identify unknown objects and incrementally learn them once annotations become available. In remote sensing imagery, object categories often exhibit latent hierarchical relationships that may be inadequately represented in the Euclidean spaces commonly adopted by existing methods, limiting unknown-object recall and incremental-learning performance. To address this issue, we investigate hyperbolic geometry for OWOD in remote sensing imagery and propose HyRS-OWOD. To improve unknown object recall, we design a two-step unknown-object discovery mechanism: a Decoupled Objectness Learning (DOL) module that disentangles foreground perception from semantic information to separate foreground proposals from background regions, followed by a Hyperbolic Uncertainty Learning (HUL) component that leverages the radius of hyperbolic embeddings as an uncertainty-aware cue for known–unknown discrimination. For incremental learning, we develop a Hyperbolic Metric Learning (HML) strategy that enhances inter-class separability, facilitating the incorporation of novel categories while mitigating catastrophic forgetting. Experiments on three remote sensing benchmarks demonstrate consistent improvements in unknown recall and incremental learning over state-of-the-art OWOD methods. Index Terms: Open-world object detection, remote sensing imagery, hyperbolic geometry, incremental learning. I Introduction Fig. 1: Comparison between closed-set and open-world object detection in remote sensing imagery. Closed-set detectors recognize objects from only a predefined set of known categories, whereas OWOD additionally identifies unseen objects as unknown and incrementally learns them as novel classes once annotations become available. Object detection is a fundamental computer vision task that underpins a wide range of remote sensing applications, including urban planning, environmental monitoring, and disaster management [1, 2]. To operate reliably in dynamic real-world scenarios, object detectors must be able to handle unknown objects that continuously emerge and may contain valuable information. However, conventional closed-set detectors are typically trained on a predefined set of categories and are expected to recognize only objects belonging to these known (labeled) classes, whereas unknown (unlabeled) objects are often misclassified as similar known categories or ignored as background, limiting the applicability of such detectors in open-world environments. In response, open-world object detection (OWOD) has emerged as a promising paradigm to bridge this gap [3, 4]. As illustrated in Fig. 1, OWOD requires a detector not only to recognize known objects but also to identify previously unseen objects as unknown and incrementally incorporate newly annotated categories across successive learning stages. Recently, OWOD has attracted growing attention in the remote sensing field [5, 6], as exemplified by the study of Tan et al. [7], which extend the unknown-aware region proposal network to handle oriented unknown objects in remote sensing imagery. Hu et al. [8] further leverage the Segment Anything Model (SAM) to discover potential unknown objects and employ a geometric-mean-based thresholding function to reduce the influence of noisy labels in the raw SAM outputs. Despite these advances, OWOD in remote sensing imagery remains challenging due to two major issues: the low recall of unknown objects and catastrophic forgetting of previously learned classes during incremental learning. The first issue stems primarily from the lack of supervision, as unknown objects, unlike known classes, are unlabeled during training and can therefore be easily confused with background regions. Furthermore, in remote sensing imagery, unknown objects may correspond to fine-grained extensions or semantic subcategories of known classes, making them prone to confusion with similar known objects, which further suppresses unknown-object recall. The second issue is mainly rooted in high inter-class similarity: the continuous introduction of novel classes shifts the input data distribution, leading to catastrophic forgetting of previously learned classes during adaptation [9]. This effect is especially pronounced when novel classes are visually similar to base classes or correspond to fine-grained variants, as their feature distributions may significantly overlap with those of previously learned classes, thereby exacerbating the problem. To tackle these challenges, our key insight is to embed the underlying hierarchical structure of remote sensing categories into hyperbolic space. Although Euclidean space has long been the de facto manifold for visual representation learning, its flat geometry cannot faithfully preserve hierarchical or tree-like relationships without considerable distortion. Remote sensing categories, however, often exhibit such hierarchical structure, where unknown objects may be semantically related to known categories at different levels of granularity. For example, a new type of destroyer can be regarded as a type of warship, while warship itself is a subcategory of ship. Embedding such data in Euclidean space may distort the underlying semantic relationships, leading to suboptimal performance. In contrast, hyperbolic space, a Riemannian manifold endowed with constant negative curvature, offers a more suitable geometry for modeling the hierarchical structure of remote sensing data. More importantly, its geometric properties are relevant to the two principal challenges of OWOD. On one hand, the hyperbolic radius of embeddings can naturally serve as a measure of uncertainty for discovering potential unknown objects [10, 11]. On the other hand, its exponential volume growth and hierarchical structure provide a suitable foundation for metric learning, helping maintain discriminative relationships among previously learned and novel classes during incremental learning, thereby mitigating catastrophic forgetting. Motivated by these observations, we introduce hyperbolic geometry into OWOD for remote sensing imagery. To improve the recall of unknown objects, we design a two-step unknown-object discovery mechanism comprising foreground–background separation and known–unknown discrimination. In the first step, we propose Decoupled Objectness Learning (DOL), which explicitly models class-agnostic objectness to distinguish foreground proposals from background regions. Although objectness is intended to be class-agnostic, supervision solely from known classes may introduce a confounding effect that suppresses potential unknown objects [12, 13, 14]. To alleviate this issue, DOL introduces a decoupling loss that reduces the statistical correlation between objectness scores and class predictions. In the second step, termed Hyperbolic Uncertainty Learning (HUL), we employ an uncertainty-based heuristic to discriminate unknown objects from known classes. Intuitively, among foreground objects, unknowns can be regarded as outliers with high predictive uncertainty. Inspired by prior studies that leverage uncertainty estimation for out-of-distribution classification [11] and segmentation tasks [15], HUL uses the hyperbolic radius of a learned embedding, measured by its distance to the origin, as a geometry-aware uncertainty cue: proposals with smaller radii tend to correspond to more abstract and ambiguous concepts. As such, foreground proposals with high objectness scores but small hyperbolic radii are more likely to be identified as unknown objects. To address catastrophic forgetting, our key idea is to learn stable and discriminative representations. Metric learning has been an effective method for learning an embedding space or distance metric in which samples from the same class are drawn closer together, while samples from different classes are pushed farther apart [16, 17]. Since remote sensing categories typically exhibit a hierarchical structure, we further perform metric learning in hyperbolic space. Compared with its Euclidean counterpart, hyperbolic space exhibits exponential volume growth, which naturally accommodates hierarchical trees with low distortion [10]. When combined with metric learning, this geometry promotes more compact intra-class representations and larger inter-class margins. Accordingly, we propose Hyperbolic Metric Learning (HML), which combines a class-balanced proposal buffer with distance-based hard-negative weighting. The proposal buffer retains representative embeddings of previously learned classes, while the weighting mechanism emphasizes visually confusing proposals from different classes. Consequently, HML reduces interference between previously learned and novel classes, thereby mitigating catastrophic forgetting during incremental learning. To summarize, our main contributions are as follows: 1. We propose HyRS-OWOD, a novel open-world object detection (OWOD) framework for remote sensing imagery that, to the best of our knowledge, is the first to investigate the effectiveness of hyperbolic geometry for OWOD in the remote sensing domain. Our method enables the detection of known classes, discovery of unknown objects, and incremental learning of novel classes in a unified manner. 2. We design a two-step unknown-object discovery mechanism comprising Decoupled Objectness Learning (DOL), which models class-agnostic objectness with a decoupling loss to separate foreground from background, and Hyperbolic Uncertainty Learning (HUL), which leverages hyperbolic uncertainty to identify unknown objects from known categories, thereby improving unknown-object recall. 3. We propose Hyperbolic Metric Learning (HML) for incremental object detection. By combining a class-balanced proposal buffer with distance-based hard-negative weighting, HML promotes intra-class compactness and inter-class separation while reducing interference between previously learned and novel classes. 4. Comprehensive experiments on three remote sensing object detection benchmarks demonstrate the effectiveness and superiority of our proposed HyRS-OWOD over existing approaches. I Related Work I-A Object Detection Deep learning has substantially advanced object detection through representative architectures such as the YOLO family [18, 19, 20, 21], two-stage detectors represented by Faster R-CNN [22, 23, 24], and transformer-based detectors such as DETR [25, 26]. In remote sensing, these architectures have been adapted to address domain-specific challenges, including small objects [27], complex backgrounds [2], and arbitrarily oriented objects [28]. Nevertheless, most existing methods are developed under a static, closed-set assumption, restricting their detection to a fixed set of predefined classes. To relax this assumption, open-set detection aims to identify objects outside the known label space [5], whereas open-vocabulary detection exploits external semantic knowledge to recognize a broader range of categories [29]. These developments highlight the unique challenges of open-world perception and motivate the need for a more continual and comprehensive approach. I-B Open-World Object Detection OWOD combines the detection of unknown objects with their incremental learning as novel classes once annotations are progressively provided. Joseph et al. [3] first formulated this task and proposed ORE, a Faster R-CNN-based framework that employs an unknown-aware pseudo-labeling scheme and an energy-based classifier to distinguish unknown objects from known classes. Following this seminal work, several studies have adopted different heuristic assumptions for discovering potential unknown objects. OW-DETR [30], for example, employs attention-driven pseudo-labeling to mine potential unknown samples. Wang et al. [13] utilize random proposal generation to mitigate the confounding effect and explore more potential proposals of unknown objects. CAT [31] decouples localization and classification with a cascade decoder and proposes a self-adaptive pseudo-labeling scheme for generating robust pseudo-labels. Sun et al. [12] further address the interference between objectness and class information via orthogonalization. Zohar et al. [32] directly learn a probabilistic objectness head from only known classes, eliminating the need for pseudo-labels. Another line of work introduces foundation-model and vision-language models [33, 34, 35, 36, 37] into OWOD for external semantic knowledge, leveraging vision-language alignment, textual descriptions, and attribute-level cues to enhance generalization to unseen categories, partially bridging OWOD with open-vocabulary object detection. Compared with generic-domain OWOD, research on OWOD in remote sensing remains relatively limited. OPODet [7] primarily improves detection of unknown rotated objects in remote sensing images. Saini et al. [38] employ multimodal large language models to assign semantic labels to unknown candidate regions generated by closed-set detectors, enabling the automatic discovery and naming of unknown classes. However, these methods focus mainly on unknown-object discovery without subsequent incremental learning, thus falling short of the full OWOD requirements. More recently, Hu et al. [8] proposed a SAM-guided OWOD framework with a label-mapping alignment mechanism to filter noisy candidate proposals. Despite this progress, existing remote sensing OWOD methods still largely overlook the latent hierarchical structure among remote sensing categories. Motivated by these observations, we explore hyperbolic geometry for OWOD in remote sensing images, aiming to better model the semantic relationships between unknown objects and known classes. I-C Hyperbolic Geometry Hyperbolic geometry has attracted increasing attention in representation learning because of its ability to represent hierarchical and tree-like structures with low distortion [39, 40]. Hyperbolic representations have been applied to natural language processing [41], knowledge graph modeling [42], and graph neural networks [43]. More recently, they have been extended to various computer vision tasks, including continual learning [44, 17], few-shot learning [45], image classification [46], semantic segmentation [47, 48], and object detection [49]. These studies suggest that hyperbolic embeddings can effectively capture latent semantic similarities and hierarchical relationships in visual data, providing a complementary alternative to conventional Euclidean representations. Moreover, another key property lies in the Poincaré model of hyperbolic spaces, where the embedding radius—i.e., the distance to the origin—can serve as a measure of model uncertainty [41, 39]. This property has been exploited in tasks such as visual anomaly recognition [11] and image segmentation [47]. Hyperbolic geometry has also recently been introduced into OWOD. Kong et al. [50] develop a hyperbolic learning framework that models hierarchical relationships between visual and caption embeddings, thereby reducing the influence of hallucinated synthetic captions. Doan et al. [51] propose Hyp-OW, which combines hyperbolic contrastive learning for object representation, a superclass regularizer for modeling class-level hierarchies, and adaptive relabeling for retrieving unknown objects based on hyperbolic distance. These studies demonstrate the potential of hyperbolic geometry for modeling category hierarchies and facilitating unknown-object discovery. However, they are primarily designed for natural-scene imagery, and the application of hyperbolic geometry to remote sensing OWOD remains underexplored. I-D Hyperbolic Learning in Remote Sensing Remote sensing imagery often contains semantically related object categories with implicit hierarchical structures, which naturally motivates the use of hyperbolic spaces. Recent studies have introduced hyperbolic geometry into remote sensing tasks, including image classification [52], semantic segmentation [53, 54], and change detection [55]. In this work, we introduce hyperbolic geometry into remote sensing OWOD to better capture the latent hierarchical relationships between known and unknown classes and formulate an uncertainty-aware unknown object discovery strategy based on the hyperbolic radius. Moreover, we further perform metric learning in hyperbolic space to preserve intra-class compactness and inter-class separability during incremental learning, thereby advancing OWOD for remote sensing imagery. I Preliminaries In this section, we begin by introducing the fundamentals of hyperbolic space, followed by a formal definition of the Poincaré ball model. We then describe several commonly used operations within this model. I-A Hyperbolic Space Hyperbolic space is defined as a complete, simply connected Riemannian manifold with constant negative sectional curvature. Among its various isometric models, the Poincaré ball model has conformal properties, which enable effective modeling of hierarchical data structures and facilitate efficient optimization. I-B Poincaré Ball Model An n-dimensional Poincaré ball model with constant negative curvature −c-c (c>0)(c>0) is defined as a Riemannian manifold cn=(ℬcn,g)M_c^n=(B_c^n,g^B), where ℬcnB_c^n denotes an open ball embedded in the Euclidean space ℝnR^n: ℬcn=∈ℝn:c‖2<1,B_c^n= \x ^n:c\|x\|^2<1 \, (1) with ∥⋅∥\|·\| the Euclidean norm. The radius of the ball is 1/c1/ c. The Riemannian metric tensor of the Poincaré ball model is conformal to the Euclidean metric and is given by g=(λc)2gE,g^B_x= ( _x^c )^2g^E, (2) where gEg^E denotes the Euclidean metric tensor and λc _x^c is the conformal factor: λc=21−c‖2. _x^c= 21-c\|x\|^2. (3) As ‖\|x\| approaches the boundary, the conformal factor grows without bound, enabling the model to capture hierarchical structures with exponentially expanding capacity near the boundary. I-C Möbius addition We adopt the formalism of Möbius gyrovector spaces, which generalize several Euclidean operations to hyperbolic geometry and enables algebraic computations within the Poincaré ball model. For ,∈ℬcnx,y _c^n, the Möbius addition ⊕c _c is defined as: ⊕c=(1+2c⟨,⟩+c‖2)+(1−c‖2)1+2c⟨,⟩+c2‖2‖2.x _cy= (1+2c ,y +c\|y\|^2 )x+ (1-c\|x\|^2 )y1+2c ,y +c^2\|x\|^2\|y\|^2. (4) I-D Exponential Map For ∈ℬcnx _c^n and ∈Tℬcnv∈ T_xB_c^n, the exponential map is defined as: expc()=⊕c(tanh(cλc‖2)c‖). _x^c(v)=x _c ( ( c _x^c\|v\|2 ) v c\|v\| ). (5) I-E Distance and Hyperbolic Radius For ,∈ℬcnx,y _c^n, the hyperbolic distance is given by dc(,)=1carccosh(1+2c‖−‖2(1−c‖2)(1−c‖2)).d_c(x,y)= 1 carccosh (1+ 2c\|x-y\|^2(1-c\|x\|^2)(1-c\|y\|^2) ). (6) The hyperbolic radius of a point x is defined as its Poincaré distance to the ball origin: rc()≜dc(,)=2carctanh(c‖).r_c(x) d_c(x,0)= 2 carctanh ( c\|x\| ). (7) The hyperbolic radius of an object proposal serves as a measure of model uncertainty: higher confidence corresponds to embeddings lying farther away from the origin in hyperbolic space, whereas lower confidence leads to embeddings located closer to the origin [10, 15]. I-F δ-Hyperbolicity Following the analysis in [10], we compute the Gromov δ-hyperbolicity of each remote sensing dataset to measure its structural properties. This evaluation is performed by first computing the Gromov product for points x,y,z∈x,y,z , which is defined as: (y,z)x=12(d(x,y)+d(x,z)−d(y,z)),(y,z)_x= 12 (d(x,y)+d(x,z)-d(y,z) ), (8) where (,d)(X,d) denotes an arbitrary metric space. For a set of points, we construct the matrix M of pairwise Gromov products. The δ-hyperbolicity is then computed as: δ=maxi,j[(M⊗M)ij−Mij],δ= _i,j [(M M)_ij-M_ij ], (9) where ⊗ denotes the min-max matrix product defined as: (A⊗B)ij=maxkminAik,Bkj.(A B)_ij= _k \A_ik,B_kj\. (10) with i,j,ki,j,k denoting the indices of matrices A and B. We further compute the relative hyperbolicity to ensure scale invariance. Its value ranges from 0 to 1, where values closer to 0 indicate stronger hierarchical structure in the data, while values closer to 1 suggest a weaker hierarchy. δrel=δmaxi,jMij. _rel= δ _i,jM_ij. (11) As shown in [10], δrel _rel can be used to estimate the manifold curvature of the Poincaré ball model: c()=(0.144δrel)2.c(X)= ( 0.144 _rel )^2. (12) We follow the above procedure to evaluate δrel _rel on remote sensing detection datasets, including NWPU VHR-10, DIOR, and DOTA. For each dataset, we extract region-level embeddings from object proposals using different detection models. We randomly sample 450 embeddings per run and the final results are reported in Table I. TABLE I: Relative δ-hyperbolicity values of embeddings extracted by different feature extractors. NWPU VHR-10 DIOR DOTA VGG16 0.2306 0.2296 0.2525 GoogLeNet 0.2196 0.2415 0.2372 ResNet50 0.2544 0.2751 0.2910 IV Methodology IV-A Problem Formulation Let t=1,2,…,CK^t=\1,2,…,C\ denote the annotated known classes, and t=C+1,…U^t=\C+1,…\ denote the unbounded set of unknown classes of interest that may appear during inference. Given a time step t, the known object classes tK^t are labeled in the dataset t=ℐt,ℒtD^t=\I^t,L^t\, which consists of N input images ℐt=I1,…,INI^t=\I_1,…,I_N\ and their corresponding label sets ℒt=L1,…,LNL^t=\L_1,…,L_N\. Each Li=li1,…,liPiL_i=\l_i1,…,l_iP_i\ contains the annotations of PiP_i object instances in image IiI_i. Each instance annotation is represented as lij=[cij,xij,yij,wij,hij]l_ij=[c_ij,x_ij,y_ij,w_ij,h_ij], where cijc_ij denotes its class label, (xij,yij)(x_ij,y_ij) denotes the center coordinates of its bounding box, and wijw_ij and hijh_ij denote its width and height, respectively. In OWOD, the trained model ℳtM^t is required to detect objects from known classes tK^t and identify unknown objects belonging to tU^t, labeling them as ”unknown”. The model then sends the discovered unknown objects to an oracle, which annotates the new classes of interest. Together with their corresponding training examples, these annotations update tD^t to t+1D^t+1 and tK^t to t+1=1,2,…,C,…,C+nK^t+1=\1,2,…,C,…,C+n\. The model incorporates these n novel classes into the set of known classes and incrementally updates itself to ℳt+1M^t+1 without retraining from scratch on the entire dataset t+1D^t+1 while preserving previously learned knowledge. This cycle continues over the model’s lifespan. IV-B Overall Architecture Fig. 2: Overview of our proposed HyRS-OWOD framework. The backbone and FPN extract multi-scale features from an input image, which are processed together with randomly generated proposals by the dynamic head to obtain proposal features if_i. These features are used for bounding-box regression and mapped into the Poincaré ball via the exponential map expc(⋅) _0^c(·). For unknown-object prediction, DOL estimates class-agnostic objectness, while HUL measures uncertainty using the hyperbolic radius; proposals with high objectness and high hyperbolic uncertainty are identified as unknown. In parallel, the classifier predicts known object categories. During incremental training, HML is additionally applied to learn discriminative representations and mitigate catastrophic forgetting. The overall architecture of the proposed HyRS-OWOD framework is illustrated in Fig. 2. HyRS-OWOD is built upon RandBox [13] and introduces three key components: (1) Decoupled Objectness Learning (DOL) (Sec. IV-C), which separates foreground from background; (2) Hyperbolic Uncertainty Learning (HUL) (Sec. IV-D), which distinguishes unknown objects from known classes; and (3) Hyperbolic Metric Learning (HML) (Sec. IV-E), which supports learning novel classes while mitigating catastrophic forgetting. Given an input image, the model first extracts feature maps and then randomly generates a set of candidate proposal boxes, which are fed into their exclusive dynamic head [56] to generate object features. The bounding box regression branch refines the proposal locations based on these object features. In parallel, the object features are projected into hyperbolic space to obtain hyperbolic embeddings. The hyperbolic embeddings are then fed into a known-class classification branch and a class-agnostic objectness branch. The objectness branch is implemented as a lightweight multilayer perceptron (MLP) that outputs an objectness score for each proposal. The hyperbolic embeddings are further used for uncertainty estimation and metric learning in the following modules. During training, we follow RandBox to employ the dynamic matcher [57] to assign proposal boxes to ground-truth instances. The matched proposals are used for known class detection, while the remaining unmatched proposals with top objectness are selected as initial candidate unknowns and used to optimize HUL. At inference, unknown objects are directly identified based on their class-agnostic objectness and hyperbolic uncertainty. During incremental learning, we retain 50 exemplars from each previously learned class following the exemplar-replay protocols adopted in [3, 12, 8]. IV-C Decoupled Objectness Learning Objectness should indicate whether a proposal corresponds to a foreground object, irrespective of its semantic category. To train the objectness branch, we adopt a binary cross-entropy loss: ℒobj=−∑b[ybfglog(ob)+(1−ybfg)log(1−ob)],L_obj=- _b [y_b^fg (o_b)+(1-y_b^fg) (1-o_b) ], (13) where ob∈[0,1]o_b∈[0,1] and ybfg∈0,1y_b^fg∈\0,1\ are the predicted objectness score and the ground-truth foreground/background label for proposal b, respectively. Because the objectness branch is supervised using only known-class annotations, its predictions may become correlated with known-class confidence. Consequently, proposals containing unknown objects may receive low objectness scores and be incorrectly suppressed as background. To reduce this semantic bias, we introduce a decoupling loss that penalizes the linear correlation between the objectness scores and known-class predictions. Inspired by [12], we measure their linear correlation using the squared correlation coefficient: ℒdec=∑c=1C(cov(pb,c,ob))2var(pb,c)var(ob),L_dec= _c=1^C (cov(p_b,c,o_b) )^2var(p_b,c)\,var(o_b), (14) where covariance and variance are computed over all proposals within a mini-batch; C denotes the number of known classes. By reducing the statistical dependence between objectness and known-class predictions, DOL encourages the objectness branch to capture class-agnostic foreground cues rather than category-specific semantic information. IV-D Hyperbolic Uncertainty Learning for Unknown Discrimination Fig. 3: Illustration of the proposed HUL strategy for known–unknown discrimination. Different colors represent proposals from different known classes, gray points denote candidate unknown proposals. (a) Candidate unknown proposals are selected for uncertainty-aware learning. (b) The HUL loss ℒHULL_HUL encourages candidate unknown proposals to lie in the high-uncertainty region near the origin. (c) After training, uncertain unknown proposals tend to lie closer to the origin than confident known proposals. The yellow dashed circle represents the normalized radius threshold ρ~=τρ ρ= _ρ used for known–unknown discrimination. We estimate the uncertainty of each proposal by its radial position in the Poincaré ball. Given an input image I, let I=ii=1NIP_I=\f_i\_i=1^N_I denote the set of proposal embeddings, where i∈ℝDf_i ^D. Each proposal embedding is mapped into a d-dimensional Poincaré ball cd=∈ℝd:c‖22<1B^d_c=\z ^d:c\|z\|_2^2<1\ with constant negative curvature −c-c by a lightweight hyperbolic projection head gϕ:ℝD→ℝdg_φ:R^D ^d, followed by a projection onto the ball: i=Πcd(gϕ(i))=tanh(c‖gϕ(i)‖2)gϕ(i)c‖gϕ(i)‖2,z_i= _B^d_c (g_φ(f_i) )= ( c\,\|g_φ(f_i)\|_2 ) g_φ(f_i) c\,\|g_φ(f_i)\|_2, (15) where i∈cdz_i ^d_c denotes the hyperbolic proposal embedding, and the zero vector case is defined by continuity. we compute the hyperbolic radius of proposal i as its distance to the origin [10, 15]: ρi=dc(i,)=2carctanh(c‖i‖2). _i=d_c(z_i,0)= 2 carctanh ( c\|z_i\|_2 ). (16) Since the scale of the hyperbolic radius may vary across images, we normalize the radius within each image: ρ~i=ρimax1≤j≤NIρj+ϵ, ρ_i= _i _1≤ j≤ N_I _j+ε, (17) where ϵε is a small constant to avoid division by zero. The hyperbolic uncertainty score is then defined as ui=1−ρ~i,u_i=1- ρ_i, (18) where a larger uiu_i indicates higher uncertainty. In this way, proposals close to the origin obtain larger uncertainty scores, whereas confident known proposals near the boundary receive smaller uncertainty scores. To explicitly encourage unknown proposals to stay in the high-uncertainty region of the Poincaré ball, we introduce a smooth hyperbolic unknown loss to regularize candidate unknown proposals: ℒHUL=1||∑i∈1βlog(1+exp(β(ρ~i−τρ))),L_HUL= 1|U| _i 1β (1+ (β( ρ_i- _ρ) ) ), (19) where U denotes the set of candidate unknown proposals, τρ _ρ is the radius threshold, and β controls the smoothness of the margin. When β becomes large, Eq. (19) approaches the hard hinge loss max(0,ρ~i−τρ) (0, ρ_i- _ρ). The overall HUL process is illustrated in Fig. 3. IV-E Hyperbolic Metric Learning for Incremental Learning In this subsection, we introduce HML to mitigate catastrophic forgetting in incremental learning. In particular, HML assigns larger weights to hard-negative proposal pairs that belong to different classes but are located close to each other. At incremental step t, let tC^t denote the set of all classes learned up to and including step t. We maintain a class-balanced proposal buffer Q, which stores hyperbolic proposal embeddings from previously learned classes. Given the foreground proposal embeddings in the current mini-batch ℬB, we construct the metric learning set as =ℬ∪.A=B . For each anchor proposal embedding i∈z_i with class label yi∈ty_i ^t, we define its positive and negative sets as (i)=p∈∖i:yp=yi,(i)=n∈:yn≠yi. \ aligned P(i)&= \p \i\:y_p=y_i \,\\ N(i)&= \n :y_n≠ y_i \. aligned . (20) Only anchors with non-empty positive sets are used for metric learning, and the corresponding anchor index set is denoted as S. In incremental learning, the most informative negative samples are typically those that lie close to the anchor in the embedding space but belong to different classes. To exploit this, we assign larger weights to hard negatives according to their Poincaré distances to the anchor: αi,n−=exp(−dc(i,n)/τh)∑m∈(i)exp(−dc(i,m)/τh),n∈(i), _i,n^-= (-d_c(z_i,z_n)/ _h ) _m (i) (-d_c(z_i,z_m)/ _h ), n (i), (21) where dc(⋅,⋅)d_c(·,·) denotes the Poincaré distance and τh _h is a temperature parameter that controls the hardness distribution. A smaller distance yields a larger weight, thereby directing the model to focus on more confusing inter-class proposal pairs. We define the positive term and the hard-negative term as Ai+=∑p∈(i)exp(−dc(i,p)/τm),Ai−=∑n∈(i)αi,n−exp(−dc(i,n)/τm). \ aligned A_i^+&= _p (i) (-d_c(z_i,z_p)/ _m ),\\ A_i^-&= _n (i) _i,n^- (-d_c(z_i,z_n)/ _m ). aligned . (22) where τm _m is the metric learning temperature. HML loss is then written as ℒHML=−1||∑i∈logAi+Ai++Ai−.L_HML=- 1|S| _i A_i^+A_i^++A_i^-. (23) This objective pulls same-class proposal embeddings closer together in the Poincaré ball while pushing hard negatives from different classes apart. By emphasizing the most confusing inter-class pairs, the model learns more discriminative class boundaries during incremental learning. The proposal buffer Q is updated after each mini-batch by storing foreground proposal embeddings along with their corresponding labels. To prevent class imbalance, we retain at most KmaxK_ embeddings per class and update the buffer in a first-in-first-out manner. IV-F Overall Training Objective The detection loss for known objects consists of a classification loss ℒclsL_cls and a bounding box regression loss ℒregL_reg: ℒkn=ℒcls+ℒreg,L_kn=L_cls+L_reg, (24) where ℒclsL_cls is the focal loss [58] for class imbalance and the ℒregL_reg is smooth L1L_1 loss [59]. The overall training objective is therefore ℒbase=ℒkn+λobjℒobj+λdecℒdec+λHULℒHUL,L_base=L_kn+ _objL_obj+ _decL_dec+ _HULL_HUL, (25) where λobj _obj, λdec _dec, and λHUL _HUL are the corresponding loss weights. In the incremental learning stage, we further introduce the weighted HML loss ℒHMLL_HML. The overall incremental training objective is defined as ℒinc=ℒbase+λHMLℒHML,L_inc=L_base+ _HMLL_HML, (26) where λHML _HML controls the contribution of the weighted HML objective. V Experiments In this section, we first describe the datasets, evaluation metrics, and implementation details. We then present quantitative and qualitative comparisons against baseline and state-of-the-art approaches. Finally, we conduct ablation studies to validate the effectiveness of the proposed components. V-A Datasets V-A1 NWPU VHR-10 This dataset consists of 650 positive images, categorized into ten geospatial object classes for object detection, including airplane (AP), vehicle (VH), ship (SH), harbor (HB), baseball diamond (BD), basketball court (BC), tennis court (TC), ground track field (GTF), storage tank (ST), and bridge (BR). The spatial resolution of the images ranges from 0.5 to 2 m. V-A2 DIOR This dataset comprises 23,463 aerial images, containing 192,472 object instances across 20 categories: airplane (AP), windmill (WM), airport (AT), train station (TS), baseball field (BF), tennis court (TC), bridge (BR), storage tank (ST), chimney (CM), ship (SH), dam (DM), expressway service area (EA), stadium (SD), expressway toll station (ES), harbor (HB), golf course (GF), vehicle (VH), ground track field (GTF), overpass (OP), and basketball court (BC). The spatial resolution ranges from 0.5 to 30 m, and each image has a fixed size of 800×800800× 800 pixels. V-A3 DOTA-v1.5 This dataset comprises 2,806 large-scale aerial images, with dimensions ranging from 800×800800× 800 to 4,000×4,0004,000× 4,000 pixels. It contains 402,089 annotated instances across 16 categories: basketball court, baseball diamond, bridge, container crane, ground track field, harbor, helicopter, large vehicle, plane, roundabout, small vehicle, ship, storage tank, soccer ball field, swimming pool, and tennis court. V-A4 Dataset Splits for OWOD For a fair comparison, we adopt the same known–unknown class partitions as in [8], including the 16+4, 10+10, and 4+16 splits on DIOR, the 8+8 split on DOTA, and the 8+2 split on NWPU VHR-10. In addition to these benchmark settings, we establish a progressive OWOD protocol on NWPU VHR-10 to further evaluate the model under sequential category discovery, as summarized in Table I. In Task 1, seven categories (AP, BC, BD, BR, GTF, HB, and SH) are used as the known classes, whereas ST, TC, and VH are regarded as unknown classes. ST, TC, and VH are then sequentially introduced as newly annotated classes in Tasks 2, 3, and 4, respectively. This protocol evaluates the model’s ability to discover unknown objects, incrementally learn novel categories, and retain knowledge of previously learned classes. Furthermore, to systematically evaluate the model’s performance, we conduct additional experiments on DIOR under the 15+5 and 19+1 settings following the protocols commonly adopted in generic-domain OWOD methods [3, 32, 12, 13]. For clarity, the novel classes are highlighted with a gray background in Table IV. V-B Evaluation Metrics We follow the evaluation protocol established in [3, 32, 12]. For known classes, we report mean average precision (mAP) to evaluate the overall detection performance. In the incremental learning setting, mAP is reported separately for known classes and novel classes. For unknown classes, we adopt unknown recall (U-Recall) as the primary metric for evaluating the model’s ability to retrieve unknown objects. We further report wilderness impact (WI) [60], which measures the influence of unknown objects on the precision of known class detection, and absolute open-set error (A-OSE) [61], which counts the number of unknown objects misclassified as known classes. V-C Implementation Details We build our model upon RandBox [13], using ResNet-50 [62] with a Feature Pyramid Network (FPN) [63] as the backbone. All experiments are conducted on a single NVIDIA GeForce RTX 5090 GPU with a batch size of 8. The model is optimized using AdamW with a base learning rate of 3×10−53× 10^-5 and a weight decay of 1×10−41× 10^-4. A warmup schedule is applied over the first 33 iterations. The learning rate starts at 3×10−73× 10^-7, corresponding to a warmup factor of 0.01, and linearly increases to the base learning rate of 3×10−53× 10^-5. The base training stage is conducted for 30 epochs on NWPU VHR-10, DIOR, and DOTA. Each incremental learning task is also trained for 30 epochs. TABLE I: Open-World Object Detection performance on DIOR (Top), DOTA (Middle), and NWPU VHR-10 (Bottom) under various settings. The best results within each setting are highlighted in bold. Task IDs (→) Task 1 (Base Stage) Task 2 (Incremental Stage) U-Recall (↑ ) mAP (↑ ) mAP (↑ ) Previously known Currently known Both DIOR (16 + 4 Setting) ORE [3] 14.18 50.62 46.30 67.11 50.46 OW-DETR [30] 18.53 51.58 49.71 67.46 53.26 PROB [32] 23.37 53.90 51.61 71.01 55.49 KTCN [64] 27.59 55.88 52.96 70.90 56.55 SGROD [65] 35.82 55.68 52.22 71.22 56.02 Hu et al. [8] 42.93 57.69 54.23 75.47 58.48 Base model 39.07 69.15 59.51 63.74 60.36 Ours 52.93 71.26 68.59 72.57 69.39 DIOR (10 + 10 Setting) ORE [3] 21.41 54.62 49.61 53.11 51.36 OW-DETR [30] 28.01 56.58 49.54 55.20 52.37 PROB [32] 27.77 57.90 54.21 55.01 54.61 KTCN [64] 33.10 59.22 56.12 58.12 57.12 SGROD [65] 39.25 59.17 55.02 61.11 58.07 Hu et al. [8] 46.33 62.51 57.09 63.21 60.15 Base model 34.56 71.13 58.31 60.69 59.50 Ours 53.72 73.49 61.14 65.97 63.56 DIOR (4 + 16 Setting) ORE [3] 16.77 58.90 59.04 53.66 54.74 OW-DETR [30] 21.08 64.62 62.44 54.60 56.17 PROB [32] 19.57 67.10 61.91 51.17 53.32 KTCN [64] 23.67 65.34 59.76 57.89 58.26 SGROD [65] 31.77 69.18 63.17 56.92 58.17 Hu et al. [8] 41.61 68.79 67.39 59.26 60.89 Base model 23.21 73.10 63.50 58.83 59.76 Ours 40.11 79.48 70.84 60.98 62.96 DOTA (8 + 8 Setting) ORE [3] 15.01 41.92 39.72 56.11 47.92 OW-DETR [30] 13.79 43.55 39.23 57.01 48.12 PROB [32] 19.05 43.09 40.06 59.01 49.54 KTCN [64] 25.02 46.80 44.99 58.60 51.80 SGROD [65] 31.90 45.11 43.69 59.96 51.82 Hu et al. [8] 36.44 47.99 45.84 61.24 53.54 Base model 28.35 45.53 41.63 49.99 45.81 Ours 36.85 49.36 47.72 58.47 53.10 NWPU VHR-10 (8 + 2 Setting) ORE [3] 27.18 83.19 81.30 81.40 81.32 OW-DETR [30] 26.52 83.58 82.15 78.70 81.46 PROB [32] 38.49 84.16 84.21 78.01 82.97 KTCN [64] 36.74 83.78 86.50 80.90 85.38 SGROD [65] 46.82 82.68 83.72 81.02 83.18 Hu et al. [8] 55.90 86.03 86.23 85.73 86.13 Base model 45.18 91.94 91.47 76.85 88.54 Ours 58.94 89.06 91.57 84.86 90.23 TABLE I: Evaluation of Unknown Object confusion on NWPU VHR-10. Task IDs (→) Task 1 Task 2 Task 3 Task 4 Method mAP U-Recall WI A-OSE mAP U-Recall WI A-OSE mAP U-Recall WI A-OSE mAP (↑)( ) (↓)( ) (↓)( ) (↑)( ) (↓)( ) (↓)( ) (↑)( ) (↓)( ) (↓)( ) (↑)( ) ORE [3] 88.97 27.33 0.000000 2 89.74 30.18 0.004090 3 87.40 34.23 0.006612 15 87.22 OW-DETR [30] 85.95 24.07 0.000000 9 85.39 25.76 0.000129 8 85.60 27.34 0.001691 7 82.98 PROB [32] 86.60 36.21 0.000000 0 82.49 31.33 0.000000 0 78.24 29.68 0.000000 0 84.88 KTCN [64] 82.62 33.34 0.000234 12 78.27 37.94 0.000311 15 77.16 36.06 0.000707 3 82.48 SGROD [65] 80.48 35.25 0.000000 0 84.24 40.46 0.000000 0 81.33 39.35 0.000000 0 83.90 Base model 90.48 38.39 0.000000 1 83.76 45.54 0.000000 13 84.56 51.97 0.000000 2 84.04 Ours 93.74 55.28 0.000578 2 94.46 60.57 0.000214 2 93.63 63.01 0.000000 0 92.39 V-D Quantitative Results Open-world object detection. We compare our method with other state-of-the-art methods on DIOR, DOTA, and NWPU VHR-10 datasets under different incremental settings and summarize the results in Table I. Following the experimental protocol of Hu et al. [8], we report U-Recall and mAP after base training, together with the mAP of previously known, currently known, and overall performance after incremental learning. For unknown object detection, our method achieves the best performance in four of the five settings, surpassing the previous best results by 10.00, 7.39, 0.41, and 3.04 percentage points in DIOR 16+4, DIOR 10+10, DOTA 8+8, and NWPU VHR-10 8+2, respectively. In terms of known object detection, our method outperforms the competing methods in most settings, achieving the highest mAP after base training in four of the five settings. After incremental learning, our method achieves the best performance on previously known classes across all five settings, with improvements of 14.36, 4.05, 3.45, 1.88, and 5.07 percentage points, respectively. Moreover, our method obtains the highest overall mAP in four of the five settings, outperforming the previous best results by 10.91, 3.41, 2.07, and 4.10 percentage points in DIOR 16+4, DIOR 10+10, DIOR 4+16, and NWPU VHR-10 8+2, respectively, further demonstrating the superiority of our method in maintaining previously learned knowledge while achieving strong overall detection performance after incremental learning. To further evaluate the confusion between unknown and known objects, we conduct additional experiments on NWPU VHR-10 using WI and A-OSE as evaluation metrics, with the results reported in Table I. Our method consistently achieves the highest U-Recall across all three open-world stages, outperforming the best competing methods by 19.07, 20.11, and 23.66 percentage points in Tasks 1–3, respectively. Meanwhile, the WI remains at a very low level, with values of only 0.000578 and 0.000214 in Tasks 1 and 2, and further decreases to zero in Task 3. A similar trend can be observed for A-OSE, where only two unknown objects are misclassified as known classes in Tasks 1 and 2, while no such confusion occurs in Task 3. These results show that the substantial gains in unknown-object recall are achieved while maintaining low absolute WI and A-OSE values. Incremental object detection. To further evaluate the class-incremental detection performance of our method, we compare it with existing OWOD approaches following the evaluation protocols adopted in [3, 30, 32]. Table IV reports the results on DIOR under two class-incremental settings. Although the RandBox baseline already achieves competitive performance, our method further improves the overall mAP by 1.7 and 4.9 percentage points in the two settings, respectively. These improvements demonstrate the effectiveness of the proposed method in learning novel classes while maintaining the performance of previously learned classes. TABLE IV: Incremental object detection results on DIOR. 15 + 5 setting AP AT BF BC BR CM DM EA ES GF GTF HB OP SH SD ST TC TS VH WM mAP ORE [3] 55.5 64.7 64.6 83.7 24.8 75.8 42.6 53.1 43.6 71.9 64.4 46.9 45.5 35.0 53.5 43.7 77.9 35.9 52.0 57.4 54.6 OW-DETR [30] 71.6 59.6 53.1 56.4 26.8 71.6 46.1 44.0 38.7 73.3 41.9 36.6 39.6 32.1 58.3 53.7 78.4 47.4 41.3 74.1 52.2 PROB [32] 81.7 63.2 83.2 61.8 28.0 81.4 48.1 45.4 49.5 66.1 56.9 37.0 42.6 28.0 71.1 63.0 74.4 38.9 43.0 47.4 55.5 KTCN [64] 72.5 56.4 70.2 80.9 27.2 76.1 45.3 51.8 50.6 70.5 66.1 44.8 45.8 70.7 50.7 60.3 82.3 25.3 46.8 68.8 58.1 SGROD [65] 85.9 64.9 85.6 55.4 14.1 87.3 51.4 33.1 39.0 70.6 59.3 3.7 47.7 30.9 66.4 76.1 85.5 46.1 55.2 67.4 56.3 Base model 89.5 61.9 83.7 67.3 27.0 86.2 47.2 45.8 51.3 65.7 58.5 44.4 47.7 11.2 74.6 79.2 87.4 42.9 70.3 67.8 60.5 Ours 89.5 68.9 88.4 71.0 29.3 86.4 50.4 58.1 53.9 72.1 57.3 40.9 44.3 22.1 74.1 76.7 86.6 37.1 66.4 70.4 62.2 19 + 1 setting AP AT BF BC BR CM DM EA ES GF GTF HB OP SH SD ST TC TS VH WM mAP ORE [3] 55.1 63.4 67.2 86.2 29.0 79.2 41.8 53.1 46.9 68.4 66.5 49.3 50.6 67.3 68.9 55.5 80.0 45.1 41.1 59.8 60.7 OW-DETR [30] 64.0 60.5 65.9 78.9 31.2 73.9 41.8 50.6 44.3 67.2 48.9 28.3 48.4 68.7 29.4 49.8 76.4 44.2 38.4 73.8 54.2 PROB [32] 83.2 55.2 86.3 70.5 28.2 81.3 34.8 43.6 52.7 51.6 55.9 12.7 41.3 52.1 64.9 72.2 82.9 40.1 49.1 45.0 55.2 KTCN [64] 77.7 57.5 70.8 85.5 28.5 75.0 45.6 52.9 52.9 67.9 66.2 47.5 49.6 70.2 62.1 60.1 80.9 33.3 39.1 68.0 59.6 SGROD [65] 89.4 51.0 90.4 74.6 22.6 87.9 38.7 59.0 53.2 61.3 56.8 13.2 48.0 29.8 71.3 75.3 84.2 42.0 51.6 63.0 58.2 Base model 79.2 66.6 87.6 78.0 35.7 88.2 51.0 62.7 55.5 76.1 56.8 55.4 55.0 11.5 86.1 75.3 85.8 47.4 45.5 76.6 63.8 Ours 90.5 75.4 92.5 79.8 39.0 84.4 50.6 65.8 56.8 78.5 68.7 46.7 57.8 68.1 77.4 78.5 88.5 39.1 58.8 76.9 68.7 V-E Qualitative Results Fig. 4: Qualitative comparison on the NWPU VHR-10 dataset. The first row shows the ground-truth annotations (green), while the second and third rows show the predictions of the baseline and our method, respectively. Orange and magenta boxes denote known and unknown objects, respectively. Fig. 5: Qualitative comparison on the DIOR dataset. The first row shows the ground-truth annotations (green), while the second and third rows show the predictions of the baseline and our method, respectively. Orange and magenta boxes denote known and unknown objects, respectively. Fig. 4 presents qualitative comparisons between our method and the RandBox baseline on NWPU VHR-10. Compared with the baseline, our method discovers more unknown objects while maintaining accurate detection of known classes. In the left example, our method identifies more tennis courts as unknown while correctly detecting the known basketball court and baseball diamonds, demonstrating improved unknown-object recall without compromising known-class detection. In the middle example, both vehicles are correctly identified as unknown with higher confidence scores, while the false-positive prediction produced by the baseline is suppressed. Similarly, in the right example, our method identifies individual tennis courts as unknown with more accurate localization while preserving reliable predictions for the known baseball diamonds and basketball courts. These qualitative observations are in line with the improvements in U-Recall, WI, and A-OSE reported in Table I, showing that our method improves unknown-object discovery while reducing the misclassification of unknown objects as known classes. Similar improvements are observed on DIOR, as illustrated in Fig. 5. Our method detects more unknown vehicles and windmills that are missed by the baseline while maintaining reliable detection of known objects and suppressing false-positive unknown predictions. These results further demonstrate the effectiveness of our method across different remote sensing datasets and object categories. V-F Ablation Study TABLE V: Ablation study of the proposed components on NWPU VHR-10 under the 8+2 setting. Task 2 DOL HUL U-Recall K-mAP WI A-OSE (↑ ) (↑ ) (↓ ) (↓ ) Base model – – 45.2 91.9 0.0002 8 ✔ ✗ 48.1 90.2 0.0001 10 ✗ ✔ 51.2 89 0 0 Ours ✔ ✔ 58.9 89.1 0.0000 1 Task 2 HML Previously Currently known known Both Base model – 91.5 76.9 88.5 Ours ✔ 91.6 91.2 91.5 Fig. 6: The t-SNE visualization of proposal embeddings learned without (a) and with (b) the HML module. In this subsection, we conduct ablation studies to analyze the contribution of each proposed component to the performance of HyRS-OWOD. All ablation experiments are conducted on NWPU VHR-10 under the 8+2 known–unknown class split described in Sec. V-A4. Effectiveness of DOL and HUL. We first evaluate the contributions of DOL and HUL to unknown object discovery. As shown in the upper part of Table V, introducing DOL alone increases U-Recall from 45.2% to 48.1%, indicating that decoupling class-agnostic objectness from semantic classification helps retrieve potential unknown objects. Incorporating HUL alone produces a larger improvement in U-Recall, from 45.2% to 51.2%, while reducing both WI and A-OSE to zero. This result demonstrates the effectiveness of the hyperbolic radius as an uncertainty cue for known–unknown discrimination. When DOL and HUL are combined, U-Recall further increases to 58.9%, outperforming the baseline by 13.7 percentage points. Meanwhile, WI remains zero and A-OSE decreases from 8 to 1. Although K-mAP decreases moderately from 91.9% to 89.1%, the combined model achieves a substantially better balance between unknown-object discovery and known-class detection. These results also indicate that DOL and HUL are complementary: DOL improves class-agnostic foreground discovery, whereas HUL facilitates the subsequent discrimination between known and unknown objects. Effectiveness of HML. We further investigate the effectiveness of HML during incremental learning. As reported in the lower part of Table V, adding HML improves the mAP of previously known classes from 91.5% to 91.6%, indicating that the model preserves the knowledge acquired in the preceding stage. More notably, the mAP of the currently introduced classes increases from 76.9% to 91.2%, corresponding to an improvement of 14.3 percentage points. Consequently, the overall mAP increases from 88.5% to 91.5%. These results suggest that HML substantially improves the learning of newly introduced classes without degrading the performance of previously known classes, thereby achieving a better balance between stability and plasticity during incremental learning. Visualization and Analysis. To further analyze the effect of HML on representation learning, we visualize the proposal embeddings learned with and without HML using t-SNE. As shown in Fig. 6, without HML, embeddings from different categories exhibit substantial overlap and relatively dispersed intra-class distributions. In contrast, incorporating HML produces more compact intra-class clusters and clearer separation between different classes. This visualization demonstrates that HML improves the discriminability of proposal representations, supporting its role in reducing interference between previously learned and novel classes during incremental learning. VI Conclusion This work investigated hyperbolic geometry for OWOD in remote sensing imagery. We proposed HyRS-OWOD, which employs DOL to disentangle class-specific information from objectness. To discriminate unknown objects from known classes, we presented the HUL strategy based on the hyperbolic radius of proposal embeddings. To mitigate the forgetting effect in incremental learning, we introduced HML, which encourages the model to learn discriminative and stable embeddings by improving intra-class compactness and inter-class separability. Extensive experiments on multiple remote sensing benchmarks demonstrated that HyRS-OWOD improves unknown-object recall, reduces known–unknown confusion, and effectively retains previously acquired knowledge during incremental learning. These results validate the potential of hyperbolic representation learning for open-world object detection in remote sensing imagery. References [1] K. Li, G. Wan, G. Cheng, L. Meng, and J. Han (2020) Object detection in optical remote sensing images: a survey and a new benchmark. ISPRS journal of photogrammetry and remote sensing 159, p. 296–307. Cited by: §I. [2] X. Zhang, T. Zhang, G. Wang, P. Zhu, X. Tang, X. Jia, and L. Jiao (2023) Remote sensing object detection meets deep learning: a metareview of challenges and advances. IEEE Geoscience and Remote Sensing Magazine 11 (4), p. 8–44. Cited by: §I, §I-A. [3] K. Joseph, S. Khan, F. S. Khan, and V. N. Balasubramanian (2021) Towards open world object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 5830–5840. Cited by: §I, §I-B, §IV-B, §V-A4, §V-B, §V-D, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV. [4] Y. Li, Y. Wang, W. Wang, D. Lin, B. Li, and K. Yap (2024) Open world object detection: a survey. IEEE Transactions on Circuits and Systems for Video Technology 35 (2), p. 988–1008. Cited by: §I. [5] C. Lang, G. Cheng, J. Wu, Z. Li, X. Xie, J. Li, and J. Han (2024) Toward open-world remote sensing imagery interpretation: past, present, and future. IEEE Geoscience and Remote Sensing Magazine 13 (4), p. 8–44. Cited by: §I, §I-A. [6] L. Fang, Z. Yang, T. Ma, J. Yue, W. Xie, P. Ghamisi, and J. Li (2024) Open-world recognition in remote sensing: concepts, challenges, and opportunities. IEEE Geoscience and Remote Sensing Magazine 12 (2), p. 8–31. Cited by: §I. [7] Z. Tan, Z. Jiang, Z. Yuan, and H. Zhang (2024) OPODet: toward open world potential oriented object detection in remote sensing images. IEEE Transactions on Geoscience and Remote Sensing 62 (), p. 1–13. Cited by: §I, §I-B. [8] M. Hu, W. Yin, W. Diao, X. Gao, and X. Sun (2026) Unveiling the unknown: a sam guided open world object detection method for remote sensing. IEEE Transactions on Geoscience and Remote Sensing 64 (), p. 1–15. External Links: Document Cited by: §I, §I-B, §IV-B, §V-A4, §V-D, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I. [9] A. Gepperth and B. Hammer (2016) Incremental learning algorithms and applications. In European symposium on artificial neural networks (ESANN), Cited by: §I. [10] V. Khrulkov, L. Mirvakhabova, E. Ustinova, I. Oseledets, and V. Lempitsky (2020) Hyperbolic image embeddings. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 6417–6427. External Links: Document Cited by: §I, §I, §I-E, §I-F, §I-F, §IV-D. [11] J. Hong, P. Fang, W. Li, J. Han, L. Petersson, and M. Harandi (2023) Curved geometric networks for visual anomaly recognition. IEEE transactions on neural networks and learning systems 35 (12), p. 17921–17934. Cited by: §I, §I, §I-C. [12] Z. Sun, J. Li, and Y. Mu (2024) Exploring orthogonality in open world object detection. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 17302–17312. External Links: Document Cited by: §I, §I-B, §IV-B, §IV-C, §V-A4, §V-B. [13] Y. Wang, Z. Yue, X. Hua, and H. Zhang (2023) Random boxes are open-world object detectors. In Proceedings of the IEEE/CVF international conference on computer vision, p. 6233–6243. Cited by: §I, §I-B, §IV-B, §V-A4, §V-C. [14] M. A. Pourhoseingholi, A. R. Baghestani, and M. Vahedi (2012) How to control confounding effects by statistical analysis. Gastroenterology and Hepatology From Bed to Bench 5, p. 79 – 83. Cited by: §I. [15] B. Chen, W. Peng, X. Cao, and J. Röning (2023) Hyperbolic uncertainty aware semantic segmentation. IEEE Transactions on Intelligent Transportation Systems 25 (2), p. 1275–1290. Cited by: §I, §I-E, §IV-D. [16] L. Vinh Tran, Y. Tay, S. Zhang, G. Cong, and X. li (2020) HyperML: a boosting metric learning approach in hyperbolic space for recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining, p. 609–617. Cited by: §I. [17] Y. Cui, Z. Yu, W. Peng, Q. Tian, and L. Liu (2023) Rethinking few-shot class-incremental learning with open-set hypothesis in hyperbolic geometry. IEEE Transactions on Multimedia 26, p. 5897–5910. Cited by: §I, §I-C. [18] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi (2016) You only look once: unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 779–788. Cited by: §I-A. [19] J. Redmon and A. Farhadi (2017) YOLO9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 7263–7271. Cited by: §I-A. [20] J. Redmon and A. Farhadi (2018) YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767. Cited by: §I-A. [21] A. Bochkovskiy, C. Wang, and H. M. Liao (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934. Cited by: §I-A. [22] R. Girshick, J. Donahue, T. Darrell, and J. Malik (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 580–587. Cited by: §I-A. [23] S. Ren, K. He, R. Girshick, and J. Sun (2015) Faster r-cnn: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28. Cited by: §I-A. [24] T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie (2017) Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 2117–2125. Cited by: §I-A. [25] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko (2020) End-to-end object detection with transformers. In European conference on computer vision, p. 213–229. Cited by: §I-A. [26] X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai (2020) Deformable detr: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159. Cited by: §I-A. [27] W. Hua and Q. Chen (2025) A survey of small object detection based on deep learning in aerial images. Artificial Intelligence Review 58 (6), p. 162. Cited by: §I-A. [28] L. Wen, Y. Cheng, Y. Fang, and X. Li (2023) A comprehensive survey of oriented object detection in remote sensing images. Expert Systems with Applications 224, p. 119960. Cited by: §I-A. [29] J. Pan, Y. Liu, Y. Fu, M. Ma, J. Li, D. P. Paudel, L. Van Gool, and X. Huang (2025) Locate anything on earth: advancing open-vocabulary object detection for remote sensing community. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 6281–6289. Cited by: §I-A. [30] A. Gupta, S. Narayan, K. J. Joseph, S. Khan, F. S. Khan, and M. Shah (2022) OW-detr: open-world detection transformer. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 9225–9234. External Links: Document Cited by: §I-B, §V-D, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV. [31] S. Ma, Y. Wang, Y. Wei, J. Fan, T. H. Li, H. Liu, and F. Lv (2023) CAT: localization and identification cascade detection transformer for open-world object detection. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 19681–19690. Cited by: §I-B. [32] O. Zohar, K. Wang, and S. Yeung (2023) PROB: probabilistic objectness for open world object detection. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 11444–11453. Cited by: §I-B, §V-A4, §V-B, §V-D, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV. [33] O. Zohar, A. Lozano, S. Goel, S. Yeung, and K. Wang (2023) Open world object detection in the era of foundation models. ArXiv abs/2312.05745. Cited by: §I-B. [34] Y. He, W. Chen, S. Wang, T. Liu, and M. Wang (2025) Recalling unknowns without losing precision: an effective solution to large model-guided open world object detection. IEEE Transactions on Image Processing 34 (), p. 729–742. External Links: Document Cited by: §I-B. [35] X. Xi, Y. Huang, Z. Zhong, and R. Luo (2024) UMB: understanding model behavior for open-world object detection. In Advances in Neural Information Processing Systems, Vol. 37, p. 74233–74261. External Links: Document Cited by: §I-B. [36] X. Xi, X. Fu, W. Wang, and R. Luo (2025) OW-VAP: visual attribute parsing for open world object detection. In Forty-second International Conference on Machine Learning, Cited by: §I-B. [37] Z. Li, Z. Xiang, J. West, and K. Khoshelham (2026) From open vocabulary to open world: teaching vision language models to detect novel objects. External Links: 2411.18207 Cited by: §I-B. [38] N. Saini, A. Dubey, D. Das, and C. Chattopadhyay (2025) Advancing open-set object detection in remote sensing using multimodal large language model. In Proceedings of the Winter Conference on Applications of Computer Vision, p. 451–458. Cited by: §I-B. [39] P. Mettes, M. Ghadimi Atigh, M. Keller-Ressel, J. Gu, and S. Yeung (2024) Hyperbolic deep learning in computer vision: a survey. International Journal of Computer Vision 132 (9), p. 3484–3508. External Links: Document, ISSN 1573-1405 Cited by: §I-C. [40] W. Peng, T. Varanka, A. Mostafa, H. Shi, and G. Zhao (2022) Hyperbolic deep neural networks: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (12), p. 10023–10044. External Links: Document Cited by: §I-C. [41] O. Ganea, G. Bécigneul, and T. Hofmann (2018) Hyperbolic neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, Red Hook, NY, USA, p. 5350–5360. Cited by: §I-C. [42] Y. Wang, H. Wang, W. Lu, and Y. Yan (2023) HyGGE: hyperbolic graph attention network for reasoning over knowledge graphs. Information Sciences 630, p. 190–205. External Links: ISSN 0020-0255, Document Cited by: §I-C. [43] I. Chami, Z. Ying, C. Ré, and J. Leskovec (2019) Hyperbolic graph convolutional neural networks. In Advances in Neural Information Processing Systems, Vol. 32, p. . Cited by: §I-C. [44] Z. Gao, C. Xu, F. Li, Y. Jia, M. Harandi, and Y. Wu (2023) Exploring data geometry for continual learning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 24325–24334. External Links: ISSN , Document Cited by: §I-C. [45] B. Zhang, H. Jiang, S. Feng, X. Li, Y. Ye, and R. Ye (2022) Hyperbolic knowledge transfer with class hierarchy for few-shot learning. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22, p. 3723–3729. Note: Main Track External Links: Document Cited by: §I-C. [46] Y. Guo, X. Wang, Y. Chen, and S. X. Yu (2022) Clipped hyperbolic classifiers are super-hyperbolic classifiers. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 1–10. External Links: ISSN , Document Cited by: §I-C. [47] B. Chen, W. Peng, X. Cao, and J. Röning (2024) Hyperbolic uncertainty aware semantic segmentation. IEEE Transactions on Intelligent Transportation Systems 25 (2), p. 1275–1290. External Links: Document Cited by: §I-C. [48] M. G. Atigh, J. Schoep, E. Acar, N. Van Noord, and P. Mettes (2022) Hyperbolic image segmentation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 4443–4452. External Links: Document Cited by: §I-C. [49] C. Lang, A. Braun, L. Schillingmann, and A. Valada (2022) On hyperbolic embeddings in object detection. In Pattern Recognition, Cham, p. 462–476. External Links: ISBN 978-3-031-16788-1 Cited by: §I-C. [50] F. Kong, Y. Chen, J. Cai, and D. Modolo (2024) Hyperbolic learning with synthetic captions for open-world detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 16762–16771. Cited by: §I-C. [51] T. Doan, X. Li, S. Behpour, W. He, L. Gou, and L. Ren (2024) Hyp-ow: exploiting hierarchical structure learning with hyperbolic distance enhances open world object detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 1555–1563. Cited by: §I-C. [52] M. Hamzaoui, L. Chapel, M. Pham, and S. Lefèvre (2024) Hyperbolic prototypical network for few shot remote sensing scene classification. Pattern Recognition Letters 177, p. 151–156. External Links: ISSN 0167-8655, Document Cited by: §I-D. [53] X. Li, F. Xu, F. Liu, R. Xia, Y. Tong, L. Li, Z. Xu, and X. Lyu (2022) Hybridizing euclidean and hyperbolic similarities for attentively refining representations in semantic segmentation of remote sensing images. IEEE Geoscience and Remote Sensing Letters 19 (), p. 1–5. External Links: Document Cited by: §I-D. [54] W. Cui, X. Xu, J. Chen, Z. Feng, H. Zhao, Y. Hao, Y. Tian, J. Wang, and C. Xia (2024) Representation of multirelations of geographic scenes based on hyperbolic space. IEEE Transactions on Geoscience and Remote Sensing 62 (), p. 1–13. External Links: Document Cited by: §I-D. [55] Q. Yang, S. Zhang, J. Li, Y. Sun, Q. Han, and Y. Sun (2024) Hyperboloid-embedded siamese network for change detection in remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 17 (), p. 9240–9252. External Links: Document Cited by: §I-D. [56] P. Sun, R. Zhang, Y. Jiang, T. Kong, C. Xu, W. Zhan, M. Tomizuka, L. Li, Z. Yuan, C. Wang, et al. (2021) Sparse r-cnn: end-to-end object detection with learnable proposals. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 14454–14463. Cited by: §IV-B. [57] S. Chen, P. Sun, Y. Song, and P. Luo (2023) DiffusionDet: diffusion model for object detection. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 19773–19786. External Links: Document Cited by: §IV-B. [58] T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár (2017) Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, p. 2980–2988. Cited by: §IV-F. [59] R. Girshick (2015) Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, p. 1440–1448. Cited by: §IV-F. [60] A. R. Dhamija, M. Günther, J. Ventura, and T. E. Boult (2020) The overlooked elephant of object detection: open set. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Vol. , p. 1010–1019. External Links: Document Cited by: §V-B. [61] D. Miller, L. Nicholson, F. Dayoub, and N. Sünderhauf (2018) Dropout sampling for robust object detection in open-set conditions. In 2018 IEEE International Conference on Robotics and Automation (ICRA), Vol. , p. 3243–3249. Cited by: §V-B. [62] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 770–778. External Links: Document Cited by: §V-C. [63] T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie (2017) Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 2117–2125. Cited by: §V-C. [64] X. Xi, Y. Huang, J. Lin, and R. Luo (2024) KTCN: enhancing open-world object detection with knowledge transfer and class-awareness neutralization. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI ’24. External Links: ISBN 978-1-956792-04-1, Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV. [65] Y. He, W. Chen, S. Wang, T. Liu, and M. Wang (2025) Recalling unknowns without losing precision: an effective solution to large model-guided open world object detection. IEEE Transactions on Image Processing 34 (), p. 729–742. External Links: Document Cited by: TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV. Wuzhou Li received the B.Sc., M.Sc., and Ph.D. degrees from Wuhan University, Wuhan, China, in 2014, 2020, and 2024, respectively. She is currently a Lecturer with the School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, China. Her research interests include computer vision and deep learning, especially on object detection in remote sensing. Jiawei Zhou received the B.Sc., M.Sc., and Ph.D. degrees from Wuhan University, Wuhan, China, in 2017, 2021, and 2025, respectively. His research interests include deep learning, computer vision, and remote sensing image object detection. Shenghang Wang received the B.S. degree in Electrical Engineering Technology from Northern Kentucky University, Highland Heights, KY, USA, in 2025. He is currently pursuing the M.S. degree with the Electrical and Computer Engineering, Ohio State University, Columbus, OH, USA. His research interests include computer vision and point cloud processing. Xiang Li received the B.Sc. degree from Wuhan University, Wuhan, China, in 2014, and the Ph.D. degree from the Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences, Beijing, China, in 2019. He is currently a Professor with with the School of Artificial Intelligence, Wuhan University, Wuhan, China. His research interests include deep learning, computer vision, and remote sensing image interpretation.