Paper deep dive
BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations
Taqi Hamoda, Hayat Rajani, Nuno Gracias
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Automated perception in side-scan sonar (SSS) imagery is severely hindered by physical acoustic artifacts, resulting in representations that inextricably mix intrinsic seabed reflectivity with transient viewing geometries. Existing self-supervised learning (SSL) frameworks rely on augmentations designed for natural images, failing to account for acoustic degradation and explicitly enforce view-invariance. To address this gap, we introduce a physics-informed self-distillation framework built upon the DINOv3 architecture utilizing a ConvNeXt-v2-Tiny backbone to maximize data efficiency. The proposed methodology enforces view-invariance through two primary mechanisms: physically motivated augmentations that simulate speckle noise, range-dependent attenuation, and radiometric miscalibration; and a Hilbert-Schmidt Independence Criterion (HSIC) penalty that explicitly decouples learned dense patch features from physical viewing parameters. Furthermore, we propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions. Extensive evaluation demonstrates that the framework natively groups complex benthic topographies into stable, noise-free semantic clusters without relying on manual annotations. During supervised downstream tasks on the S3Seg dataset, the fused representations exhibited exceptional data efficiency, achieving 96% of its absolute peak performance using only 10% of the available annotated data, ultimately reaching a mean Intersection over Union (mIoU) of 71.4% and an overall accuracy of 86.5%.
Tags
Links
- Source: https://arxiv.org/abs/2608.23215v1
- Canonical: https://arxiv.org/abs/2608.23215v1
Trouble viewing inline? Open PDF directly →
Full Text
55,715 characters extracted from source content.
Expand or collapse full text
BenthicDINO: Physics-Informed Self-Distillation for View-Invariant Side-Scan Sonar Representations Taqi Hamoda Hayat Rajani Nuno Gracias Thanks: All authors are with the Computer Vision and Robotics Research Institute (ViCOROB) of the University of Girona, Spain. Thanks: Corresponding author: Hayat Rajani (hayat.rajani@udg.edu). Abstract Automated perception in side-scan sonar (S) imagery is severely hindered by physical acoustic artifacts, resulting in representations that inextricably mix intrinsic seabed reflectivity with transient viewing geometries. Existing self-supervised learning (SSL) frameworks rely on augmentations designed for natural images, failing to account for acoustic degradation and explicitly enforce view-invariance. To address this gap, we introduce a physics-informed self-distillation framework built upon the DINOv3 framework with a ConvNeXt-v2-Tiny backbone to maximize data efficiency. The proposed methodology enforces view-invariance through two primary mechanisms: physically motivated augmentations that simulate speckle noise, range-dependent attenuation, and radiometric miscalibration; and a Hilbert-Schmidt Independence Criterion (HSIC) penalty that explicitly decouples learned dense patch features from physical viewing parameters. Furthermore, we propose a dense, hierarchical feature fusion strategy across all four network stages to preserve fine-grained sediment details alongside deep semantic abstractions. Extensive evaluation demonstrates that the framework natively groups complex benthic topographies into stable, noise-free semantic clusters without relying on manual annotations. During supervised downstream tasks on the S3Seg dataset, the fused representations exhibited exceptional data efficiency, achieving 96% of its absolute peak performance using only 10% of the available annotated data, ultimately reaching a mean Intersection over Union (mIoU) of 71.4% and an overall accuracy of 86.5%. Index Terms: side-scan sonar, self-supervised learning, view-invariant representation learning, physics-guided augmentation, Hilbert-Schmidt independence criterion, benthic habitat mapping, DINOv3, ConvNeXt. I Introduction Underwater robotic perception is severely constrained by the marine environment. Due to absorption and scattering, electromagnetic radiation attenuates rapidly in water, limiting high-resolution optical sensors to ranges under 10 m [15, 12]. Conversely, acoustic energy propagates with low attenuation, enabling sound waves to travel hundreds of metres even in turbid or low-light conditions. As such, side-scan sonar (S) has become the primary modality for large-scale seafloor mapping, long-range target detection, and autonomous navigation [3, 29]. Unlike optical cameras, S image formation is governed by acoustic acquisition dynamics [3, 16]. Laterally mounted transducers on a moving platform emit fan-shaped pulses perpendicular to the trajectory. The returning echoes are recorded over time and converted to range assuming a constant sound speed of 1500 m s−11500\,m\,s^-1, yielding a continuous 2D backscatter intensity map [3, 29]. Traditionally, this process is approximated by a Lambertian model: I(x,y)=ρ(x,y)⋅cos(θ(x,y))⋅L(x,y)I(x,y)=ρ(x,y)· (θ(x,y))· L(x,y) (1) where the recorded intensity I(x,y)I(x,y) is a function of the intrinsic seabed reflectivity ρ(x,y)ρ(x,y), the local incidence angle θ(x,y)θ(x,y), and the acoustic propagation loss L(x,y)L(x,y) [16, 3]. However, this simplified model fails to capture true acoustic complexity. Real S transducers emit non-uniform power profiles that diminish at the beam edges and vary across side lobes [6]. Furthermore, topographic occlusions create acoustic shadows, and multiplicative speckle noise inherently corrupts the signal [12, 27]. The resulting imagery is therefore highly viewpoint-dependent: the same seafloor structure produces drastically different intensity patterns and shadow geometries when surveyed from a different direction or range. Because a large portion of the recorded intensity encodes the acquisition geometry rather than the seabed itself, features extracted from raw imagery inextricably mix intrinsic reflectivity (ρ) with transient viewing conditions. These physical artefacts severely bottleneck automated perception. Viewpoint dependence and non-uniform intensity result in poor repeatability for traditional handcrafted keypoint extractors (e.g., SIFT [21], SURF [1]), causing high matching errors and rendering real-time analytical interpretation unreliable [15, 12, 8, 29]. The most direct consequence is unreliable cross-view correspondence: when the same location is imaged from two different headings, the views are difficult to associate, weakening loop closure, mosaicking, and change detection over repeated surveys. A view-invariant representation—one that responds to the structural seabed and not the sensor geometry—is therefore a prerequisite for robust large-scale S perception. To overcome these limitations, recent research has turned toward deep self-supervised learning (SSL). By leveraging large-scale unlabelled sonar data, SSL frameworks extract invariant features without relying on scarce manual annotations [36, 27]. Modern self-distillation methods like DINOv3 [36] promote generic invariance through multi-crop augmentation. However, these standard augmentations are designed for natural scenes and fail to model the physical causes of S view dependence. Consequently, the learned features still absorb geometric information. No existing SSL approach for S explicitly forces the learned representations to be statistically independent of the acquisition parameters. We address this gap with a physics-guided self-supervised framework built upon the DINOv3 framework. We replace the default Vision Transformer (ViT) with a ConvNeXt-v2-Tiny [40] backbone, leveraging its convolutional inductive biases and global response normalization to stabilize masked latent learning on small S datasets. The framework operates on two complementary fronts. First, physics-guided augmentations reproduce dominant S nuisance factors during training: bounded additive Gaussian perturbations act as a denoising regularizer against speckle, linear synthetic Time-Varying Gain (TVG) profiles enforce invariance to uncompensated attenuation, and radiometric jitter simulates gain miscalibration. Second, a Hilbert-Schmidt Independence Criterion (HSIC) [9] penalty explicitly minimizes the statistical dependence between the learned dense patch features and the measured viewing parameters, effectively scrubbing residual geometry information that augmentations alone cannot suppress. Finally, a novel hierarchical MLP fusion strategy aggregates features across all four network stages, preserving high-frequency sediment details alongside deep semantic abstractions. The main contributions of this work are as follows: • We adapt the DINOv3 framework to acoustic data by integrating a ConvNeXt-v2-Tiny backbone and proposing a dense, multi-scale hierarchical feature fusion that retains fine-grained seabed textures. • We formulate view-invariance in S as a statistical independence problem, introducing an HSIC-based objective—efficiently estimated via random Fourier features—that explicitly decouples learned dense representations from acquisition geometries (slant range and incidence angle). • We design a set of physics-guided augmentations derived from the S acquisition model, safely injecting speckle, linear range-dependent attenuation, and radiometric variation to enforce physical invariances without manual annotation. • We demonstrate through extensive representation-quality analyses that the resulting embeddings are view-invariant, significantly improving cross-view correspondence reliability compared to existing self-supervised baselines, while transferring effectively to downstream perception tasks. I Related Work I-A Representation Learning for Sonar Self-supervised learning (SSL) has seen increasing adoption for sonar perception, driven by the high cost of expert annotations and the poor transferability of models pre-trained on natural optical images [38]. Early investigations explored fundamental pretext tasks—such as rotation prediction, denoising autoencoders, and jigsaw solving—demonstrating that SSL significantly narrows the performance gap with supervised methods in low-label regimes for forward-looking sonar (FLS) [27]. For synthetic-aperture sonar (SAS), subsequent works adapted momentum-contrast pre-training [35] and surveyed both contrastive and generative SSL paradigms for processing and recognition [33]. Within the S domain, recent research has evaluated modern architectures, comparing ViTs against convolutional networks (including ConvNeXt variants) for classification, highlighting SSL as a critical next step for acoustic data [34]. Concurrently, joint-embedding predictive architectures have been applied for in-domain mine-like object classification [14]. More advanced self-distillation frameworks, such as DINOv3, have proven highly effective at generating robust dense features [36], inspiring related FLS applications that leverage self-supervised feature-space transformations to suppress speckle and enhance target regions [41]. Despite this rapid progress, existing sonar SSL pipelines inherit augmentations and pretext tasks designed for natural imagery. None explicitly enforce statistical invariance to the physical acquisition geometry or acoustic degradation models. This fundamental gap motivates our approach, which uniquely embeds an explicit independence constraint and physics-guided augmentations directly within a dense self-distillation framework. I-B Physics-Informed Learning for Sonar The scarcity of paired, high-quality real sonar data has driven significant interest in physics-informed learning, largely focused on realistic image synthesis and closing the domain gap between simulated and real imagery. Explicit acoustic simulators, such as S3Simulator [11] and ACOUSIM [24], model the acquisition process directly to generate benchmark datasets or measure statistical alignment without generative models. Another major line of work combines learned generators with physical priors to augment small datasets: Li et al. explored zero-shot and few-shot S synthesis for target detection [17], Peng et al. trained multi-view generative adversarial networks [26], Koo et al. utilized CycleGANs for synthetic generation [13], and Ma et al. embedded random-fusion strategies within diffusion frameworks for segmentation [22]. A third group of methods grounds the physics directly in the acquisition geometry to normalize or transform backscatter intensity. For instance, Liu and Ye proposed a gray-scale correction method accounting for rugged seafloors [19], Stewart et al. mapped SAS intensity to seabed elevation via image-to-height translation [37], and recent works have employed physics-based backscatter correction to reduce view-dependent variations prior to feature extraction [16]. While these approaches demonstrate that encoding the sensor model improves data realism and mitigates intensity inconsistencies, they universally apply physics during pre-processing or image synthesis. They do not embed physical constraints directly into the representation learning objective. Consequently, the resulting deep features are not mathematically guaranteed to be invariant to the acquisition geometry. Our work departs from this paradigm by enforcing geometric view-invariance directly in the latent space during pre-training, ensuring the network natively decouples intrinsic seabed structure from transient viewing conditions. I Methodology We build on the DINOv3 self-distillation framework and adapt it to the acoustic characteristics of S. The framework has three parts: a base self-distillation model with a ConvNeXt-v2-Tiny encoder (Sections I-A and I-B); a set of physics-guided augmentations that reproduce the dominant S nuisance factors during training (Section I-C); and an explicit statistical-independence penalty that decouples the learned dense features from the acquisition geometry (Section I-D). Furthermore, we introduce a set of training adaptations that make self-distillation stable under small-batch distributed training on S data that are described later with the experimental settings (Section IV-B). I-A DINOv3 Self-Distillation Framework DINOv3 employs a self-distillation paradigm where a student network learns to match the output distributions of a teacher network. The teacher’s weights are updated using an exponential moving average (EMA) of the student’s weights, ensuring a stable target representation [36, 25]. The overall pre-training objective is a composite loss function designed to capture both global semantic understanding and fine-grained local structures: ℒPre=ℒDINO+ℒiBOT+0.5⋅ℒGram+0.1⋅ℒKoLeoL_Pre=L_DINO+L_iBOT+0.5·L_Gram+0.1·L_KoLeo I-A1 Global Discriminative Loss The DINO objective enforces view-invariance at the global image level by aligning representations across different augmented views of the S input. Specifically, the student network processes a comprehensive set of global and local crops, denoted as V, while the teacher network is restricted to processing only global crops VglobalV_global [36]. The outputs from both networks are transformed into probability distributions over K dimensions via a projection head followed by a softmax activation. To effectively prevent representational collapse, Sinkhorn-Knopp centering is used for the teacher’s distributions, replacing the original centering and sharpening mechanism used in DINOv1 [5, 25]. The resulting loss minimizes the cross-entropy between the student distribution Ps(xv)P_s(x_v) and the centered teacher distribution Pt(xv′)P_t(x_v ). ℒDINO=−∑v∈V∑v′∈VglobalPt(xv′)logPs(xv)L_DINO=- _v∈ V _v ∈ V_globalP_t(x_v ) P_s(x_v) Minimizing this objective promotes the learning of global features that are robustly invariant to geometric distortions and sensor noise. I-A2 Masked Latent Reconstruction To instill dense prediction capabilities and enhance the model’s robustness to acoustic shadows, an iBOT masked image modeling loss is used to complement the global objective [25]. This patch-level objective forces the student network to reconstruct the features of masked tokens using the teacher’s representations of the corresponding unmasked patches as learning targets. Consistent with the global DINO objective, Sinkhorn-Knopp centering is also applied to these patch-level projections. For a given set of masked patch indices ℳM, the reconstruction loss is formulated as the cross-entropy between the teacher’s patch distribution PtipatchP_t_i^patch and the student’s prediction PsipatchP_s_i^patch. ℒiBOT=−∑i∈ℳPtipatchlogPsipatchL_iBOT=- _i P_t_i^patch P_s_i^patch I-A3 Feature Regularisation To prevent feature collapse and encourage a uniform distribution across the hypersphere, KoLeo regularization is applied on the dense embeddings of the student patches [25, 36]. ℒKoLeo=−1n∑i=1nln(minj≠i∥zi−zj∥)L_KoLeo=- 1n _i=1^n ( _j≠ i\|z_i-z_j\| ) (2) This term maximises the minimum distance between samples, which improves class separability in downstream benthic tasks. In ℒPreL_Pre we use its distributed variant, which computes the same quantity across the distributed batch. I-A4 Gram Anchoring Long training schedules and high resolutions often degrade dense feature maps, causing patch-level inconsistency and noisy representations that harm S matching and registration [36, 12]. DINOv3 introduces Gram anchoring to preserve local structure [36]. The Gram matrix captures the pairwise correlations of the L2-normalised patch features. The student is regularised by matching its Gram matrix to that of a stable “Gram teacher”, a checkpoint from earlier training. ℒGram=|XSXS⊤−XGXG⊤|F2L_Gram=|X_SX_S -X_GX_G |_F^2 (3) Gram anchoring stabilises dense features, accelerates iBOT convergence, and improves performance on dense S tasks such as feature matching and segmentation, while preserving fine sediment textures [36, 29]. (a) (b) Fig. 1: ConvNeXt-v2 adapted for DINOv3 (Left). Outputs from stages 1–4 are upsampled to match stage 1, concatenated, and compressed via a fusion MLP into a 768-D embedding. The ConvNeXt-v2 block structure (Right) utilizes global response normalization to stabilize masked learning. I-B ConvNeXt-v2 Backbone Standard implementations of the DINO framework typically rely heavily on ViTs as the backbone. However, ViTs lack the strong inductive biases inherent to CNNs, such as translation equivariance and local connectivity, and therefore demand massive amounts of training data to generalize effectively. Given that high-quality, diverse S data is notoriously scarce and our dataset is relatively small, we opted for a convolutional architecture to maximize data efficiency. Furthermore, selecting the tiny variant was a deliberate regularization tactic; its constrained parameter space prevents the model from severely overfitting to the limited acoustic dataset while remaining computationally viable for AUV deployment. The original ConvNeXt [20] modernized the standard ResNet block by incorporating ViT-inspired design choices, including large 7×77× 7 depthwise convolutions, inverted bottlenecks, and replacing Batch Normalization with Layer Normalization. While highly effective for supervised learning, applying mask-based self-supervised learning, such as the iBOT objective, directly to ConvNeXt induces severe representation collapse [40]. When trained on masked inputs, the dimension-expansion MLP layers produce dead or saturated feature maps, leading to redundant channel activations and degraded feature diversity [40]. To mitigate this, our framework utilizes ConvNeXt-v2 [40], which introduces Global Response Normalization (GRN). Inserted directly after the MLP expansion layer, GRN enhances inter-channel competition by aggregating spatial features into a channel-wise L2L_2-norm vector and applying divisive normalization [40]. This calibration successfully maintains feature diversity and prevents channel saturation during the masked latent reconstruction of sonar patches. Furthermore, moving beyond previous multi-scale approaches that discarded early features, our framework extracts and aggregates hierarchical representations from all four stages of the network. This is as depicted in Fig. 1. Let F(s)F^(s) denote the feature map from stage s∈1,2,3,4s∈\1,2,3,4\. Feature maps from stages 2, 3, and 4 are upsampled via bilinear interpolation to match the spatial resolution of stage 1. The upsampled feature maps are then concatenated along the channel dimension. To effectively fuse these multi-scale representations without losing spatial context, we introduce a 3-layer Multi-Layer Perceptron (implemented via 1×11× 1 convolutions) that compresses the concatenated features into a unified 768-dimensional descriptor: ℋi,j=MLP1×1(⨁s=14Fi,j(s))∈ℝ768H_i,j=MLP_1× 1 ( _s=1^4F^(s)_i,j ) ^768 This dense feature fusion effectively preserves the fine-grained, high-frequency sediment details captured in the initial stage, while seamlessly integrating the deep, robust semantic abstraction generated in the deeper stages. Fig. 2: Effects of TVG Attenuation on five sample S patches, progressing from the original patches (top row) to moderate (middle row) and extreme (bottom row) attenuation levels. I-C Physics-Guided Augmentations DINOv3 induces its target invariances through the augmentations applied to each crop. The default photometric augmentations are designed for natural photographs and do not accurately represent the physical factors that affect S imagery. We therefore incorporate three physically motivated augmentations, each targeting a dominant S nuisance factor: speckle noise, range-dependent attenuation, and radiometric miscalibration. Each operation is applied independently to every crop. Consequently, the student and teacher process varying acoustic conditions for the same seabed patch, compelling the network to learn features invariant to these physical artifacts. Let I∈ℝH×WI ^H× W denote an input tile, with x representing the along-track axis and r representing the across-track (range) axis. I-C1 Speckle Perturbation Coherent acoustic imaging produces speckle, a grainy multiplicative fluctuation that corrupts the backscatter signal [27]. To improve robustness to this degradation, we perturb the tile with additive Gaussian noise: I~(x,r)=I(x,r)+η(x,r),η(x,r)∼(0,σ2) I(x,r)=I(x,r)+η(x,r), η(x,r) (0,σ^2) (4) While a multiplicative variant is closer to the true physical speckle model, the additive formulation acts as a highly effective denoising regularizer that forces the network to learn robust structural representations rather than over-indexing on raw pixel intensities [27]. Unlike previous assumptions of a zero-based uniform distribution, our implementation explicitly bounds the standard deviation to prevent catastrophic signal destruction. Specifically, σ is sampled per tile from a uniform distribution σ∼(0.01,0.05)σ (0.01,0.05), and the augmentation is applied with a probability of p=0.5p=0.5. I-C2 Synthetic Time-Varying Gain (TVG) Recorded acoustic intensity decays with range due to geometric spreading and absorption. While hardware TVG is normally applied to compensate for this loss, the compensation is often imperfect, leaving residual range-dependent trends that vary drastically between survey passes [2, 3]. To enforce invariance to uncompensated transmission loss, we multiply the tile by a synthetic residual gain curve [3]. Diverging from exponential decibel-based models, our augmentation simulates this artifact by applying a linear decay gradient across the range of the tile: I~(x,r)=g(r)⋅I(x,r),g(r)=1.0−(1.0−β)rrmax I(x,r)=g(r)· I(x,r), g(r)=1.0-(1.0-β) rr_ (5) where rmaxr_ is the maximum range of the tile, and β∼(0.3,0.7)β (0.3,0.7) represents the retention factor at the farthest edge (i.e., the far edge retains between 30%30\% and 70%70\% of its original intensity). Applied with a probability of p=0.5p=0.5, this linear decay simulates severe, uncompensated profile fading, driving the network to decouple structural features from spatial acoustic attenuation. This is as depicted in Fig. 2. I-C3 Radiometric Jitter Operator adjustments and automatic gain control (AGC) consistently change the overall brightness and contrast of a survey, altering the dynamic range independently of the underlying seabed structure [3]. To make the representations invariant to these radiometric calibration errors, we apply a global brightness and contrast perturbation using standard jitter implementations: I~(x,r)=κ(I(x,r)−I¯)+I¯+δ I(x,r)=κ (I(x,r)- I )+ I+δ (6) where I¯ I is the mean tile intensity, κ∼(1−c,1+c)κ (1-c,1+c) is a contrast scaling factor with c=0.2c=0.2, and δ∼(−b,b)δ (-b,b) is a brightness offset with b=0.1b=0.1. By artificially perturbing the global intensity scale, the network is forced to isolate the intrinsic seabed reflectivity (ρ) rather than relying on transient intensity shifts [15]. The output is subsequently clipped to maintain valid, normalized intensity boundaries. I-D Enforcing View-Invariance While the self-distillation framework of DINOv3 inherently promotes a degree of invariance through multi-crop augmentations, explicitly decoupling the learned representations from the physical viewing geometry can further isolate the intrinsic seafloor properties. The physics-guided augmentations above do encourage invariance, but only implicitly. Residual geometric information can still leak into the features. To remove it, we enforce statistical independence between the learned representations and the viewing parameters using the Hilbert-Schmidt Independence Criterion (HSIC) [9]. Unlike mutual information, which lacks a notion of geometry in the feature space and can be difficult to estimate directly, HSIC incorporates geometry via kernel choice and can be directly estimated from mini-batches without restrictive data assumptions [18]. For two random variables X and Y, HSIC measures the squared Hilbert-Schmidt norm of the cross-covariance operator between their non-linear feature mappings in reproducing kernel Hilbert spaces (RKHS): HSIC(X,Y)=‖[ϕ(X)⊗ψ(Y)]−[ϕ(X)]⊗[ψ(Y)]‖HS2HSIC(X,Y)= \|E[φ(X) ψ(Y)]-E[φ(X)] [ψ(Y)] \|_HS^2 where ϕφ and ψ are the feature transformations induced by the chosen kernels for X and Y. Crucially, for a wide range of characteristic kernels, HSIC(X,Y)=0HSIC(X,Y)=0 if and only if X and Y are strictly independent. Let Z denote the dense patch features outputted by the DINOv3 backbone for the global views. For each corresponding patch, let V=(θ,rs)V=(θ,r_s) represent the physical viewing geometry, where θ is the incidence angle and rsr_s is the slant range; both are obtained from the recorded acquisition geometry and navigation data. To incentivize the model to output features that rely solely on the intrinsic reflectivity and structure of the seabed, we want to minimize the statistical dependence between the patch features Z and the viewing geometry V. We can achieve this by introducing an HSIC-based regularization loss during training: ℒview-inv=HSIC(Z,V)L_view-inv=HSIC(Z,V) Minimizing this term penalizes the network whenever its feature distribution covaries with the incidence angle or the slant range. The objective acts as an information bottleneck that filters out the geometric and attenuation artifacts specific to a given sonar pass. It integrates directly into the DINOv3 pre-training objective, yielding the overall loss: ℒTotal=ℒPre+γHSIC(Z,V)L_Total=L_Pre+ (Z,V) where ℒPreL_Pre represents the combination of the global discriminative loss (ℒDINOL_DINO), masked latent reconstruction (ℒiBOTL_iBOT), and other regularizers, while γ is a hyperparameter controlling the strength of the view-invariance constraint. However, a naive empirical estimation of HSIC requires computing exact kernel matrices over the mini-batch, resulting in a computational complexity of (B2)O(B^2), where B is the batch size [18]. Given that S self-distillation often operates under small-batch distributed training constraints, this quadratic complexity can become a bottleneck. To maintain training efficiency, we utilize Random Fourier Features (RFF) [28] to approximate a Gaussian Radial Basis Function (RBF) kernel. By projecting the data into a randomized low-dimensional Fourier space before computing the covariance, the computational complexity of the HSIC regularizer is reduced to (B)O(B), scaling linearly with the batch size [18]. Furthermore, to ensure stable gradients, the bandwidth of the Gaussian RBF kernel is dynamically calibrated during each forward pass using the median distance heuristic of the batch. This implementation ensures that enforcing strict statistical view-invariance adds minimal computational overhead to the DINOv3 training pipeline. IV Experimental Setup IV-A Dataset and Preprocessing We utilize the BenthiCat dataset, a large-scale opti-acoustic dataset for benthic classification and habitat mapping [29], to train and evaluate our framework. The dataset contains a subset comprising approximately one million S tiles for self-supervised representation learning, collected along the coast of Catalonia, Spain, which covers a wide variety of benthic habitats. The raw 12-bit S waterfall data underwent rigorous preprocessing to ensure stable feature learning. First, the data were logarithmically compressed and normalized to the range [0,1][0,1] to preserve low-intensity structural details: Inormalized′=ln(1+Iraw)max(ln(1+Iraw)).I _normalized= (1+I_raw) ( (1+I_raw)). (7) Subsequently, slant-range correction was applied under a flat-seafloor assumption to convert the slant range rsr_s to ground range rgr_g: rg=rs2−h2,r_g= r_s^2-h^2, (8) where h denotes the sensor altitude above the seabed. This crucial step removes nadir compression and the acoustic blind zone. Finally, images were extracted into overlapping 384×384384× 384 pixel patches with a 192192-pixel stride. During the self-supervised training pipeline, these patches are further processed by our multi-crop augmentation strategy, producing global crops of 224×224224× 224 pixels and local crops of 96×9696× 96 pixels. IV-B Training Details Direct application of DINO-style self-distillation to S imagery is inherently challenging due to acoustic-specific statistics (e.g., speckle noise and gain variations) and the small-batch distributed training constraints typical in this domain. . We introduce two adaptations that stabilise training under these conditions. First, because the original DINOv3 high-dimensional output (K=65,536K=65,536) produces near-zero gradients at smaller batch sizes, we drastically reduced the projection-head dimensionality to K=4,096K=4,096. Second, the standard Sinkhorn-Knopp algorithm frequently produces zero rows or columns under these batch constraints, leading to numerical instability. Therefore, for the global-to-local distillation loss, we reverted to the DINOv1 centering mechanism [5]: Ct←mCt−1+(1−m)1B∑i=1BOteacher(xi)C_t← m\,C_t-1+(1-m)\, 1B _i=1^BO_teacher(x_i) (9) where the teacher output center C is subtracted prior to the softmax activation. The iBOT masked-image-modeling head retains Sinkhorn-Knopp, as its dense patch tokens provide sufficient statistical density to prevent collapse. The network was trained across two NVIDIA Quadro RTX 6000 GPUs with an effective batch size of 8,192 (4,096 per GPU × 2). Optimization was performed using a cosine learning rate scheduler with linear warmup. IV-C Unsupervised Feature Space Evaluation To rigorously assess the quality and robustness of the learned representations without relying on downstream task biases, we conduct a comprehensive unsupervised analysis of the fused, dense feature space (ℝ768R^768) utilizing several geometric and topological metrics: IV-C1 Dimensional Collapse via SVD and Effective Rank To ensure the network utilizes its full capacity and avoids dimensional collapse, we compute the Singular Value Decomposition (SVD) on the covariance matrix of the patch embeddings. For a mean-centered feature matrix ∈ℝN×DX ^N× D, we decompose =TX=U ^T [10]. The effective rank is then measured using the Shannon entropy of the normalized singular values σi _i, quantifying the uniformity of variance distribution across the latent dimensions. IV-C2 Intrinsic Dimension We estimate the intrinsic dimension of the learned manifold using the Two-N algorithm [7]. This method relies on the ratio of distances to the first and second nearest neighbours, estimating the minimal number of parameters required to describe the local data distribution without assuming a global Euclidean structure. IV-C3 Clustering and Manifold Metrics To evaluate the separability and semantic grouping of the representations, we apply both K-Means and HDBSCAN clustering over the embeddings. The clustering quality is quantified using the Silhouette Coefficient [31], defined for a single sample as: s=b−amax(a,b)s= b-a (a,b) (10) where a is the mean intra-cluster distance and b is the mean nearest-cluster distance. Values approaching 11 indicate dense, well-separated spherical clusters in Euclidean space. For non-spherical density estimations via HDBSCAN (applied post-UMAP projection [23]), we track the Noise Ratio (percentage of unclustered diffuse samples) and Mean Confidence (probability of cluster membership) [4]. Finally, we measure Trustworthiness [39] to ensure that the local neighborhoods of the high-dimensional feature space are faithfully preserved when projected or clustered, penalizing the false preservation of distant points as nearest neighbors. V Results and Discussion V-A Unsupervised Feature Space Evaluation To understand the geometric and semantic properties of the learned representations prior to any downstream supervised fine-tuning, we evaluated the 768-dimensional fused, dense feature space utilizing the metrics outlined in our experimental setup. Fig. 3: The log of singular values against the singular value rank index of the fused representation. V-A1 Dimensionality and Feature Collapse A critical failure mode in self-supervised masked modeling is dimensional collapse, where the network projects inputs into a trivial low-dimensional subspace, limiting representation capacity. Our evaluation of the feature covariance matrix yielded an effective rank of 53.8153.81. While lower than the ambient dimension of 768768, the log-singular value spectrum (derived from the SVD) exhibits a smooth, continuous decay rather than a sharp cutoff [10]. This confirms that the ConvNeXt-v2 backbone successfully leverages a highly expressive subspace, avoiding representation collapse despite the acoustic homogeneity of S imagery. Furthermore, the Two-N algorithm estimated the intrinsic dimension of the data manifold to be 8.808.80. This indicates that while the features reside in a high-dimensional space, the local semantic neighborhoods can be defined by approximately 9 degrees of freedom. This strong compression suggests the model successfully filtered out high-variance, viewpoint-dependent acoustic noise (e.g., speckle and transient geometry) and isolated the underlying intrinsic benthic structures. V-A2 Manifold Geometry and Clustering We assessed the structural grouping of the unannotated representations using both K-Means (an isotropic, distance-based algorithm) and HDBSCAN (a density-based algorithm). A K-Means sweep across k∈[4,64]k∈[4,64] revealed consistently high Trustworthiness, rapidly scaling to 1.00001.0000 for k≥32k≥ 32. This indicates that the local topological neighbourhoods are perfectly preserved in the latent space. However, the K-Means Silhouette score remained relatively modest, peaking at 0.13360.1336 for k=56k=56. This discrepancy implies that the feature space is not partitioned into simple, spherical Euclidean clusters. When applying HDBSCAN—which is capable of discovering non-linear, arbitrarily shaped manifolds—the clustering performance improved dramatically. Upon projecting the space to its intrinsic dimension, HDBSCAN identified 6 distinct, highly dense clusters with an impressive filtered Silhouette score of 0.53590.5359. Most notably, the algorithm reported a 0.0%0.0\% noise ratio alongside a mean cluster membership confidence of 0.9950.995. This demonstrates that the self-distillation objective, coupled with our physics-guided augmentations, forces the representations into highly stable, well-separated density basins. There are virtually no diffuse or ambiguous samples bridging the gaps between these semantic clusters, proving the framework learns highly discriminative textures completely unsupervised. V-B Supervised Downstream Evaluation on S3Seg To evaluate the semantic discriminative power of our self-supervised representations, we conduct extensive supervised ablation studies on the S3Seg dataset, which provides pixel-wise annotations for four distinct benthic classes: Sand Ripples, Fine Sediments, Rocks, and Maerl [30]. We freeze the pre-trained ConvNeXt-v2 backbone and evaluate the representations using two standard protocols: Linear Probing (LP) and K-Nearest Neighbors (K-N). To rigorously assess the data-efficiency of our model, we simulate a few-shot learning environment by strictly limiting the amount of annotated training data available to the evaluators, sweeping across 1%1\%, 5%5\%, 10%10\%, 50%50\%, and 100%100\% subsets of the training split. V-B1 Feature Representation and Stage-by-Stage Progression A primary focus of our ablation study is tracking the evolution of the feature representations through the network to justify our multi-scale feature fusion. Table I presents the Accuracy, Macro F1, and Mean Intersection over Union (mIoU) for features extracted at individual backbone stages compared against our 3-layer MLP fusion module (the Fused Representation). TABLE I: Stage-by-Stage Feature Progression (100% Data, Linear Probe) Representation Accuracy (%) Macro F1 mIoU (%) Stage 1 70.6 0.597 45.4 Stage 2 78.3 0.704 56.9 Stage 3 84.5 0.791 67.3 Stage 4 83.8 0.789 67.0 Fused Representation 86.5 0.822 71.4 Using the full dataset Linear Probe as a baseline, the quality of the representations improves dramatically through the network. Stage 1 captures basic textures but struggles with semantic boundaries (45.4%45.4\% mIoU). Stage 2 exhibits a strong +11.5%+11.5\% jump to 56.9%56.9\% mIoU as low-level features, such as acoustic shadows and highlights, begin to group. Stage 3 provides another massive +10.4%+10.4\% improvement (67.3%67.3\% mIoU). However, performance stalls and plateaus at Stage 4 (67.0%67.0\% mIoU). This plateau is expected in heavily downsampled convolutional networks, where intermediate layers lose fine-grained spatial details before the fusion layers solidify global concepts while restoring granularity. Our concatenated and MLP-compressed Fused Representation strictly dominates, peaking at 71.4% mIoU and 86.5% accuracy. By preserving the fine-grained high-frequency acoustic details from Stage 1 alongside the deep semantic context from later stages, the final fusion layer successfully consolidates the features for optimal class separation. V-B2 Classifier Generalization: Linear Probe vs. K-N Our results, detailed in Table I, definitively prove that Linear Probing is vastly superior to K-N for navigating this specific embedding space. Across nearly every stage and few-shot scale, K-N heavily overfits: it achieves a near-perfect Train mIoU (∼0.998 0.998) during evaluation but collapses on the Test set (dropping to 59.7%59.7\% mIoU at full data scale). TABLE I: Few-Shot Generalization: Linear Probe vs. K-N using the Fused Representation Evaluator Metric 1% Data 5% Data 10% Data 50% Data 100% Data Linear Probe Accuracy (%) 80.9 83.7 85.0 86.4 86.5 Macro F1 0.744 0.786 0.802 0.819 0.822 mIoU (%) 61.8 66.6 68.8 71.0 71.4 K-N Accuracy (%) 73.8 77.2 78.5 80.8 81.6 Macro F1 0.608 0.671 0.680 0.703 0.717 mIoU (%) 47.6 54.1 55.4 58.3 59.7 Because the DINO self-distillation objective produces high-dimensional embeddings (768768-D), K-N likely falls victim to the curse of dimensionality—memorizing exact training vectors without learning a smooth, generalizing decision boundary. Conversely, the Linear Probe forces a simpler, hyperplane-based decision that ignores noisy micro-textures, generalizing far better to unseen sonar swaths and maintaining a Test mIoU of 71.4%71.4\%. V-B3 Data Efficiency and the Power of Self-Supervision One of the primary promises of the DINO methodology is extreme sample efficiency, which our model delivers. Looking at how the Linear Probe Test mIoU scales for our Fused Representation in Table I, we observe steady, rapid growth from 1%1\% to 10%10\% data scale. Remarkably, by utilizing just 10%10\% of the labeled data, the model achieves 96%96\% of its absolute peak performance (68.8%68.8\% vs 71.4%71.4\% mIoU). Furthermore, scaling from 50%50\% to 100%100\% yields a microscopic +0.004+0.004 gain in mIoU. This confirms the success of our unsupervised pre-training: the learned representations are natively well-separated in the latent space, requiring very few supervised labels to draw accurate classification boundaries. V-B4 Class-Level Bottleneck Analysis To understand the remaining error modes, we conducted a class-level deep dive on our best configuration (Fused Representation, 100% Data, Linear Probe). The per-class metrics are reported in Table I. TABLE I: Per-Class Performance for the Fused Representation (100% Data, Linear Probe) Benthic Class Precision Recall F1-Score IoU (%) Sand Ripples 0.942 0.885 0.913 83.9 Fine Sediments 0.907 0.934 0.920 85.2 Rocks 0.580 0.683 0.627 45.7 Maerl 0.826 0.833 0.829 70.8 Clearly, the Rocks class serves as the primary bottleneck dragging down the overall dataset mIoU. Analysis of the normalized confusion matrix reveals that true Rocks are misclassified as Sand Ripples 20.7%20.7\% of the time, and as Maerl 6.5%6.5\% of the time. We attribute this to two primary factors. First, acoustic similarity: in side-scan sonar imagery, both rocks and sand ripples create distinct high-intensity returns followed by hard acoustic shadows. Given the low spatial resolution of the extracted patch (i.e. 256×256256× 256), the input data fails to capture the broader, repetitive structural pattern of a ripple field versus the isolated geometric nature of a rock causing the embedding space to confuse them. Second, class imbalance: the support size for Rocks is significantly smaller than for Sand Ripples or Fine Sediments. The linear probe is therefore naturally biased toward predicting the majority classes to minimize global cross-entropy loss. VI Conclusion and Future Work In this work, we introduced a novel, physics-informed self-distillation framework tailored for Side-Scan Sonar (S) imagery. Recognizing the inherent limitations of applying standard vision models to acoustic data, we adapted the DINO methodology to overcome small-batch distributed training constraints and acoustic-specific statistics. Our primary architectural contribution—a multi-scale feature fusion module utilizing a 1×11× 1 convolutional MLP—successfully bridged the gap between fine-grained, high-frequency acoustic textures (extracted from shallow layers) and deep, context-aware semantic abstractions (from deeper layers). Our unsupervised feature space analysis demonstrated that the network avoids dimensional collapse and natively groups complex benthic topographies into highly dense, noise-free semantic clusters. During supervised downstream evaluation on the S3Seg dataset, the fused representation strictly dominated individual stage outputs. Notably, the framework exhibited exceptional data efficiency: utilizing a simple Linear Probe on just 10%10\% of the annotated data, the model achieved 96%96\% of its peak performance, ultimately reaching 71.4%71.4\% mIoU and 86.5%86.5\% overall accuracy at full data scale. This proves that self-supervised pre-training can effectively eliminate the massive annotation bottleneck traditionally associated with S habitat mapping. While these results establish a strong baseline for deep learned S representations, several avenues remain for future investigation: • Comparative SSL Framework Evaluation: While our DINO-based self-distillation proved highly effective, future work will benchmark our representations against other leading Self-Supervised Learning (SSL) paradigms. Specifically, we aim to evaluate our framework against the standard Masked Autoencoder (MAE) approach, which was the original pre-training methodology proposed for the ConvNeXt-v2 backbone, to determine which objective better captures acoustic scattering physics [40]. • Generalization to Public Datasets: The S3Seg dataset provided a rigorous testbed for benthic habitat classification; however, to fully validate the generalizability of our pre-trained backbone across different sensor payloads and distinct marine environments, we plan to evaluate the model on publicly available datasets, such as the AI4Shipwrecks dataset [32]. • Non-Linear Segmentation Heads: Our current ablation studies utilized Linear Probing and K-Nearest Neighbors to strictly measure the quality of the frozen latent space. However, as observed in the confusion between acoustically similar classes (e.g., Rocks and Sand Ripples), a linear hyperplane is sometimes insufficient. Future iterations will utilize a lightweight multi-layer perceptron (MLP) or a dedicated segmentation decoder to capture non-linear decision boundaries and further elevate the final segmentation performance on the S3Seg dataset. Acknowledgments This work was supported by Spanish Government through the project ”Automated Seabed Analysis through Self-Supervised Deep Learning Sonar Technology (ASSiST)” under grant PID2023-149413OB-I00. References [1] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool (2008) Speeded-up robust features (SURF). Computer Vision and Image Understanding 110 (3), p. 346–359. Cited by: §I. [2] P. Blondel (2009) The handbook of sidescan sonar. Springer (Praxis), Berlin, Heidelberg. Cited by: §I-C2. [3] A. Burguera and G. Oliver (2016) High-resolution underwater mapping using side-scan sonar. PLOS ONE 11 (1), p. 1–41. External Links: Document, Link Cited by: §I, §I, §I, §I-C2, §I-C3. [4] R. J. G. B. Campello, D. Moulavi, and J. Sander (2013) Density-based clustering based on hierarchical density estimates. In Advances in Knowledge Discovery and Data Mining, J. Pei, V. S. Tseng, L. Cao, H. Motoda, and G. Xu (Eds.), Berlin, Heidelberg, p. 160–172. Cited by: §IV-C3. [5] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin (2021) Emerging properties in self-supervised vision transformers. External Links: 2104.14294, Link Cited by: §I-A1, §IV-B. [6] E. Coiras, Y. Petillot, and D. M. Lane (2007) Multiresolution 3-d reconstruction from side-scan sonar images. IEEE Transactions on Image Processing 16 (2), p. 382–390. External Links: Link, Document Cited by: §I. [7] E. Facco, M. d’Errico, A. Rodriguez, and A. Laio (2017) Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports 7 (1). External Links: ISSN 2045-2322, Link, Document Cited by: §IV-C2. [8] Y. Fu, X. Luo, X. Qin, H. Wan, J. Cui, and Z. Huang (2025) Deep learning-based feature matching algorithm for multi-beam and side-scan images. Remote Sensing 17 (4). External Links: Link, ISSN 2072-4292, Document Cited by: §I. [9] A. Gretton, O. Bousquet, A. Smola, and B. Sch”olkopf (2005) Measuring statistical dependence with Hilbert-Schmidt norms. In International Conference on Algorithmic Learning Theory (ALT), LNAI, Vol. 3734, p. 63–77. Cited by: §I, §I-D. [10] L. Jing, P. Vincent, Y. LeCun, and Y. Tian (2022) Understanding dimensional collapse in contrastive self-supervised learning. External Links: 2110.09348, Link Cited by: §IV-C1, §V-A1. [11] S. Kamal Basha and A. Nambiar (2024) S3simulator: a benchmarking side scan sonar simulator dataset for underwater image analysis. In International Conference on Pattern Recognition, p. 219–235. Cited by: §I-B. [12] O. Katrusha, D. Prylipko, and K. Yefremov (2025) Change detection in side-scan sonar imagery based on deep learning feature matching methods. Eastern-European Journal of Enterprise Technologies 6 (2 (138)), p. 52–62. External Links: Link, Document Cited by: §I, §I, §I, §I-A4. [13] S. Koo, S. Youm, and J. Shin (2024) Cycle-gan-based synthetic sonar image generation for improved underwater classification. In Ocean Sensing and Monitoring XVI, Vol. 13061, p. 69–83. Cited by: §I-B. [14] T. Kwon, Y. Choi, H. Kim, M. Cho, J. Choi, and M. H. Kim (2026) Mine-jepa: in-domain self-supervised learning for mine-like object classification in side-scan sonar. arXiv preprint arXiv:2604.00383. Cited by: §I-A. [15] C. Lei, H. Rajani, N. Gracias, R. Garcia, and H. Wang (2025) A geometrically consistent matching framework for side-scan sonar mapping. External Links: 2509.11255, Link Cited by: §I, §I, §I-C3. [16] C. Lei, H. Rajani, N. Gracias, R. Garcia, and H. Wang (2026) PhysDNet: physics-guided decomposition network of side-scan sonar imagery. IEEE Geoscience and Remote Sensing Letters. Cited by: §I, §I, §I-B. [17] L. Li, Y. Li, H. Wang, C. Yue, P. Gao, Y. Wang, and X. Feng (2024) Side-scan sonar image generation under zero and few samples for underwater target detection. Remote Sensing 16 (22), p. 4134. Cited by: §I-B. [18] Y. Li, R. Pogodin, D. J. Sutherland, and A. Gretton (2021) Self-supervised learning with kernel dependence maximization. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. W. Vaughan (Eds.), Vol. 34, p. 15543–15556. External Links: Link Cited by: §I-D, §I-D. [19] Y. Liu and X. Ye (2023) A gray scale correction method for side-scan sonar images considering rugged seafloor. IEEE Transactions on Geoscience and Remote Sensing 61, p. 1–10. Cited by: §I-B. [20] Z. Liu, H. Mao, C. Wu, C. Feichtenhofer, T. Darrell, and S. Xie (2022) A convnet for the 2020s. In 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), p. 11966–11976. Cited by: §I-B. [21] D. G. Lowe (2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60 (2), p. 91–110. Cited by: §I. [22] Z. Ma, W. Meng, X. Zhao, and L. Jiang (2025) Enhancing sonar image segmentation with random fusion in a diffusion model framework: z. ma et al.. The Visual Computer 41 (11), p. 8369–8383. Cited by: §I-B. [23] L. McInnes, J. Healy, and J. Melville (2020) UMAP: uniform manifold approximation and projection for dimension reduction. External Links: 1802.03426, Link Cited by: §IV-C3. [24] A. Nambiar et al. (2026) Physics-informed simulation framework for realistic sonar image generation and statistical validation. arXiv preprint arXiv:2605.19712. Cited by: §I-B. [25] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski (2024) DINOv2: learning robust visual features without supervision. External Links: 2304.07193, Link Cited by: §I-A1, §I-A2, §I-A3, §I-A. [26] Y. Peng, H. Li, W. Zhang, J. Zhu, L. Liu, and G. Zhai (2025) Multi-view sonar image generation via gan trained with limited data for underwater object classification and detection. Expert Systems with Applications, p. 129452. Cited by: §I-B. [27] A. Preciado-Grijalva, B. Wehbe, M. B. Firvida, and M. Valdenegro-Toro (2022) Self-supervised learning for sonar image classification. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), p. 1498–1507. Cited by: §I, §I, §I-A, §I-C1, §I-C1. [28] A. Rahimi and B. Recht (2007) Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems (NIPS), Vol. 20, p. 1177–1184. Cited by: §I-D. [29] H. Rajani, V. Franchi, B. M. Valles, R. Ramos, R. Garcia, and N. Gracias (2025) BenthiCat: an opti-acoustic dataset for advancing benthic classification and habitat mapping. External Links: 2510.04876, Link Cited by: §I, §I, §I, §I-A4, §IV-A. [30] H. Rajani, N. Gracias, and R. Garcia (2023) A convolutional vision transformer for semantic segmentation of side-scan sonar data. Ocean Engineering 286, p. 115647. External Links: ISSN 0029-8018, Document, Link Cited by: §V-B. [31] P. J. Rousseeuw (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, p. 53–65. External Links: ISSN 0377-0427, Document, Link Cited by: §IV-C3. [32] A. V. Sethuraman, A. Sheppard, O. Bagoren, C. Pinnow, J. Anderson, T. C. Havens, and K. A. Skinner (2024) Machine learning for shipwreck segmentation from side scan sonar imagery: dataset and benchmark. External Links: Document, Link Cited by: 2nd item. [33] B. W. Sheffield, F. E. Bobe, B. Marchand, and M. S. Emigh (2023) Advances in self-supervised learning for synthetic aperture sonar data processing, classification, and pattern recognition. In OCEANS 2023-MTS/IEEE US Gulf Coast, p. 1–5. Cited by: §I-A. [34] B. Sheffield, J. Ellen, and B. Whitmore (2024) On vision transformers for classification tasks in side-scan sonar imagery. arXiv preprint arXiv:2409.12026. Cited by: §I-A. [35] B. Sheffield (2023) Self-supervised learning for improved synthetic aperture sonar target recognition. arXiv preprint arXiv:2307.15098. Cited by: §I-A. [36] O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jégou, P. Labatut, and P. Bojanowski (2025) DINOv3. External Links: 2508.10104, Link Cited by: §I, §I-A, §I-A1, §I-A3, §I-A4, §I-A4, §I-A. [37] D. Stewart, A. Kreulach, S. F. Johnson, and A. Zare (2023) Image-to-height domain translation for synthetic aperture sonar. IEEE Transactions on Geoscience and Remote Sensing 61, p. 1–13. Cited by: §I-B. [38] M. Valdenegro-Toro, A. Preciado-Grijalva, and B. Wehbe (2021) Pre-trained models for sonar images. In OCEANS 2021: San Diego–Porto, p. 1–8. Cited by: §I-A. [39] L. van der Maaten (2009) Learning a parametric embedding by preserving local structure. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, D. van Dyk and M. Welling (Eds.), Proceedings of Machine Learning Research, Vol. 5, Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA, p. 384–391. External Links: Link Cited by: §IV-C3. [40] S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie (2023) Convnext v2: co-designing and scaling convnets with masked autoencoders. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 16133–16142. Cited by: §I, §I-B, §I-B, 1st item. [41] Z. Zhang, P. Zhang, F. Wang, L. Ma, and F. Sun (2025) Self-supervised enhancement of forward-looking sonar images: bridging cross-modal degradation gaps through feature space transformation and multi-frame fusion. arXiv preprint arXiv:2504.10974. Cited by: §I-A. Fig. 4: PCA visualization of feature maps across network stages. The first three principal components are mapped to RGB channels to visualize the high-dimensional feature spaces. Rows (from top to bottom) display: original image patches, intermediate outputs from Stages 1 through 4, and the final fused representation. Fig. 5: Top five matches for a sample of the centroids obtained by running K-Means clustering on 56 clusters. The centroid label is outlined on top of each respective image. Fig. 6: Continued: Top five matches for a sample of the centroids obtained using the K-Means clustering.