Paper deep dive
RINO: Renormalization Group Distillation with No Labels
Zichun Hao, Raghav Kansal, Abhijith Gandrakota, Chang Sun, Haoyang Li, Zihan Zhao, Ngadiuba Jennifer, Javier Duarte, Maria Spiropulu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel $H \to 4q$ by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.15919v1
- Canonical: https://arxiv.org/abs/2610.15919v1
Trouble viewing inline? Open PDF directly →
Full Text
122,646 characters extracted from source content.
Expand or collapse full text
RINO: Renormalization Group Distillation with No Labels Zichun Hao Raghav Kansal †thanks: Also affiliated with the Fermi National Accelerator Laboratory, Batavia, IL 60510, USA†thanks: Now at Bexorg†thanks: Inc. Chang Sun Maria Spiropulu Affiliation: Division of Physics, Mathematics and Astronomy Affiliation: California Institute of Technology Affiliation: Pasadena, CA 91125 Email: zhao,rkansal,chsun,smaria@caltech.edu Abhijith Gandrakota Jennifer Ngadiuba Affiliation: Physics Division Affiliation: Fermi National Accelerator Laboratory Affiliation: Batavia, IL 60510 Email: abhijith,ngadiuba@fnal.gov Haoyang Li Zihan Zhao Javier Duarte Affiliation: Department of Physics Affiliation: University of California San Diego Affiliation: La Jolla, CA 92093 Email: hal113,ziz078,jduarte@ucsd.edu Abstract Supervised machine learning in high-energy physics (HEP) requires training on labeled simulations of particle collisions, which are known to be unfaithful to the underlying physics. This leads to significant inaccuracies and biases in the transfer of trained models to real data, limiting the power of deep learning methods. Self-supervised learning (SSL) is a promising alternative for pretraining directly on real, unlabeled data. However, existing techniques largely operate with inductive biases inspired by domains such as images and natural language. We propose RINO (Renormalization Group DIstillation with NO Labels), which, for the first time, exploits the fundamentally multi-scale renormalization group structure of high-energy jet formation as a physically grounded source of SSL views. We define teacher-student representations across energy scales using different levels of hierarchical clustering, aligned via DINO-style self-distillation. To mirror the background-dominated nature of real collider data, RINO pretrains exclusively on jets from quantum chromodynamics (QCD) processes and is evaluated on out-of-distribution (OOD) transfer to rare signal classes. In our controlled data-efficiency study, RINO matches fully supervised training on JetNet after finetuning on only 1% of the labeled data. With the full labeled dataset, RINO improves OOD classification accuracy on top-quark jets by 5.9% and, without retraining, improves AUC on the unseen Higgs boson decay channel H→4qH→ 4q by 4.1% over the strongest SSL baseline. Scale-structure diagnostics reveal that DINO-style self-distillation permits our backbone to retain scale-dependent structure. Ablation studies show that this accounts for an 8.6% OOD accuracy gap with respect to alternative SSL paradigms such as SimCLR. Code is available at https://github.com/zichunhao/RINO. 1 Introduction The discrepancy between simulated training data and real measurements is a fundamental bottleneck for machine learning (ML) in the physical sciences. In high-energy physics (HEP), particles produced in real collisions cannot be labeled with absolute certainty, so supervised classifiers are trained using Monte Carlo (MC) simulations with well-defined collision contents (i.e., labels). However, due to the resulting domain shift, trained models can lose 20% to 40% of their performance when applied to real data (CMS Collaboration, 2025c; CMS Collaboration, 2025b). This gap stems from imperfect modeling of effects such as nonperturbative quantum chromodynamics (QCD) and detector response. Self-supervised learning (SSL) offers a way to reduce the reliance on simulated data by pretraining directly on unlabeled real data and finetuning on labeled samples (Hendrycks et al., 2019; Asano et al., 2020; Gui et al., 2024). To do so, most SSL methods construct augmented views of the same input and train the model to treat them as equivalent, with the choice of views implicitly determining the inductive bias of the learned representation. A number of physics-motivated SSL methods have been developed for HEP (Dillon et al., 2022; Zhao et al., 2024; Golling et al., 2024; Leigh et al., 2025; Birk et al., 2024; Katel et al., 2024; Bardhan et al., 2025), utilizing pretext tasks such as learning to align augmented views based on symmetries (Dillon et al., 2022), predicting masked particle constituents (Golling et al., 2024; Leigh et al., 2025), and joint-embedding predictions of held-out subjets or constituents (Katel et al., 2024; Bardhan et al., 2025). These designs encode valuable symmetries; however, they all operate on a single physical scale, ignoring an important structural property of the input. A central object of study in HEP is the jet: a collimated spray of particles produced by quark and gluon emission at high-energy colliders such as the CERN Large Hadron Collider (LHC) (Andersson, 1987). From an ML perspective, a jet is a variable-length point cloud, while from a physics perspective, it is the observable end of QCD radiation, a stochastic cascade whose kinematic structure is multi-scale by construction. Identifying the source of a jet, called “jet tagging”, is itself a well-studied ML task with a rich body of supervised methods (Komiske et al., 2019; Battaglia et al., 2016; Moreno et al., 2020; Qu and Gouskos, 2020; Qu et al., 2022b; Gong et al., 2022; Bogatskiy et al., 2024; Spinner et al., 2024; Brehmer et al., 2025; Krause et al., 2026). Concretely, the same jet can be described at different resolutions: as a fine-grained cloud of O(30–100) particles; a coarse-grained set of 2–4 subjets that capture only the high-level substructure; or anywhere in between. These descriptions correspond to physically meaningful coarse-graining of the same underlying object, effectively the renormalization group (RG) flow from the high-energy collision scale down to the hadronization scale (Nagy and Soper, 2022). RG structure has been widely studied across the physical sciences, such as cosmology (Pietroni et al., 2012; Ivanov, 2024; Calzetta et al., 1992), materials science (Kmiecik et al., 2016; Wang et al., 2019; Ingólfsson et al., 2014), and fluid dynamics (Frisch, 1995; Grinstein, 2024; Berendsen, 2007), suggesting that methods designed around physical coarse-graining may generalize beyond HEP. ppppqqq¯ qHadronizationParton shower / RG flowkTk_T clustering (approx. inverse)Hard scaleHadronization(final-state particles)N=2N=2N=3N=3N=4N=4N=6N=6N=8N=8N=16N=16uncl.uncl.N=2N=2N=6N=6uncl.uncl.Teacher / Student (a) (b) Figure 1: (a) A QCD parton shower (here, a qq¯q q pair radiating softer gluons and quarks) is analogous to a renormalization group (RG) flow down to the hadronization scale. The kTk_T clustering approximately inverts this process, so stopping at different numbers of subjets N generates the multi-scale views used for RINO self-distillation. (b) RINO training pipeline (DINO term of Eq. 2). The same jet is clustered at two kTk_T stopping points (here teacher N=6N=6, student N=3N=3). The teacher produces soft labels for the student to match via cross-entropy. Teacher parameters are updated from the student via an exponential moving average (EMA). This figure shows one representative student-teacher pair; in practice the loss aligns each student view to every teacher view at a different scale (Sec. 3.2). To exploit this fundamental multi-scale property of jets, we propose RINO (Renormalization Group DIstillation with NO Labels), an SSL framework for learning rich and robust jet representations. RINO uses teacher-student self-distillation (Caron et al., 2021; Oquab et al., 2024; Siméoni et al., 2025; Zhang et al., 2019) over jet views generated at different energy scales by a hierarchical clustering algorithm called kTk_T clustering (Catani et al., 1993). Training across these scales encourages representations aligned across resolutions, while retaining scale-structured geometry in the backbone. Our contributions are the following: • A physics-informed multi-scale SSL framework. To the best of our knowledge, RINO is the first SSL framework to align representations of a jet across multiple kTk_T clustering scales. • A realistic controlled comparison. RINO is pretrained exclusively on QCD jets, the dominant background process at hadron colliders, in contrast to prior HEP SSL work that pretrains on simulated mixtures with signal classes overrepresented relative to real data. To probe transfer to deployment-like conditions, we finetune on the smaller JetNet dataset and evaluate out-of-distribution (OOD) on the larger JetClass, where JetNet stands in for limited labeled simulation data and JetClass for the larger target distribution, with a distribution shift between the two. This protocol finetunes without target-domain labels, matching the realistic downstream task. Under this protocol, RINO outperforms baselines on OOD top tagging by 5.9% over the strongest baseline with lower seed-to-seed variance, and exhibits ∼3× 3× smaller 1-Wasserstein distance (W1W_1) and 6–7× smaller squared maximum mean discrepancy (MMD2) between the finetuning and evaluation domains. • Ablation studies identifying both DINO and scale-based distillation as core improvements. We observe that both DINO and SimCLR (Chen et al., 2020) achieve comparable cross-scale invariance as a whole system (i.e., backbone + projection head), but differ in where. DINO delegates it to the projection head while retaining scale-dependent structure in the backbone, whereas the backbone of our JetCLR-scale model is itself nearly scale-invariant. This difference is associated with an 8.6%8.6\% OOD accuracy gap. We further find that the multi-scale view construction contributes +11.4%+11.4\% to +14.0%+14.0\% over single-scale alternatives (Fig. 3). 2 Related Work Jet Tagging. Jet tagging has been the focus of numerous ML advances using deep sets, graph networks, transformers, and equivariant architectures (Komiske et al., 2019; Battaglia et al., 2016; Qu and Gouskos, 2020; Qu et al., 2022b; Gong et al., 2022; Bogatskiy et al., 2024; Spinner et al., 2024; Brehmer et al., 2025; Krause et al., 2026). These advances are largely supervised and architecture-driven, and therefore depend on labeled MC simulations whose imperfect modeling of detector and physics effects creates a persistent gap when the trained models are deployed on real collision data. RINO instead focuses on the pretraining signal that produces transferable jet representations, learning directly from unlabeled data and using jet tagging only as its primary downstream benchmark. Learning from Augmented Views. Several SSL methods learn representations by aligning differently augmented views of the same input (Chen et al., 2020; He et al., 2020; Caron et al., 2020; Bardes et al., 2022; Zbontar et al., 2021). DINO (Caron et al., 2021) advances this via teacher-student self-distillation (Zhang et al., 2019), and DINOv2 (Oquab et al., 2024) adds iBOT-style (Zhou et al., 2022) masked prediction on patch tokens. In computer vision, views come from random cropping and photometric augmentation of dense pixel grids. However, these augmentations do not transfer directly to jets, which are variable-length point clouds with no fixed spatial topology, so the choice of view generation shapes the inductive bias. SSL in HEP. There has been growing interest in applying SSL techniques to HEP, spanning contrastive (Dillon et al., 2022), masked (Golling et al., 2024; Leigh et al., 2025), generative (Birk et al., 2024), and joint-embedding paradigms (Katel et al., 2024; Bardhan et al., 2025). JetCLR (Dillon et al., 2022; Zhao et al., 2024) uses SimCLR-style contrastive learning with physics-motivated augmentations such as rotations, translations, and soft and collinear augmentations. MPMv1 (Golling et al., 2024) tokenizes jet constituents via a VQ-VAE (van den Oord et al., 2017) and predicts discrete token IDs, in analogy with masked language modeling (Devlin et al., 2019), while MPMv2 (Leigh et al., 2025) reconstructs masked constituent features directly. OmniJet-α (Birk et al., 2024) trains an autoregressive decoder on VQ-VAE tokens of jet constituents. J-JEPA (Katel et al., 2024) and HEP-JEPA (Bardhan et al., 2025) adapt the JEPA framework (Assran et al., 2023) to jets, predicting the embeddings of held-out subjets or constituents from the remaining ones. In parallel, jBOT (Tsoi and Rankin, 2026) adapts iBOT to jets and evaluates in-distribution on JetNet. Other approaches include supervised pretraining on auxiliary physics tasks (Li and others, 2024; Mikuni and Nachman, 2025b; Mikuni and Nachman, 2025a; Mokhtar et al., 2025) and learning from simulation resampling (Harris et al., 2025). While these approaches introduce valuable domain-specific adaptations, they generate views at a single physical resolution. RINO constructs views from the unclustered particle cloud down to a few subjets via kTk_T clustering and evaluates transfer from labeled simulation to a distinct target, mirroring on-data use. Multi-Scale and Hierarchical SSL. Multi-scale view construction is a recurring theme in vision SSL. DINO (Caron et al., 2021) combines large global crops with small local crops to encourage local-global consistency, and hierarchical vision transformers such as Swin (Liu et al., 2021) build features at multiple resolutions. These approaches define scale spatially (through pixel support) or architecturally, relying on artificial augmentations as a proxy for the underlying generative process. By contrast, RINO defines scale through the QCD-motivated kTk_T clustering sequence: the number of remaining subjets indexes a physically ordered sequence of resolution scales. The scale axis is therefore inherited from the physics that produces the data, and we analyze the resulting scale structure in Sec. 5.1. RG Flow and Jet Clustering. A parton shower is a stochastic realization of a QCD renormalization-group flow, in which partons from the hard scattering successively branch into lower-scale partons until the evolution reaches the hadronization scale (Nagy and Soper, 2018). Jet clustering algorithms approximately invert this process by sequentially combining nearby constituents. At each step, the kTk_T algorithm (Catani et al., 1993) merges the pair that minimizes dij=min(pTi2,pTj2)ΔRij2R2,d_ij= \! (p_Ti^2,\,p_Tj^2 )\, R_ij^2R^2, (1) or removes constituent i if its beam distance diB=pTi2d_iB=p_Ti^2 is smallest, where ΔRij2=(Δηij)2+(Δϕij)2 R_ij^2=( _ij)^2+( _ij)^2 and R is the jet radius parameter, so soft and collinear pairs are merged first to mimic the reverse of QCD splitting (Salam, 2010; Thaler and Van Tilburg, 2011; CMS Collaboration, 2025a; Kogler et al., 2019; Bonilla et al., 2022). Stopping at N subjets yields a coarse-grained jet at a characteristic scale (App. A.5; Nagy and Soper, 2022), and varying N traces a physically ordered multi-scale axis for the same jet. We use “RG flow” as a heuristic analogy: kTk_T clustering provides a sequence of coarse-grained jet descriptions ordered by an emission scale. RINO converts this coarse-graining sequence into an SSL strategy by aligning representations across clustering scales. 3 Method: The RINO Framework 3.1 Pretraining and Finetuning Data We pretrain RINO exclusively on the QCD-jet subset of JetClass (Qu et al., 2022a), excluding the simulated signal classes (e.g., top quark, W boson, Z boson, and Higgs boson jets). While prior HEP SSL work pretrains on simulated mixtures of signal classes, real collider data is overwhelmingly composed of QCD jets. Hence, restricting pretraining to QCD jets is more realistic for real-data SSL in HEP. This also allows downstream evaluation to measure generalization to rare signal classes unseen during pretraining. Meanwhile, JetClass provides ground-truth labels for the held-out signal classes used in downstream evaluation, which would otherwise require signature-dependent triggers and physics cuts on real data that could themselves introduce biases. For finetuning, we use the smaller JetNet (Kansal et al., 2021) as a proxy for limited labeled simulation data and evaluate OOD on the held-out signal classes of JetClass, the larger target distribution. The two datasets differ in pTp_T range, constituent count, and detector simulation, inducing a distribution shift between finetuning and evaluation (App. A.4). 3.2 Jet Clustering as Multi-Scale View Construction RINO follows DINO’s self-distillation framework (Caron et al., 2021; Oquab et al., 2024; Siméoni et al., 2025), but generates physics-informed views, each of which is a kinematic description of the same jet at a different scale. We construct these views by stopping the exclusive kTk_T clustering sequence at varying numbers of subjets N, which integrates out soft and collinear radiation below a corresponding resolution scale (Sec. 2, Fig. 1(a)). Varying N therefore produces a sequence of views of the same jet at physically ordered resolution scales. We use the discrete ladder N∈2,3,4,6,8,16N∈\2,3,4,6,8,16\ in addition to the fully unclustered particle cloud (denoted uncl.uncl.), with N∈2,3,4N∈\2,3,4\ covering the typical hard prong multiplicities at the LHC. Clustering histories are precomputed with FastJet (Cacciari et al., 2012) and cached. During training, the teacher and student receive asymmetric view sets. In the DINO term, the teacher sees only the intermediate-scale views (N∈6,8,16N∈\6,8,16\), while the student additionally sees the views at the two extremes of the ladder: coarse-grained (N∈2,3,4N∈\2,3,4\) and unclustered (uncl.uncl.). The DINO loss aligns each student view’s representation with the teacher’s intermediate-scale representations, encouraging multi-scale substructure encoding rather than single-scale specialization. We ablate this choice in App. A.8. 3.3 Setup: Training Objective and Architecture We adopt the composite pretraining objective of DINOv2 (Oquab et al., 2024) (Fig. 1(b)), with the teacher parameters tracking the student via an exponential moving average (EMA) (Holt, 2004; Tarvainen and Valpola, 2017; He et al., 2020). The RINO objective is ℒ=λDINOℒDINO+λiBOTℒiBOT+λKoLeoℒKoLeo.L= _DINOL_DINO+ _iBOTL_iBOT+ _KoLeoL_KoLeo. (2) The first term, ℒDINOL_DINO, aligns representations across the view sets (Sec. 3.2) along the parton shower hierarchy, with Sinkhorn–Knopp centering (Sinkhorn and Knopp, 1967; Caron et al., 2020) to prevent collapse. The second term, ℒiBOTL_iBOT, provides local supervision via masked-particle prediction: the student predicts the teacher’s representation of each particle at masked positions. This serves a dual role: a single-scale masked-prediction baseline analogous to MPM-style methods (Golling et al., 2024; Leigh et al., 2025) that we ablate against in Sec. 4.5, and an auxiliary co-training objective that sharpens representation quality alongside DINO (App. A.8). The third term, ℒKoLeoL_KoLeo, is the Kozachenko–Leonenko (KoLeo) differential entropy estimator (Beirlant et al., 1997; Delattre and Fournier, 2017; Sablayrolles et al., 2019), which regularizes the embeddings for diversity on the hypersphere. Appendices A.1 and A.2 give the hyperparameters and the explicit loss terms. Each jet is a point cloud of particle momentum vectors (an Np×3N_p× 3 tensor) preprocessed into 77 kinematic features following the Particle Transformer (ParT) (Qu et al., 2022b), fed to a shared student-teacher transformer encoder (Vaswani et al., 2017). A polar-binned positional encoding (PE) indexed by each constituent’s (Δη,Δϕ)( η, φ) position breaks the degeneracy among masked iBOT tokens while preserving permutation equivariance. Architecture details are provided in App. A.3. 4 Experiments 4.1 Setup and Controlled Comparison Protocol All comparisons follow a controlled protocol, a simulated proxy for the intended workflow on experimental data (App. A.13), designed to isolate the contribution of the pretraining strategy. Methods share the same jet transformer encoder backbone, except for the decoder-based OmniJet-α baseline, where we add causal masking to support its GPT-style task (App. A.17). SSL methods are finetuned with the LP-FT (Kumar et al., 2022) strategy, while the supervised baseline is trained end-to-end from random initialization. All methods use the same pretraining data (namely the JetClass QCD class, a proxy for QCD-dominated real data), downstream data splits, finetuning hyperparameters (optimizer, learning-rate scheduler, weight decay; App. A.6), and number of finetuning seeds. We evaluate top tagging (hadronic t→bqqt→ bqq vs. QCD jets) under the JetNet→ OOD protocol of Sec. 3.1, with binary classification accuracy on the JetClass test set as the primary metric. For each pretrained model, we report two downstream-evaluation levels, each with two metrics. On the one hand, we probe the embedding quality by directly running a k-nearest-neighbors (k-N; k=20k=20) classifier and a linear probe on the frozen pretrained backbone after re-introducing JetClass top jets and labels.11 1 We would like to point out that such labels are unavailable in deployment on real, unlabeled data, so these two metrics serve only to demonstrate and diagnose embedding quality. On the other hand, a linear classification head and a two-layer MLP head are finetuned together with the backbone on JetNet labels and evaluated on JetClass’s top-tagging task to measure OOD generalization.22 2 In this case, JetClass top jets and all JetClass labels (QCD and top) are not used in pretraining or finetuning. This hierarchy disentangles representation quality from OOD generalization. Unless otherwise noted, numbers are reported as mean ± standard deviation over 1010 finetuning seeds; for pretrained backbones, the k-N and linear-probe uncertainties are over probe fits on disjoint data subsets. 4.2 OOD Top Tagging Table 1 summarizes top-tagging accuracy on JetClass for RINO, four recent SSL baselines (JetCLR, MPMv1, MPMv2, and OmniJet-α), and supervised training from scratch on JetNet. To further isolate the contribution of multi-scale view construction, we include JetCLR-scale, a controlled variant that replaces JetCLR’s augmentations with the same clustering-scale views used by RINO while keeping the contrastive objective (see App. A.17 for details). RINO is the only method whose MLP head significantly exceeds its linear head (t=+3.0t=+3.0, p≈0.01p≈ 0.01); no baseline shows the same improvement, and JetCLR and MPMv2 in particular show significant degradation (t<−3t<-3; App. A.15). This indicates that RINO’s representations are robust enough that the additional capacity of the MLP head improves, rather than degrades, OOD accuracy. With the MLP head, RINO demonstrates a 5.9%5.9\% relative improvement over the strongest baseline (MPMv1) and 7.8%7.8\% over the supervised model trained on JetNet. We also find that replacing JetCLR’s augmentations with kTk_T views (JetCLR-scale) improves accuracy on the two OOD metrics, demonstrating the benefits of our scale-based augmentations under domain shift. RINO also has the lowest seed-to-seed variance with the MLP head (±0.003± 0.003, versus at least ±0.016± 0.016 for every other method), and its worst accuracy over five weight-decay and dropout settings (0.8430.843) exceeds the best of supervised training, MPMv1, and MPMv2 (App. A.7), demonstrating robustness to common regularization hyperparameters (weight decay and dropout). When the domain shift is smaller (Top Quark Tagging→ ), the leading methods saturate near 0.930.93 in accuracy (App. A.11), yet RINO’s proxy scale factors stay closest to unity (App. A.12). Table 1: Top-tagging accuracy on JetClass (top vs. QCD). The linear and MLP heads are finetuned on JetNet only (OOD); k-N and the linear probe are fit on JetClass labels and measure frozen-embedding quality. The bottom block groups all methods using kTk_T-clustering views (ours), differing only in the training loss. Each entry is μ±σμ±σ; bold marks the best per column; underline marks entries within 1σ1σ of the best. Method k-N Linear probe Linear head MLP head Supervised 0.722±0.0220.722± 0.022 0.714±0.0300.714± 0.030 0.784±0.0330.784± 0.033 0.786±0.0230.786± 0.023 SSL baselines JetCLR (Dillon et al., 2022) 0.780±0.0020.780± 0.002 0.793±0.0010.793± 0.001 0.791±0.0480.791± 0.048 0.700±0.0310.700± 0.031 OmniJet-α (Birk et al., 2024) 0.859±0.0010.859± 0.001 0.869±0.0020.869± 0.002 0.749±0.0800.749± 0.080 0.751±0.0500.751± 0.050 MPMv1 (Golling et al., 2024) 0.882±0.0020.882± 0.002 0.907±0.0020.907± 0.002 0.809±0.0080.809± 0.008 0.800±0.0190.800± 0.019 MPMv2 (Leigh et al., 2025) 0.896±0.0010.896± 0.001 0.912±0.0010.912± 0.001 0.817±0.0270.817± 0.027 0.760±0.0160.760± 0.016 kTk_T-view SSL (ours) JetCLR-scale 0.884±0.0010.884± 0.001 0.915±0.001¯ 0.915± 0.001 0.803±0.0170.803± 0.017 0.780±0.0230.780± 0.023 RINO 0.908±0.0010.908± 0.001 0.916±0.0010.916± 0.001 0.838±0.0090.838± 0.009 0.847±0.0030.847± 0.003 4.3 Data Efficiency We sweep the JetNet finetuning data fraction over 0.1,1,10,50,100%\0.1,1,10,50,100\\% and compare RINO with representative baselines, under the same finetuning protocol (App. A.6) to ensure the comparison reflects representational differences rather than tuning effort. Figure 2 reports OOD top-tagging accuracy on JetClass. RINO finetuned on 1%1\% of labels reaches 0.797±0.0150.797± 0.015 accuracy, matching fully supervised training on the entire JetNet dataset (0.786±0.0230.786± 0.023) within uncertainty. RINO has the leading OOD accuracy at all data fractions and improves monotonically. In contrast, some models are non-monotonic: MPMv2, for example, peaks in OOD performance at 10%10\% of the finetuning dataset. This is consistent with the feature-distortion mechanism of Kumar et al. (2022): when pretrained features are strong and the OOD shift is large, finetuning distorts features along in-distribution (ID) directions. More ID labels drive further distortion, which can explain this non-monotonic behavior. Moreover, JetCLR-scale outperforms JetCLR at all label fractions, so the benefit of cross-view alignment persists across data regimes. Full curves are in App. A.14. Figure 2: OOD top-tagging accuracy versus JetNet data fraction. RINO with 1%1\% of labels matches supervised training on 100%100\% within uncertainty. Bands show ±1σ± 1σ over 1010 finetuning seeds (MLP head); the dotted line marks supervised training on 100%100\% of labels. 4.4 Cross-Topology Generalization To test whether the learned representations capture general jet substructure rather than task- or dataset-specific features, we evaluate all methods on unseen JetClass signal topologies after finetuning only on top tagging from JetNet with a linear head (Table 2). No model sees Higgs samples during any training. We also report AUC as a metric, since the classification threshold is calibrated for the finetuning task and does not necessarily transfer to unseen signals and topologies. RINO achieves the highest AUC on both H→4qH→ 4q (0.894±0.0070.894± 0.007) and H→bb¯H→ b b (0.788±0.0160.788± 0.016). The H→4qH→ 4q result demonstrates genuine cross-topology generalization. Although finetuned only on 3-prong top-quark jets (3 hard subjets from t→bqqt→ bqq), RINO’s linear classifier identifies 4-prong Higgs decays (H→4qH→ 4q) with low variance, demonstrating that kTk_T-view pretraining learns multi-prong substructure itself rather than only topology-specific features. On H→bb¯H→ b b, the leading methods cluster within ∼2% 2\% AUC of each other. This is expected: H→bb¯H→ b b discrimination relies on track-level features (impact parameters and secondary-vertex displacements) that distinguish b-quarks from light quarks. Our kinematic-only inputs cannot access these flavor signatures, so all methods are bounded by what kinematic substructure each backbone preserves. Table 2: Cross-topology transfer (JetNet→ , linear head). Models are finetuned on top tagging only and evaluated on unseen JetClass signal topologies. Bold marks the best per column; underline marks entries within one standard deviation of the best. Method H→4qH→ 4q H→bb¯H→ b b ACC AUC ACC AUC Supervised 0.707±0.0260.707± 0.026 0.800±0.0290.800± 0.029 0.623±0.0200.623± 0.020 0.726±0.0300.726± 0.030 SSL baselines JetCLR (Dillon et al., 2022) 0.727±0.0670.727± 0.067 0.803±0.0420.803± 0.042 0.706±0.0410.706± 0.041 0.782±0.018¯ 0.782± 0.018 MPMv1 (Golling et al., 2024) 0.741±0.0270.741± 0.027 0.819±0.0180.819± 0.018 0.612±0.0260.612± 0.026 0.705±0.0170.705± 0.017 MPMv2 (Leigh et al., 2025) 0.757±0.0330.757± 0.033 0.859±0.0180.859± 0.018 0.648±0.0230.648± 0.023 0.778±0.010¯ 0.778± 0.010 OmniJet-α (Birk et al., 2024) 0.692±0.0530.692± 0.053 0.797±0.0340.797± 0.034 0.603±0.0260.603± 0.026 0.709±0.0320.709± 0.032 kTk_T-view SSL (ours) JetCLR-scale 0.630±0.0200.630± 0.020 0.819±0.0140.819± 0.014 0.575±0.0150.575± 0.015 0.771±0.0100.771± 0.010 RINO 0.821±0.0080.821± 0.008 0.894±0.0070.894± 0.007 0.687±0.025¯ 0.687± 0.025 0.788±0.0160.788± 0.016 4.5 Design Ablations Cross-scale DINO objective. Table 3 progressively adds RINO components on top of an iBOT-only variant (single-scale masked prediction, no cross-scale DINO objective). Adding DINO-based multi-scale distillation across kTk_T views yields +14.0%+14.0\%, while a final teacher-schedule refinement adds another +1.0%+1.0\%. Therefore, most of the improvement comes from the cross-scale DINO objective. The full progressive decomposition, including view-assignment and PE choices, is in App. A.8. Multi-scale view alignment. JetCLR-scale and RINO use the same multi-scale views from kTk_T but differ in the training objective (Table 1). JetCLR-scale outperforms the original JetCLR (0.7800.780 vs. 0.7000.700, +11.4%+11.4\%), confirming that kTk_T views account for most of the gain regardless of the SSL family. The remaining gap to RINO (+7.6%+7.6\%; +8.6%+8.6\% for the final model) reflects the benefit of self-distillation over contrastive alignment (discussed in detail in Sec. 5.1).33 3 We compare against the model before teacher tuning (0.8390.839, Table 3) because it shares the pretraining recipe of the iBOT-only variant, whereas teacher tuning was applied only to the final model. Clustering algorithm. To isolate the role of the energy-ordered kTk_T scale sequence, we train variants using Cambridge/Aachen (C/A) clustering (Dokshitzer et al., 1997), which groups by angular proximity, and anti-kTk_T (Cacciari et al., 2008), which clusters the hardest constituents first. Under the same training objective, C/A yields 0.796±0.0120.796± 0.012 and anti-kTk_T yields 0.693±0.0260.693± 0.026, compared to kTk_T’s 0.839±0.0050.839± 0.005. All three algorithms produce multi-scale views, but they differ in how they define resolution: kTk_T coarsens by energy scale (approximately reversing the parton shower), C/A by angular proximity, and anti-kTk_T by hardness. The ranking kTk_T >> C/A >> anti-kTk_T is consistent with the finding that kTk_T subjets align more closely with the prongs from the hard process than C/A or anti-kTk_T subjets (CMS Collaboration, 2025a), confirming that the SSL views help most when the coarse-graining axis matches the physical process that generated the data. Table 3: Ablation: contribution of DINO self-distillation. All variants share the same polar-binned PE, backbone, and finetuning protocol. Arrows give the relative gain over the previous row. Configuration OOD top ACC iBOT only 0.736±0.0280.736± 0.028 + Cross-scale self-distillation 0.839±0.0050.839± 0.005 ↑14.0% 14.0\% + Teacher HP tuning 0.847±0.0030.847± 0.003 ↑1.0% 1.0\% 5 Discussion 5.1 Why Cross-Scale Training Helps Two controlled comparisons isolate the contributions of kTk_T views and the SSL objective, as shown in Fig. 3. Replacing JetCLR’s augmentations with kTk_T views while keeping the contrastive loss (JetCLR-scale) improves accuracy from 0.7000.700 to 0.7800.780 (+11.4%+11.4\%). Adding cross-scale distillation over kTk_T views to the iBOT-only variant gives +14.0%+14.0\% (0.736→0.8390.736→ 0.839). Switching from contrastive alignment to self-distillation adds a further +5.1%+5.1\% (JetCLR → iBOT-only) and +7.6%+7.6\% (JetCLR-scale → RINO before teacher tuning, 0.8390.839; Table 3). JetCLR0.7000.700JetCLR-scale0.7800.780iBOT-only0.7360.736RINO0.8390.839kTk_T scale views+11.4%+11.4\%kTk_T scale views+14.0%+14.0\%self-distillation+5.1%+5.1\%self-distillation+7.6%+7.6\%Single-scaleMulti-scale (kTk_T)ContrastiveSelf-distillationSelf-distillationSelf-distillation Figure 3: Factorization of RINO’s OOD accuracy gain. The top horizontal arrow replaces JetCLR augmentations with kTk_T scale views; the bottom horizontal arrow adds DINO-based multi-scale distillation across kTk_T views. Vertical arrows compare contrastive and self-distillation objectives. RINO is shown before teacher tuning (0.8390.839, Table 3); the final model reaches 0.8470.847 (Table 1). Perhaps surprisingly, the mean cosine similarity across kTk_T clustering scales (App. A.9, Fig. 5) shows that RINO’s backbone is not scale-invariant, while the entire model (backbone and head combined) is one of the most invariant. Mean cosine similarity alone does not tell the whole story, though, since low similarity can also be inherited from input differences; the structure of the similarity matrix matters. Unlike the scale-ignorant MPMv2 backbone (monotonic decay with distance along the kTk_T ladder) and the scale-invariant JetCLR-scale backbone (all cosines near 11), RINO’s backbone is scale-aware: it shows a block structure that follows the student-teacher role assignment. This is permitted by projection-head shielding (Bordes et al., 2023): the loss only constrains h∘fh f, so the backbone f is free to develop any geometry the projection head h can absorb. RINO’s invariance is therefore non-trivial: the head realizes invariance on top of a scale-aware backbone. By contrast, the JetCLR-scale backbone is itself nearly scale-invariant (all cosines ≥0.89≥ 0.89). For OOD top tagging, the 8.6%8.6\% accuracy gap between RINO and JetCLR-scale (same views, same architecture) suggests that scale-aware backbones outperform scale-invariant ones. Preserving scale information matters: for example, the top-tagging discriminative signal is how jet substructure varies with scale. Top-quark jets resolve into three prongs at N∼3N\! \!3 and develop characteristic soft-radiation patterns at relatively fine scales, whereas QCD jets do neither. 5.2 Why RINO Generalizes OOD To measure the JetNet→ domain shift in representation space, we compute the 1-Wasserstein distance (W1W_1) and squared maximum mean discrepancy (MMD2) between JetNet and JetClass finetuned embeddings, separately for QCD and top-quark jets (implementation details in App. A.16). Both metrics quantify how far source-domain finetuned representations sit from the target distribution. Table 4 shows that RINO has the smallest W1W_1 and MMD2 of all methods (W1=0.043W_1=0.043 and MMD2=0.007^2=0.007 for QCD, versus at least 0.1320.132 and 0.0450.045 for the baselines), alongside the highest OOD accuracy. RINO’s shift is already small before finetuning and does not grow during finetuning (App. A.16). Table 4: Domain shift in finetuned representations (JetNet vs. JetClass). Lower W1W_1 and MMD2 indicate more domain-invariant representations. Method W1W_1 (QCD) ↓ W1W_1 (top) ↓ MMD2 (QCD) ↓ MMD2 (top) ↓ OOD ACC ↑ Supervised 0.7460.746 0.7280.728 0.5150.515 0.5640.564 0.786±0.0230.786± 0.023 JetCLR 0.1320.132 0.1270.127 0.1960.196 0.1910.191 0.700±0.0310.700± 0.031 MPMv1 0.2410.241 0.2620.262 0.1330.133 0.1620.162 0.800±0.0190.800± 0.019 MPMv2 0.3750.375 0.3480.348 0.3380.338 0.3050.305 0.760±0.0160.760± 0.016 OmniJet-α 0.8150.815 0.7520.752 0.5710.571 0.5520.552 0.751±0.0500.751± 0.050 JetCLR-scale 0.1330.133 0.1450.145 0.0450.045 0.0480.048 0.780±0.0230.780± 0.023 RINO 0.0430.043 0.0440.044 0.0070.007 0.0070.007 0.847±0.0030.847± 0.003 5.3 Limitations First, our simulation-to-simulation proxy (JetNet→ ) may not faithfully reflect the actual simulation-to-data shift. Appendix A.13 outlines the intended workflow on experimental data, but direct real-data validation requires involved physics analyses (e.g., control-region selection, tag-and-probe) and is beyond this work’s scope. Second, SSL methods pretrain on JetClass QCD, so pretraining partially overlaps with the test domain; the genuine OOD challenge comes from unseen signal topologies and the mismatch in simulation settings between JetNet and JetClass QCD. Third, the kTk_T inductive bias is task-dependent: RINO’s advantage is largest for multi-prong substructure (t→bqqt→ bqq, H→4qH→ 4q) and narrows for tasks driven by other signatures (e.g., H→bb¯H→ b b). 6 Conclusion RINO constructs SSL views at multiple kTk_T clustering scales of the same jet and trains via DINO-style self-distillation. To simulate real-data conditions, we pretrain on JetClass QCD only and finetune on the smaller labeled JetNet before evaluating OOD on JetClass. RINO outperforms baselines on OOD top tagging (Table 1), matches supervised training with 100×100× fewer labels (Fig. 2), and transfers without retraining to unseen H→4qH→ 4q jets (Table 2). A factorization study (Fig. 3) attributes this to both multi-scale view alignment and self-distillation, the latter preserving scale-aware backbone geometry (Sec. 5.1). AI use statement We used generative AI tools for the four purposes selected in the submission form, detailed below together with how the authors checked the results. Aiding or polishing writing. We used generative AI tools to edit the paper for readability and to format references. Every AI-suggested edit was shown to the authors as a diff and applied only after their review, modification, and approval. Retrieval and discovery. We used generative AI tools to search for and identify references. The authors required the tools to quote the supporting passage of each source verbatim, verified these quotes against the source, and checked that each citation supports the claim it is attached to. Every reference entry was checked against its arXiv, INSPIRE, or DOI record. Research ideation or execution. We used generative AI tools to: • execute and provide feedback on experiments; • implement methods in a limited capacity, namely small fixes, bug fixes, and speed-only training optimizations to the authors’ code in https://github.com/zichunhao/RINO, adapting the open-source baseline implementations to our shared training and evaluation framework, and adapting JetCLR to the kTk_T-view JetCLR-scale variant; • fix bugs in the data-preprocessing code; • assist in interpreting results and in checking interpretations, including writing scripts to compute summary statistics (means and standard deviations); • create and edit software code (e.g., scripts, job configurations, and plotting code); and • create and modify scientific figures. The core RINO implementation and the preprocessing pipeline were written by the authors and were largely complete before generative AI tools were used in this project. AI-assisted code, including plotting code, was reviewed and tested by the authors and validated through training and evaluation runs. Derived numbers (e.g., relative gains and test statistics) were recomputed with scripts rather than by hand and checked against the experiment outputs, and implementation details stated in the paper were checked against the code and configuration files. Methodology and results were discussed in weekly group meetings, so that multiple authors reviewed them. We have not used generative AI tools to propose or refine hypotheses, develop theoretical models or conceptual frameworks, or assist with translation; generating synthetic datasets, formulating or proving mathematical claims, and qualitative or thematic data analysis are not applicable to this work. Drafting sections of the paper. We used generative AI tools to draft parts of the text, which the authors substantially rewrote. Every drafted change was reviewed and approved by the authors before inclusion, and multiple authors then independently reviewed the full manuscript or its core parts. Review and responsibility. The authors have reviewed all AI-assisted work as detailed above. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI. Ethics statement RINO aims to reduce reliance on MC simulation by enabling pretraining on real, unlabeled collider data, with no direct societal risks specific to this work. Reproducibility statement Code is available at https://github.com/zichunhao/RINO. Appendix A.1 lists the pretraining hyperparameters, App. A.2 the loss terms, App. A.3 the architecture and input features, App. A.4 the datasets and preprocessing, App. A.6 the finetuning protocol, App. A.17 the baseline implementations, and App. A.18 the compute resources. All datasets are public (Qu et al., 2022a; Kansal et al., 2021; Kasieczka et al., 2019b). Acknowledgments We thank Yue Ma and Aobo Li for thoughtful feedback on the experimental design and presentation. We are grateful to Joseph Lykken for valuable discussions and insightful comments that sharpened the narrative of this work. Z.H., R.K., C.S., and M.S. are supported by the California Institute of Technology High Energy Physics under Contract DE-SC0011925 with the United States Department of Energy, Office of High Energy Physics. Z.H. would like to thank the Fermilab LPC Guests and Visitors program. A.G., J.N., and J.D. are supported by the DOE Office of Science, Award No. DE-SC0023524, FermiForward Discovery Group, LLC under Contract No. 89243024CSC000002 with the U.S. Department of Energy, Office of Science, Office of High Energy Physics, LDRD L2024-066-1, Fermilab, DOE Office of Science, Office of High Energy Physics “Designing efficient edge AI with physics phenomena” Project (DE-FOA-0002705), DOE Office of Science, Office of Advanced Scientific Computing Research under the “Real-time Data Reduction Codesign at the Extreme Edge for Science” Project (DE-FOA-0002501). J.D. is also supported by the Research Corporation for Science Advancement (RCSA) under grant #CS-CSA-2023-109, Alfred P. Sloan Foundation under grant #FG-2023-20452, DOE, Office of Science, Office of High Energy Physics Early Career Research program under Award No. DE-SC0021187, DOE, Office of Science, Office of High Energy Physics under Award No. DE-SC0009919, and the U.S. National Science Foundation (NSF) Harnessing the Data Revolution (HDR) Institute for Accelerating AI Algorithms for Data Driven Discovery (A3D3) under Cooperative Agreement PHY-2117997. R.K. and J.N. are also supported by the AI2050 program at Schmidt Futures (Grant G-23-64934). This work was performed using the National Research Platform Nautilus HyperCluster supported by NSF awards CNS-1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2112167, CNS-2100237, CNS-2120019, the University of California Office of the President, and the University of California San Diego’s California Institute for Telecommunications and Information Technology/Qualcomm Institute. References Andersson (1987) B. Andersson The Lund Model. Nucl. Phys. A 461, p. 513C. External Links: Document Cited by: §1. Asano et al. (2020) Y. M. Asano, C. Rupprecht, and A. Vedaldi A critical analysis of self-supervision, or what we can learn from a single image. In International Conference on Learning Representations, External Links: 1904.13132, Link Cited by: §1. Assran et al. (2023) M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y. LeCun, and N. Ballas Self-supervised learning from images with a joint-embedding predictive architecture. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 15619. External Links: 2301.08243, Link, Document Cited by: §2. Bardes et al. (2022) A. Bardes, J. Ponce, and Y. LeCun VICReg: variance-invariance-covariance regularization for self-supervised learning. In International Conference on Learning Representations, External Links: Link Cited by: §2. Bardhan et al. (2025) J. Bardhan, R. Agrawal, A. Tilak, C. Neeraj, and S. Mitra HEP-JEPA: A foundation model for collider physics using joint embedding predictive architecture. Note: External Links: 2502.03933 Cited by: §A.17, §1, §2. Barnard et al. (2017) J. Barnard, E. N. Dawe, M. J. Dolan, and N. Rajcic Parton Shower Uncertainties in Jet Substructure Analyses with Deep Neural Networks. Phys. Rev. D 95 (1), p. 014018. External Links: 1609.00607, Document Cited by: §A.14, §A.16. Battaglia et al. (2016) P. W. Battaglia, R. Pascanu, M. Lai, D. Rezende, and K. Kavukcuoglu Interaction networks for learning about objects, relations and physics. In Advances in Neural Information Processing Systems, D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (Eds.), Vol. 29. External Links: 1612.00222, Link Cited by: §1, §2. Beirlant et al. (1997) J. Beirlant, E. J. Dudewicz, L. Györfi, and E. C. van der Meulen Nonparametric entropy estimation: an overview. International Journal of Mathematical and Statistical Sciences 6 (1), p. 17–39. Cited by: §3.3. Berendsen (2007) H. J. C. Berendsen Coarse graining from particles to fluid dynamics. In Simulating the Physical World: Hierarchical Modeling from Quantum Mechanics to Fluid Dynamics, p. 279–296. Cited by: §1. Birk et al. (2024) J. Birk, A. Hallin, and G. Kasieczka OmniJet-α: the first cross-task foundation model for particle physics. Mach. Learn. Sci. Tech. 5, p. 035031. External Links: Document, 2403.05618 Cited by: §A.17, Table 14, §1, §2, Table 1, Table 2. Bogatskiy et al. (2024) A. Bogatskiy, T. Hoffman, D. W. Miller, J. T. Offermann, and X. Liu Explainable equivariant neural networks for particle physics: PELICAN. JHEP 03, p. 113. External Links: 2307.16506, Document Cited by: §1, §2. Bonilla et al. (2022) J. Bonilla, G. Chachamis, B. M. Dillon, S. V. Chekanov, R. Erbacher, L. Gouskos, A. Hinzmann, S. Höche, B. T. Huffman, A. V. Kotwal, D. Kar, R. Kogler, C. Lange, M. LeBlanc, R. Lemmon, C. McLean, B. Nachman, M. S. Neubauer, T. Plehn, S. Rappoccio, D. Roy, J. Roloff, G. Stark, N. Tran, M. Vos, C. Yeh, and S. Yu Jets and Jet Substructure at Future Colliders. Frontiers in Physics 10, p. 897719. External Links: 2203.07462, Document Cited by: §2. Bordes et al. (2023) F. Bordes, R. Balestriero, Q. Garrido, A. Bardes, and P. Vincent Guillotine regularization: why removing layers is needed to improve generalization in self-supervised learning. Transactions on Machine Learning Research (TMLR). External Links: 2206.13378, Link Cited by: §5.1. Brehmer et al. (2025) J. Brehmer, V. Bresó, P. de Haan, T. Plehn, H. Qu, J. Spinner, and J. Thaler A Lorentz-equivariant transformer for all of the LHC. SciPost Phys. 19 (4), p. 108. External Links: 2411.00446, Document Cited by: §1, §2. Cacciari et al. (2008) M. Cacciari, G. P. Salam, and G. Soyez The anti-ktk_t jet clustering algorithm. JHEP 04, p. 063. External Links: 0802.1189, Document Cited by: §4.5. Cacciari et al. (2012) M. Cacciari, G. P. Salam, and G. Soyez FastJet User Manual. Eur. Phys. J. C 72, p. 1896. External Links: 1111.6097, Document Cited by: §A.4, §3.2. Calzetta et al. (1992) E. Calzetta, M. Castagnino, and R. Scoccimarro Coarse graining approach to quantum cosmology. Phys. Rev. D 45, p. 2806–2813. External Links: Document Cited by: §1. Caron et al. (2020) M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin Unsupervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 9912. External Links: 2006.09882, Link Cited by: §A.2, §2, §3.3. Caron et al. (2021) M. Caron, H. Touvron, I. Misra, H. Jegou, J. Mairal, P. Bojanowski, and A. Joulin Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 9630–9640. External Links: Document Cited by: §A.2, §1, §2, §2, §3.2. Catani et al. (1993) S. Catani, Yu.L. Dokshitzer, M.H. Seymour, and B.R. Webber Longitudinally-invariant kTk_T-clustering algorithms for hadron-hadron collisions. Nucl. Phys. B 406, p. 187. External Links: ISSN 0550-3213, Document Cited by: §A.5, §1, §2. Chen et al. (2020) T. Chen, S. Kornblith, M. Norouzi, and G. Hinton A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, H. D. I and A. Singh (Eds.), Vol. 119, p. 1597. External Links: 2002.05709, Link Cited by: 3rd item, §2. CMS Collaboration (2025a) CMS Collaboration A method for correcting the substructure of multiprong jets using the Lund jet plane. JHEP 11, p. 038. External Links: 2507.07775, Document Cited by: §2, §4.5. CMS Collaboration (2025b) CMS Collaboration Calibration of the top and W jet tagging efficiency in 13 TeV data collected by the CMS experiment in 2016–2018. External Links: Link Cited by: §A.13, §1. CMS Collaboration (2025c) CMS Collaboration Performance of heavy-flavour jet identification in Lorentz-boosted topologies in proton-proton collisions at s=13 s=13 TeV. Journal of Instrumentation 20 (11), p. P11006. External Links: Document, Link Cited by: §1. Darcet et al. (2024) T. Darcet, M. Oquab, J. Mairal, and P. Bojanowski Vision transformers need registers. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §A.3. Delattre and Fournier (2017) S. Delattre and N. Fournier On the Kozachenko–Leonenko entropy estimator. Journal of Statistical Planning and Inference 185, p. 69–93. Cited by: §3.3. Devlin et al. (2019) J. Devlin, M. Chang, K. Lee, and K. Toutanova BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio (Eds.), Minneapolis, Minnesota, p. 4171–4186. External Links: Link, Document Cited by: §2. Dillon et al. (2022) B. M. Dillon, G. Kasieczka, H. Olischlager, T. Plehn, P. Sorrenson, and L. Vogel Symmetries, safety, and self-supervision. SciPost Phys. 12, p. 188. External Links: 2108.04253, Document Cited by: §A.17, Table 14, §1, §2, Table 1, Table 2. Dokshitzer et al. (1997) Yu.L. Dokshitzer, G.D. Leder, S. Moretti, and B.R. Webber Better jet clustering algorithms. Journal of High Energy Physics 1997 (08), p. 001. External Links: Document, Link Cited by: §4.5. Frisch (1995) U. Frisch Turbulence: the legacy of a. n. kolmogorov. Cambridge University Press. Cited by: §1. Golling et al. (2024) T. Golling, L. Heinrich, M. Kagan, S. Klein, M. Leigh, M. Osadchy, and J. A. Raine Masked particle modeling on sets: towards self-supervised high energy physics foundation models. Mach. Learn. Sci. Tech. 5, p. 035074. External Links: 2401.13537, Document Cited by: §A.17, Table 10, Table 14, §1, §2, §3.3, Table 1, Table 2. Gong et al. (2022) S. Gong, Q. Meng, J. Zhang, H. Qu, C. Li, S. Qian, W. Du, Z. Ma, and T. Liu An efficient Lorentz equivariant graph neural network for jet tagging. Journal of High Energy Physics 2022 (7). External Links: ISSN 1029-8479, Link, Document Cited by: §1, §2. Grinstein (2024) F. F. Grinstein Coarse grained simulation and dynamic bridging for turbulent mixing predictions. Computers & Fluids 271, p. 106185. External Links: ISSN 0045-7930, Document, Link Cited by: §1. Gui et al. (2024) J. Gui, T. Chen, J. Zhang, Q. Cao, Z. Sun, H. Luo, and D. Tao A survey on self-supervised learning: algorithms, applications, and future trends. IEEE Trans. Pattern Anal. Mach. Intell. 46, p. 9052. External Links: 2301.05712, Document Cited by: §1. Harris et al. (2025) P. Harris, J. Krupa, M. Kagan, B. Maier, and N. Woodward Resimulation-based self-supervised learning for pretraining physics foundation models. Phys. Rev. D 111, p. 032010. External Links: 2403.07066, Document Cited by: §2. He et al. (2020) K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vol. , p. 9726–9735. External Links: Document Cited by: §2, §3.3. Hendrycks et al. (2019) D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. External Links: 1906.12340, Link Cited by: §1. Holt (2004) C. C. Holt Forecasting seasonals and trends by exponentially weighted moving averages. Int. J. Forecast. 20, p. 5. External Links: Document Cited by: §3.3. Ingólfsson et al. (2014) H. I. Ingólfsson, C. A. Lopez, J. J. Uusitalo, D. H. de Jong, S. M. Gopal, X. Periole, and S. J. Marrink The power of coarse graining in biomolecular simulations. WIREs Computational Molecular Science 4 (3), p. 225–248. External Links: Document Cited by: §1. Ivanov (2024) M. M. Ivanov Effective field theory for large-scale structure. In Handbook of Quantum Gravity, C. Bambi, L. Modesto, and I. Shapiro (Eds.), p. 213–260. External Links: ISBN 978-981-99-7681-2, Document, Link Cited by: §1. Kansal et al. (2021) R. Kansal, J. Duarte, H. Su, B. Orzari, T. Tomei, M. Pierini, M. Touranakou, J. Vlimant, and D. Gunopulos Particle cloud generation with message passing generative adversarial networks. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. W. Vaughan (Eds.), Vol. 34, p. 23858. External Links: Link, 2106.11535 Cited by: §A.4, §3.1, §6. Kansal et al. (2023) R. Kansal, C. Pareja, Z. Hao, and J. Duarte JetNet: A Python package for accessing open datasets and benchmarking machine learning methods in high energy physics. Journal of Open Source Software 8 (90), p. 5789. External Links: Document, Link Cited by: §A.4. Kasieczka et al. (2019a) G. Kasieczka, T. Plehn, A. Butter, K. Cranmer, D. Debnath, B. M. Dillon, M. Fairbairn, D. A. Faroughy, W. Fedorko, C. Gay, L. Gouskos, J. F. Kamenik, P. Komiske, S. Leiss, A. Lister, S. Macaluso, E. Metodiev, L. Moore, B. Nachman, K. Nordström, J. Pearkes, H. Qu, Y. Rath, M. Rieger, D. Shih, J. Thompson, and S. Varma The machine learning landscape of top taggers. SciPost Physics 7 (1). External Links: ISSN 2542-4653, Link, Document Cited by: §A.11, §A.4. Kasieczka et al. (2019b) G. Kasieczka, T. Plehn, J. Thompson, and M. Russel Top Quark Tagging Reference Dataset. Zenodo. External Links: Document, Link Cited by: §A.11, §A.4, §6. Katel et al. (2024) S. Katel, H. Li, Z. Zhao, R. Kansal, F. Mokhtar, and J. Duarte Learning symmetry-independent jet representations via jet-based joint embedding predictive architecture. External Links: 2412.05333, Link Cited by: §A.17, §1, §2. Kmiecik et al. (2016) S. Kmiecik, D. Gront, M. Kolinski, L. Wieteska, A. E. Dawid, and A. Kolinski Coarse-grained protein models and their applications. Chemical Reviews 116 (14), p. 7898–7936. External Links: ISSN 0009-2665, Document, Link Cited by: §1. Kogler et al. (2019) R. Kogler, B. Nachman, A. Schmidt, L. Asquith, E. Winkels, M. Campanelli, C. Delitzsch, P. Harris, A. Hinzmann, D. Kar, C. McLean, J. Pilot, Y. Takahashi, N. Tran, C. Vernieri, and M. Vos Jet substructure at the Large Hadron Collider. Reviews of Modern Physics 91 (4). External Links: ISSN 1539-0756, Link, Document Cited by: §2. Komiske et al. (2019) P. T. Komiske, E. M. Metodiev, and J. Thaler Energy Flow Networks: Deep Sets for Particle Jets. JHEP 01, p. 121. External Links: 1810.05165, Document Cited by: §1, §2. Krause et al. (2026) C. Krause, R. Winterhalder, M. Feickert, B. Nachman, J. Raine, and HEP ML Community A living review of machine learning for particle physics. Zenodo. External Links: Document, Link Cited by: §1, §2. Kumar et al. (2022) A. Kumar, A. Raghunathan, R. Jones, T. Ma, and P. Liang Fine-tuning can distort pretrained features and underperform out-of-distribution. In International Conference on Learning Representations, External Links: 2202.10054, Link Cited by: §A.16, §A.6, §4.1, §4.3. Leigh et al. (2025) M. Leigh, S. Klein, F. Charton, T. Golling, L. Heinrich, M. Kagan, I. Ochoa, and M. Osadchy Is tokenization needed for masked particle modeling?. Mach. Learn. Sci. Tech. 6, p. 025075. External Links: 2409.12589, Document Cited by: §A.17, Table 10, Table 14, §1, §2, §3.3, Table 1, Table 2. Li et al. (2024) C. Li et al. Accelerating Resonance Searches via Signature-Oriented Pre-training. External Links: 2405.12972, Link Cited by: §2. Liu et al. (2021) Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo Swin Transformer: hierarchical vision transformer using shifted windows. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 9992–10002. External Links: Document Cited by: §2. Mikuni and Nachman (2025a) V. Mikuni and B. Nachman Method to simultaneously facilitate all jet physics tasks. Phys. Rev. D 111 (5), p. 054015. External Links: 2502.14652, Document Cited by: §2. Mikuni and Nachman (2025b) V. Mikuni and B. Nachman Solving key challenges in collider physics with foundation models. Phys. Rev. D 111, p. L051504. External Links: 2404.16091, Document Cited by: §2. Mokhtar et al. (2025) F. Mokhtar, J. Pata, D. Garcia, E. Wulff, M. Zhang, M. Kagan, and J. Duarte Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders. Phys. Rev. D 111 (9), p. 092015. External Links: 2503.00131, Document Cited by: §2. Moreno et al. (2020) E. A. Moreno, T. Q. Nguyen, J. Vlimant, O. Cerri, H. B. Newman, A. Periwal, M. Spiropulu, J. M. Duarte, and M. Pierini Interaction networks for the identification of boosted H→bb¯H→ b b decays. Phys. Rev. D 102 (1), p. 012010. External Links: 1909.12285, Document Cited by: §1. Nagy and Soper (2018) Z. Nagy and D. E. Soper What is a parton shower?. Phys. Rev. D 98, p. 014034. External Links: Document, Link Cited by: §A.14, §A.16, §2. Nagy and Soper (2022) Z. Nagy and D. E. Soper Multivariable evolution in final state parton shower algorithms. Phys. Rev. D 105, p. 054012. External Links: Document, Link Cited by: §1, §2. Oquab et al. (2024) M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P. Huang, S. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski DINOv2: learning robust visual features without supervision. Trans. Mach. Learn. Res.. External Links: 2304.07193, Link Cited by: §A.2, §A.3, §1, §2, §3.2, §3.3. Pietroni et al. (2012) M. Pietroni, G. Mangano, N. Saviano, and M. Viel Coarse-Grained Cosmological Perturbation Theory. JCAP 01, p. 019. External Links: 1108.5203, Document Cited by: §1. Qu and Gouskos (2020) H. Qu and L. Gouskos Jet tagging via particle clouds. Physical Review D 101 (5). External Links: ISSN 2470-0029, Link, Document Cited by: §1, §2. Qu et al. (2022a) H. Qu, C. Li, and S. Qian JetClass: a large-scale dataset for deep learning in jet physics. Zenodo. External Links: Document Cited by: §A.4, §3.1, §6. Qu et al. (2022b) H. Qu, C. Li, and S. Qian Particle Transformer for Jet Tagging. In Proceedings of the 39th International Conference on Machine Learning, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, and S. Sabato (Eds.), Vol. 162, p. 18281. External Links: 2202.03772, Link Cited by: §A.4, §1, §2, §3.3. Ren et al. (2023) Y. Ren, S. Guo, W. Bae, and D. J. Sutherland How to prepare your task head for finetuning. In International Conference on Learning Representations, External Links: 2302.05779, Link Cited by: §A.6. Sablayrolles et al. (2019) A. Sablayrolles, M. Douze, C. Schmid, and H. Jégou Spreading vectors for similarity search. In International Conference on Learning Representations, External Links: 1806.03198, Link Cited by: §A.2, §3.3. Salam (2010) G. P. Salam Towards jetography. The European Physical Journal C 67 (3-4), p. 637–686. External Links: ISSN 1434-6052, Link, Document Cited by: §A.14, §A.16, §2. Shazeer (2020) N. Shazeer GLU variants improve transformer. External Links: 2002.05202, Link Cited by: §A.3. Siméoni et al. (2025) O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, F. Massa, D. Haziza, L. Wehrstedt, J. Wang, T. Darcet, T. Moutakanni, L. Sentana, C. Roberts, A. Vedaldi, J. Tolan, J. Brandt, C. Couprie, J. Mairal, H. Jégou, P. Labatut, and P. Bojanowski DINOv3. External Links: 2508.10104, Link Cited by: §1, §3.2. Sinkhorn and Knopp (1967) R. Sinkhorn and P. Knopp Concerning nonnegative matrices and doubly stochastic matrices.. Pacific Journal of Mathematics 21 (2), p. 343–348. External Links: Document Cited by: §A.2, §3.3. Spinner et al. (2024) J. Spinner, V. Bresó, P. de Haan, T. Plehn, J. Thaler, and J. Brehmer Lorentz-Equivariant Geometric Algebra Transformers for High-Energy Physics. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 22178. External Links: 2405.14806, Link Cited by: §1, §2. Tarvainen and Valpola (2017) A. Tarvainen and H. Valpola Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4–9, 2017, Long Beach, CA, USA, I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett (Eds.), p. 1195–1204. External Links: Link Cited by: §3.3. Thaler and Van Tilburg (2011) J. Thaler and K. Van Tilburg Identifying Boosted Objects with N-subjettiness. JHEP 03, p. 015. External Links: 1011.2268, Document Cited by: §2. Touvron et al. (2021) H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve, and H. Jegou Going deeper with Image Transformers. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , Los Alamitos, CA, USA, p. 32–42. External Links: ISSN , Document, Link Cited by: §A.3. Tsoi and Rankin (2026) H. F. Tsoi and D. Rankin jBOT: semantic jet representation clustering emerges from self-distillation. SciPost Physics 21 (3). External Links: ISSN 2542-4653, Link, Document Cited by: §2. van den Oord et al. (2017) A. van den Oord, O. Vinyals, and K. Kavukcuoglu Neural discrete representation learning. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30, p. . External Links: Link Cited by: §2. Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin Attention is all you need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. External Links: 1706.03762, Link Cited by: §3.3. Wang et al. (2019) J. Wang, S. Olsson, C. Wehmeyer, A. Pérez, N. E. Charron, G. de Fabritiis, F. Noé, and C. Clementi Machine learning of coarse-grained molecular dynamics force fields. ACS Central Science 5 (5), p. 755–767. Note: PMID: 31139712 External Links: Document, Link, https://doi.org/10.1021/acscentsci.8b00913 Cited by: §1. Zbontar et al. (2021) J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny Barlow Twins: self-supervised learning via redundancy reduction. In Proceedings of the 38th International Conference on Machine Learning, M. Meila and T. Zhang (Eds.), Proceedings of Machine Learning Research, Vol. 139, p. 12310–12320. External Links: Link Cited by: §2. Zhang and Sennrich (2019) B. Zhang and R. Sennrich Root mean square layer normalization. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §A.3. Zhang et al. (2019) L. Zhang, J. Song, A. Gao, J. Chen, C. Bao, and K. Ma Be your own teacher: improve the performance of convolutional neural networks via self distillation. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Vol. , p. 3712–3721. External Links: Document Cited by: §1, §2. Zhao et al. (2024) Z. Zhao, F. Mokhtar, R. Kansal, H. Li, and J. Duarte Large-Scale Pretraining and Finetuning for Efficient Jet Classification in Particle Physics. In 22nd International Workshop on Advanced Computing and Analysis Techniques in Physics Research, External Links: 2408.09343 Cited by: §1, §2. Zhou et al. (2022) J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong iBOT: image BERT pre-training with online tokenizer. International Conference on Learning Representations (ICLR). Cited by: §A.2, §2. Appendix A Appendix The code is provided in https://github.com/zichunhao/RINO. A.1 Pretraining Hyperparameters Table 5 summarizes the hyperparameters used for RINO pretraining on the QCD-jet subset of JetClass. The teacher EMA momentum and the learning-rate scheduler follow cosine schedules over the full 100100 training epochs, with warmup as indicated. The pretraining recipe was held fixed across all RINO ablation variants reported in Sec. 4.5, with only the view-construction parameters and the input positional encoding varied between variants. The headline RINO model in Table 1 (0.847±0.0030.847± 0.003) uses a tuned teacher schedule, a fixed teacher temperature and a lower final EMA momentum (Table 5), chosen by the OOD top-vs-QCD accuracy on JetClass; this step contributes the final +1.0%+1.0\% row of Table 3, and RINO before tuning (0.839±0.0050.839± 0.005) already exceeds every baseline (App. A.17). Values listed in Table 5 below are those of this final configuration. Table 5: RINO pretraining hyperparameters (production configuration). Group Hyperparameter Value Optimization Optimizer AdamW (β1=0.9 _1=0.9, β2=0.999 _2=0.999) Peak learning rate 3×10−33× 10^-3 Weight decay 0.050.05 Gradient clipping (max norm) 3.03.0 Schedule Total epochs 100100 LR warmup epochs 00–1919, 10−6→1.010^-6→ 1.0 (cosine) LR main schedule epochs 2020–9999, 1.0→0.051.0→ 0.05 (cosine) Layer-wise LR decay 0.90.9 per layer (backbone) Batching Total batch size 30003000 jets Per-GPU batch size 15001500 jets (22 GPUs, data parallel) DINO loss Student temperature τs _s 0.100.10 (fixed) Teacher temperature τt _t 0.040.04 (fixed) ablation variants 0.04→0.070.04→ 0.07, epochs 00–1919 (cosine) Teacher centering Sinkhorn–Knopp Loss weight λDINO=1.0 _DINO=1.0 iBOT loss Mask ratio 0.250.25 (unclustered view) Mask embedding learnable dmodeld_model-dim vector Loss weight λiBOT=1.0 _iBOT=1.0 KoLeo regularizer Applied to student [CLS] embeddings (teacher-scale views) Loss weight λKoLeo=0.1 _KoLeo=0.1 Teacher EMA Update rule θ¯←mθ¯+(1−m)θ θ← m\, θ+(1-m)\,θ, per batch Momentum schedule m:0.995→0.997m:0.995→ 0.997 over 100100 epochs (cosine) ablation variants m:0.995→0.999m:0.995→ 0.999 over 100100 epochs (cosine) A.2 Pretraining Loss Terms This section writes out the three terms of Eq. 2; hyperparameter values are listed in Table 5. For a view x of a jet, Ps(x)P_s(x) and Pt(x)P_t(x) denote the student and teacher output distributions over the K prototypes of a projection head (App. A.3), obtained with temperatures τs _s and τt _t. The teacher distribution is normalized with Sinkhorn–Knopp (Sinkhorn and Knopp, 1967; Caron et al., 2020) and receives no gradient. We write the cross-entropy as H(a,b)=−∑k=1Kaklogbk,H(a,b)=- _k=1^Ka_k b_k, (3) and each loss term below is averaged over the jets in a mini-batch. DINO term. Let x(v)x^(v) be the jet at view v, =6,8,16T=\6,8,16\ the teacher views, and S the student views (the teacher views plus three student-only views sampled per batch from 2,3,4,uncl.\2,3,4,uncl.\). Using the DINO head on the [CLS] output, ℒDINO=1||∑(t,s)∈H(Pt(x(t)),Ps(x(s))),=(t,s)∈×:s≠t,L_DINO= 1|P| _(t,s) H (P_t(x^(t)),\,P_s(x^(s)) ), =\(t,s) ×S:s≠ t\, (4) so each student view is aligned with every teacher view (Caron et al., 2021) at a different scale. iBOT term. The iBOT term acts on the unclustered view x only (App. A.8). A random set ℳM of 25%25\% of the particles is masked in the student’s input, giving x~ x, and the teacher sees the unmasked jet. Using the iBOT head on the output token of each masked particle i (Zhou et al., 2022), ℒiBOT=1|ℳ|∑i∈ℳH(Pt(x)i,Ps(x~)i).L_iBOT= 1|M| _i H (P_t(x)_i,\,P_s( x)_i ). (5) KoLeo term. Following DINOv2 (Oquab et al., 2024; Sablayrolles et al., 2019), for the ℓ2 _2-normalized student [CLS] representations z^1,…,z^B z_1,…, z_B of the B jets in a mini-batch at the same teacher-scale view, ℒKoLeo=−1B∑i=1Blog(minj≠i∥z^i−z^j∥2),L_KoLeo=- 1B _i=1^B ( _j≠ i\, z_i- z_j _2 ), (6) averaged over teacher-scale views. It spreads different jets within a view and never compares two views of the same jet. A.3 Architecture Details Backbone. The shared transformer encoder backbone has 88 layers with hidden dimension dmodel=256d_model=256, 1616 attention heads, and a feed-forward expansion ratio of 44 (FFN hidden dimension 10241024), totaling 8.48.4M trainable parameters (plus 2.62.6M parameters in the polar-binned positional encoding lookup table). Activations are SwiGLU (Shazeer, 2020) in the FFN sub-layers, normalization is RMSNorm (Zhang and Sennrich, 2019) applied pre-residual, and residual paths use LayerScale (Touvron et al., 2021) with learnable per-channel scaling initialized to 0.010.01. A learnable [CLS] token is prepended to each input sequence and serves as the global jet representation. Following Darcet et al. (2024), we additionally append four learnable register tokens that are not used as outputs but stabilize the attention maps. Variable-length particle sequences are handled with attention masking; the maximum sequence length is 9090 particles per jet (the 9090 highest-pTp_T particles are kept). Input features (per particle). Each constituent is represented by the seven features listed in Table 6. All features are pre-standardized through a fixed affine map x↦(x−μ)/σx (x-μ)/σ. The per-feature centering μ and scale σ are estimated once from the median and interquartile range (IQR) of the corresponding distribution on the JetClass QCD pretraining set, rather than from the empirical mean and standard deviation; this robust estimator is insensitive to the long tails typical of jet kinematic spectra. The resulting (μ,σ)(μ,σ) values are held fixed during both training and inference, and used unchanged on all downstream datasets, so that the same backbone can be applied without renormalizing the input distribution. Quantities quoted in (Δη,Δϕ)( η, φ) space elsewhere in this appendix (notably the radial parameters of the positional encoding) are in these standardized units, so for example Δη=1 η=1 corresponds to a raw rapidity offset of 0.140.14. Table 6: Per-particle input features and the fixed standardization (μ,σ)(μ,σ) applied by the input pipeline. The model receives (x−μ)/σ(x-μ)/σ for each feature. Feature Definition μ σ logpT p_T logpT,i p_T,i, constituent transverse momentum 1.71.7 1.81.8 logE E logEi E_i, constituent energy 2.02.0 1.81.8 log(pT/pTjet) (p_T/p_T^jet) log(pT,i/pTjet) (p_T,i/p_T^jet), relative pTp_T −4.7-4.7 1.81.8 log(E/Ejet) (E/E^jet) log(Ei/Ejet) (E_i/E^jet), relative energy −4.7-4.7 1.81.8 Δη η ηi−ηjet _i- _jet, rapidity offset 0.00.0 0.140.14 Δϕ φ ϕi−ϕjet _i- _jet (wrapped to [−π,π)[-π,π)), azimuthal offset 0.00.0 0.140.14 ΔR R (Δηi)2+(Δϕi)2 ( _i)^2+( _i)^2, angular distance to jet axis 0.140.14 0.250.25 Positional encoding. The polar-binned positional encoding maps each constituent’s (Δη,Δϕ)( η, φ) position to a learned embedding via a ×100100\!×\!100 angular grid (radial × azimuthal bins) with logarithmic radial spacing centered on the jet axis. Concretely, the radial coordinate r=(Δη)2+(Δϕ)2r= ( η)^2+( φ)^2 is mapped through log(1+r/rscale) (1+r/r_scale) before quantization into 100100 bins on the interval [0,log(1+rmax/rscale)][0,\, (1+r_max/r_scale)], while the azimuthal coordinate α=arctan2(Δϕ,Δη)α=arctan2( φ, η) is binned uniformly on [−π,π)[-π,π). The two radial parameters are not free: rscaler_scale is set to the median of the per-constituent radial distribution on the pretraining set (so half of all constituents fall in the high-resolution region near the knee of the log compression), and rmaxr_max is set to the 99th percentile of the same distribution (so extreme tail constituents are clamped rather than allowed to consume a large share of bins). The two radial parameters are quoted in the standardized (Δη,Δϕ)( η, φ) units of Table 6; on the JetClass QCD pretraining set this gives rscale≈0.96r_scale≈ 0.96 and rmax≈5.60r_max≈ 5.60, corresponding to ∼0.13 0.13 and ∼0.78 0.78 in raw (Δη,Δϕ)( η, φ) space, within the R=0.8R=0.8 jet cone. The combination of polar parameterization and log radial binning therefore concentrates resolution where the constituent density is highest, by construction. Bin collisions matter because the role of the positional encoding in an encoder-only iBOT setup is precisely to break what would otherwise be a complete degeneracy among masked constituents. With a generative decoder (as in MAE-style methods), positional queries fed to the decoder disambiguate which masked token to reconstruct. With our encoder-only architecture, the only signal a masked token carries is its mask embedding plus its positional embedding; without PE, all masked tokens are identical and iBOT cannot distinguish them. A bin collision restores this degeneracy locally: two masked constituents that fall in the same bin become indistinguishable. Empirically, adding polar-binned PE helps with training stability and improves OOD accuracy from 0.8280.828 to 0.8390.839 on cluster views (Table 12), consistent with reduced bin collisions aiding iBOT training. We measure, for each candidate scheme, the per-particle collision rate (fraction of real constituents sharing a bin with at least one other constituent of the same jet) and the per-jet collision rate (fraction of jets containing at least one such collision). The polar grid is markedly more bin-efficient than a Cartesian grid of comparable size because the constituent (Δη,Δϕ)( η, φ) distribution is concentrated near the jet axis (the median jet has 3232 constituents, see App. A.4). A representative comparison is shown in Table 7. Table 7: Bin-collision rates (%) for positional-encoding schemes across grid sizes, measured on ×1055\!×\!10^5 JetClass QCD jets. Lower collision rates reduce ambiguity among masked iBOT tokens. The production choice is Polar ×100100\!×\!100 (bold). Scheme ×2020\!×\!20 ×5050\!×\!50 ×8080\!×\!80 ×100\!×\!100 ×120120\!×\!120 (patches) 400 2,500 6,400 10,000 14,400 Per-particle collision rate (%) Cartesian 64.76 35.98 22.19 16.82 13.13 Log-Cartesian 42.70 13.61 6.27 4.19 3.00 Polar 32.96 9.87 4.52 3.03 2.16 Per-jet collision rate (%) Cartesian 99.93 99.44 97.49 94.55 90.13 Log-Cartesian 99.13 83.55 59.55 46.79 37.26 Polar 90.01 59.14 39.59 31.19 24.94 Mask–mask collision rate (% of masked particles, 25%25\% mask) Cartesian 38.37 14.71 7.42 5.17 3.80 Log-Cartesian 16.99 3.82 1.59 1.02 0.72 Polar 12.38 2.78 1.16 0.76 0.53 Three observations emerge. First, all three schemes improve monotonically with grid size, and the same ranking (Polar best, Log-Cartesian intermediate, Cartesian worst) holds at every grid size and on all three metrics. Second, most of the improvement over uniform Cartesian comes from density-matched binning (Log-Cartesian alone reduces the ×100100\!×\!100 per-particle rate from 16.82%16.82\% to 4.19%4.19\%, a 4×4× reduction); the polar parameterization adds a further ∼1.4× \!1.4× refinement (4.19%→3.03%4.19\%→ 3.03\%) on top, by aligning the bin boundaries with the rotational symmetry of the jet. Third, the mask–mask rate (bottom panel) is the operationally relevant figure for iBOT, and it is substantially smaller than the per-particle rate at every grid size: at the production Polar ×100100\!×\!100, only 0.76%0.76\% of masked particles share a bin with another masked particle (vs. 3.03%3.03\% per-particle overall), so the degeneracy iBOT actually faces during training is roughly 4×4× lower than the full per-particle rate would suggest. Polar ×100100\!×\!100 is a deliberate middle ground. Going further to Polar ×120120\!×\!120 would reduce the per-particle collision rate only from 3.03%3.03\% to 2.16%2.16\% and the per-jet rate from 31.19%31.19\% to 24.94%24.94\%, while requiring 44%44\% more patches and a correspondingly larger learnable PE table (∼1.13 \!1.13M extra parameters at dmodel=256d_model=256). Conversely, dropping to Polar ×8080\!×\!80 would save 36%36\% of the patches but increase the per-particle rate from 3.03%3.03\% to 4.52%4.52\%. The ×100100\!×\!100 choice therefore lies at the knee of the diminishing-returns curve, capturing nearly all of the achievable reduction without paying for the long tail of marginal improvements. Projection heads. Following DINOv2 (Oquab et al., 2024), the DINO and iBOT projection heads are independent three-layer MLPs with hidden width 10241024 and bottleneck dimension 256256, GELU activations, an ℓ2 _2-normalization layer, and a weight-normalized linear output of dimension 20482048. The DINO head operates on the [CLS] representation, whereas the iBOT head operates on per-particle outputs at masked positions. A.4 Dataset Details JetClass. JetClass (Qu et al., 2022a) is a large-scale jet classification benchmark with 1010 classes spanning QCD, Higgs, electroweak, and top topologies. The hard process is generated with MadGraph5_aMC@NLO, parton showering and hadronization use Pythia, and detector effects are modeled with Delphes in the CMS configuration; jets are clustered from Delphes E-Flow objects with the anti-kTk_T algorithm at R=0.8R=0.8 and selected with pT∈[500,1000]p_T∈[500,1000] GeV and |η|<2|η|<2. See Qu et al. (2022a); Qu et al. (2022b) for full details. We use the QCD subset (1010M training jets) for pretraining and the signal classes for downstream evaluation. Pretraining set (QCD only). We pretrain exclusively on the QCD subset (1010M training jets), excluding all signal classes. The median QCD jet contains 3232 constituents (mean 35.2±17.235.2± 17.2; Fig. 4): only 11%11\% of jets have ≤16≤ 16 constituents and 0.13%0.13\% have ≤6≤ 6, confirming that the coarse views (N∈2,3,4N∈\2,3,4\) are genuine coarse-grainings for nearly all jets. Subjet clustering is computed once during preprocessing with FastJet (Cacciari et al., 2012) using the exclusive kTk_T algorithm with jet radius R=0.8R=0.8, and the four-momenta of the resulting subjets are stored on disk for the stopping points N∈2,3,4,6,8,16N∈\2,3,4,6,8,16\. Figure 4: Cumulative distribution of constituent multiplicity for QCD jets in JetClass (12.512.5M jets across train, validation, and test splits). The shaded region marks the kTk_T view range N∈2,…,16N∈\2,…,16\; only 11%11\% of jets have ≤16≤ 16 constituents, confirming that the coarse views are genuine coarse-grainings. Downstream evaluation sets. We finetune on JetNet (Kansal et al., 2021; Kansal et al., 2023) (∼360 \!360K jets covering gluon, light-quark, and top topologies, with jet pT≈1p_T≈ 1 TeV, restricted to 0.80.8–1.61.6 TeV, and at most 3030 constituents per jet, zero-padded). JetNet parton-level events are generated with MadGraph5_aMC@NLO, then showered and hadronized with Pythia (Monash 2013 tune), with a simplified detector-level description instead of Delphes. It therefore differs from JetClass in pTp_T range and detector-level simulation. We additionally use the Top Quark Tagging dataset (Kasieczka et al., 2019a; Kasieczka et al., 2019b) (∼1.2 \!1.2M training jets: top vs. QCD, with pT∈[550,650]p_T∈[550,650] GeV and up to 200200 constituents), which uses Pythia for shower/hadronization and Delphes for detector effects with a different (ATLAS-like) configuration than JetClass. For OOD evaluation we test on the JetClass test set, restricted to the relevant signal class versus QCD (e.g., top vs. QCD for the main comparison). All downstream evaluation samples are disjoint from the pretraining sample; the model never sees their labels during pretraining and never sees their non-QCD signal classes at all. The JetNet→ transfer is OOD in three respects: different pTp_T range, different constituent count distribution (at most 3030 vs. up to 183183), and different detector-level simulation (simplified vs. Delphes). A.5 Physical Scales of the kTk_T Views The hierarchy is not learned but imposed by the number of subjets per view, N. Each clustering step does indeed have a meaningful energy scale: the N+1→N+1→ N merge scale qN=dmerge(N)q_N= d_merge(N) (Catani et al., 1993), where dmerge(N)d_merge(N) is the kTk_T distance (dijd_ij of Eq. 1 or diBd_iB) at the clustering step from N+1N+1 to N subjets. For JetClass QCD jets with more than 1616 constituents, we report the median and IQR of qNq_N and of pTsoft/pTjetp_T^soft/p_T^jet, the transverse momentum of the softest of the N subjets divided by that of the parent jet (Table 8). Table 8: Physical scales of the exclusive-kTk_T views: merge scale qNq_N and softest-subjet momentum fraction, median [IQR] over JetClass QCD jets with more than 1616 constituents. N qNq_N [GeV] pTsoft/pTjetp_T^soft/p_T^jet 22 9.91[6.07,17.09]9.91\ [6.07,17.09] 0.23[0.12,0.36]0.23\ [0.12,0.36] 33 5.70[3.69,9.01]5.70\ [3.69,9.01] 0.088[0.042,0.156]0.088\ [0.042,0.156] 44 3.89[2.57,5.99]3.89\ [2.57,5.99] 0.044[0.021,0.084]0.044\ [0.021,0.084] 66 2.25[1.48,3.39]2.25\ [1.48,3.39] 0.018[0.009,0.034]0.018\ [0.009,0.034] 88 1.49[0.95,2.26]1.49\ [0.95,2.26] 0.010[0.006,0.019]0.010\ [0.006,0.019] 1616 0.46[0.24,0.76]0.46\ [0.24,0.76] 0.0033[0.0021,0.0054]0.0033\ [0.0021,0.0054] The median scale qNq_N falls monotonically from 9.919.91 to 0.460.46 GeV, while the resolved softest-subjet fraction falls from 0.230.23 to 0.00330.0033. Thus, the fixed N does indeed correspond to a meaningful distribution of physical scales. Two results show that RINO benefits from this ordering. First, the scale-structure analysis (App. A.9) finds that the backbone retains resolution information while the projection head aligns across scales. Second, with all other choices fixed, the intended teacher 6,8,16\6,8,16\/student-only 2,3,4,uncl.\2,3,4,uncl.\ assignment gives 0.839±0.0050.839± 0.005, versus 0.823±0.0060.823± 0.006 when the roles are swapped and 0.730±0.0090.730± 0.009 when both roles mix scales (Table 12). A.6 Finetuning Protocol Table 9 gives the locked finetuning recipe used in all main results in Sec. 4, identical across pretrained backbones. The supervised baseline shares the same optimizer, scheduler, and head architecture but is trained end-to-end from random initialization without the frozen-backbone warmup. For SSL methods we adopt the LP-FT (linear-probe-then-finetune) strategy (Kumar et al., 2022; Ren et al., 2023): the classification head is trained for the first 5 epochs with the backbone frozen, after which the backbone is unfrozen and trained jointly with the head at the same learning rate. This was found to give substantially smaller seed-to-seed variance than full finetuning from epoch 00, while matching its peak accuracy. Accuracy uses a fixed decision threshold of 0.50.5. The JetClass test sets are class-balanced, while JetNet has a QCD:top ratio of 2:12:1, which the positive-class weight of 2.02.0 compensates (Table 9). Table 9: Finetuning hyperparameters (locked across all backbones and datasets). Hyperparameter Value Optimizer AdamW (β1=0.9 _1=0.9, β2=0.999 _2=0.999) Head learning rate 1×10−41× 10^-4 Backbone learning-rate factor (after unfreeze) 1.01.0 (i.e., same as head) Weight decay 0.050.05 LP-FT: backbone freeze epochs 55 LR warmup epochs 1010 LR scheduler cosine with warmup Dropout (in head) 0.10.1 Label smoothing 0.00.0 Early-stopping patience 55 epochs (on validation loss) BCE positive-class weight 2.02.0 for JetNet; 1.01.0 for Top Quark Tagging Head architecture 2-layer MLP, hidden [256,128][256,128], ReLU, dropout 0.10.1 Pooling mean over particle tokens (for backbones with PE) Seeds per configuration 1010 for JetNet and 66 for Top Quark Tagging A.7 Robustness to Finetuning Regularization All main results use one finetuning recipe for every method (Table 9).44 4 The supervised baseline uses the same hyperparameters but is trained end-to-end, without the LP-FT frozen-backbone warmup (App. A.6). To test whether RINO’s lead depends on this shared recipe, we vary the weight decay and dropout over five (weight decay, dropout) settings, (0.01,0.1)(0.01,0.1), (0.05,0.1)(0.05,0.1), (0.10,0.1)(0.10,0.1), (0.05,0.3)(0.05,0.3), and (0.05,0.5)(0.05,0.5), for supervised training, MPMv1, MPMv2, and RINO with the default backbone and 100%100\% of the JetNet labels, keeping all other settings of Table 9. Table 10 reports the default setting (the Table 1 value) together with the lowest (Worst) and highest (Best) mean OOD accuracy across the five settings. Best is selected on JetClass accuracy, which requires target labels that are unavailable in applications to real data, so it is a retrospective best case rather than a deployable choice. For supervised training and MPMv1, no setting improves on the default, so the shared recipe does not disadvantage them. RINO’s worst setting (0.8430.843) exceeds the best setting of every other method, and its default leads the strongest target-selected baseline (MPMv2, 0.8100.810) by 0.0370.037. Its accuracy spans 0.0090.009 across the settings, comparable to MPMv1 (0.0100.010) and smaller than supervised training (0.0150.015) and MPMv2 (0.0500.050). Table 10: Sensitivity of OOD top-tagging accuracy (μ±σμ±σ over 1010 finetuning seeds; JetNet→ , MLP head, 100%100\% of JetNet labels) to the finetuning weight decay and dropout. Default is the Table 1 value (weight decay 0.050.05, dropout 0.10.1); Worst and Best are the lowest and highest mean over the five settings of App. A.7. Best is selected on JetClass accuracy and is a retrospective upper bound. Bold marks the best per column. Method Default Worst Best (target-selected) Supervised 0.786±0.0230.786± 0.023 0.771±0.0330.771± 0.033 0.786±0.0230.786± 0.023 MPMv1 (Golling et al., 2024) 0.800±0.0190.800± 0.019 0.790±0.0180.790± 0.018 0.800±0.0190.800± 0.019 MPMv2 (Leigh et al., 2025) 0.760±0.0160.760± 0.016 0.760±0.0160.760± 0.016 0.810±0.0170.810± 0.017 RINO 0.847±0.0030.847± 0.003 0.843±0.0050.843± 0.005 0.852±0.0030.852± 0.003 A.8 Extended Ablation Results Table 11 traces the progressive construction of the RINO recipe. Table 12 reports independent comparisons along the positional-encoding, view-assignment, and clustering-algorithm axes. In the view sets, uncl.uncl. denotes the unclustered jet (all particles). OOD ACC (accuracy) entries are mean ± standard deviation over 1010 seeds with an MLP head; the k-N and linear-probe (LP) columns are the frozen-backbone diagnostics of Sec. 4.1. The final model (row 5) uses a refined teacher schedule (fixed τt=0.04 _t=0.04, final EMA momentum 0.9970.997) compared to rows 1–4, which use a cosine warmup to τt=0.07 _t=0.07 and final momentum 0.9990.999. Table 11: Progressive construction of the RINO recipe. Each row builds on the previous row. Step Configuration k-N LP OOD ACC 1. DINO only, mixed views, no PE 0.8590.859 0.9230.923 0.677±0.0160.677± 0.016 2. + iBOT 0.8970.897 0.9180.918 0.776±0.0180.776± 0.018 ↑14.6% 14.6\% 3. + kTk_T multi-scale views, pTp_T-rank PE 0.9050.905 0.9160.916 0.828±0.0040.828± 0.004 ↑6.7% 6.7\% 4. + Polar-binned PE 0.9190.919 0.9190.919 0.839±0.0050.839± 0.005 ↑1.3% 1.3\% 5. + Teacher HP tuning (production) 0.9080.908 0.9160.916 0.847±0.0030.847± 0.003 ↑1.0% 1.0\% Table 12: Independent comparisons of design choices. Each block varies one axis while holding others fixed. Arrows give the relative change (%) with respect to the first row of each block. Axis Configuration k-N LP OOD ACC PE choice ∣ mixed views, DINO+iBOT No PE 0.8970.897 0.9180.918 0.776±0.0180.776± 0.018 pTp_T-rank 0.9020.902 0.9190.919 0.777±0.0280.777± 0.028 ↑0.1 0.1 Polar-binned 0.9090.909 0.9220.922 0.730±0.0090.730± 0.009 ↓5.9 5.9 PE choice ∣ kTk_T multi-scale views, DINO+iBOT pTp_T-rank 0.9050.905 0.9160.916 0.828±0.0040.828± 0.004 Polar-binned 0.9190.919 0.9190.919 0.839±0.0050.839± 0.005 ↑1.3 1.3 View assignment ∣ polar-binned PE, DINO+iBOT (sets shown are teacher views and student-only views; the student also sees the teacher’s views) Teacher 6,8,16\6,8,16\, student 2,3,4,uncl.\2,3,4,uncl.\ 0.9190.919 0.9190.919 0.839±0.0050.839± 0.005 Swapped: teacher 2,3,4\2,3,4\, student 6,8,16,uncl.\6,8,16,uncl.\ 0.9030.903 0.9220.922 0.823±0.0060.823± 0.006 ↓1.9 1.9 Mixed-scale: teacher 2,4,8,16\2,4,8,16\, student 3,6,uncl.\3,6,uncl.\ 0.9090.909 0.9220.922 0.730±0.0090.730± 0.009 ↓13.0 13.0 Clustering algorithm ∣ polar-binned PE, DINO+iBOT kTk_T 0.9190.919 0.9190.919 0.839±0.0050.839± 0.005 C/A 0.8780.878 0.9090.909 0.796±0.0120.796± 0.012 ↓5.1 5.1 Anti-kTk_T 0.7990.799 0.9060.906 0.693±0.0260.693± 0.026 ↓17.4 17.4 Mixed-scale views. The “mixed-scale” view assignment reshuffles clustering scales between teacher and student so that both roles span the full range: the teacher sees 2,4,8,16\2,4,8,16\ and the student additionally sees 3,6,uncl.\3,6,uncl.\. This preserves the use of multi-scale kTk_T views but breaks the hierarchy in which the teacher is restricted to resolved intermediate scales and the student-only views sit at both extremes (few subjets and unclustered). The 13.0%13.0\% drop from hierarchical to mixed-scale views under the same DINO+iBOT loss (Table 12) shows that the specific hierarchical assignment is the important design choice. iBOT loss scale. In both RINO and the iBOT-only ablation, the iBOT loss is applied only on the full unclustered particle view. We do not train iBOT to reconstruct masked constituents in the N-subjet clustered views, so that the multi-scale part of RINO is isolated in the DINO objective. Applying masked prediction across clustered views is a natural future extension. DINO and iBOT are complementary. Neither ℒDINOL_DINO nor ℒiBOTL_iBOT produces a strong representation alone: DINO-only (mixed views, no PE) reaches 0.6770.677 and iBOT-only reaches 0.7360.736 (Table 11, row 1; Table 3). Combined, they reach 0.7760.776 (mixed views) and 0.8390.839 (kTk_T views), exceeding what either component achieves on its own. ℒiBOTL_iBOT thus acts as an auxiliary co-training signal that sharpens representation quality when paired with ℒDINOL_DINO. PE interaction with views. Polar-binned PE hurts on mixed views (0.7300.730 vs. 0.7760.776 without PE) but helps on cluster views (0.8390.839 vs. 0.8280.828 with pTp_T-rank PE). This interaction is specific to the view assignment: polar-binned PE encodes spatial position within each subjet view, and its benefit appears only when the teacher-student hierarchy is respected. Under the mixed assignment, polar-binned PE degrades accuracy by 5.9%5.9\%, while under the hierarchical assignment it improves accuracy by 1.3%1.3\%. A.9 Scale Structure Figure 5: Per-jet mean cosine similarity across kTk_T clustering scales. Left to right, top to bottom: MPMv2 shows monotonic decay (input-similarity pattern); JetCLR-scale converges to near-invariance; RINO backbone retains scale-dependent geometry organized along the student/teacher role assignment; RINO backbone + projection head realizes cross-scale invariance in the 20482048-dimensional head output, including the unclustered view. A natural question is how models’ representations encode the parton-shower hierarchy. Figure 5 compares the per-jet mean cosine similarity matrix between clustering scales for three models. A scale-ignorant baseline such as MPMv2 shows the pattern produced by input similarity alone, where, for a given clustering scale, cosine similarity decreases monotonically as the other scale moves away along the kTk_T ladder in either direction, ranging from 0.920.92–0.970.97 between adjacent clustered scales to 0.350.35 at the most distant pair (N=2N\!=\!2 vs. the unclustered jet). This monotonic decay reflects the trivial correlation between subjet count and input feature overlap, without any learned cross-scale structure. In contrast, JetCLR-scale produces near-invariant representations across scales, with all cross-scale cosines in [0.89,1.00][0.89,1.00]. RINO shows a third, scale-aware pattern that matches neither baseline. The N=2N\!=\!2 row is particularly informative: N=2N\!=\!2 is more similar to the unclustered view (0.540.54) and N=16N\!=\!16 (0.530.53) than to the intermediate teacher-target scales N=6N\!=\!6 (0.360.36) and N=8N\!=\!8 (0.370.37), despite N=6N\!=\!6 and N=8N\!=\!8 being closer to N=2N\!=\!2 in subjet count. This non-monotonic structure tracks the student-teacher role assignment instead of the trivial input-similarity axis. Adding the projection head makes RINO nearly scale-invariant, with all cross-scale cosines ≥0.88≥ 0.88 in the head output, including the unclustered view (Fig. 5, bottom right). Swapped teacher-student roles. To test whether this non-monotonic backbone geometry is causally shaped by the teacher-student role assignment or is a spurious artifact of representation quality, we train a variant with swapped roles: teacher 2,3,4\2,3,4\, with the student additionally seeing 6,8,16,uncl.\6,8,16,uncl.\. Figure 6 compares the [CLS] cosine similarity matrices side by side. Figure 6: [CLS] cosine similarity across kTk_T scales for the standard (left) and swapped (right) teacher-student assignments. Red = teacher scales, blue = student scales. The grouping of N=4N\!=\!4 changes with its role: as a student-only scale (left) it is closest to N=3N\!=\!3 (cosine 0.850.85, vs. 0.650.65 to N=6N\!=\!6); as a teacher scale (right) it is closest to N=6N\!=\!6 (cosine 0.870.87, vs. 0.510.51 to N=3N\!=\!3). Under the standard assignment, the student-only scale N=4N\!=\!4 is closest to N=3N\!=\!3 (0.850.85), with cosines 0.650.65, 0.640.64, and 0.790.79 to the teacher scales N=6N\!=\!6, 88, and 1616. Under the swapped assignment, where N=4N\!=\!4 is a teacher scale, it is closest to N=6N\!=\!6, 88, and 1616 (0.870.87, 0.820.82, 0.770.77) and separates from N=3N\!=\!3 (0.510.51). The scales N=6N\!=\!6, 88, 1616 remain mutually similar under both assignments (0.720.72–0.850.85 standard, 0.870.87–0.940.94 swapped). The grouping of N=4N\!=\!4 thus follows its role, showing that the backbone geometry is shaped by the role assignment. A.10 Cross-Topology Transfer with MLP Head Table 13 reports the MLP head variant of the cross-topology evaluation, including the finetuning topology t→bqqt→ bqq. RINO leads on t→bqqt→ bqq. On H→4qH→ 4q, RINO and JetCLR-scale are statistically tied (0.828±0.0060.828± 0.006 vs. 0.824±0.0200.824± 0.020), as are RINO and MPMv2 on H→bb¯H→ b b (0.697±0.0090.697± 0.009 vs. 0.692±0.0290.692± 0.029). JetCLR-scale shows a large gap between MLP and linear heads (H→4qH→ 4q: 0.8240.824 vs. 0.6300.630), while RINO’s are nearly identical (0.8280.828 vs. 0.8210.821), indicating that self-distillation produces linearly accessible class boundaries. Table 13: Cross-topology transfer ACC (JetNet→ , MLP head). Method t→bqqt→ bqq H→4qH→ 4q H→bb¯H→ b b Supervised 0.786±0.0230.786± 0.023 0.699±0.0380.699± 0.038 0.623±0.0280.623± 0.028 SSL baselines JetCLR 0.700±0.0310.700± 0.031 0.696±0.0280.696± 0.028 0.686±0.0150.686± 0.015 MPMv1 0.800±0.0190.800± 0.019 0.769±0.0060.769± 0.006 0.674±0.0170.674± 0.017 MPMv2 0.760±0.0160.760± 0.016 0.780±0.0250.780± 0.025 0.692±0.0290.692± 0.029 OmniJet-α 0.751±0.0500.751± 0.050 0.684±0.0280.684± 0.028 0.600±0.0170.600± 0.017 kTk_T-view SSL (ours) JetCLR-scale 0.780±0.0230.780± 0.023 0.824±0.0200.824± 0.020 0.639±0.0140.639± 0.014 RINO 0.847±0.0030.847± 0.003 0.828±0.0060.828± 0.006 0.697±0.0090.697± 0.009 A.11 Top Quark Tagging→ Transfer Results Table 14 reports top-tagging accuracy on the JetClass test set for models finetuned on the Top Quark Tagging dataset (Kasieczka et al., 2019a; Kasieczka et al., 2019b). This is a transfer between datasets with a smaller domain gap than JetNet→ , not the in-distribution Top Quark Tagging benchmark, so the values are not comparable to results on the standard Top Quark Tagging test set. The leading methods converge to ∼0.93 \!0.93 accuracy, indicating that this benchmark is less discriminating than the JetNet→ setting used for the main comparison. Nevertheless, scale factors (App. A.12), which directly quantify how strongly a classifier depends on simulation-specific structure, separate the methods, with RINO’s closest to unity. Table 14: Top Quark Tagging→ transfer: top-tagging accuracy (μ±σμ±σ over 66 seeds) on the JetClass test set of models finetuned on Top Quark Tagging. These are transfer results, not comparable to in-distribution results on the standard Top Quark Tagging test set. The leading methods saturate near 0.930.93. Bold marks the best per column; underline marks entries within 1σ1σ of the best. Method MLP head Linear head Supervised 0.929±0.0010.929± 0.001 0.929±0.0010.929± 0.001 SSL baselines JetCLR (Dillon et al., 2022) 0.921±0.0010.921± 0.001 0.921±0.0010.921± 0.001 MPMv1 (Golling et al., 2024) 0.929±0.0020.929± 0.002 0.927±0.0010.927± 0.001 MPMv2 (Leigh et al., 2025) 0.932±0.0010.932± 0.001 0.933±0.001¯ 0.933± 0.001 OmniJet-α (Birk et al., 2024) 0.760±0.0090.760± 0.009 0.756±0.0080.756± 0.008 kTk_T-view SSL (ours) JetCLR-scale 0.936±0.0030.936± 0.003 0.935±0.0030.935± 0.003 RINO 0.933±0.001¯ 0.933± 0.001 0.933±0.001¯ 0.933± 0.001 Table 15 extends this to cross-topology transfer: models finetuned on Top Quark Tagging for top tagging are evaluated on the unseen H→4qH→ 4q signal from JetClass. When finetuned on Top Quark Tagging (smaller domain gap with overlapping pTp_T range), cross-topology transfer is much more uniform: the four methods evaluated span H→4qH→ 4q AUC 0.8770.877–0.9000.900, narrower than the 0.8030.803–0.8940.894 of the same four methods on JetNet→ (Table 2), and RINO (0.896±0.0020.896± 0.002) is within one standard deviation of the leading MPMv2 (0.900±0.0050.900± 0.005). This contrasts with JetNet→ (Table 2), where RINO leads by +4.1%+4.1\%, consistent with its advantage being specific to larger domain shifts. Table 15: Top Quark Tagging→ cross-topology AUC (linear head, 6 seeds). With the smaller domain gap, the four methods reach similar AUC. Method t→bqqt→ bqq H→4qH→ 4q MPMv2 0.982±0.0010.982± 0.001 0.900±0.0050.900± 0.005 RINO 0.981±0.0000.981± 0.000 0.896±0.0020.896± 0.002 MPMv1 0.980±0.0000.980± 0.000 0.889±0.0010.889± 0.001 JetCLR 0.976±0.0010.976± 0.001 0.877±0.0050.877± 0.005 A.12 Scale-Factor Proxy (Top Quark Tagging→ ) Scale factors (SFs) are a standard downstream component of collider analyses: efficiencies measured in data control regions are divided by their simulated values and used to correct simulated yields (App. A.13). They therefore directly quantify how strongly a classifier depends on simulation-specific structure. We additionally use the Top Quark Tagging dataset to construct a proxy experiment that measures them, using Top Quark Tagging as source (“simulation”) and JetClass as target (“data”), with the Top Quark Tagging-finetuned models of Table 14. We partition the source signal-score cumulative distribution function (CDF) into six disjoint intervals (the last containing the highest-scoring 5%5\% of jets), apply the same score boundaries to the target, and compute the target/source efficiency ratio in each. For each finetuning run, let s be the signal score, with CDF F over the Top Quark Tagging test signal jets and w-quantile qwq_w: F(s)=Psrc(s′≤s∣signal),qw=F−1(w),F(s)=P_src\! (s ≤ s ), q_w=F^-1(w), (7) with q0=−∞q_0=-∞ and q1=+∞q_1=+∞. An interval [a,b)[a,b) thus selects jets with qa≤s<qbq_a≤ s<q_b, i.e., source signal efficiencies 1−F(s)1-F(s) between 1−b1-b and 1−a1-a; the last interval, [0.95,1][0.95,1], is the tightest working point (1−F(s)≤0.051-F(s)≤ 0.05). For each interval [a,b)[a,b), the signal efficiency in domain d∈src,tgtd∈\src,tgt\ and the scale factor are ε[a,b)d=Pd(qa≤s<qb∣signal),SF[a,b)=ε[a,b)tgtε[a,b)src, ^d_[a,b)=P_d\! (q_a≤ s<q_b ), _[a,b)= ^tgt_[a,b) ^src_[a,b), (8) so SF=1SF=1 means that the target populates that interval exactly as the source does. Table 16 lists the scale factors per bin and their mean absolute departure from unity. Table 16: Top Quark Tagging→ scale factors (Eq. 8) per interval of the source signal-score CDF (the last containing the highest-scoring 5%5\% of jets) and their mean absolute departure from unity (last column). Entries are mean ± standard deviation over finetuning runs; bold marks the value closest to unity per bin. Method [0,0.3)[0,0.3) [0.3,0.5)[0.3,0.5) [0.5,0.7)[0.5,0.7) [0.7,0.9)[0.7,0.9) [0.9,0.95)[0.9,0.95) [0.95,1][0.95,1] ⟨|SF−1|⟩↓ |SF-1| Supervised 1.120±0.0531.120± 0.053 0.947±0.0220.947± 0.022 0.933±0.0230.933± 0.023 0.960±0.0410.960± 0.041 0.958±0.0730.958± 0.073 0.972±0.1280.972± 0.128 0.0590.059 SSL baselines JetCLR 1.134±0.0321.134± 0.032 0.942±0.0120.942± 0.012 0.903±0.0200.903± 0.020 0.898±0.0230.898± 0.023 0.964±0.0290.964± 0.029 1.256±0.0841.256± 0.084 0.1140.114 MPMv1 1.162±0.0471.162± 0.047 0.966±0.0130.966± 0.013 0.892±0.0140.892± 0.014 0.884±0.0300.884± 0.030 0.947±0.0430.947± 0.043 1.117±0.0831.117± 0.083 0.0980.098 MPMv2 0.928±0.0240.928± 0.024 0.867±0.0210.867± 0.021 0.898±0.0080.898± 0.008 1.019±0.0221.019± 0.022 1.278±0.0481.278± 0.048 1.999±0.1121.999± 0.112 0.2670.267 OmniJet-α 1.073±0.0291.073± 0.029 0.938±0.0200.938± 0.020 0.949±0.0310.949± 0.031 0.975±0.0260.975± 0.026 1.001±0.0461.001± 0.046 1.110±0.1591.110± 0.159 0.0540.054 kTk_T-view SSL (ours) JetCLR-scale 0.980±0.0510.980± 0.051 0.871±0.0300.871± 0.030 0.884±0.0200.884± 0.020 0.980±0.0280.980± 0.028 1.186±0.0861.186± 0.086 1.967±0.2821.967± 0.282 0.2400.240 RINO 1.029±0.0571.029± 0.057 0.963±0.0270.963± 0.027 0.978±0.0180.978± 0.018 1.001±0.0591.001± 0.059 1.031±0.0841.031± 0.084 1.029±0.1151.029± 0.115 0.0250.025 RINO’s 2.5%2.5\% average deformation is less than half the next-lowest value (5.4%5.4\%), indicating substantially less dependence on simulation-specific structure. A.13 Path to Experimental Data Figure 7 shows the intended use of RINO in an experimental analysis, in three stages. 1. Pretrain on collision data unlabeled jets 2. Finetune on labeled simulations signal and background 3. Calibrate in data scale factors and uncertainties from control regions This work: JetClass QCD (simulated) This work: JetNet, Top Quark Tagging This work: labeled top and QCD test jets from JetClass; SF proxy Figure 7: Intended deployment of RINO. Labels enter only in stage 2, and stage 3 is the standard calibration used for supervised taggers. The bottom row lists the simulated stand-ins used in this work, where JetClass top jets and all JetClass labels enter only in stage 3, for evaluation. 1. Pretrain on collision data. RINO can learn jet representations from unlabeled experimental data, since its pretraining uses no labels. Nevertheless, in this work, for more straightforward evaluations, pretraining uses simulated JetClass QCD jets to mimic the QCD-dominated data. Including a fraction of simulated jets and also specialized control regions to isolate more signal-like topologies for the pretraining are both interesting areas for future work. 2. Finetune on labeled simulations. The pretrained backbone is finetuned on simulated signal and background jets, where labels are available, as in our JetNet and Top Quark Tagging experiments. 3. Calibrate in data. Signal efficiencies (CMS Collaboration, 2025b) and background mistag rates are measured in dedicated control regions. For top tagging, semileptonic-tt¯t t tag-and-probe can measure signal efficiency, while QCD-enriched multijet regions constrain mistag rates. The resulting data-to-simulation scale factors (SFs) correct the predicted yields; detector, shower, and hadronization variations enter the final likelihood as systematic uncertainties. Appendix A.12 gives example SFs measured in a simulation-to-simulation proxy. A.14 Full Data-Efficiency Curves Figure 8 shows the complete data-efficiency curves for all seven methods evaluated in Sec. 4.3. This includes the JetCLR-scale variant, which isolates the effect of replacing JetCLR’s generic augmentations with kTk_T scale views within the same contrastive objective family. OmniJet-α is omitted from the main-text figure for clarity; its behavior is qualitatively similar to most other baselines (non-monotonic, high variance). Figure 8: Full data-efficiency curves (all seven methods) with ±1σ± 1σ bands over 1010 seeds. Non-monotonic behavior under distribution shift. Several baselines exhibit non-monotonic data-efficiency curves: MPMv2 peaks at 10%10\% and declines at 5050–100%100\%. The mechanism is discussed in Sec. 4.3. To rule out a batch-size artifact, we re-ran MPMv2 at 50%50\% with batch size 256256 (vs. the default 10241024): accuracy changed from 0.7890.789 to 0.761±0.0270.761± 0.027, indicating that the non-monotonicity is not simply a batch-size effect. We would like to emphasize that the supervised baseline gains no JetClass accuracy beyond 10%10\% of the JetNet labels (0.7860.786, 0.7790.779, and 0.7860.786 at 10%10\%, 50%50\%, and 100%100\%), which is not classical overfitting: its held-out in-distribution JetNet (source) accuracy stays flat or rises with the label fraction. The model is thus not memorizing its training set but specializing to simulation-specific features that do not transfer well. RINO is designed to prevent this: physically, kTk_T coarsening absorbs low-scale radiation into harder subjets (Salam, 2010), so requiring agreement across scales discourages reliance on soft, low-scale details, precisely where parton-shower and hadronization modeling is approximate and generator-dependent (Nagy and Soper, 2018; Barnard et al., 2017). RINO improves monotonically across all label fractions and leads at every fraction; the only other monotonic curve, JetCLR, stays far lower (0.700±0.0310.700± 0.031 at 100%100\%). This is consistent with RINO’s ∼3× 3× smaller domain shift in representation space (Table 4). A.15 MLP vs. Linear Head Diagnostic Table 17 compares MLP and linear head performance on the JetNet→ OOD benchmark. The significance column reports the Welch t-statistic t=(μMLP−μLinear)/σMLP2/n+σLinear2/nt=( _MLP- _Linear)/ _MLP^2/n+ _Linear^2/n with n=10n=10 seeds per condition. Degrees of freedom follow the Welch–Satterthwaite approximation (≈11≈ 11 for RINO, whose two variances differ, and ≈12≈ 12–1717 for the other methods). RINO is the only method with a significantly positive t (+3.0+3.0, two-tailed p≈0.01p≈ 0.01), indicating exploitable nonlinear structure; JetCLR and MPMv2 show clear degradation (t<−3t<-3, p<10−3p<10^-3). Table 17: MLP vs. linear head on OOD top tagging (JetNet→ ). Method MLP Linear t RINO 0.847±0.0030.847± 0.003 0.838±0.0090.838± 0.009 +3.0+3.0 Supervised 0.786±0.0230.786± 0.023 0.784±0.0330.784± 0.033 +0.2+0.2 MPMv1 0.800±0.0190.800± 0.019 0.809±0.0080.809± 0.008 −1.4-1.4 MPMv2 0.760±0.0160.760± 0.016 0.817±0.0270.817± 0.027 −5.7-5.7 JetCLR 0.700±0.0310.700± 0.031 0.791±0.0480.791± 0.048 −5.0-5.0 JetCLR-scale 0.780±0.0230.780± 0.023 0.803±0.0170.803± 0.017 −2.5-2.5 OmniJet-α 0.751±0.0500.751± 0.050 0.749±0.0800.749± 0.080 +0.1+0.1 A.16 Domain Shift in Representation Space Protocol. Both metrics are computed on 5,0005,000 jets per domain. W1W_1 is averaged over embedding dimensions; the squared maximum mean discrepancy (MMD2, biased estimator) uses an RBF kernel exp(−γ∥x−y∥2) (-γ x-y ^2) with fixed γ=1/dγ=1/d, where d is the embedding dimension. Table 18 reports the same metrics as Table 4 on the pretrained backbones, before finetuning, with signed changes indicating whether finetuning increases (++) or decreases (−-) the gap. RINO already has the smallest pretrained shift on three of the four metrics (e.g., W1W_1 (QCD): 0.0550.055, vs. at least 0.0920.092 for the baselines). JetCLR-scale, which shares RINO’s kTk_T views, is the other method with a small pretrained shift: the two kTk_T-view methods have the two smallest values on every metric. JetCLR-scale’s shift increases during finetuning but stays relatively low (Table 4), while RINO’s decreases further on all four metrics (e.g., W1W_1 (QCD): 0.055→0.0430.055→ 0.043). MPMv2 is the only other method whose gap decreases on all four. The remaining baselines do not consistently benefit from finetuning: the changes are mixed for JetCLR and MPMv1. These trends connect to the feature-distortion picture of Kumar et al. (2022) (Sec. 4.3): finetuning updates the features of in-distribution (JetNet) jets more than those of OOD (JetClass) jets, which can widen the gap between the two domains. For RINO, whose pretrained representations are already close across domains, finetuning instead brings them even closer together. This suggests that the small pretrained shift comes from the kTk_T views, and provides further evidence for the physical picture of App. A.14: kTk_T coarsening absorbs low-scale radiation into harder subjets (Salam, 2010), so requiring agreement across scales discourages reliance on soft, low-scale details, where parton-shower and hadronization modeling is approximate and generator-dependent (Nagy and Soper, 2018; Barnard et al., 2017). Table 18: Domain shift in pretrained representations (JetNet vs. JetClass), before finetuning. Colored values give the change after finetuning (Table 4): −- (green) = finetuning reduces the gap, ++ (red) = finetuning increases it. Method W1W_1 (QCD) ↓ W1W_1 (top) ↓ MMD2 (QCD) ↓ MMD2 (top) ↓ JetCLR 0.1280.128 +.004+.004 0.1250.125 +.002+.002 0.2230.223 −.027-.027 0.2120.212 −.021-.021 OmniJet-α 0.1810.181 +.634+.634 0.2460.246 +.506+.506 0.0950.095 +.476+.476 0.1930.193 +.359+.359 MPMv1 0.2320.232 +.009+.009 0.2740.274 −.012-.012 0.1200.120 +.013+.013 0.1830.183 −.021-.021 MPMv2 0.4090.409 −.034-.034 0.3740.374 −.026-.026 0.4270.427 −.089-.089 0.4090.409 −.104-.104 JetCLR-scale 0.0920.092 +.041+.041 0.0710.071 +.074+.074 0.0170.017 +.028+.028 0.0090.009 +.039+.039 RINO 0.0550.055 −.012-.012 0.0520.052 −.008-.008 0.0120.012 −.005-.005 0.0110.011 −.004-.004 A.17 Baseline Implementation Details All SSL baselines are implemented under a controlled implementation and finetuning setup so that the main variables across methods are the pretraining objective and view-construction strategy. This setup reduces confounders from depth, width, normalization, and activation choices that would otherwise conflate model capacity with pretraining quality. Table 19 summarizes the key implementation choices. We do not include J-JEPA (Katel et al., 2024) or HEP-JEPA (Bardhan et al., 2025) in our main comparison: J-JEPA reports modest gains over training from scratch, and HEP-JEPA’s code has not been released, precluding direct reproduction in our controlled-backbone protocol. Among existing methods, J-JEPA is the closest conceptual relative of RINO: it reclusters each jet into C/A subjets and predicts the embeddings of target subjets from context subjets within a single jet, but does not align representations across different coarse-graining scales. Our ablation (Sec. 4.5) shows that cross-scale kTk_T views account for the majority of the gain (+11.4%+11.4\%) regardless of the SSL family, suggesting that the multi-scale view construction dominates the choice of joint-embedding objective. Table 19: Baseline implementation summary. All methods share the same 7 kinematic input features (Table 6), the same QCD pretraining data, and the same finetuning recipe (App. A.6). Method Backbone Attention Input Loss RINO JTE enc. Bidir. Continuous DINO+iBOT Supervised JTE enc. Bidir. Continuous Cross-ent. MPMv1 JTE enc. Bidir. Continuous Masked VQ pred. MPMv2 JTE enc. Bidir. Continuous Masked K-means + L1 + flow JetCLR JTE enc. Bidir. Continuous NT-Xent JetCLR-scale JTE enc. Bidir. Continuous NT-Xent+MAE OmniJet-α JTD dec. Causal VQ tokens Next-token pred. Shared components. The jet transformer encoder (JTE) backbone is identical across all encoder-based methods: 88 layers, dmodel=256d_model\!=\!256, 1616 heads, SwiGLU FFN, RMSNorm, LayerScale, and 44 register tokens. MPMv1 and OmniJet-α share a VQ-VAE tokenizer with a 512512-code codebook, trained once on the QCD pretraining set and frozen during SSL pretraining. All methods are pretrained on the same QCD-only data. Hyperparameter parity. Each baseline uses the core hyperparameters from its original publication (loss weights, masking ratios, temperature), which we assume are near-optimal for the respective objectives. RINO’s teacher temperature and EMA momentum were selected on the OOD top-vs-QCD accuracy on JetClass. This step contributes +1.0%+1.0\% (Table 3), and RINO before tuning (0.839±0.0050.839± 0.005) already exceeds every baseline’s MLP-head accuracy in Table 1 (best: MPMv1, 0.800±0.0190.800± 0.019). All encoder-based methods share the same backbone, and all methods share the same data and finetuning protocol, so the comparison isolates the pretraining objective and view construction as much as possible. OmniJet-α. OmniJet-α is the only decoder-based method. The original model (Birk et al., 2024) uses a 33-layer GPT-style decoder with 88 heads and an 8,1948,194-token vocabulary. Our adaptation replaces this with an 88-layer jet transformer decoder (JTD) that mirrors the JTE in depth, width, normalization, and activation, differing only in the causal attention mask required by the autoregressive objective. This controls for model capacity when comparing the decoder baseline to encoder-based methods. We use the shared 512512-code VQ-VAE tokenizer rather than the original vocabulary. Because causal attention imposes an ordering on an inherently unordered set (jet constituents have no natural sequence), the decoder can only aggregate information left-to-right. We use mean pooling over all token outputs as the jet representation, which empirically outperforms the last_token alternative. MPMv1 and MPMv2. Both follow the masked-particle modeling framework of Golling et al. (2024) and Leigh et al. (2025), adapted to use the shared JTE backbone. The backbone receives continuous input features (the same 7 kinematic features as RINO), projected through a linear embedding layer. MPMv1 uses the VQ-VAE only as a target generator for a cross-entropy classification loss at masked positions. MPMv2 uses no VQ-VAE: at masked positions, it predicts each constituent’s cluster in the 16,38416,384-centroid K-means codebook released with Leigh et al. (2025), regresses the continuous features directly (L1 loss), and models them with a conditional normalizing flow. Mean pooling is used because MPM’s per-particle masked-prediction objective provides no training signal for a [CLS]. JetCLR and JetCLR-scale. JetCLR (Dillon et al., 2022) uses the NT-Xent contrastive loss with standard jet augmentations (rotations, translations, pTp_T-dependent smearing, and collinear splittings). JetCLR-scale isolates the contribution of multi-scale view construction: it replaces JetCLR’s generic augmentations with the same kTk_T clustering views used by RINO, while keeping a contrastive objective. Scales are grouped into three tiers: coarse (N∈2,3,4N∈\2,3,4\), intermediate (N∈6,8N∈\6,8\), and fine (N=16N=16 or unclustered). Each training batch samples one scale from each of two randomly chosen tiers, producing a two-view pair with a guaranteed cross-scale gap (e.g., N=3N\!=\!3 vs. N=16N\!=\!16). The NT-Xent loss encourages agreement across these paired scales, and an MAE reconstruction auxiliary at the fine scale stabilizes low-data training. The JetCLR-to-JetCLR-scale comparison (0.7000.700 vs. 0.7800.780) confirms that the kTk_T views account for most of the gain. A.18 Compute Pretraining a single RINO model on the QCD subset of JetClass (∼107 10^7 jets, 100100 epochs) takes approximately 4545 minutes per epoch on 2×2× NVIDIA A100-80GB, for a wall-clock cost of ∼80 80 hours per pretraining run. Finetuning on JetNet takes ∼2 2 minutes per epoch on a single NVIDIA 3090 or A10, for at most 5555 epochs with early stopping; runs typically converge within ∼25 25 epochs (∼50 50 minutes wall-clock per seed). Finetuning on the Top Quark Tagging dataset takes ∼15 15 minutes per epoch on 2×2× 3090/A10, again capped at 5555 epochs with early stopping (∼25 25 epochs and ∼6 6 wall-clock hours per seed in practice). Table 20 compares per-method pretraining wall-clock cost on the same QCD-only JetClass subset. All encoder-based methods use the shared JTE backbone (OmniJet-α uses the matched JTD). MPMv1 and OmniJet-α additionally require a VQ-VAE tokenizer (∼8 8h, trained once and shared). Table 20: Per-method pretraining wall-clock cost (QCD-only JetClass subset, estimated for 100100 epochs). † Requires a shared VQ-VAE tokenizer (∼8 8h additional, trained once). Method Time/epoch (min) Total (h) JetCLR ∼10 10 ∼20 20 JetCLR-scale ∼25 25 ∼45 45 MPMv1† ∼30 30 ∼50 50 MPMv2 ∼40 40 ∼65 65 OmniJet-α† ∼20 20 ∼30 30 iBOT-only ∼50 50 ∼80 80 RINO ∼45 45 ∼80 80