Paper deep dive
ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning
Noah Videcrantz, Mostafa Mehdipour Ghazi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/1/2026, 5:11:51 AM
Summary
The paper introduces ActiveAugment, a framework that treats data augmentation selection as an online active learning problem. Instead of static or random augmentation, it scores candidate augmented views based on the model's predictive uncertainty and feature discrepancy. The most fragile augmentations are selected per sample to enforce invariance via a joint supervised classification and contrastive objective. ActiveAugment outperforms baselines like AutoAugment and RandAugment, particularly in low-label-budget and medical imaging scenarios.
Entities (16)
Relation Signals (14)
ActiveAugment → evaluatedon → ISIC-2019
confidence 95% · medical imaging datasets (BRISC, BUSI, FETAL-PLANES-DB, ISIC-2019)
ActiveAugment → evaluatedon → MNIST
confidence 95% · We evaluate ActiveAugment on eight benchmark datasets... (CIFAR-10/100, STL-10, MNIST)
ActiveAugment → evaluatedon → CIFAR-10
confidence 95% · We evaluate ActiveAugment on eight benchmark datasets... (CIFAR-10/100, STL-10, MNIST)
ActiveAugment → outperforms → AutoAugment
confidence 95% · ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets
ActiveAugment → outperforms → RandAugment
confidence 95% · ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets
ActiveAugment → outperforms → TrivialAugment
confidence 95% · ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets
ActiveAugment → uses → Feature Discrepancy
confidence 95% · scores each candidate using a combination of the model's predictive uncertainty and the feature discrepancy induced by the augmentation
ActiveAugment → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, either relying on expensive dataset-specific policy search or applying transformations uniformly at random, regardless of what the model has already learned. We introduce ActiveAugment, a unified framework that treats augmentation selection as an online active learning problem. For each training minibatch, ActiveAugment generates a pool of candidate augmented views and scores each candidate using a combination of the model's predictive uncertainty and the feature discrepancy induced by the augmentation. The augmentation under which the current model is most fragile is selected per sample, and the model is then trained with a joint supervised classification and supervised contrastive objective that enforces intra-class invariance to the selected augmentations while maintaining inter-class separation. We evaluate ActiveAugment on eight benchmark datasets spanning natural and medical imaging, using CNN and transformer architectures across three training regimes (training from scratch, full fine-tuning, and linear probing), and comparing eight active selection strategies for augmentation scoring. ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where data is scarce and domain shift relative to natural-image pretrained models is large, ActiveAugment achieves higher test F1 than all baselines, demonstrating strong cross-domain adaptability. Our analysis reveals that the augmentation selection policy evolves meaningfully during training and that strategy choice has a direct impact on generalisation. Code is available at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.28923v1
- Canonical: https://arxiv.org/abs/2608.28923v1
Trouble viewing inline? Open PDF directly →
Full Text
55,479 characters extracted from source content.
Expand or collapse full text
Active–Augment ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning Noah Videcrantz Mostafa Mehdipour Ghazi Abstract Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, either relying on expensive dataset-specific policy search or applying transformations uniformly at random, regardless of what the model has already learned. We introduce ActiveAugment, a unified framework that treats augmentation selection as an online active learning problem. For each training minibatch, ActiveAugment generates a pool of candidate augmented views and scores each candidate using a combination of the model’s predictive uncertainty and the feature discrepancy induced by the augmentation. The augmentation under which the current model is most fragile is selected per sample, and the model is then trained with a joint supervised classification and supervised contrastive objective that enforces intra-class invariance to the selected augmentations while maintaining inter-class separation. We evaluate ActiveAugment on eight benchmark datasets spanning natural and medical imaging, using CNN and transformer architectures across three training regimes (training from scratch, full fine-tuning, and linear probing), and comparing eight active selection strategies for augmentation scoring. ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where data is scarce and domain shift relative to natural-image pretrained models is large, ActiveAugment achieves higher test F1 than all baselines, demonstrating strong cross-domain adaptability. Our analysis reveals that the augmentation selection policy evolves meaningfully during training and that strategy choice has a direct impact on generalisation. Code is available at: https://github.com/noahvide/ActiveAugment. †email: novi@di.ku.dk†email: ghazi@di.ku.dk†affiliation: Pioneer Centre for AI University of Copenhagen 1 Introduction Training deep learning models typically requires large volumes of labelled data, and this need is only intensifying as models grow in size and complexity [Wang et al.(2024)Wang, Han, Wang, Song, Tian, and Huang]. Yet, acquiring large amounts of annotated data is not always feasible due to constraints such as costly expert labelling or access to biased corpora [Chi et al.(2020)Chi, Ma, Wu, Chen, Lu, and Gu, Vădineanu et al.(2024)Vădineanu, Pelt, Dzyubachyk, and Batenburg]. Consequently, data-efficient deep learning has become an essential research direction aimed at maximising the utility of limited available samples [Wang et al.(2021b)Wang, Gao, Long, and Wang]. Two complementary strategies have emerged for maximising data efficiency. Active Learning (AL) reduces the number of required labels by identifying the most informative instances from an unlabelled pool for annotation [Settles(2009)]. Data augmentation increases the effective utility of existing labels by synthesising additional training views [Shorten and Khoshgoftaar(2019)]. While data augmentation is a cornerstone of modern training pipelines, it is treated as a static step rather than an adaptive component of the learning process, a mismatch that motivates this work. Several challenges arise when designing an augmentation pipeline. A central one is choosing which augmentations to apply: a horizontal flip may be label-preserving for many natural-image tasks yet label-destructive in others (e.gdigit recognition or anatomically oriented medical imaging tasks). Another challenge is deciding on the intensity of each transformation. Furthermore, as the model learns, it may become invariant to some augmentations while remaining fragile to others, so a static pipeline becomes increasingly misaligned with the model’s current state. Given the combinatorial space of augmentation types and intensities, together with the dynamic nature of training, there is a need for improved augmentation. Automatic augmentation selection has been explored previously. AutoAugment [Cubuk et al.(2019)Cubuk, Zoph, Mané, Vasudevan, and Le] treats the problem as a reinforcement-learning search over a discrete policy space and yields a strong, dataset-specific policy. However, the search is computationally expensive and the resulting policy is static, making it model-agnostic. RandAugment [Cubuk et al.(2020)Cubuk, Zoph, Shlens, and Le] reduces the search space to two scalar hyperparameters (pipeline length and global intensity), making the search cheaper while achieving comparable accuracy. Nevertheless it remains dataset-specific and model-agnostic. TrivialAugment [Müller and Hutter(2021)] eliminates the search entirely by sampling a single augmentation and intensity uniformly at random for each image, achieving state-of-the-art results on several benchmarks. While dataset-agnostic, it is also model-agnostic. This leaves a gap between simple, model-blind random sampling and expensive, static policy searches. The technical core of adaptive augmentation selection lies in identifying the model’s empirical blind spots. Rather than treating data augmentation purely as a heuristic tool for dataset expansion, it can be formally viewed through the lens of domain adaptation: clean training data acts as the source distribution, and augmented variants represent a proxy target domain. Classic generalisation bounds establish that target-domain performance is upper-bounded by the source error and the divergence between the two feature distributions [Ben-David et al.(2010)Ben-David, Blitzer, Crammer, Kulesza, Pereira, and Vaughan, Mansour et al.(2009)Mansour, Mohri, and Rostamizadeh]. Uniform or random augmentation strategies fail to optimise this bound because they apply transformations blindly. Without a mechanism to identify where the model is fragile, training steps are spent on uninformative augmentations that do not tighten the generalisation gap. We propose that augmentation selection should be treated as an active process, using informativeness measures from the AL literature to select the most beneficial augmentations online during training. Therefore, we introduce ActiveAugment, a framework for active invariance learning via selective augmentation. The framework decouples training into two phases: a Selection phase and a Learning phase. During selection, a pool of candidate augmented views is generated; each candidate is scored by a combination of the model’s predictive uncertainty and the feature discrepancy induced by the augmentation. This allows the model to identify and prioritise augmentations under which it is currently most fragile. During learning, the selected augmentations are used within a supervised contrastive objective that simultaneously enforces intra-class invariance and inter-class separation. We evaluate ActiveAugment using convolutional and transformer backbones across three training regimes (training from scratch, full fine-tuning, and linear probing) on eight datasets, including natural image benchmarks (CIFAR-10/100, STL-10, MNIST) and medical imaging datasets (BRISC, BUSI, FETAL-PLANES-DB, ISIC-2019). Our main contributions are: • A novel unified framework, ActiveAugment, that reformulates augmentation selection as an online AL problem, combining predictive uncertainty and feature discrepancy into a principled, theoretically grounded selection score. • A joint training objective that pairs active augmentation selection with supervised contrastive learning, enforcing invariance to model-fragile augmentations while maintaining inter-class discriminability. • A systematic translation of eight AL strategies (uncertainty-based, diversity-based, and hybrid) into online augmentation-scoring strategies, with detailed analysis of their selection behaviour and its performance consequences. • Comprehensive empirical evaluation across eight datasets, two backbone architectures, and three training regimes, demonstrating consistent gains over AutoAugment, RandAugment, and TrivialAugment, particularly at low labelling budgets and in the medical imaging domain where domain-adaptive augmentation selection is most beneficial. 2 Related Work 2.1 Data Augmentation Data augmentation addresses two central challenges in supervised learning: insufficient training data and overfitting. By augmenting samples and appending them to the training pool, one artificially increases the dataset size; by applying a different transformation each time a sample is presented, the network is forced to rely on robust, semantically meaningful features rather than surface-level memorisation. The earliest augmentations were geometric transformations applied when training LeNet-5 [LeCun et al.(1998)LeCun, Bottou, Bengio, and Haffner], including flipping, shearing, and rotation. When training AlexNet, photometric augmentations altering pixel-level properties like brightness and contrast were introduced alongside geometric ones [Krizhevsky et al.(2012)Krizhevsky, Sutskever, and Hinton]. Subsequent work added augmentations like Cutout [DeVries and Taylor(2017)] and Mixup [Zhang et al.(2018)Zhang, Cisse, Dauphin, and Lopez-Paz], further expanding the augmentation repertoire. These augmentations were selected largely by heuristics, motivating the development of automatic augmentation methods. AutoAugment [Cubuk et al.(2019)Cubuk, Zoph, Mané, Vasudevan, and Le] was the first to frame augmentation selection as a search problem. It parameterises the policy search space with a recurrent neural network controller trained via policy gradient on a small proxy task. While effective, the search is computationally expensive. RandAugment [Cubuk et al.(2020)Cubuk, Zoph, Shlens, and Le] reduced the search space to two hyperparameters, achieving comparable results at a fraction of the cost; augmentations are drawn uniformly at random from a fixed pool, all at the chosen intensity. TrivialAugment [Müller and Hutter(2021)] removed the need for search entirely: for each sample, it uniformly samples a single augmentation type and a random intensity, achieving strong results with zero hyperparameter tuning. Related approaches explicitly optimize augmentation hardness. MaxUp [Gong et al.(2021)Gong, Ren, Ye, and Liu] generates multiple randomly augmented views and trains on the view with the largest loss, whereas AugMax [Wang et al.(2021a)Wang, Xiao, Kossaifi, Yu, Anandkumar, and Wang] constructs adversarial mixtures of randomly sampled augmentations to improve robustness. ActiveAugment differs by framing augmentation choice as online active selection: candidate views are scored using predictive informativeness and their relative representation shift, and standard AL acquisition strategies translate to the augmentation-selection problem. 2.2 Active Learning Active learning is a framework designed to maximise data efficiency under labelling-budget constraints by querying the most informative instances from an unlabelled pool [Settles(2009)]. Two key considerations govern the selection of samples to label. First, one should prioritise uncertain samples, those for which the current model is least confident, since labelling them yields the largest expected reduction in generalisation error. Methods operating on this principle are called uncertainty-based methods. Second, selecting a batch of highly similar uncertain samples is wasteful; the selection should also be diverse so that it covers the available input space. Methods operating on this principle are called diversity-based methods. Among uncertainty-based methods, Least Confident sampling [Settles(2009)] selects the instances whose maximum predicted class probability is lowest. Margin Sampling focuses on the decision boundary, prioritising instances whose two highest class probabilities are closest. Bayesian Active Learning by Disagreement (BALD) [Gal et al.(2017)Gal, Islam, and Ghahramani] uses Monte Carlo Dropout to estimate the mutual information between predictions and model parameters. Coreset [Sener and Savarese(2018)] and TypiClust [Hacohen et al.(2022)Hacohen, Dekel, and Weinshall] are representative diversity-based methods: Coreset selects samples that maximise the minimum distance to already-labelled instances, ensuring high coverage, while TypiClust uses clustering to find typical representatives that avoid overly difficult samples in early rounds. Hybrid approaches combine both criteria; BADGE [Ash et al.(2020)Ash, Zhang, Krishnamurthy, Langford, and Agarwal] operates in gradient-embedding space, using gradient magnitude as a proxy for uncertainty and gradient direction as a proxy for diversity, selecting a batch that is simultaneously uncertain and diverse. Although AL methods were originally formulated for unlabelled sample selection, most methods do not inherently depend on whether the informativeness measure is applied to a novel image or an augmented view of an existing one. This opens the door to translating sample selection directly into augmentation selection, which is the central idea of this work. 2.3 Contrastive and Invariance Learning Actively selecting informative augmentations provides the model with challenging training views; however, learning robust representations from those views requires an appropriate objective. Standard cross-entropy loss is susceptible to shortcut learning, where the model exploits superficial correlations that do not generalise [Geirhos et al.(2020)Geirhos, Jacobsen, Michaelis, Zemel, Brendel, Bethge, and Wichmann]. Contrastive learning methods address this by restructuring the geometry of the latent space: representations of related samples are pulled together while those of unrelated samples are pushed apart. SimCLR [Chen et al.(2020)Chen, Kornblith, Norouzi, and Hinton] and MoCo [He et al.(2020)He, Fan, Wu, Xie, and Girshick] are influential self-supervised contrastive frameworks. Each sample is augmented in two independent ways to produce a positive pair; the model is optimised to maximise agreement within positive pairs while repelling other samples in the batch as negatives. While effective for unsupervised training, instance-discrimination objectives treat instances of the same class as negatives, which is suboptimal in supervised learning. Supervised Contrastive Learning (SupCon) [Khosla et al.(2020)Khosla, Teterwak, Wang, Sarna, Tian, Isola, Maschinot, Liu, and Krishnan] addresses this limitation: it treats all augmented views that share the same class label as positives, while views from different classes serve as negatives. This enforces intra-class invariance while maintaining inter-class separation. Existing implementations sample positive augmented views at random, which may spend computation on views that provide little additional learning signal. ActiveAugment instead prioritises informative views from a predefined, domain-valid augmentation space. 3 Method 3.1 Augmentation Space We define an augmentation as a parameterised function aI(x)a^I(x), where x is an input image and I∈L,M,HI∈\ L,\, M,\, H\ denotes the intensity level (low, medium, high). Since each intensity level corresponds to a range of raw transformation strengths, the exact magnitude is sampled uniformly at random from within that range at each application. Most augmentations are controlled by a single continuous strength parameter; augmentations such as random horizontal flipping have no associated magnitude and therefore no intensity level. The intensity ranges for each augmentation are fixed before training. We use heuristics to choose initial ranges and verify them using the Structural Similarity Index Measure (SSIM) [Wang et al.(2004)Wang, Bovik, Sheikh, and Simoncelli] to confirm that each level produces a qualitatively distinct degree of image corruption. Note that intensity does not map monotonically to perceptual corruption for all augmentations: for example, low Brightness yields a dark image, while medium Brightness produces an image close to the original. Note that the candidate space is defined before training using domain knowledge; ActiveAugment therefore assumes, rather than automatically verifies, that candidate transformations are label-preserving for the task. SSIM is used only to verify distinct perturbation levels. We sample the low/medium/high intensity during selection for tractability, although the framework can naturally extend to joint augmentation–intensity selection. This discrete intensity parameterisation limits the effective search space to ||×3|A|× 3 (minus the intensity-free augmentations), making online selection tractable. 3.2 Active Invariance Learning via Selective Augmentation The goal is to select, at each training step, the augmentation under which the model is most fragile, and to enforce invariance to that augmentation during training. Let =(xi,yi)i=1ND=\(x_i,y_i)\_i=1^N be a labelled dataset, fθ:→ℝdf_θ:X ^d an encoder parameterised by θ, gϕg_φ a classification head parameterised by ϕφ, and A the augmentation space. Classical invariance learning enforces: fθ(x)≈fθ(a(x)),∀a∈,∀x∈.f_θ(x)≈ f_θ\! (a(x) ), ∀\,a ,\;∀\,x . (1) Enforcing invariance uniformly across A is suboptimal: augmentations affect the model disproportionately. Within a predefined domain-valid augmentation space, different transformations affect the model disproportionately: some expose model fragilities whereas others provide little additional learning signal. Hence, we decouple training into two phases: • Selection: For each sample xix_i, identify the augmentation a∈a from the predefined domain-valid candidate space, under which the model is currently most non-invariant. • Learning: Enforce invariance specifically to the selected augmentations. We combine two complementary informativeness signals during selection. Predictive uncertainty (uiku_ik). We define predictive uncertainty as a score indicating how far the model is from a confident, stable prediction when presented with augmented view x^i,k=akI(xi) x_i,k=a_k^I(x_i). Any AL acquisition function can serve this role; we adapt existing AL strategies to score augmented views rather than unlabelled samples (see Section 3.3). Feature discrepancy (dikd_ik). We define feature discrepancy as the distance between the representation of the original sample and that of its augmented counterpart in the encoder’s embedding space: dik=D(fθ(xi),fθ(x^i,k)),d_ik=D\! (f_θ(x_i),\,f_θ( x_i,k) ), (2) where DD is a distance metric such as cosine distance. This measures how much augmentation aka_k displaces the representation relative to the clean sample. To prevent the score from being dominated by globally large or small embedding norms, we normalise dikd_ik by the mean discrepancy across samples for the same augmentation: d¯k=1B∑i=1Bdik, d_k= 1B _i=1^Bd_ik, (3) where B is the minibatch size. The combined selection score is: sik=dikd¯k⋅uik.s_ik= d_ik d_k· u_ik. (4) The selected augmentation for sample xix_i is: ai∗=argmaxk∈1,…,Ksik.a_i^*= *arg\,max_k∈\1,…,K\s_ik. (5) Theoretical motivation. The selection criterion in Eq. (4) can be grounded in a generalisation bound under augmentation-induced distribution shift. Let P_X denote the clean data distribution and P_A the distribution over augmented inputs. The generalisation error of fθf_θ under augmentation shift is bounded by [Ben-David et al.(2010)Ben-David, Blitzer, Crammer, Kulesza, Pereira, and Vaughan, Mansour et al.(2009)Mansour, Mohri, and Rostamizadeh]: ε(fθ)≤ε(fθ)+dℋ(P,P)⏟representation shift+λ∗⏟ideal joint risk, _A(f_θ)≤ _X(f_θ)+ d_H(P_X,P_A)_representation shift+ λ^*_ideal joint risk, (6) where dℋd_H is the ℋH-divergence between the feature distributions of clean and augmented data. The discrepancy term dikd_ik is a sample-level estimator of this divergence; minimising i[dik∗]E_i[d_ik^*] via the contrastive objective tightens the bound for the selected augmentations. By a standard AL argument [Settles(2009)], training on samples where the model exhibits high predictive entropy yields the largest expected reduction in loss per update, as gradient contributions scale with prediction error. Combining both signals, the selection score siks_ik approximately maximises: Δℒik∝uik⏟expected loss reduction⋅dik/d¯k⏟relative shift contribution, _ik\; \; u_ik_expected loss reduction· d_ik/ d_k_relative shift contribution, (7) which can be interpreted as a per-sample, per-augmentation expected improvement in the generalisation bound. The greedy selection rule ai∗=argmaxksika_i^*= *arg\,max_ks_ik is a principled approximation to the problem of jointly optimising augmentation selection to maximise bound tightening. Algorithm. Figure 1 provides an overview of ActiveAugment. For each minibatch ℬ⊂B of size B, we perform online selection as follows. For each sample xi∈ℬx_i , we sample an intensity level I∼L,M,HI \ L, M, H\ and generate K=||K=|A| candidate augmented views: x^i,kk=1K=akI(xi)|ak∈.\ x_i,k\_k=1^K= \a_k^I(x_i)\; |\;a_k \. (8) For each candidate x^i,k x_i,k we compute the score siks_ik using the current model state, and select the highest-scoring augmentation: ai∗=argmaxksik.a_i^*= *arg\,max_k\,s_ik. (9) The model is then trained on the expanded minibatch ℬfinal=(xi,yi)∪(ai∗(xi),yi)i=1BB_final=\(x_i,y_i)\∪\(a_i^*(x_i),y_i)\_i=1^B, updating encoder parameters θ and classifier parameters ϕφ. Figure 1: Overview of ActiveAugment. For each minibatch, every sample is augmented with all available augmentations at a randomly sampled intensity level. An active selection strategy scores each candidate and selects the most informative augmentation per sample. The model is then trained on both the original and selected augmented samples using the combined supervised classification and contrastive loss. 3.3 Selection Strategies AL provides the scoring framework for the selection phase. For uncertainty-based strategies, the translation from sample scoring to augmentation scoring is straightforward: unlabelled samples are simply replaced by augmented views of labelled samples. This holds for Entropy Sampling, Least Confident Sampling, Margin Sampling, and BALD [Gal et al.(2017)Gal, Islam, and Ghahramani, Settles(2009)]. Diversity-based strategies typically operate on the global labelled pool. We use the local clean minibatch as the reference distribution, making diversity computation tractable in the online setting. We apply this local adaptation to Coreset, BADGE, and TypiClust [Ash et al.(2020)Ash, Zhang, Krishnamurthy, Langford, and Agarwal, Sener and Savarese(2018), Hacohen et al.(2022)Hacohen, Dekel, and Weinshall]. For BAL [Li et al.(2024)Li, Chen, Yu, Liu, and Jia], instead of performing global k-means clustering and scoring based on the Cluster Distance Difference, we compute a batch-adapted Cluster Distance Difference: for each augmented candidate, we measure its distance to its own clean source image versus its distance to the nearest other clean image in the minibatch. In total we implement and compare eight active learning strategies for augmentation scoring. 3.4 Training Objective We train the model with a combined supervised classification and supervised contrastive objective, with the balance controlled by hyperparameter λ>0λ>0: ℒ=ℒsup+λℒcon.L=L_sup+λ\,L_con. (10) This joint objective prevents feature collapse, encourages invariance to the selected augmentations within classes, and maintains discriminative structure across classes. Supervised classification loss. ℒsupL_sup is the standard cross-entropy loss ℓ applied to both the original samples and their selected augmented counterparts: ℒsup=1B∑i=1B[ℓ(gϕ(fθ(xi)),yi)+ℓ(gϕ(fθ(ai∗(xi))),yi)].L_sup= 1B _i=1^B [ \! (g_φ(f_θ(x_i)),\,y_i )+ \! (g_φ(f_θ(a_i^*(x_i))),\,y_i ) ]. (11) Supervised contrastive loss. ℒconL_con is the SupCon loss [Khosla et al.(2020)Khosla, Teterwak, Wang, Sarna, Tian, Isola, Maschinot, Liu, and Krishnan] applied over the expanded minibatch of size 2B2B (original plus augmented samples), using cosine similarity as the kernel: ℒcon=−12B∑i=12B1|P(i)|∑j∈P(i)logexp(sim(zi,zj)/τ)∑n=1,n≠i2Bexp(sim(zi,zn)/τ),L_con=- 12B _i=1^2B 1|P(i)| _j∈ P(i) \! (sim(z_i,z_j)/τ ) _n=1,\,n≠ i^2B \! (sim(z_i,z_n)/τ ), (12) where zi=gϕ(fθ(xi))z_i=g_φ(f_θ(x_i)), sim(u,v)=u⊤v/(‖u‖‖v‖)sim(u,v)=u v/(\|u\|\|v\|) is cosine similarity, τ>0τ>0 is a temperature hyperparameter, and P(i)=j≠i:yj=yiP(i)=\j≠ i:y_j=y_i\ is the set of positive pairs, i.eall samples in the expanded minibatch that share the same class label as sample i. The SupCon loss enforces intra-class invariance while maintaining inter-class separation. In this work, gϕg_φ serves as the classification head for ℒsupL_sup and the projection for ℒconL_con; a unified head simplifies the architecture and was found empirically sufficient for the scales studied here. 3.5 Backbone Architectures We evaluate ActiveAugment using two backbone architectures that represent the two dominant paradigms in modern image recognition: ResNet-18 [He et al.(2016)He, Zhang, Ren, and Sun], a classic residual convolutional network, and TinyViT [Wu et al.(2022)Wu, Zhang, Wang, Wei, Xie, Chen, and Hu], a compact vision transformer. Both backbones have approximately 11 million parameters. A classification head, using a single fully connected layer with output dimension equal to the number of classes, is appended to each backbone. We consider three training regimes: random initialisation, where all weights are initialised randomly and updated throughout training from scratch; full fine-tuning, where weights are initialised from pre-trained networks and all parameters are updated; linear probing, where pre-trained weights are frozen and only the classification head is updated. 3.6 Data Pre-processing All images pass through a unified pre-processing pipeline. When using pre-trained weights, greyscale images are converted to three-channel RGB by replicating the single channel, so as to match the domain of the pre-trained features. This channel replication step is omitted for random-initialisation experiments. All images are resized to 256×256256× 256 pixels for consistency across datasets. Pixels are then cast to float32 in the range [0,1][0,1]. Augmentations are applied at this stage, after which pixel values are clamped back to [0,1][0,1] to handle any out-of-range values introduced by intensity transformations. Finally, images are normalised using the per-channel mean and standard deviation computed from each specific training dataset; the same dataset-specific statistics are used regardless of whether pre-trained weights are employed, ensuring a consistent normalisation scheme across all experimental conditions. 4 Experiments and Results 4.1 Datasets We evaluate on eight datasets from two main domains: natural image datasets (CIFAR-10, CIFAR-100 [Krizhevsky(2009)], STL-10 [Coates et al.(2011)Coates, Ng, and Lee], MNIST [LeCun et al.(1998)LeCun, Bottou, Bengio, and Haffner]) and medical image datasets (BRISC [Fateh et al.(2026)Fateh, Rezvani, Moayedi, Rezvani, Fateh, Fateh, and Abolghasemi], BUSI [Vallez et al.(2025)Vallez, Bueno, Deniz, Rienda, and Pastor], FETAL-PLANES-DB [Burgos-Artizzu et al.(2020)Burgos-Artizzu, Coronado-Gutierrez, Valenzuela-Alcaraz, Bonet-Carne, Eixarch, Crispi, and Gratacós], ISIC-2019 [Codella et al.(2018)Codella, Gutman, Celebi, Helba, Marchetti, Dusza, Kalloo, Liopyris, Mishra, Kittler, and Halpern, Hernández-Pérez et al.(2024)Hernández-Pérez, Combalia, Podlipnik, Codella, Rotemberg, Halpern, Reiter, Carrera, Barreiro, Helba, Puig, Vilaplana, and Malvehy, Tschandl et al.(2018)Tschandl, Rosendahl, and Kittler]). Medical imaging benchmarks are particularly relevant because data acquisition and annotation are often expensive, and access to large volumes of labelled data is uncommon, making low-budget performance critical. 4.2 Experimental Protocol For each dataset, we sample 1,000 images using stratified sampling to ensure class balance, following the same controlled low-label protocol across datasets. We allocate 80% (800) to the training pool, 10% (100) to validation, and 10% (100) to the held-out clean test set. For robustness evaluation, every held-out test image is additionally evaluated under every augmentation and available intensity. We refer to the aggregate of the clean and systematically augmented views as the overall test set; clean performance is reported separately on the 100 original held-out images. Thus, although the clean test contains 100 images by design, the robustness evaluation contains substantially more test views. The training pool is split into budgets of size 10,20,50,100,200,500,800\10,20,50,100,200,500,800\, constructed by cumulatively sampling stratified subsets. This cumulative design ensures that performance improvements from one budget to the next are attributable solely to the additional labelled samples. Models are retrained independently for each budget level. Each model is trained for 50 epochs with the Adam optimiser [Kingma and Ba(2015)] at a learning rate of 10−410^-4. We set λ=0.1λ=0.1 and τ=0.07τ=0.07 and run five independent trials with different random seeds, reporting mean performance with ±1± 1 standard deviation intervals. The model checkpoint with the highest macro-averaged F1-score on the validation set is used for test evaluation. ActiveAugment uses a batch size of 16, which is expanded to an effective size of 32 after augmentation; all baselines use a batch size of 32. All experiments are run on either an NVIDIA TITAN RTX (24 GB VRAM) or a Quadro RTX 6000 (24 GB VRAM). We compare against AutoAugment [Cubuk et al.(2019)Cubuk, Zoph, Mané, Vasudevan, and Le], RandAugment [Cubuk et al.(2020)Cubuk, Zoph, Shlens, and Le], and TrivialAugment [Müller and Hutter(2021)], all accessed via the torchvision transforms API. For RandAugment, the pipeline length is set to 2, and our intensity levels are mapped to the corresponding magnitude values. TrivialAugment requires no hyperparameter configuration. We also include a No Augmentation baseline for reference. All baselines use standard cross-entropy loss with a batch size of 32. Primarily, we report results using ResNet-18 under the full fine-tuning regime with the BADGE strategy, as it was found to be consistently among the top-performing strategies (see Section 4.4). However, consistent patterns are also observed with TinyViT (under random initialisation and linear probing), as well as in a continual learning setting where increasing budgets are accumulated progressively during training. 4.3 Results on Natural Image Datasets Figure 2 compares ActiveAugment with the SOTA baselines and the no-augmentation baseline across natural image datasets under full fine-tuning with ResNet-18 and BADGE strategy. Figure 2: Test results on natural image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Shaded bands indicate ±1± 1 standard deviation. Overall: test set including augmented views. Clean: original test images only. Figure 3: Per-augmentation test F1 on MNIST (ResNet-18 backbone, full fine-tuning). BADGE, Margin, Entropy, and Coreset substantially outperform BAL, Least Confident, and TypiClust, with the largest gap observed under the Flip augmentation. On the overall test set, ActiveAugment matches or outperforms all baselines. The difference is most pronounced at low budgets: training without augmentations is clearly the weakest strategy, the three SOTA methods perform comparably to one another, and ActiveAugment achieves a small but consistent improvement. The advantage shrinks at larger budgets, as the increased amount of labelled data reduces the relative benefit of adaptive augmentation selection. The pattern is particularly clear on CIFAR-100, where ActiveAugment maintains a visible margin over all SOTA baselines across the full budget range. On the clean test set, the picture is more nuanced. For STL-10 and MNIST, augmentation confers no advantage over the no-augmentation baseline; this is consistent with the nature of these datasets, where the dominant discriminative features are not substantially affected by the augmentations we employ. For CIFAR-10 and CIFAR-100, augmentation clearly improves clean-image performance, and ActiveAugment is competitive with SOTA across all budgets. These results indicate that the gains from ActiveAugment on the overall test set stem from improved robustness to the specific augmentation types considered, rather than solely from improved clean-image accuracy. It learns to prioritise augmentations under which the model is fragile, thereby improving robustness without sacrificing generalisation to clean data. We additionally test generalisation to held-out transformations by excluding one augmentation type from the training pool and applying it only at test time. ActiveAugment remains competitive with or improves over the random and augmentation baselines, although its gains are smaller than for transformations observed during training. To illustrate how the selection strategies affect test-time generalisation performance, Figure 3 reports the per-augmentation test F1 on MNIST. BADGE consistently ranks among the top-performing strategies alongside Margin, Entropy, and Coreset. In contrast, BAL, Least Confident, and TypiClust consistently underperform, particularly under the Flip and Gaussian Noise augmentations, while BALD and Random also show weaker performance in several settings. The most striking gap appears under the Flip augmentation, where the best strategies reach near-perfect F1 at moderate budgets while BAL, Least Confident, Random and TypiClust lag significantly behind throughout training. Figure 4: Left: Augmentation selection frequency (%) on natural image datasets (ResNet-18 backbone, full fine-tuning), aggregated over all datasets and budgets. Each column corresponds to one active selection strategy; the rightmost column is the random baseline. Random frequencies are non-uniform because the final score is weighted by feature discrepancy. Strategies that favour Gaussian Blur consistently underperform (see Section 4.3). Right: Evolution of augmentation selection frequencies (top) and selection confidence (bottom) over normalised training epochs, aggregated over all natural image datasets and budgets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Confidence is measured as the coefficient of variation of the spread (Std/Mean) and of the margin (Margin/Mean) across augmentation scores. Both confidence metrics rise monotonically, indicating that the selection policy becomes increasingly decisive as the encoder representations stabilise. 4.4 Augmentation Selection Frequency Since ActiveAugment selects a different augmentation for each sample, we can analyse which ones each informativeness strategy prefers. Figure 4 (left) reports selection frequencies aggregated over all natural image datasets and budgets under full fine-tuning with ResNet-18. Effect of feature discrepancy. The random baseline (uniform augmentation sampling) yields non-uniform selection frequencies because the final score is weighted by feature discrepancy. In particular, Flip is over-selected (22.5% vs. the expected 16.7%), while Gaussian Blur is substantially under-selected (13.4%). This reveals that horizontal/vertical flipping induces a large representation shift in fine-tuned ResNet-18 features, while Gaussian Blur has comparatively little effect on the embedding, despite being metrically perceptible. Strategy comparison. BADGE, BALD, Coreset, Entropy, and Margin all exhibit a strong preference for Flip, followed by Gaussian Noise and Rotation, and largely ignore Gaussian Blur. In contrast, BAL, Least Confident, and TypiClust preferentially select Gaussian Blur. Crucially, this behavioural difference has a performance consequence: BAL, Least Confident, and TypiClust consistently underperform the other five strategies across datasets, backbones, and training regimes. These results provide strong evidence that Gaussian Blur, despite being perceptibly distinct, does not challenge the model’s learned feature representations in a way that promotes generalisation. A good selection strategy should therefore avoid it in favour of transformations that expose genuine model fragilities. 4.5 Policy Evolution During Training Figure 4 (right) shows how the selection policy of ActiveAugment using BADGE evolves over the course of training, together with a measure of selection confidence. Results are aggregated over all natural image datasets and budgets. We measure confidence using two metrics: the coefficient of variation (CV) of the spread (standard deviation over mean score) and the CV of the margin (gap between the highest and mean score). At the start of training, confidence is low and the strategy is essentially uncertain about which augmentation to prioritise. As training progresses, confidence rises monotonically, reflecting the model’s increasingly stable representations. This emergent learning of a stable selection policy is a direct consequence of the feature discrepancy becoming more informative as the encoder matures. In terms of augmentation preference, Flip is initially selected less than 20% of the time but rises to 30–35% by the end of training, becoming the single most preferred augmentation. At low intensity levels, Gaussian Noise and Rotation are preferred in early epochs but are displaced by Brightness and Contrast augmentations in later stages. Notably, GaussianBlur (L), Brightness (M), and Contrast (M) are virtually never selected across all epochs. Inspection of image corruption metrics confirms that these three configurations have minimal perceptual impact on the image, explaining why the model does not benefit from learning invariance to them. This consistency between the SSIM-based analysis and the selection behaviour provides a meaningful validation of the feature discrepancy component. 4.6 Results on Medical Image Datasets Figure 5 compares ActiveAugment with the baselines across the four medical image datasets under full fine-tuning with ResNet-18 and BADGE strategy. The overall pattern mirrors the natural image results: ActiveAugment achieves a clear advantage on the overall test set, and the gap narrows on the clean test set. On the clean test set, ActiveAugment is consistently among the top-performing methods across all four medical datasets and budget levels, whereas on natural image datasets, the clean-set gap was less pronounced. We attribute this to the out-of-domain nature of these datasets relative to the ImageNet pre-trained models: the SOTA baselines use augmentation policies optimised for natural images (less well-suited to the medical imaging domain), whereas ActiveAugment’s data-driven selection adapts automatically to the statistical properties of the target domain. This cross-domain adaptability is a key practical advantage of the proposed framework. Figure 5: Test results on medical image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Shaded bands indicate ±1± 1 standard deviation. Overall: test set including augmented views. Clean: original test images only. ActiveAugment is consistently among the top-performing methods on the clean test set across all four datasets and all budgets. On BRISC, ActiveAugment reaches approximately 0.95 macro-F1 on the clean test set at the highest budget, outperforming all baselines, while on BUSI, the overall and clean curves show a clear separation from AutoAugment and the no-augmentation baseline. On ISIC-2019, which is a challenging dataset due to class imbalance, all methods show wider variance, but ActiveAugment remains competitive even at very low budgets (10–50 samples). To illustrate how selection strategies affect test-time generalisation performance in the medical domain, Figure 6 reports the per-augmentation test F1 on FETAL-PLANES. The ordering of strategies is consistent with the natural image results: BADGE, Margin, Entropy, and Coreset consistently outperform BAL, Least Confident, and TypiClust. The most pronounced gap is observed under Gaussian Noise, reinforcing the finding that noise-based augmentation is a particularly informative perturbation for medical image models. Figure 6: Per-augmentation test F1 on FETAL-PLANES (ResNet-18 backbone, full fine-tuning). The performance gap between top and bottom strategies is largest under Gaussian Noise, suggesting that noise robustness is discriminative for medical image recognition. Selection frequency analysis on medical data (Figure 7 (left)) shows broadly consistent trends with the natural image results. The preference for Flip is somewhat less pronounced, and Gaussian Noise is selected as frequently as Flip by several strategies, suggesting that additive noise is a more discriminative perturbation in the medical imaging domain than in natural imaging. This difference is consistent with the nature of medical images, where structural content is often more orientation-invariant and noise sensitivity is higher. Policy evolution on medical data (Figure 7 (right)) also shows very similar dynamics to the natural image case, with confidence rising steadily throughout training, confirming that the selection mechanism is domain-agnostic and adapts reliably across image modalities. Figure 7: Left: Augmentation selection frequency (%) on medical image datasets (ResNet-18 backbone, full fine-tuning), aggregated over all datasets and budgets. Compared to natural images, Gaussian Noise is selected more frequently (on par with Flip for several strategies). Right: Policy evolution on medical image datasets (ResNet-18 backbone, full fine-tuning, BADGE strategy). Selection confidence rises monotonically as on natural image data, confirming the domain-agnostic behaviour of the selection mechanism. 5 Conclusion We have presented ActiveAugment, a unified framework that reformulates data augmentation selection as an online active learning problem. ActiveAugment scores augmented views per sample using predictive uncertainty and normalised feature discrepancy, and trains the model with a joint classification and supervised contrastive objective that enforces invariance to the selected, model-fragile augmentations. The selection criterion is grounded in domain adaptation theory. By maximising the per-sample product of uncertainty and relative feature shift, ActiveAugment tightens the augmentation-induced generalisation bound at each step. ActiveAugment consistently outperforms AutoAugment, RandAugment, and TrivialAugment under the controlled augmentation shifts considered, across datasets, backbone architectures, and training regimes, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where ImageNet-pretrained models face a larger domain shift and existing augmentation policies are optimised for natural images, ActiveAugment demonstrates strong cross-domain adaptability by learning the augmentation policy from the evolving model state. Analysis of selection behaviour reveals that the policy evolves meaningfully during training, becoming increasingly decisive as encoder representations stabilise, and that strategy choice directly shapes generalisation: strategies that favour Gaussian Blur underperform those that favour Flip and Gaussian Noise. This constitutes an empirical validation of the feature discrepancy component and provides practical guidance for strategy selection. Acknowledgements This project is supported by the Pioneer Centre for AI, funded by the Danish National Research Foundation (grant number P1). References [Ash et al.(2020)Ash, Zhang, Krishnamurthy, Langford, and Agarwal] Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations, 2020. [Ben-David et al.(2010)Ben-David, Blitzer, Crammer, Kulesza, Pereira, and Vaughan] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79(1):151–175, 2010. [Burgos-Artizzu et al.(2020)Burgos-Artizzu, Coronado-Gutierrez, Valenzuela-Alcaraz, Bonet-Carne, Eixarch, Crispi, and Gratacós] Xavier P. Burgos-Artizzu, David Coronado-Gutierrez, Brenda Valenzuela-Alcaraz, Elisenda Bonet-Carne, Elisenda Eixarch, Fatima Crispi, and Eduard Gratacós. FETAL_PLANES_DB: Common maternal-fetal ultrasound images. Zenodo, 2020. [Chen et al.(2020)Chen, Kornblith, Norouzi, and Hinton] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, pages 1597–1607, 2020. [Chi et al.(2020)Chi, Ma, Wu, Chen, Lu, and Gu] Weicheng Chi, Lin Ma, Junjie Wu, Mingli Chen, Weiguo Lu, and Xuejun Gu. Deep learning-based medical image segmentation with limited labels. Physics in Medicine & Biology, 65(23):235001, 2020. [Coates et al.(2011)Coates, Ng, and Lee] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 215–223, 2011. [Codella et al.(2018)Codella, Gutman, Celebi, Helba, Marchetti, Dusza, Kalloo, Liopyris, Mishra, Kittler, and Halpern] Noel C. F. Codella, David Gutman, M. Emre Celebi, Brian Helba, Michael A. Marchetti, Stephen W. Dusza, Aadi Kalloo, Konstantinos Liopyris, Nabin Mishra, Harald Kittler, and Allan Halpern. Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging), hosted by the international skin imaging collaboration. In 2018 IEEE 15th International Symposium on Biomedical Imaging, pages 168–172, 2018. [Cubuk et al.(2019)Cubuk, Zoph, Mané, Vasudevan, and Le] Ekin D. Cubuk, Barret Zoph, Dandelion Mané, Vijay Vasudevan, and Quoc V. Le. AutoAugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 113–123, 2019. [Cubuk et al.(2020)Cubuk, Zoph, Shlens, and Le] Ekin D. Cubuk, Barret Zoph, Jon Shlens, and Quoc Le. RandAugment: Practical automated data augmentation with a reduced search space. In Advances in Neural Information Processing Systems, volume 33, pages 18613–18624, 2020. [DeVries and Taylor(2017)] Terrance DeVries and Graham W. Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017. [Fateh et al.(2026)Fateh, Rezvani, Moayedi, Rezvani, Fateh, Fateh, and Abolghasemi] Amirreza Fateh, Yasin Rezvani, Sara Moayedi, Sadjad Rezvani, Fatemeh Fateh, Mansoor Fateh, and Vahid Abolghasemi. BRISC: Annotated dataset for brain tumour segmentation and classification. Scientific Data, 13(1):361, 2026. [Gal et al.(2017)Gal, Islam, and Ghahramani] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep Bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1183–1192, 2017. [Geirhos et al.(2020)Geirhos, Jacobsen, Michaelis, Zemel, Brendel, Bethge, and Wichmann] Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020. [Gong et al.(2021)Gong, Ren, Ye, and Liu] Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. MaxUp: Lightweight adversarial training with data augmentation improves neural network training. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2474–2483. IEEE, 2021. [Hacohen et al.(2022)Hacohen, Dekel, and Weinshall] Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on a budget: Opposite strategies suit high and low budgets. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 8175–8195, 2022. [He et al.(2016)He, Zhang, Ren, and Sun] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016. [He et al.(2020)He, Fan, Wu, Xie, and Girshick] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2020. [Hernández-Pérez et al.(2024)Hernández-Pérez, Combalia, Podlipnik, Codella, Rotemberg, Halpern, Reiter, Carrera, Barreiro, Helba, Puig, Vilaplana, and Malvehy] C. Hernández-Pérez, Marc Combalia, S. Podlipnik, Noel C. F. Codella, Veronica Rotemberg, Allan C. Halpern, Ofer Reiter, C. Carrera, A. Barreiro, Brian Helba, Susana Puig, Veronica Vilaplana, and Josep Malvehy. BCN20000: Dermoscopic lesions in the wild. Scientific Data, 11(1):641, 2024. [Khosla et al.(2020)Khosla, Teterwak, Wang, Sarna, Tian, Isola, Maschinot, Liu, and Krishnan] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, pages 18661–18673, 2020. [Kingma and Ba(2015)] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. [Krizhevsky(2009)] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. [Krizhevsky et al.(2012)Krizhevsky, Sutskever, and Hinton] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, volume 25, pages 1097–1105, 2012. [LeCun et al.(1998)LeCun, Bottou, Bengio, and Haffner] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. [Li et al.(2024)Li, Chen, Yu, Liu, and Jia] Jingyao Li, Pengguang Chen, Shaozuo Yu, Shu Liu, and Jiaya Jia. BAL: Balancing diversity and novelty for active learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):3653–3664, 2024. [Mansour et al.(2009)Mansour, Mohri, and Rostamizadeh] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In Proceedings of the 22nd Conference on Learning Theory, pages 19–30, 2009. [Müller and Hutter(2021)] Samuel G. Müller and Frank Hutter. TrivialAugment: Tuning-free yet state-of-the-art data augmentation. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 754–762, 2021. [Sener and Savarese(2018)] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018. [Settles(2009)] Burr Settles. Active learning literature survey. Technical Report 1648, University of Wisconsin-Madison, 2009. [Shorten and Khoshgoftaar(2019)] Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):60, 2019. [Tschandl et al.(2018)Tschandl, Rosendahl, and Kittler] Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data, 5(1):180161, 2018. [Vădineanu et al.(2024)Vădineanu, Pelt, Dzyubachyk, and Batenburg] Serban Vădineanu, Daniël M. Pelt, Oleh Dzyubachyk, and Kees Joost Batenburg. Reducing manual annotation costs for cell segmentation by upgrading low-quality annotations. Journal of Imaging, 10(7):172, 2024. [Vallez et al.(2025)Vallez, Bueno, Deniz, Rienda, and Pastor] Noelia Vallez, Gloria Bueno, Oscar Deniz, Miguel Angel Rienda, and Carlos Pastor. BUS-UCLM: Breast ultrasound lesion segmentation dataset. Scientific Data, 12(1):242, 2025. [Wang et al.(2021a)Wang, Xiao, Kossaifi, Yu, Anandkumar, and Wang] Haotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, and Zhangyang Wang. AugMax: Adversarial composition of random augmentations for robust training. Advances in Neural Information Processing Systems, 34:237–250, 2021a. [Wang et al.(2021b)Wang, Gao, Long, and Wang] Ximei Wang, Jinghan Gao, Mingsheng Long, and Jianmin Wang. Self-tuning for data-efficient deep learning. In Proceedings of the 38th International Conference on Machine Learning, pages 10738–10748, 2021b. [Wang et al.(2024)Wang, Han, Wang, Song, Tian, and Huang] Yulin Wang, Yizeng Han, Chaofei Wang, Shiji Song, Qi Tian, and Gao Huang. Computation-efficient deep learning for computer vision: A survey. Cybernetics and Intelligence, 1:9390002, 2024. [Wang et al.(2004)Wang, Bovik, Sheikh, and Simoncelli] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. [Wu et al.(2022)Wu, Zhang, Wang, Wei, Xie, Chen, and Hu] Kan Wu, Jinnian Zhang, Huiyu Wang, Yanbing Wei, Guodong Xie, Zhenzhong Chen, and Han Hu. TinyViT: Fast pretraining distillation for small vision transformers. In European Conference on Computer Vision, 2022. [Zhang et al.(2018)Zhang, Cisse, Dauphin, and Lopez-Paz] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.