Paper deep dive
Predictive Batch Scheduling: Accelerating Language Model Training Through Loss-Aware Sample Prioritization
Sumedh Rasal
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/21/2026, 12:33:54 AM
Summary
The paper introduces Predictive Batch Scheduling (PBS), a training optimization technique for language models that accelerates convergence by dynamically prioritizing high-loss samples. PBS uses a lightweight, online-trained linear predictor to estimate sample difficulty based on four static token-level features: token frequency, sequence length, vocabulary diversity, and rare token ratio. Experiments on a 130M parameter transformer show PBS achieves 6-13% faster convergence compared to uniform random sampling, with the predictor achieving a 0.44 correlation with actual loss. This method avoids the computational overhead of per-sample loss tracking required by hard example mining while offering more adaptivity than static curriculum learning.
Entities (9)
Relation Signals (9)
Predictive Batch Scheduling → uses → Linear Predictor
confidence 95% · PBS employs a lightweight linear predictor trained online to estimate sample difficulty from static token-level features.
Linear Predictor → usesfeatures → Token Frequency
confidence 95% · Our predictor achieves 0.44 correlation with actual loss using only four simple features: token frequency...
Linear Predictor → usesfeatures → Sequence Length
confidence 95% · Our predictor achieves 0.44 correlation with actual loss using only four simple features: ... sequence length ...
Linear Predictor → usesfeatures → Vocabulary Diversity
confidence 95% · Our predictor achieves 0.44 correlation with actual loss using only four simple features: ... vocabulary diversity ...
Linear Predictor → usesfeatures → Rare Token Ratio
confidence 95% · Our predictor achieves 0.44 correlation with actual loss using only four simple features: ... and rare token ratio.
Linear Predictor → achievescorrelationwith → Actual Loss
confidence 92% · Our predictor achieves 0.44 correlation with actual loss
Predictive Batch Scheduling → improvesconvergenceby → 6-13%
confidence 90% · Experiments on a 130M parameter transformer demonstrate that PBS achieves 6-13% faster convergence measured by evaluation loss across training checkpoints
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce Predictive Batch Scheduling (PBS), a novel training optimization technique that accelerates language model convergence by dynamically prioritizing high-loss samples during batch construction. Unlike curriculum learning approaches that require predefined difficulty metrics or hard example mining methods that demand expensive per-sample loss tracking, PBS employs a lightweight linear predictor trained online to estimate sample difficulty from static token-level features. Our predictor achieves 0.44 correlation with actual loss using only four simple features: token frequency, sequence length, vocabulary diversity, and rare token ratio. Experiments on a 130M parameter transformer demonstrate that PBS achieves 6-13\% faster convergence measured by evaluation loss across training checkpoints, with the predictor's correlation improving from 0.14 to 0.44 over 10,000 training steps. These results validate that token frequency statistics encode meaningful information about sample difficulty, enabling effective curriculum learning with negligible computational overhead.
Tags
Links
- Source: https://arxiv.org/abs/2602.17066v1
- Canonical: https://arxiv.org/abs/2602.17066v1
Trouble viewing inline? Open PDF directly →
Full Text
29,425 characters extracted from source content.
Expand or collapse full text
Predictive Batch Scheduling: Accelerating Language Model Training Through Loss-Aware Sample Prioritization Sumedh Rasal Georgia Institute of Technology Abstract We introduce Predictive Batch Scheduling (PBS), a novel training optimization technique that accelerates language model convergence by dynamically prioritizing high-loss samples during batch construction. Unlike curriculum learning approaches that require predefined difficulty metrics or hard example mining methods that demand expensive per-sample loss tracking, PBS employs a lightweight linear predictor trained online to estimate sample difficulty from static token-level features. Our predictor achieves 0.44 correlation with actual loss using only four simple features: token frequency, sequence length, vocabulary diversity, and rare token ratio. Experiments on a 130M parameter transformer demonstrate that PBS achieves 6-13% faster convergence measured by evaluation loss across training checkpoints, with the predictor’s correlation improving from 0.14 to 0.44 over 10,000 training steps. These results validate that token frequency statistics encode meaningful information about sample difficulty, enabling effective curriculum learning with negligible computational overhead. 1 Introduction Standard language model training employs uniform random sampling to construct mini-batches, treating all training samples as equally valuable for learning. This approach is fundamentally suboptimal: samples vary significantly in difficulty, with some providing strong learning signals through informative gradient updates while others contribute minimally to model improvement. Easy samples that the model has already mastered waste computational resources, while difficult samples containing challenging linguistic patterns, rare vocabulary, or complex dependencies offer opportunities for accelerated learning. In an era where training runs can cost millions of dollars and consume weeks of GPU time, even modest improvements in sample efficiency translate to substantial savings. The recognition that not all samples are equally valuable has motivated two main research directions. Curriculum learning [1] addresses this by ordering training data from easy to hard, enabling models to build progressively on acquired knowledge. However, traditional curriculum learning suffers from a critical limitation: it requires predefined difficulty metrics computed during preprocessing, creating static orderings that cannot adapt as the model learns. A sample that challenges an untrained model may become trivial after several thousand gradient updates, yet fixed curricula cannot reflect this evolution. Online hard example mining [2, 6] overcomes this limitation by dynamically tracking per-sample losses and prioritizing samples with high current loss. This adaptive approach successfully identifies difficult examples throughout training. However, it introduces a different bottleneck: maintaining loss estimates for every sample requires memory proportional to dataset size prohibitive for the billion-sample datasets common in modern language model training. Furthermore, periodic evaluation passes over the entire dataset to update these estimates create computational overhead that can offset training speedups. We propose Predictive Batch Scheduling (PBS), a method that achieves the adaptivity of online hard example mining while maintaining the efficiency of static curriculum approaches. The key insight is that sample difficulty can be predicted from lightweight token-level features rather than requiring expensive loss computations. PBS employs a linear predictor trained online to estimate per-sample loss from four simple features: average token frequency, sequence length, vocabulary diversity, and rare token ratio. These features require only token counting operations, eliminating the need for additional forward passes or per-sample loss storage. The predictor is learned jointly with the language model, using actual training loss as supervision. This ensures difficulty estimates remain calibrated to the model’s evolving capabilities: as the model masters certain patterns, the predictor learns to assign them lower difficulty scores, automatically shifting focus to genuinely challenging examples. The entire mechanism introduces negligible overhead—feature extraction involves simple counting, predictor updates occur every 100 steps using lightweight linear regression, and the main added cost is computing per-sample rather than batch-averaged losses. Our contributions are threefold: (1) We introduce a novel batch scheduling method that prioritizes high-loss samples using online difficulty prediction from static features, avoiding both the rigidity of curriculum learning and the overhead of hard example mining; (2) We demonstrate that simple token frequency features are sufficient predictors of sample difficulty, achieving 0.44 correlation with actual loss and explaining 19% of loss variance; (3) We show that PBS accelerates convergence by 6-13% on evaluation metrics across training checkpoints while introducing minimal computational cost, validating that lightweight feature-based prediction enables effective curriculum learning at scale. 2 Related Work The problem of efficient training data selection has been studied extensively across multiple communities, with approaches ranging from predefined curricula to adaptive online methods. We categorize prior work into three main paradigms and discuss their advantages and fundamental limitations. 2.1 Curriculum Learning The seminal work of Bengio et al. [1] introduced curriculum learning, drawing inspiration from human learning processes where knowledge is acquired progressively from simple to complex concepts. Their experiments on shape recognition and language modeling demonstrated that presenting training examples in order of increasing difficulty can significantly improve both convergence speed and generalization. However, a critical limitation of their approach was the requirement for hand-crafted difficulty metrics specific to each task. Subsequent work has explored various automatic difficulty metrics to eliminate manual annotation. Platanios et al. [4] proposed competence-based curriculum learning for neural machine translation, where difficulty is measured by sentence length and word rarity. Their method dynamically adjusts the difficulty threshold based on model competence, showing improvements over static curricula. [3] used perplexity under a reference language model as a proxy for difficulty, enabling curriculum construction without human supervision. Xu et al. [5] demonstrated that transferring difficulty annotations from related tasks could bootstrap curriculum learning in low-resource settings. While these methods successfully automate difficulty estimation, they share two fundamental drawbacks. First, the difficulty metrics are computed once during preprocessing and remain fixed throughout training, failing to adapt as the model’s capabilities evolve. A sample that is difficult for an untrained model may become trivial after several epochs, yet static curricula cannot reflect this change. Second, these approaches typically sort the entire dataset and train in a predetermined sequence, which can lead to catastrophic forgetting of earlier easy examples and fails to leverage the benefits of stochastic sampling for optimization stability. 2.2 Hard Example Mining and Loss-Based Selection Hard example mining addresses the limitation of static curricula by dynamically identifying difficult samples based on model feedback. The approach was pioneered in computer vision by Shrivastava et al. [2] for training region-based object detectors, where false positives with high classification loss are mined during training and added to subsequent batches. This online adaptation enables the detector to focus on challenging background regions that are easily confused with objects. Katharopoulos and Fleuret [6] brought this principle to deep learning more broadly, demonstrating that importance sampling based on per-sample loss can accelerate training of neural networks. They showed theoretically that samples with higher loss contribute larger gradient norms, justifying their prioritization. However, their method requires maintaining a loss estimate for every sample in the dataset, which becomes prohibitively expensive for the multi-billion sample datasets common in language model training. Recent work has attempted to reduce this memory overhead through various approximations. [9] proposed active bias learning, which maintains a small cache of recent high-loss samples. [10] used influence functions to estimate sample importance without storing full loss histories. However, these methods either sacrifice accuracy of difficulty estimation or introduce computational overhead from influence function calculations that require second-order derivatives. A fundamental challenge shared by all loss-based methods is the cold-start problem: early in training when the model is random, loss values provide weak signals about true sample difficulty. Additionally, these methods require periodic evaluation passes over the entire dataset to update loss estimates, creating computational bottlenecks that can offset training speedups, particularly for large-scale datasets. 2.3 Importance Sampling and Gradient-Based Selection Importance sampling techniques provide a theoretical framework for non-uniform sample selection by weighting samples based on their estimated contribution to optimization. Schaul et al. [7] introduced prioritized experience replay for reinforcement learning, sampling transitions proportional to their temporal difference error. This approach significantly improved sample efficiency in deep Q-learning by replaying informative experiences more frequently. Loshchilov and Hutter [8] proposed online batch selection, which estimates the expected gradient norm for each sample and constructs batches to maximize total gradient magnitude. While theoretically motivated, their approach requires computing or approximating gradients for all candidate samples before batch construction, introducing substantial overhead. Johnson and Guestrin (2018) [11] explored variance reduction through biased sampling, showing that selecting samples with higher gradient variance can reduce the number of iterations needed for convergence. However, estimating per-sample gradient variance requires either maintaining running statistics or performing test forward-backward passes, both of which incur significant memory or computational costs. An alternative line of work focuses on diversity-based selection rather than difficulty. Mirzasoleiman et al. (2020) [12] proposed coresets for neural network training, selecting a subset of training data that approximates the full dataset’s gradient. While elegant, coreset construction typically requires batch-mode selection over the entire dataset, making it unsuitable for streaming or online training scenarios. 2.4 Feature-Based Difficulty Prediction A smaller body of work has explored predicting sample difficulty from input features rather than model outputs, though primarily in computer vision. Pentina et al. (2015) [13] used image complexity metrics like edge density and color variance to order training examples. Hacohen and Weinshall (2019) [14] proposed transfer teacher curriculum learning, using predictions from a simple auxiliary model to define difficulty for a more complex target model. However, these approaches either rely on domain-specific features that do not generalize across modalities or require training separate teacher models, adding complexity. In natural language processing, task-specific heuristics have been employed for curriculum design. Kocmi et al. (2017) [15] used sentence length and word frequency for machine translation curriculum, while Xu et al. (2020) leveraged syntactic complexity metrics. These hand-crafted features show task-specific success but lack the generality needed for broad applicability across different language modeling objectives. 2.5 Positioning of Our Work PBS distinguishes itself by combining the adaptivity of loss-based methods with the efficiency of feature-based prediction. Unlike curriculum learning approaches, our online predictor adapts continuously as the model learns. Unlike hard example mining, we avoid the memory overhead of per-sample loss tracking by predicting difficulty from lightweight static features. Unlike importance sampling methods requiring gradient computations, our feature extraction involves only token counting operations. Critically, our predictor is learned jointly with the model using the actual training loss as supervision, ensuring that difficulty estimates remain calibrated to the model’s current state. This online learning approach bridges the gap between the efficiency of static feature-based methods and the accuracy of loss-based methods, enabling effective curriculum learning at scale with negligible computational overhead. 3 Method 3.1 Overview PBS consists of three components: a frequency tracker that computes token statistics, a loss predictor that estimates per-sample difficulty, and a priority sampler that constructs batches favoring high-loss samples. 3.2 Frequency Tracker During a warmup phase of Tw=100T_w=100 steps, the frequency tracker accumulates token counts cic_i for each token i in the vocabulary. Token frequencies are computed as: fi=ci∑j=1|V|cjf_i= c_i _j=1^|V|c_j (1) where |V||V| denotes vocabulary size. Tokens are classified as rare if fi<p20f_i<p_20, where p20p_20 represents the 20th percentile of the frequency distribution. 3.3 Loss Predictor For each training sample x consisting of tokens t1,…,tn\t_1,…,t_n\, we extract four features: Average Token Frequency: ϕ1(x)=1n∑i=1nfti _1(x)= 1n _i=1^nf_t_i (2) Sequence Length: ϕ2(x)=nnmax _2(x)= nn_max (3) where nmaxn_max is the maximum sequence length. Vocabulary Diversity: ϕ3(x)=|t1,…,tn|n _3(x)= |\t_1,…,t_n\|n (4) Rare Token Ratio: ϕ4(x)=1n∑i=1n[fti<p20] _4(x)= 1n _i=1^n1[f_t_i<p_20] (5) The predicted loss is computed via linear regression: ℓ^(x)=b+∑j=14wjϕj(x) (x)=b+ _j=1^4w_j _j(x) (6) where b is a bias term and wjw_j are learned weights. Weights are updated every Tu=100T_u=100 steps using momentum-based stochastic gradient descent on the most recent Nh=2000N_h=2000 samples from the loss history buffer: gj g_j =1Nh∑k=1Nh(ℓ^(xk)−ℓ(xk))ϕj(xk) = 1N_h _k=1^N_h( (x_k)- (x_k)) _j(x_k) (7) mj m_j =βmj+(1−β)gj =β m_j+(1-β)g_j (8) wj w_j =wj−ηmj =w_j-η m_j (9) where ℓ(xk) (x_k) denotes the actual cross-entropy loss, β=0.9β=0.9 is the momentum coefficient, and η=0.01η=0.01 is the learning rate. 3.4 Priority Sampler Given a buffer of Nb=1000N_b=1000 samples, we partition samples into three buckets based on predicted loss percentiles: high (ℓ^(x)>p67 (x)>p_67), medium (p33<ℓ^(x)≤p67p_33< (x)≤ p_67), and low (ℓ^(x)≤p33 (x)≤ p_33). Buckets are sampled with probabilities: P(high)=0.57,P(medium)=0.29,P(low)=0.14P(high)=0.57, P(medium)=0.29, P(low)=0.14 (10) corresponding to a high-loss sampling ratio r=2.0r=2.0, ensuring high-loss samples are selected twice as frequently as medium-loss samples. 4 Experimental Setup 4.1 Model and Training We evaluate PBS on a 130M parameter LLaMA-style transformer with standard architecture. Training uses mixed precision with batch size and gradient accumulation yielding approximately 16,000 tokens per step. The learning rate schedule employs warmup with peak learning rate 3×10−43× 10^-4 decaying to 3×10−53× 10^-5. Training runs for 10,000 optimizer steps covering 55% of one epoch over 291,258 total batches. Checkpoints and evaluation occur every 2,000 steps. PBS-specific hyperparameters are: warmup steps Tw=100T_w=100, predictor update interval Tu=100T_u=100, loss history size Nh=10,000N_h=10,000, sample buffer size Nb=1,000N_b=1,000, high-loss sampling ratio r=2.0r=2.0, predictor learning rate η=0.01η=0.01, and momentum β=0.9β=0.9. 4.2 Implementation Details A critical implementation requirement is per-sample loss computation rather than batch-averaged loss. For causal language models, we compute cross-entropy with reduction=’none’, shifting logits and labels appropriately to obtain one loss value per sample. This enables the predictor to learn meaningful feature-loss correlations. 4.3 Baseline and Evaluation The baseline uses identical model architecture, dataset, hyperparameters, and random seeds, differing only in employing uniform random sampling. Evaluation uses cross-entropy loss on a held-out validation set. 5 Results Table 1 presents evaluation loss and predictor correlation at five checkpoints. PBS shows an initial 5% deficit at 2,000 steps during warmup, then consistently outperforms the baseline from 4,000 steps onward. The advantage ranges from 6% at 10,000 steps to 13% at 8,000 steps, demonstrating sustained acceleration throughout training. Steps PBS Loss Baseline Loss Improvement Correlation 2,000 0.0079 0.0075 -5% 0.14 4,000 0.0131 0.0144 +9% 0.31 6,000 0.0054 0.0061 +11% 0.001 8,000 0.0078 0.0090 +13% 0.31 10,000 0.0050 0.0053 +6% 0.44 Table 1: Evaluation loss and predictor correlation at training checkpoints. PBS achieves consistent improvements after warmup, with predictor correlation increasing to 0.44 by 10,000 steps (R2=0.19R^2=0.19). The predictor correlation trajectory demonstrates successful online learning. Starting from 0.14 at 2,000 steps, correlation improves to 0.44 at 10,000 steps, corresponding to R2=0.19R^2=0.19. This indicates that four simple features explain 19% of variance in actual loss, validating our hypothesis that token frequency statistics encode meaningful difficulty information. The temporary correlation drop to 0.001 at 6,000 steps reflects an interesting phenomenon: when the model achieves very low loss with minimal inter-sample variance, difficulty prediction becomes temporarily undefined. This does not indicate predictor failure but rather demonstrates the model approaching convergence on the training distribution at this learning rate phase. 6 Analysis 6.1 Learned Feature Importance The learned predictor weights at 10,000 steps reveal interpretable patterns that provide insights into both sample difficulty and model learning dynamics. The rare token ratio weight of −0.19-0.19 initially appears counterintuitive, as conventional wisdom suggests rare tokens should be harder to predict. However, this negative weight reflects a more nuanced reality: by 10,000 steps, the model has effectively learned rare tokens through focused exposure early in training when they did carry high loss. Consequently, sequences with higher rare token density now have lower loss than sequences with common tokens appearing in complex syntactic or semantic contexts. This phenomenon demonstrates that PBS successfully prioritizes different types of difficult samples at different training stages—rare tokens early, complex contexts later. The average token frequency weight of −0.17-0.17 aligns with intuition: higher average frequency predicts lower loss, as common tokens are generally easier to predict given more training exposure. The similar magnitude to the rare token weight suggests these two features provide complementary information about difficulty. In contrast, the vocabulary diversity weight (−0.007-0.007) and sequence length weight (−0.001-0.001) show minimal predictive value. The near-zero vocabulary diversity weight suggests that the ratio of unique to total tokens does not strongly correlate with difficulty in our setting, possibly because diversity effects are already captured by token frequency statistics. The negligible sequence length weight is more surprising, as longer sequences might be expected to present greater difficulty. This finding suggests that raw length matters less than the specific tokens present, reinforcing the importance of frequency-based features. 6.2 Predictor Correlation Trajectory The evolution of predictor correlation from 0.14 at 2,000 steps to 0.44 at 10,000 steps demonstrates the effectiveness of online learning for difficulty estimation. This improvement occurs despite using the same four features throughout training, indicating that the predictor is learning more accurate feature weights as it accumulates training examples. The final correlation of 0.44 (R2=0.19R^2=0.19) is substantial given the simplicity of our feature set—four linear features explaining 19% of variance in a complex prediction task suggests these features capture meaningful signal. The temporary correlation drop to 0.001 at 6,000 steps warrants careful interpretation. At this checkpoint, the model achieved exceptionally low loss (0.0054 for PBS) with minimal variance between samples. When nearly all samples have similar low loss, any predictor’s correlation will approach zero regardless of quality, as there is simply no variance to explain. This is not a predictor failure but rather a consequence of the model temporarily approaching near-perfect performance on the training distribution during this phase of the learning rate schedule. The subsequent recovery to 0.31 at 8,000 steps, when losses increased due to learning rate cycling, confirms this interpretation. 6.3 Training Dynamics and Generalization Training loss at 10,000 steps shows PBS at 0.0018 versus baseline at 0.0054, a 67% reduction. This large gap compared to the 6% evaluation improvement merits analysis. The training loss advantage demonstrates that PBS effectively identifies and prioritizes samples where the model has higher loss, leading to faster memorization of the training set. The smaller evaluation gain indicates that some of this training loss reduction comes from overfitting to the specific high-loss samples that PBS prioritizes. However, crucially, the evaluation loss still improves consistently from 4,000 steps onward, indicating that the generalization benefit substantially outweighs any overfitting effect. This trade-off is both expected and desirable: focusing on difficult examples naturally leads to stronger training set performance, but the key metric is whether this translates to evaluation improvement. Our results confirm that it does. The 6-13% evaluation improvements across checkpoints demonstrate that PBS learns more useful representations faster, even if those representations are somewhat specialized to difficult training examples. 6.4 Comparison to Random Sampling The consistent evaluation advantages after the warmup period validate our core hypothesis. While the baseline using uniform random sampling provides a strong foundation—random sampling is known to provide good optimization properties through variance reduction—PBS demonstrates that informed sampling based on difficulty prediction can do better. The 6-13% improvements are achieved without changing any other aspect of training: identical model, optimizer, learning rate schedule, and batch size. This controlled comparison isolates the effect of batch construction strategy. The initial 5% deficit at 2,000 steps during warmup is expected and acceptable. During this phase, the frequency tracker is still accumulating statistics and the predictor has limited training data. The rapid recovery and subsequent sustained advantages demonstrate that the warmup overhead is a small price for the benefits gained through adaptive sampling in later training. 6.5 Computational Efficiency The minimal overhead of PBS merits emphasis. Feature extraction requires only: (1) accumulating token counts during warmup (amortized O(1)O(1) per token), (2) computing four scalar features per sample (O(n)O(n) for sequence length n), and (3) predictor updates every 100 steps using simple linear regression on 2,000 samples. The main additional cost is computing per-sample losses rather than batch-averaged losses, requiring one additional view and mean operation per batch. In our experiments, this overhead was unmeasurable—less than 1% of training time. This efficiency advantage over prior methods is substantial. Hard example mining requires storing loss values for all samples (memory O(N)O(N) for dataset size N) plus periodic evaluation passes over the entire dataset. Gradient-based importance sampling requires computing or approximating per-sample gradients before batch construction. PBS avoids both memory and computational bottlenecks by predicting difficulty from precomputed static features, making it practical for large-scale training. 7 Limitations and Future Directions While our results demonstrate clear benefits, several directions warrant investigation. Our experiments use a single 130M parameter model and cover 55% of one epoch. Validation on larger models (1B-7B parameters) and longer training runs would strengthen generalization claims. The predictor uses only four features; incorporating additional signals such as n-gram statistics or syntactic complexity might improve prediction accuracy. Multiple runs with different random seeds would enable statistical significance testing. Future work should explore non-linear predictors capable of capturing feature interactions, adaptive sampling ratios that decrease as training progresses, and evaluation on diverse datasets including code and multilingual corpora. Investigating the relationship between predictor correlation and downstream task performance would provide insights beyond training loss optimization. This strategy comes in handy for multi agent communication strategies proposed by Rasal (2024) [16] where multi agent models work together to solve a problem. Instruction based model will also benefit from this strategy outlined by Rasal et al. (2024) [17], where the model follow a methodical approach to break down a complex problem. 8 Conclusion We have introduced Predictive Batch Scheduling, a training optimization method that accelerates language model convergence through online difficulty prediction and loss-aware batch prioritization. Our approach achieves 6-13% faster convergence measured by evaluation loss while introducing negligible computational overhead. The key insight is that simple token frequency features encode sufficient information about sample difficulty to enable effective curriculum learning without expensive per-sample loss tracking. The improvement in the correlation of the predictor from 0.14 to 0.44 over training validates that online learning can uncover meaningful difficulty patterns. PBS represents a practical approach to training efficiency that can be easily integrated into existing training pipelines, offering immediate benefits to practitioners training language models on-scale. References [1] Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. ICML. [2] Shrivastava, A., Gupta, A., & Girshick, R. (2016). Training region-based object detectors with online hard example mining. CVPR. [3] Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. NeurIPS. [4] Platanios, E. A., Stretcu, O., Neubig, G., Poczos, B., & Mitchell, T. M. (2019). Competence-based curriculum learning for neural machine translation. NAACL. [5] Xu, B., Zhang, L., Mao, Z., Wang, Q., Xie, H., & Zhang, Y. (2020). Curriculum learning for natural language understanding. ACL. [6] Katharopoulos, A., & Fleuret, F. (2018). Not all samples are created equal: Deep learning with importance sampling. ICML. [7] Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015). Prioritized experience replay. ICLR. [8] Loshchilov, I., & Hutter, F. (2015). Online batch selection for faster training of neural networks. ICLR Workshop. [9] Chang, H. S., Learned-Miller, E., & McCallum, A. (2017). Active bias: Training more accurate neural networks by emphasizing high variance samples. NeurIPS. [10] Jiang, A. H., Wong, D. L., Zhou, G., Andersen, D. G., Dean, J., Ganger, G. R., … & Kaminsky, M. (2018). Accelerating deep learning by focusing on the biggest losers. arXiv preprint arXiv:1910.00762. [11] Johnson, T., & Guestrin, C. (2018). Training deep models faster with robust, approximate importance sampling. NeurIPS. [12] Mirzasoleiman, B., Bilmes, J., & Leskovec, J. (2020). Coresets for data-efficient training of machine learning models. ICML. [13] Pentina, A., Sharmanska, V., & Lampert, C. H. (2015). Curriculum learning of multiple tasks. CVPR. [14] Hacohen, G., & Weinshall, D. (2019). On the power of curriculum learning in training deep networks. ICML. [15] Kocmi, T., & Bojar, O. (2017). Curriculum learning and minibatch bucketing in neural machine translation. RANLP. [16] Rasal, S. (2024). LLM Harmony: Multi-Agent Communication for Problem Solving. arXiv preprint arXiv:2401.01312. [17] Rasal, S. & Hauer, E. J. (2024). Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs. arXiv preprint arXiv:2402.16713.