Paper deep dive
A Sovereign, Indic-First, Reasoning Model
Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik, Nayanjit Sarkar, Avinash Benki
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA's Nemotron-3-Nano-30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved reading ability but is measured on a self-selected subset. Two unreleased studies test repairs. A scoped Indic reward restores the language convention. A 26B-A4B generative judge outperforms a 235B-parameter model on RM-Bench, and distillation restores 81% of the capabilities displaced by optimizing against it. Relative to the architecture-identical starting checkpoint, the released model gains 5.2 points on an Indic aggregate while giving up 1.7 points of English retention. The unreleased Indic-aware branch trades further, gaining 14.6 Indic points at a cost of 5.4 in English. A planned FP8 release reduces the artifact to 32GB with near-zero mean change across 32 paired benchmarks.
Tags
Links
- Source: https://arxiv.org/abs/2610.15371v1
- Canonical: https://arxiv.org/abs/2610.15371v1
Trouble viewing inline? Open PDF directly →
Full Text
117,707 characters extracted from source content.
Expand or collapse full text
A Sovereign, Indic-First, Reasoning Model Tokenizer expansion, continual pretraining, supervised fine-tuning, and reinforcement learning. Core Team: Sutanshu Saksena Raj, Prasanjith Pasunuti, Parijat Chatterjee, Daksh Maan, Purusharth Malik, Nayanjit Sarkar Technical Lead: Avinash Benki Gnani.AI Release family Released RL checkpoint LicenseApache License 2.0 Report status Technical Report Model artifacts: https://huggingface.co/gnani/gnani-evon-v3.3-30B-A3B arXiv:2610.15371v1 [cs.CL] 11 Oct 2026 Gnani-Evon-v3.3-30B-A3BTechnical Report Abstract Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba–Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA’s Nemotron 3 Nano 30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English–Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved reading ability but is measured on a self-selected subset. Two unreleased studies test repairs. A scoped Indic reward restores the language convention. A 26B-A4B generative judge outperforms a 235B-parameter model on RM-Bench, and distillation restores 81% of the capabilities displaced by optimizing against it. Relative to the architecture-identical starting checkpoint, the released model gains 5.2 points on an Indic aggregate while giving up 1.7 points of English retention. The unreleased Indic-aware branch gains 14.6 for 5.4. A planned FP8 release reduces the artifact to 32GB with near-zero mean change across 32 paired benchmarks. Keywords: Indic language models, continual pretraining, tokenizer expansion, mixture of experts, Mamba, long context, supervised fine-tuning, GRPO, multilingual reinforcement learning. i Gnani-Evon-v3.3-30B-A3BTechnical Report Contents 1 Introduction1 1.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Starting Point: An Efficient Reference Architecture2 2.1 Why reuse Nemotron 3 Nano? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 NoPE and the meaning of 128K context . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 Tokenizer Expansion4 3.1 The language tax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2 Continued BPE and normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3 Fertility results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.4 An exploratory 64K vocabulary study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.5 Native versus romanized text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.6 Initializing the new matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.7 A tokenizer pathology worth reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4 Embedding Warmup7 4.1 Why a separate stage is needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.2 Decoupled weight decay can move frozen rows . . . . . . . . . . . . . . . . . . . . . . . . . 7 4.3 Recovery trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5 Continual Pretraining8 5.1 Data preparation and quality control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.2 Choosing the learning rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.3 Measured CPT performance ceiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.4 The realized training ladder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 5.5 Checkpoint evaluation across CPT phase 1 and annealing . . . . . . . . . . . . . . . . . . 12 5.6 Ablation-guided language allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.7 Long-context mixture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 6 Supervised Fine-Tuning14 i Gnani-Evon-v3.3-30B-A3BTechnical Report 6.1 Role in the model story . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 6.2 Selected Evon-Instruct checkpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.3 Data construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.4 Masking and long-sequence packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 6.5 What SFT establishes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 7 Reinforcement Learning16 7.1 GRPO in one equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 7.2 Released Evon-RL-Gen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 7.3 Capability versus convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 8 Release Evaluation19 8.1 Comparison with open models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 8.2 Safety evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 9 Experimental Research Branches (Not Released)21 9.1 A sibling Indic-aware branch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 9.2 Learning from a judge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 9.3 Recovering a convention by distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 9.4 Weight-space alternatives to distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 10 FP8 Quantization and Serving26 10.1 Selective E4M3 W8A8 recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 10.2 Calibration and validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 10.3 Serving performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 10.4 Matched BF16 versus FP8 quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 11 Compute and Infrastructure29 12 Responsible Deployment29 12.1 Intended use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 12.2 Risks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 12.3 Recommended deployment controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 i Gnani-Evon-v3.3-30B-A3BTechnical Report 13 Release and Reproducibility30 14 Conclusion31 A Architecture and Artifact Accounting31 B Exploratory 64K Vocabulary Study32 B.1 Production tokenizer chain of custody . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 C Training-Time and Throughput Diagnostics33 D CPT Ablations and Non-lineage Validation Runs34 D.1 Operational validation runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.2 The 120B mixing ablation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.3 Uplift-weighted planning table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 E Detailed CPT Configuration36 E.1 Production CPT training traces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 E.2 CPT data composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 F CPT Checkpoint Evaluation Series39 F.1 HELMET long-context evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 G SFT Corpus Composition Across Retained Stages41 G.1 Selected checkpoint accounting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 G.2 Source-family evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 H SFT Training Traces43 I Matched Evaluation of the Released Checkpoint46 J Indic Extractive-QA Failure Taxonomy48 K Sibling Indic-Aware RL Study (Not Released)49 K.1 Branch definition and reward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 K.1.1 Why the experimental mix retains an English guardrail pool . . . . . . . . . . . . 49 K.1.2 Experimental mixture and configuration . . . . . . . . . . . . . . . . . . . . . . . . 50 iv Gnani-Evon-v3.3-30B-A3BTechnical Report K.1.3 Experimental matched results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 K.1.4 Direct comparison with the released checkpoint . . . . . . . . . . . . . . . . . . . . 54 K.1.5 Experimental tool-use decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 55 L Reinforcement-Learning Configuration, Side by Side55 M Reward Model and Distillation Study (Not Released)55 M.1 Preference data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 M.2 The judge’s reward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 M.3 Reinforcement learning against the judge . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 M.4 Length control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 M.5 The 70:30 mixture that failed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 M.6 Distillation configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 N Matched BF16 and FP8 Benchmark Results59 O Glossary61 v Gnani-Evon-v3.3-30B-A3BTechnical Report 1 Introduction Large language models do not process words directly. They first split text into tokens, and the quality of that split depends on which languages were well represented when the tokenizer was built. For many Indian scripts, a tokenizer optimized primarily for English and code can turn one word into several fragments. Training, context capacity, latency, and serving cost are all paid per token, so this fragmentation creates a practical language tax. Evon v3.3 was built around a simple question: how much Indic capability can be added to an efficient, already capable base model without discarding its English reasoning, code, or deployment advantages? The answer required more than adding translated data [11]. Four linked interventions were needed, following the pattern of other sovereign-language adaptation programmes [2]: 1. expand the vocabulary so that Indian scripts are represented efficiently; 2. adapt the new embedding and output rows without disturbing the inherited model; 3. continue pretraining on a quality-controlled English–Indic curriculum and teach the model directly at 128K context; 4.post-train for instruction following and reasoning, measure native-script retention, and study corrective language rewards in a separate experimental branch. Table 1 gives the development path. In brief, tokenizer adaptation and continual pretraining produce Evon- Base; supervised fine-tuning produces Evon-Instruct; and the released RL stage produces Evon-RL-Gen. The sibling Indic-aware branch is experimental, not an ancestor of the release. 1.1 Contributions This report makes six main contributions. The lineage they describe runs Evon-Base→Evon-Instruct→ Evon-RL-Gen, with Evon-RL-Indic branching from Evon-Instruct as a research arm. •An Indic-aware tokenizer upgrade. The vocabulary is expanded in place rather than replaced, improving eight-language fertility by 38.6% while preserving inherited token identities. •A guarded embedding-adaptation recipe. Decoupled weight decay can move nominally frozen rows, while an Indic-only warmup can disturb English output calibration even with a frozen backbone. •A measured continual-pretraining curriculum. Per-language uplift informs the data mixture, and direct training establishes the model’s 128K context length. •A reward-mixture diagnosis. The released RL run improves reasoning but weakens unrewarded native-script conventions. A separate Indic-aware experiment tests a scoped repair. •A reward model beyond verifiable tasks. An Indic-capable generative judge beats a nine-times- larger model on RM-Bench. Distillation then restores 81% of the capabilities displaced by optimizing against that judge while retaining most of its gains. • Negative results with operational value. RL can erase output conventions without erasing capability; perplexity can select the wrong retention rate; and tool metrics can hide call policies. 1 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 1: Production ancestry of the released RL checkpoint and its planned FP8 derivative. Stage ArtifactWhat changed 0NVIDIA base checkpoint Nemotron 3 Nano 30B-A3B Base BF16, upstream revision97ab8012, with a 131,072-token vocabulary. 1Tokenizer and initialization A 32,000-entry continued-BPE extension, portable tokenizer backend, and constituent-mean initialization produce the 163,072-row vocabulary matrices. 2Embedding warmupNew vocabulary rows are adapted on a 90% Indic / 10% English anchor mixture while inherited parameters remain fixed. 3CPT phase 1Broad 8,192-token English–Indic continual pretraining. 4CPT annealingHigh-quality 8,192-token continuation from the selected phase-1 checkpoint. 5CPT long contextDirect 131,072-token NoPE training through iteration 372. 6SFT-phase-1First retained supervised curriculum stage. 7SFT-phase-2Second retained supervised stage. 8SFT-rebuild-3 epoch 1 First pass over the rebuilt SFT-rebuild-3 corpus. Iteration 2,000 is the retained branch point. 9SFT-rebuild-3 epoch 2 / Evon-Instruct Epoch-2 continuation from SFT-rebuild-3 epoch-1 iteration 2,000; its iteration-2,400 export initializes both RL studies. 10Evon-RL-GenReleased four-environment GRPO checkpoint. 11Planned Evon-FP8Selectively quantized derivative of Evon-RL-Gen, planned for release. 2 Starting Point: An Efficient Reference Architecture 2.1 Why reuse Nemotron 3 Nano? The upstream model is NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 [1]. It combines Mamba-2 sequence mixers [7], sparse attention, and granular mixture-of-experts layers. This matters for two reasons. First, only a small fraction of the total parameters is active for each token. Second, only six attention layers maintain a growing key–value cache, while the Mamba layers carry sequence state compactly. Sparse experts provide capacity, while routing activates only a small subset for each token. The released model contains 31.75B parameters, of which 3.31B are active per token under NVIDIA’s convention. Expanding both untied vocabulary matrices adds 172.03M parameters without changing the backbone. Table 2 summarizes the design, and Appendix A provides the exact accounting. 2.2 NoPE and the meaning of 128K context Evon v3.3 uses no explicit positional embedding, so its 128K support comes from direct training on 131,072-token sequences rather than positional scaling. That establishes a trained sequence length, not uniform capability across it. At the CPT checkpoint, HELMET in-context learning rises from 59.2 at 8K to 81.7 at 128K, while exact retrieval falls from 95.9 to 14.0; retrieval-augmented generation and long-form question answering remain between those extremes. 2 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 2: Architecture inherited from the Nemotron 3 Nano reference configuration. HyperparameterValue Total parameters31,749,975,232 (31.75B) Active parameters 3,313,773,760 (3.31B; NVIDIA convention) Layer count52 total: 23 Mamba-2, 23 MoE, 6 GQA Model dimension2,688 Attention heads / KV heads32 / 2 Attention head dimension128 Mamba-2 state dimension / groups128 / 8 Mamba-2 heads / head dimension64 / 64 Routed / activated / shared expert modules128 / 6 / 1 Routed / shared expert hidden dimension1,856 / 3,712 (2× width) MoE activationsquared ReLU (relu2) Routerlearned MLP with sigmoid gating Position encodingnone (NoPE) NormalizationRMSNorm Input embedding / output projectionuntied Evon vocabulary size163,072 Maximum supported sequence length131,072 tokens 3 Gnani-Evon-v3.3-30B-A3BTechnical Report Under the same completion prompting, the released checkpoint raises average recall from 57.1 to 84.0. Table 3 gives the remaining post-training results. Long-context ability therefore depends strongly on the task and prompt format, even within the supported length. Table 3: Long-context results under matched completion prompting. Evon-RL-Indic is an unreleased research branch. MeasureCPT checkpoint Evon-RL-Gen Evon-RL-Indic Recall average57.184.081.2 Abstention on unanswerable questions98.199.899.7 IndicQA F1, HELMET reader slice54.360.560.1 3 Tokenizer Expansion 3.1 The language tax We measure tokenizer fertility as the number of tokens produced per whitespace-delimited word: F (D) = P x∈D N tok (x) P x∈D N word (x) .(1) Lower fertility is better: the same sentence occupies fewer model tokens. This increases usable context and reduces the amount of token-level work needed to process the sentence. The upstream vocabulary contains 131,072 entries. We continue its BPE merge process on data from Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Tamil, and Telugu, including romanized text during construction. The selected tokenizer adds 32,000 entries, producing a vocabulary of 163,072. The released vocabulary contains 162,071 ordinary BPE entries and 1,001 added tokens. Of the added tokens, 19 have named functions and 982 are reserved placeholders. English and existing special tokens retain their upstream identities. 3.2 Continued BPE and normalization Training a completely new tokenizer would break the mapping learned by the base model. Instead, continued BPE preserves inherited pieces and appends merges for sequences that remain fragmented. This follows the in-place expansion pattern studied by Smith et al. [3] and uses adaptive mixture ideas related to AdaptMix [4]. The production Round-09 tokenizer is built on the E3 corpus: Unicode NFKC normalization followed by newline/space cleanup and an Indic-aware regex canonicalization. NFKC folds compatibility variants into a canonical form before BPE learning, while the regex stage aligns better with intended boundaries. E1 (NFKC only) and E3 (NFKC plus regex) have identical average fertility to four decimal places, 2.1823, in the controlled normalization sweep. Fertility could not arbitrate between them. E3 is the shipped normalization. It changes segmentation and token identities at nearly the same sequence length. 4 Gnani-Evon-v3.3-30B-A3BTechnical Report The production lineage uses the Round-09 32K extension and a 163,072-entry vocabulary. A later 64K extension achieved lower fertility but remained exploratory and was not adopted (Appendix B). 3.3 Fertility results Figure 1 reports the production tokenizer on the common comparison protocol: Samanantar, 2,000 samples per language, seed 42. The eight-language mean falls from 3.5472 to 2.1773 tokens per word, a 38.6% reduction. Gujarati improves most (47.1%), followed by Malayalam (44.3%). Figure 1: Tokenizer fertility on the held-out Samanantar evaluation slice. Lower is better. Percent labels show change relative to the upstream tokenizer. Table 4: Per-language tokenizer fertility for the 163,072-entry tokenizer carried by the documented model lineage. LanguageUpstream Evon v3.3 Change Hindi2.05131.6568 −19.2% Bengali3.05982.1416 −30.0% Gujarati3.60861.9099 −47.1% Kannada3.86432.3377 −39.5% Malayalam4.96542.7684 −44.3% Marathi3.09811.9387 −37.4% Tamil3.77052.2948 −39.1% Telugu3.95952.3706 −40.1% Eight-language mean3.54722.1773 −38.6% 5 Gnani-Evon-v3.3-30B-A3BTechnical Report 3.4 An exploratory 64K vocabulary study A larger 64,000-entry extension was trained and measured before the production vocabulary was fixed. It reaches lower fertility than the shipped tokenizer, at a cost of a second embedding matrix and a regression on one language, and it was not adopted. Appendix B gives the measurements and the reasoning. 3.5 Native versus romanized text Romanized Indic remains substantially more expensive than native script. Across the new eight-language run, the macro romanized-to-native fertility ratio is 1.84. Seven languages fall in a range of approximately 1.59–1.94 times native fertility. Tamil is the clear outlier at 2.58 times: 2.314 tokens per word in native script versus 5.964 when romanized. This does not invalidate the production tokenizer’s native-script improvement. It identifies its weak flank. Applications expecting heavy romanized Tamil or code-mixed Latin-script traffic should account for the larger token budget and may benefit from a future tokenizer revision. Table 5: Native-script versus controlled ITRANS romanized fertility for the production tokenizer. LanguageNative T/W Romanized T/W Ratio Bengali2.1063.3551.59 Gujarati1.9173.1681.65 Hindi1.6492.7311.66 Kannada2.3434.2191.80 Malayalam2.8324.9581.75 Marathi1.9443.7671.94 Tamil2.3145.964 2.58 Telugu2.3634.0641.72 Macro average2.1834.0281.84 The native column reproduces the standalone Round-09 fertility result closely (macro 2.183 versus 2.1773, with the largest per-language difference 0.064). The romanized samples are controlled ITRANS transliterations, not naturally occurring Latin-script web text, so the ratios isolate script conversion rather than broader domain differences. The expansion did not directly target Punjabi or Odia. Their tokenizer efficiency should be measured separately before extending the eight-language fertility conclusions to them. 3.6 Initializing the new matrices The input embedding and output projection are separate matrices (tie_word_embeddings=false). Each new BPE piece has an exact decomposition into upstream pieces. Its new row is initialized as the mean of those constituent rows, independently for the input and output matrices: e new (t) = 1 |S(t)| X s∈S(t) e old (s), w new (t) = 1 |S(t)| X s∈S(t) w old (s).(2) 6 Gnani-Evon-v3.3-30B-A3BTechnical Report A new token starts near the representation of the old tokens that used to spell it. This is a much better starting point than a random vector, while preserving every inherited row exactly. 3.7 A tokenizer pathology worth reporting One continued-BPE entry represented a degenerate space-plus-newline sequence. It could still be emitted through direct vocabulary lookup even after its merge path was removed. Because its initialized row was pathological, this single high-frequency whitespace entry contaminated many English encodings. The slot was made unreachable without changing vocabulary size and was later repurposed as the atomic <|file_sep|>token at ID 144891. This preserves tensor alignment and provides a transparent provenance explanation for a special token located inside the vocabulary-extension range. The broader lesson is simple: after tokenizer surgery, audit frequent encodings in every retention language. Merge-table reasoning alone is insufficient because production tokenizer backends may emit atomic vocabulary entries independently of the merge path expected by the model author. 4 Embedding Warmup 4.1 Why a separate stage is needed Vocabulary expansion modifies only a small part of the model, but those rows participate in every read and every output softmax. The first adaptation stage therefore freezes the inherited backbone and inherited vocabulary rows while updating the new rows in both untied matrices. The warmup consumes approximately 3B tokens from a 90% Indic / 10% English-anchor mixture. The English slice is small enough to keep the objective focused on the new entries, but large enough to preserve the competition between new and inherited output rows. Table 6: Embedding-only warmup configuration. SettingValue Trainable parametersNew vocabulary rows in the untied input and output matrices Frozen parametersBackbone and all inherited vocabulary rows Data mixture90% Indic, 10% English anchor Training tokens∼3B Peak learning rate1× 10 −4 ParallelismTP=1, EP=8, P=1 Retention gateHellaSwag plus bit-exact checks on frozen rows 4.2 Decoupled weight decay can move frozen rows Setting a gradient to zero does not necessarily freeze a parameter under AdamW. The decoupled weight- decay update contains a direct parameter term: 7 Gnani-Evon-v3.3-30B-A3BTechnical Report θ t+1 = θ t − η b m t − ηλθ t .(3) If the gradient-derived term b m t is zero, the final term can still moveθ t . The first warmup implementation therefore allowed inherited rows to drift by about 4% in relative magnitude over 8,000 iterations, enough to collapse HellaSwag near random performance. The fix snapshots inherited rows before each optimizer step and restores them immediately afterward. The guard is tensor-parallel aware, and a bit-exact comparison is run as a training invariant. 4.3 Recovery trajectory Figure 2 shows the sequence of failures and corrections. Constituent-mean initialization reduced HellaSwag from 79.0% to 71.5%. The freeze-leak run reached 34.0%. An Indic-only warmup recovered to 50.0%, while the final anchored warmup returned to 79.0%. Figure 2: English-quality recovery across tokenizer initialization and embedding-only warmup. This result is sometimes summarized incorrectly as a consequence of tied embeddings. The model is untied. The observed effect is instead about calibration: introducing and updating thousands of output rows changes the softmax competition faced by inherited tokens. A small retention-language anchor prevents the adaptation objective from optimizing only the new-language distribution. 5 Continual Pretraining 5.1 Data preparation and quality control The CPT corpus combines English skill-carrying data, including code, mathematics, reasoning, and high-quality web text, with Indic web, academic, reference, and translated material [10]. The available Indic pool used for planning contains approximately 691B tokens. 8 Gnani-Evon-v3.3-30B-A3BTechnical Report Data processing applies exact deduplication followed by inter-document fuzzy deduplication. The implementation follows the practical ordering used in DataComp-LM processing: remove byte- or content- identical records first, then identify near-duplicate documents using fuzzy signatures [5]. Doing exact deduplication first reduces both compute and the number of trivial collisions seen by the fuzzy stage. Every Indic document receives a continuous quality score and a discrete high-, medium-, or low-quality tier. The scorer combines structural length, script purity, character validity, repetition, boilerplate, URL density, language-model perplexity, and safety signals. Calibrated thresholds on the soft score assign the high, medium, and low tiers. Independent hard gates can reject a document regardless of its soft score. Table 7: Quality-scoring signal groups. GroupExamplesRole Length and structureword count, mean word length, mean line lengthsoft score Script qualitytarget-script ratio, Latin ratio, invalid-character ratio soft score / gate Repetitionrepeated words, characters, and n-gramssoft score Web noisesymbol ratio, URL density, boilerplate densitysoft score Language fitper-language KenLM log perplexitysoft score SafetyNSFW and unsafe-content ratioshard gate Synthetic translations and rewrites are down-weighted in broad CPT and excluded from the highest-quality annealing slice. The aim is not to avoid synthetic data entirely. It is to prevent repeated translation style from dominating the late-stage distribution. 5.2 Choosing the learning rate A four-arm, 1B-token-per-arm sweep compared peak learning rates of 10 −4 , 5×10 −5 , 10 −5 , and 5×10 −6 . Validation perplexity preferred 10 −4 , but HellaSwag retention preferred 10 −5 . The disagreement is explainable. Validation perplexity was computed on a holdout close to the 90%-Indic adaptation mixture. It therefore rewards rapid Indic fit even when that movement costs English retention. The deployment objective was multilingual gain under a retention constraint, so 10 −5 was selected. Here, lower Indic-heavy validation perplexity did not imply a better multilingual model. The selected schedule uses a peak learning rate of 10 −5 , a minimum floor of 10 −6 , a linear warmup, and a Warmup–Stable–Decay schedule with a minus-square-root decay tail. Broad CPT uses a global batch of 2,048 sequences at length 8,192. 5.3 Measured CPT performance ceiling The production topology uses 256 H200 GPUs. Expert parallelism keeps each expert group within one eight-GPU node and its NVLink fabric. Table 8 gives the stage-level settings and measured throughput; Appendix E records the full configuration. Aggregate throughput peaks at 2.87M tokens/s during annealing and falls to 1.68M during long-context training. Utilization still rises because attention adds more work per token. 9 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 3: Learning-rate selection. Perplexity and downstream retention rank the candidates differently. Table 8: Measured steady-state performance across the three CPT stages. MFU uses the 989 TFLOP/s BF16 peak. Stages/iter GFLOP/tok tok/s/GPU TFLOP/s/GPU MFU Broad CPT, 8k, CP=16.0522.7910,826246.8 24.95% Annealing, 8k, CP=15.8422.7911,226255.9 25.87% Long context, 128k, CP=8 159.7740.916,563268.5 27.15% 10 Gnani-Evon-v3.3-30B-A3BTechnical Report Appendix C reports timing and utilization, while Appendix D records non-lineage checks. 5.4 The realized training ladder The training ladder contains three distinct roles. The broad phase builds Indic capacity without abandoning English skills. Annealing concentrates the distribution on higher-quality sources. The last phase trains directly on long documents at 131,072 tokens. Figure 4: Realized production CPT token counts. The 128K extension consumed 99.86B of its 100B budget. Table 9: Planned and realized continual-pretraining stages. StageContextRealized tokens Role Broad CPT8,192∼974.49BDiverse English–Indic skill and coverage phase. HQ annealing8,192194.89BHigher-quality continuation initialized from the best broad-CPT validation checkpoint. Long-context extension 131,072∼99.86BDirect NoPE long-document training; 372 of ∼373 target iterations. The long-context stage consumed 99.86B tokens and stopped one iteration short of its nominal 100B budget. Table 9 gives the batch, length, and stage totals. Across broad CPT, annealing, and long-context training, the documented realized total is 1.269T tokens. Appendix E.1 reproduces the available training panels for all three CPT stages, including learning rate, gradient norm, language-model loss, and MoE load balancing. The phase-1 panel covers the exported interval, not every retained checkpoint. 11 Gnani-Evon-v3.3-30B-A3BTechnical Report 5.5 Checkpoint evaluation across CPT phase 1 and annealing The matched checkpoint series shows where CPT changes the model. Figure 5 traces the trajectory, Table 10 gives selected endpoints, and Appendix F reports the complete 21-benchmark series. Absolute scores from this series should not be mixed with the earlier embedding-warmup evaluation. Figure 5: Matched CPT checkpoint trajectory. The upper panel shows Indic uplift and English retention; the lower panel shows FLORES-IN bits per byte. The divider marks the start of annealing. Phase 1 produces most of the movement: broad Indic gains with a measurable English-retention cost. Annealing preserves those gains and further improves FLORES-IN bits per byte, while English results remain mixed. Because later SFT and RL evaluations use different protocols, only the trajectory within this series should be compared. 5.6 Ablation-guided language allocation A 120B-token ablation showed that the weakest starting languages gained most, while a more Indic-heavy mixture did not improve the aggregate. These non-lineage studies informed later language allocation; Appendix D gives the results and planning weights. 5.7 Long-context mixture The long-context manifest balances long Indic and English documents with mathematics, long-context SFT-style data, code, retention data, and a smaller multilingual component. 12 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 10: Selected matched results across the CPT stages. The long-context column is Evon-Base. BenchmarkMetricBasePhase 1AnnealingLong context Indic accuracy macroaccuracy (%) 41.0946.8346.6346.83 MILUaccuracy (%) 40.6353.5453.5253.24 ARC-Challenge-Indicaccuracy (%) 35.2348.2048.2047.09 IndicGLUE-selectedscore (%)52.1257.8657.2257.97 IndicMMLU-Proaccuracy (%) 14.4420.0320.0219.44 Indic BPB overallbits/byte ↓ 0.84530.61890.61320.6177 English 9-bench meanaccuracy (%) 70.6267.9868.4067.95 MMLUaccuracy (%) 78.5075.2075.1175.28 HellaSwagaccuracy (%) 85.5783.5283.6983.69 Table 11: Composition of the 100B-token long-context pool. The selected run consumed ∼99.86B tokens. BucketWeight Shards Pool tokens Indic long documents0.22922.0B Indic high-quality0.1513215.0B English long documents0.20720.0B Mathematics and STEM0.14114.0B Long-context supervised data0.10310.0B Code0.0818.0B Base-retention data0.0727.0B Other multilingual0.0424.0B 13 Gnani-Evon-v3.3-30B-A3BTechnical Report We call the post-long-context checkpoint Evon-Base, and all later training starts from it. Appendix F compares it with the earlier CPT checkpoints. 6 Supervised Fine-Tuning 6.1 Role in the model story Continual pretraining leaves Evon-Base fluent in the target languages and domains, but that does not by itself define a helpful assistant. SFT teaches the chat protocol: how to follow a request, produce a final answer, call tools, maintain a conversation, and use the requested language and script. We call the public-facing SFT stage Evon-Instruct. Its retained path is SFT-phase-1→SFT-phase-2 →SFT-rebuild-3→corrected epoch 2. A continuation after SFT-phase-2 was discarded because of a data-splitting error, and the curriculum was rebuilt as SFT-rebuild-3. Experimental runs outside this path are not ancestors of the released RL model. Table 12: SFT lineage relevant to the released RL checkpoint. StepDispositionRole SFT-phase-1retained ancestor first supervised curriculum stage SFT-phase-2retained ancestor second supervised stage intervening continuation discardedaffected by data-point splitting; not in the released lineage SFT-rebuild-3 epoch 1 retained branch point first pass over the rebuilt SFT-rebuild-3 corpus. Iteration 2,000 is the branch point for epoch 2 Evon-Instruct (SFT-rebuild-3 epoch 2) selected checkpointepoch-2 continuation from SFT-rebuild-3 epoch-1 iteration 2,000; † direct initialization for RL † Epoch 2 restarts from SFT-rebuild-3 epoch-1 iteration 2,000 with fresh optimizer state; later epoch-1 checkpoints showed elevated MoE load-balancing loss. The retained SFT stages also differ in corpus composition. Table 13 gives the row-level summary, and Appendix G gives the language mix and approximate source-family shares. The selected Evon-Instruct checkpoint uses the SFT-rebuild-3 corpus, whose materialized rows are 64.0% English. Table 13: Materialized corpus summary across the retained SFT lineage. StageMaterialized rows English share Principal change SFT-phase-1∼12.5M50.7% Era-1 phase-1 pool; instruction following, mathematics, and code dominate. SFT-phase-2∼14.7M54.6% Era-1 phase-2 pool; code, science, and agentic shares increase. SFT-rebuild-3∼15.2M64.0% Era-2 rebuild with a larger English share. 14 Gnani-Evon-v3.3-30B-A3BTechnical Report 6.2 Selected Evon-Instruct checkpoint RL initializes from the iteration-2,400 export of the corrected SFT-rebuild-3 epoch-2 run. It represents 80.53B scheduled sequence positions, including padding. A direct sample of the assistant-only loss mask estimates roughly 15B loss-bearing positions. The run is stable at this point, with a recorded language-model loss of 0.614. Appendix G.1 gives the full export, packing, and schedule accounting. 6.3 Data construction The SFT mixture covers English and eight Indic languages in native and romanized scripts [12,13,14]. It includes question answering, reasoning, mathematics, summarization, instruction following, tool interaction, and multi-turn dialogue. The final SFT-rebuild-3 build is 64.0% English and 36.0% Indic by materialized rows. Appendix G gives the source families; these row shares do not measure token exposure. Gnani-published datasets use the Apache-2.0 terms stated on the Evon repository. Upstream datasets retain their original terms and attribution requirements. Translation is structure-aware. User-facing prose can be translated, while code, mathematical spans, tool schemas, function names, and machine-readable arguments are preserved byte-for-byte. Failed or incomplete translations are quarantined rather than replaced with English text. Conversations are normalized into a common schema with visible answer content separated from reasoning content. Deduplication is applied after normalization so that equivalent conversations stored in different source formats can match. 6.4 Masking and long-sequence packing Training loss is applied only to assistant spans that should be learned. System and user history remain visible as context but are masked from the objective. Tool chains remain intact so that intermediate assistant actions and tool observations form one coherent training example. Packing places multiple conversations into 131,072-token sequences while preserving their boundaries. Table 14 records the production topology and optimizer. Table 14: SFT configuration for the long-context production family. SettingValue Maximum sequence length 131,072 PrecisionBF16 GPUs256 NVIDIA H200 ParallelismTP=1, P=1, CP=4, EP=8, DP=64 Global / micro batch256 / 1 OptimizerAdam, β 2 = 0.95, ε = 10 −8 Weight decay / clipping 0.1 / 1.0 Packing objectivehigh utilization with assistant-only loss Downstream roleinitialization for both RL investigations 15 Gnani-Evon-v3.3-30B-A3BTechnical Report Appendix H reproduces the available Weights & Biases panels for SFT-phase-1, SFT-phase-2, and the final SFT-rebuild-3 stage. The stage-wise loss levels should not be compared as if they came from one stationary data distribution: each curriculum rebuild changes the source mixture, packing, and assistant-token composition. 6.5 What SFT establishes The SFT initialization begins RL with strong native-script output: measured in-script rates range from 93.5% to 99.1% in one multilingual harness, and a later matched IndicIFEval-ground evaluation records 99.9%. It also establishes the response format and tool-call syntax expected by the RL parsers. The purpose of reporting these pre-RL conventions is not cosmetic. RL optimizes whatever is rewarded by its environments. If a convention is present in SFT but absent from every RL reward, it can disappear quickly even while benchmark reasoning improves. 7 Reinforcement Learning 7.1 GRPO in one equation The released RL stage uses group relative policy optimization (GRPO) [6]. For each prompt it samples G= 16 completions, centers each reward against the other fifteen, and scales all advantages by the whole group’s sample standard deviation: μ −i = 1 G− 1 X j̸=i r j ,(4) σ = v u u t 1 G− 1 X j (r j −r) 2 ,(5) A i = r i − μ −i σ + ε , ε = 10 −6 ,(6) The division is applied only forσ >0. Equal-reward groups contain no relative signal and are resampled. Appendix L records the estimator details and the later experimental floor. The model learns which completion was better than the other completions for the same prompt. It does not require a separately trained critic to assign an absolute value to each response. Dynamic sampling discards and replaces groups in which every completion receives the same reward. Consequently, a dataset’s nominal row share is not its effective training share: s effective,e ∝ s nominal,e p mixed,e ,(7) wherep mixed,e is the fraction of prompt groups in environmentethat contain more than one reward value. Realized shares must therefore be measured from rollout logs. 16 Gnani-Evon-v3.3-30B-A3BTechnical Report 7.2 Released Evon-RL-Gen Evon-RL-Gen is the public BF16 checkpoint and the source of the planned FP8 artifact. It initializes from Evon-Instruct and trains on four NeMo-Gym task environments: • math with judge: final-answer equivalence after boxed-answer extraction; • structured outputs: JSON validity and schema conformance; • workplace assistant: correctness of an iterative tool-call sequence; • MCQA: exact match on a selected option. No custom reward shaping is applied in the released run. Each environment returns its own binary or task-specific scalar. There is no language reward, length reward, or repetition penalty. Table 15: Released Evon-RL-Gen training configuration. SettingValue InitializationEvon-Instruct AlgorithmGRPO with leave-one-out baseline and whole-group standard-deviation normalization Resolved estimator flagsuse_leave_one_out_baseline=true; normalize_rewards=true Generations per prompt16 Dynamic samplingenabled; zero-variance groups resampled Sampling temperature 1.0 Inference placementcolocated vLLM and training workers Tool-call parserQwen3 XML parser Reasoning parserNemotron-v3 parser Custom reward shaping none The largest gains appear on tasks represented directly in the reward mixture: MATH rises 36.07 points and GPQA Diamond 26.84. Competition mathematics moves little because the SFT checkpoint is already strong there. Table 36 gives the matched results; extractive-QA comparisons use F1 because the exact-match rule changed between recorded runs. The regressions cluster in Indic extractive question answering even as Indic free generation improves. On XQuAD-Indic, the share answered in English rises from 8.7% to 91.7% [15]. Conditional accuracy in the target script roughly doubles, although this is a self-selected subset rather than a controlled estimate. English outputs are classified before grounding, so the lower ungrounded count is not independent evidence of better reading. The evidence points mainly to output language, although a smaller capability loss remains possible (Appendix J). This behavior follows the reward boundary. The run contains no language reward, and English completions receive 0.1681 more reward on average, almost entirely from structured-output and workplace-assistant tasks. IndicIFEval is sensitive to evaluation state; the canonical release-protocol value is 33.05, and Appendix J records the other readings. 17 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 6: Released-run validation rewards. Structured output and workplace assistance improve most; the curves are optimization diagnostics, not benchmark accuracies. 18 Gnani-Evon-v3.3-30B-A3BTechnical Report 7.3 Capability versus convention The released RL study separates capability from emission convention. On 5,000 MATH problems, accuracy rises from 51.00 to 85.80, while conditional accuracy on boxed answers barely changes. Changing only the boxed-answer emission mix reproduces 94% of the gain; Appendix I gives the decomposition. The exact decomposition depends on where the response is partitioned, but the conclusion does not: a benchmark swing can be carried largely by an output convention. Appendix I records the alternate partition and the unboxed-answer counts. RL can destabilize a cheap output convention without erasing the capability underneath. Track in-script rate, boxed-answer rate, parse success, and tool-call count alongside benchmark accuracy. This observation motivates a separate experimental branch: every product-critical convention needs a non-zero, correctly scoped training signal. 8 Release Evaluation 8.1 Comparison with open models We evaluate Evon v3.3 and six open checkpoints on the same 35 benchmarks. The suite combines English knowledge, reasoning, and code benchmarks [20,21,22,23,24,25,26] with Indic question answering, instruction following, and generation benchmarks [16,17,18,19]. Reference models use their published sampling settings, so the most reliable reading is movement within the Evon lineage, not a controlled ranking across providers. Two reference checkpoints emit a non-standard whitespace marker on code tasks; those code results are excluded. Figure 7 summarizes equal-weight English-retention and Indic-capability aggregates. Relative to its architecture-identical starting point, the released model gains 5.2 Indic points while giving up 1.7 English points. The unreleased Indic-aware branch gains 14.6 for 5.4. Larger open models lead both aggregates; the figure is meant to show the movement produced by the pipeline. 8.2 Safety evaluation IndicSafe evaluates 5,965 examples across 12 Indian languages. Overall accuracy is 86.50%. The model is strongest on harmless controls and ambiguous-but-safe prompts. The hardest categories are political manipulation and gender/caste discrimination. BBQ moves in opposite directions by context type. Disambiguated accuracy rises from 94.13 to 97.70 and bias falls from 0.99 to 0.60; ambiguous accuracy falls from 92.29 to 88.69 and bias rises from 4.33 to 7.39. The model appears more willing to commit, which helps when context resolves the answer and increases stereotyped completions when it does not. Safety scores are not a deployment guarantee. They measure one dataset and one decoding configuration. Applications should add domain-specific testing, moderation, logging, and human escalation. 19 Gnani-Evon-v3.3-30B-A3BTechnical Report 657075808590 English retention aggregate 45 50 55 60 65 Indic capability aggregate continual pretraining, fine-tuning, and RL +14.6 Indic / −5.4 English Evon-RL-Gen Evon-RL-Indic Evon-Instruct Nemotron-3-Nano-30B Gemma-4-26B-A4B Gemma-4-31B Qwen3.8-27B Sarvam-30B Sarvam-105B Figure 7: English retention and Indic capability. The arrow shows movement from the architecture-identical starting checkpoint; reference models use their published sampling settings. Table 16: IndicSafe accuracy by harm category. CategoryAccuracy (%) Harmless control97.93 Tricky / ambiguous97.77 Offensive or hate speech89.80 Harmful instructions85.62 Misinformation / conspiracy83.50 Health misinformation80.41 Religious hate / provocation80.08 Political manipulation72.49 Gender and caste discrimination70.26 20 Gnani-Evon-v3.3-30B-A3BTechnical Report 9 Experimental Research Branches (Not Released) Everything in this section belongs to research lineage, not the public release. The experiments are included because their negative results transfer and clarify the limitation in Section 7.3. 9.1 A sibling Indic-aware branch Evon-RL-Indic starts independently from the same Evon-Instruct export as the released run. It adds an Indic instruction-following environment, a scoped language reward, and a repetition penalty. On IndicIFEval-ground, the SFT score of 40.52 falls to 33.05 under the released mixture but rises to 60.00 under the experimental one. XQuAD-Indic F1 similarly moves from 36.51 to 17.19 and 50.44, respectively. Because several ingredients change together, these gains cannot be attributed to the language reward alone. They do show that the released regression is a property of its reward mixture rather than an inevitable effect of RL. The branch also learns a one-tool-call-per-turn policy, showing that a targeted repair can create another convention. Appendix K gives the full analysis. Thirteen of fourteen Indic benchmarks improve over the starting model, and eleven gain more than ten points. The losses cluster in competition mathematics, code, and instruction following (Figure 8). The branch reaches 60.0 on grounded Indic instruction following, compared with 48.9 for the strongest open reference. It is competitive, rather than leading, on Indic extractive QA (Figure 9). 9.2 Learning from a judge Verifiable rewards only reach tasks with a checkable answer. Mathematics, multiple choice, and schema conformance can be graded by a program. Open-ended helpfulness cannot. To extend reinforcement learning past that boundary we trained a reward model, a judge, and then optimized the policy against it. This section describes that judge, what training against it bought, and what it cost. The branch is a research lineage continuing from Evon-RL-Indic. It is not part of the released model. An Indic preference set had to be built first. We combine native prompts with translated HelpSteer3-style prompts in a 70:30 ratio. Two models generate candidate responses and a third labels the preferred one. Of 98,744 pairs in Hindi, Telugu, Tamil, and Malayalam, 98,327 receive labels; Appendix M gives the construction details. The judge writes a verdict rather than emitting a scalar. Initialized from a 26B-A4B model, it reasons over two responses and returns calibrated helpfulness scores plus a forced relative ranking. It scores 90.8 on RM-Bench against 87.3 for a 235B-parameter judge, with most of the margin on the hard slice. The comparison is against model-generated labels, however, and the judge has not been validated against human Indic preferences. The instrument had the same disease as the patient. At a 4,096-token cap, JudgeBench falls from 88.3 to 83.7 because some responses never reach the verdict tag. Thus 4.6 apparent quality points come from the judge’s own emission convention; Appendix M gives the completion-rate analysis. 21 Gnani-Evon-v3.3-30B-A3BTechnical Report −40−30−20−10010203040 Evon-RL-Indic minus Nemotron 3 Nano 30B (points) HMMT February LiveCodeBench IFBench AIME 2026 HMMT November MMLU-Pro AIME 2025 GPQA-Diamond AGIEval English MMLU IFEval-Hindi GSM8K BBQ IFEval MBPP+ Beyond-AIME MATH MATH-500 ARC-Challenge HumanEval+ MGSM-English IndicIFEval translated CrossSum-IN FLORES-IN IndicParam IndicIFEval grounded IndicQA IndicMMLU-Pro MILU BhashaBench-Multi XorQA-Indic BharatMath MGSM-Indic XQuAD-Indic XQuAD-English -19.8 -16.4 -15.3 -14.6 -13.7 -13.7 -11.5 -8.5 -7.3 -6.3 -4.8 -4.5 -3.8 -3.3 -2.4 -2.2 -1.6 -1.6 -0.8 -0.4 +2.0 +5.9 +6.7 +10.3 +10.7 +11.1 +11.5 +12.4 +14.1 +17.6 +18.7 +24.4 +24.4 +29.7 +33.9 Figure 8: Per-benchmark change for the unreleased Indic-aware branch. Positive values favour the branch. 22 Gnani-Evon-v3.3-30B-A3BTechnical Report 0204060 Qwen3.8-27B Evon-RL-Gen Gemma-4-26B-A4B Sarvam-105B Sarvam-30B Gemma-4-31B Evon-Instruct Nemotron-3-Nano-30B Evon-RL-Indic 25.9 33.0 34.9 35.5 35.8 38.2 40.5 48.9 60.0 Grounded Indic instructions 0204060 Evon-RL-Gen Nemotron-3-Nano-30B Sarvam-30B Evon-Instruct Gemma-4-31B Evon-RL-Indic Sarvam-105B Gemma-4-26B-A4B Qwen3.8-27B 17.2 20.8 30.0 36.5 42.1 50.4 50.9 55.2 59.1 XQuAD-Indic (F1) 010203040 Evon-RL-Gen Sarvam-30B Gemma-4-31B Nemotron-3-Nano-30B Evon-Instruct Sarvam-105B Evon-RL-Indic Gemma-4-26B-A4B Qwen3.8-27B 17.7 19.2 20.7 26.1 26.4 32.4 37.5 39.1 40.3 IndicQA (F1) 02040 Evon-Instruct Nemotron-3-Nano-30B Sarvam-30B Gemma-4-31B Sarvam-105B Gemma-4-26B-A4B Evon-RL-Indic Evon-RL-Gen Qwen3.8-27B 16.2 20.2 22.0 29.0 32.6 33.7 38.9 44.1 44.9 XorQA-Indic (F1) Figure 9: Grounded Indic instruction following and Indic extractive QA for the unreleased branch. 23 Gnani-Evon-v3.3-30B-A3BTechnical Report A reward model is measured by benchmarks that carry the same emission-convention confound as the policy’s own. Check a judge’s parse and completion rates before trusting its score, and re-check them at the generation cap you actually train with. What training against the judge bought. The largest gains are on LiveCodeBench (+13.2) and MMLU-Pro (+8.8). Knowledge, reasoning, grounded Indic instruction following, and mathematics also improve, while empty or unterminated outputs fall on most gated benchmarks. Table 48 gives the complete set of headline gains; Appendix M also describes the ring comparison used for judging. What it cost: the answer, wrapped in a sentence. Extractive question answering loses 24–38 F1 points even though answer spans remain present at similar rates. The parent returns a bare span; the judge-trained checkpoint wraps it in a sentence, adds a median of 9–17 tokens, and reduces overlap precision. Conversational elaboration has transferred to a terse-answer task. What it cost: the requested language. FLORES-IN falls 17 chrF++ points for a different reason. English-target script compliance falls from 98.6% to 24.5%, and the model often copies the Indic source instead. Monolingual extraction changes the form of a largely preserved answer; cross-lingual tasks can lose the answer itself. A length bonus drove reasoning to zero. Because the judge never sees hidden reasoning, reasoning length drifted. Rewarding the shortest trace drove its mean from 6,020 characters to zero while the judge score improved. A two-sided target still shortened reasoning by 60% and cost more than 25 AIME points. The adopted design is neutral below a generous budget and penalizes only traces above it. Any bonus placed on the extreme of a property the model controls cheaply will be collected, and a two-sided length target is an attractor. Constrain such properties with a one-sided bound outside a permitted range rather than a reward inside it. Mixing verifiable rewards back in did not work. A 70:30 mix of judge and binary-verifier prompts also failed. All-pass and all-fail verifier groups carry no relative signal, so dynamic sampling discarded 35–49% of them while every judge group survived. The intended mixture became roughly 80:20 at the gradient. The underlying pool contained too many problems the model almost always solved or almost always missed; Appendix M.5 gives the full analysis. When two reward sources are mixed, the realized mixture is the one that survives filtering, not the one specified. Measure the share each source contributes at the gradient, and instrument every environment rather than the one most likely to be healthy. 9.3 Recovering a convention by distillation The judge-trained checkpoint improves knowledge, code, and reasoning but loses useful conventions. On-policy distillation recovers those conventions without another reward model or verifier. The parent teaches the child. The student starts from the judge-trained checkpoint, while Evon- RL-Indic serves as its teacher. On the student’s own responses, the teacher supplies a top-64 next-token distribution at every position: L(θ) =E x, y∼π S h 1 |y| P t w KL p (t) T ∥p (t) S + (1− w) KL p (t) S ∥p (t) T i , w = 0.5.(8) 24 Gnani-Evon-v3.3-30B-A3BTechnical Report The two KL directions are weighted equally. One restores probability to forms retained by the teacher; the other suppresses forms the teacher rejects. Teacher and student share an architecture and tokenizer, and no reward is optimized during this stage. Most of what was lost comes back, and most of what was gained stays. Across the six degraded benchmarks, distillation recovers 81% of the loss; extractive QA recovers 74–99%. Cross-lingual QA and translation return to their starting values within measurement noise. The student also retains all of the MMLU-Pro gain, about three quarters of the code and mathematics gains, at least half on MMLU, IndicMMLU-Pro, and AGIEval, and one third on GPQA Diamond. The repair did not come from the repaired tasks. The 29,988 distillation prompts contain mathematics and multiple-choice questions, but no extraction, translation, or code. The repair therefore comes from the teacher’s token distribution rather than task-specific examples. Unlike a sparse reward, that distribution supervises every position, even when the desired form has become rare. It is not free. Distillation moves the whole distribution, including behavior that did not need repair. IFEval falls 2.6 points, translated IndicIFEval 1.9, and Indic grade-school mathematics 4.8. Non- termination and empty-output rates also worsen on some tasks. A convention lost to reward optimization can be restored by distilling from the checkpoint that still has it, without re-specifying the reward. The cost is that distillation moves everything, including behavior that did not need repair. 9.4 Weight-space alternatives to distillation Two cheaper weight-space controls require no generation: equal interpolation with the parent and extrapolation beyond the judge-trained checkpoint. Neither is a release candidate. Table 17 reports how much of the extractive question-answering loss each method returns. Distillation recovers 84% of it across the four benchmarks, interpolation recovers 41%, and extrapolation does not recover any of it. Extrapolation instead moves those benchmarks further in the direction the judge pushed them, ending well below the checkpoint it was meant to repair. Table 17: Share of extractive-QA loss recovered by each repair. Negative values indicate further loss. BenchmarkLoss to repair Distillation Interpolation Extrapolation XQuAD-English38.25+74.5%+65.8%−61.2% XQuAD-Indic32.77+89.9%+32.1%−25.2% IndicQA24.45+73.7%+21.0%−24.0% XorQA-Indic24.46+99.1%+46.1%−34.6% Macro+84.3%+41.3%−36.2% Interpolation preserves more of the judge’s gains but repairs only half as much extractive QA as distillation. Extrapolation makes the loss worse. Pushing farther along a reward direction therefore accelerates damage to behavior that the reward ignores. 25 Gnani-Evon-v3.3-30B-A3BTechnical Report 10 FP8 Quantization and Serving The FP8 checkpoint planned for release is produced from the frozen BF16 Evon-RL-Gen export using post-training quantization, not quantization-aware training. The pipeline uses NVIDIA Model Optimizer 0.45 through Megatron-Bridge [8]. No pruning, distillation, or INT4 weight-only compression is applied. 10.1 Selective E4M3 W8A8 recipe Quantized sites use static E4M3 FP8 for both weights and inputs, with per-tensor symmetric scaling and no affine zero-point term. A max calibrator records the absolute maximuma max . WritingF max for the largest finite E4M3 magnitude andQ E4M3 for rounding to the nearest representable E4M3 value with finite-range saturation, the conceptual quantize–dequantize operation is s = a max F max , q = Q E4M3 x s , b x = sq.(9) Unlike INT8, E4M3 is a floating-point format with a non-uniform representable grid determined by its exponent and mantissa. Quantization is therefore an FP8 cast after scaling, not uniform rounding to an integer code. Zero is represented directly by the floating-point format rather than introduced through an affine quantization zero point. Each quantized tensor uses one scale that maps its largest observed magnitude into the E4M3 range. The stored values use roughly half the bytes of BF16, while sensitive modules remain in BF16. The conservative recipe quantizes the routed and shared MoE expert input/output projections across all 23 MoE layers. Attention projections, attention matrix multiplications, Mamba projections and convolutions, routers, embeddings, final normalization, and the output head remain BF16. Table 18: Precision allocation in the FP8 artifact planned for release. ComponentPrecisionRationale Routed expert FC1/FC2FP8 E4M3 W8A8dominant parameter and compute block Shared expert FC1/FC2 FP8 E4M3 W8A8same expert-matrix structure Attention QKV/output and BMM BF16excluded by conservative recipe Mamba input/output projections and convolution BF16protects recurrent sequence-mixing path Routers / routing gates BF16protects token-to-expert decisions Embeddings, final norm, LM head BF16protects vocabulary and output calibration FP8 covers approximately 94% of parameters and reduces the artifact from 63.5GB to 32GB, while routers remain in BF16. Appendix A provides tensor-level accounting. 26 Gnani-Evon-v3.3-30B-A3BTechnical Report 10.2 Calibration and validation Calibration requested 1,024 multilingual examples but exhausted the source at 253 packed rows. All artifact and paired-quality gates still pass, although a production recalibration should fill the intended sample. Weight scales use weight maxima; activation scales come from the calibration pass. Artifact gates verify FP8 expert weights, BF16 exclusions, scale tensors, the hybrid layer pattern, tokenizer and EOS behavior, and the expected 32–34GB disk size. A no-op quantization that misses fused three-dimensional expert tensors remains near 62GB and fails this gate. 10.3 Serving performance BF16 and FP8 are served with the same stack on one H200 GPU. At concurrency 16, FP8 raises aggregate throughput from 1,165 to 2,135 tokens per second and roughly doubles usable concurrency. The complete latency and rate measurements appear in Table 19 and Figure 10. The gain differs from that of a dense transformer. Only six of the 52 layers contribute a growing key-value cache; the Mamba layers carry context-independent state. FP8 saves resident weight memory, increasing concurrency rather than usable context length. Figure 10: Paired BF16 and FP8 serving throughput on one NVIDIA H200 using the same serving stack. 10.4 Matched BF16 versus FP8 quality Across 32 paired accuracy benchmarks, the mean FP8-minus-BF16 delta is +0.0006, with seventeen positive and fifteen negative results. Only three absolute deltas exceed 0.02, all on samples of at most 541 items. IFBench is excluded because empty-response failures differ between the two arms. Appendix N lists every result. These are within-pair quantization deltas; their absolute scores should not be mixed with the release-evaluation campaign. FLORES-IN and CrossSum-IN move slightly downward, as do HumanEval+ and MBPP+; Indic MCQ and QA are otherwise flat to slightly positive. On IFBench, 11.7% of FP8 and 10.3% of BF16 responses are 27 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 19: Paired one-H200 serving measurements. MetricBF16FP8 Warm TTFT, concurrency 1348ms341ms Warm generation rate, concurrency 1345.7 tok/s 387.3 tok/s Generation-rate p50, concurrency 1341.6 tok/s 382.9 tok/s Aggregate generation rate, concurrency 81,059 tok/s 1,406 tok/s Aggregate generation rate, concurrency 16 1,165 tok/s 2,135 tok/s Usable concurrency under latency/rate gate∼8∼16 Disk footprint∼63.5GB32GB Figure 11: Selected paired accuracy deltas after FP8 PTQ. Small benchmark samples should not be over- interpreted. 28 Gnani-Evon-v3.3-30B-A3BTechnical Report empty, on only partly overlapping items, so the difference mixes instruction following with non-termination and is not included in the aggregate. 11 Compute and Infrastructure Training uses NVIDIA H200 clusters, Megatron-Bridge for CPT and SFT, and NeMo-RL with NeMo-Gym for reinforcement learning. Expert and context parallelism distribute sparse routing and long sequences. Table 20 reports GPU-hours for the released lineage. It excludes tests, discarded runs, retries, ablations, translation inference, and experimental branches. The SFT entry covers the retained curriculum. Table 20: Recorded GPU-hours for relevant checkpoints in the released lineage. Released-lineage training scopeRecorded GPU-hours Embedding warmup139 CPT broad 974B phase∼25,147 CPT annealing phase∼4,988 Complete 128K CPT long-context stage, allocated cluster time4,977 Entire retained SFT curriculum∼17,205 Released Evon-RL-Gen production run763 FP8 quantization and calibration8 The 128K ledger uses allocated cluster time, including startup and recovery. The RL value covers the released step-40 run, and Appendix C gives detailed timing and MFU. 12 Responsible Deployment 12.1 Intended use Evon v3.3 is intended for multilingual chat, long-context retrieval-augmented generation, English and Indic instruction following, mathematical reasoning, coding assistance, and tool-using agents. It can be deployed globally and self-hosted under the release license. The model is not recommended as an autonomous authority for medical, legal, financial, employment, credit, or other high-stakes decisions. Deployers should evaluate the exact model artifact, quantization, prompt template, languages, and domain used in production. 12.2 Risks The training corpus contains web and synthetic data and can reproduce factual errors, social bias, offensive material, or unsafe instructions. Multilingual safety is especially difficult because a safety behavior learned in one language may not transfer uniformly to another script, dialect, or code-mixed setting. 29 Gnani-Evon-v3.3-30B-A3BTechnical Report Reasoning traces can also expose unnecessary internal content, repeat sensitive prompt material, or create a false impression of certainty. Applications should decide explicitly whether reasoning is enabled and should not treat a generated chain of thought as an auditable proof. 12.3 Recommended deployment controls • run target-language and target-domain safety evaluations before launch; • apply input/output moderation appropriate to the jurisdiction and application; • verify requested output language and script downstream for strict workflows; • sandbox tools, validate arguments, and require confirmation for consequential actions; • log model version, quantization, template, and decoding settings for incident analysis; • disclose to users when they are interacting with an AI system; • retain human review for high-impact or irreversible decisions. 13 Release and Reproducibility The public BF16 weights are hosted athttps://huggingface.co/gnani/gnani-evon-v3.3-30B-A3B under Apache License 2.0 [9]. Inherited components retain their original terms, documented in the release licence inventory. An FP8 derivative is planned; Evon-Base and intermediate checkpoints are not. The release bundle records checkpoint and tokenizer identifiers, the chat template, the evaluation commit, and decoding settings. It also distinguishes report measurements from external results. 30 Gnani-Evon-v3.3-30B-A3BTechnical Report 14 Conclusion Evon v3.3 treats Indic capability as an end-to-end systems problem. The tokenizer reduces the cost of Indian scripts; embedding warmup integrates the new vocabulary; continual pretraining adds language capacity and direct 128K experience; SFT defines the assistant protocol; and the released RL stage improves reasoning and agentic performance. The central lesson is that an optimizer cannot reliably preserve behavior that no environment rewards. The released mixture improves mathematics, structured output, multiple choice, and tool use, while native-script compliance becomes less reliable. An unreleased Indic-aware branch reverses that trend, but remains separate from claims about the public model. The reward-model branch repeats the pattern: optimizing against a learned judge improves knowledge, code, and reasoning but weakens bare-span answers and English-target translation. The judge itself also loses 4.6 measured points when it fails to emit a verdict before the generation cap. Distillation from an earlier checkpoint restores most of the lost conventions without task-specific repair data. Remaining work includes router-health traces for the shipped expert count, the exact NVIDIA source- repository identifier, checkpoint-specific SFT token accounting, the FP8 artifact identifier, and serving measurements beyond one device. A Architecture and Artifact Accounting The released model contains 31,749,975,232 parameters. Under NVIDIA’s convention, with six routed experts active and input embeddings excluded, 3,313,773,760 are active per token. Including the input embedding gives 3,752,111,296. The vocabulary extension adds 32,000 rows to each of two untied matrices at model dimension 2,688: ∆P vocab = 2× 32,000× 2,688 = 172,032,000.(10) Each MoE layer contains 128 routed experts of width 1,856 and one shared expert of width 3,712. With arelu2MLP, this gives 258 expert tensors per layer, or 5,934 across 23 layers. Routers remain separate and in high precision. Ignoring biases, a routed expert contains 9,977,856 parameters and the shared module contains 19,955,712. The expert weights therefore contribute P MoE = (128× 9,977,856 + 19,955,712)× 23 = 29,833,789,440,(11) P MoE,active = (6× 9,977,856 + 19,955,712)× 23 = 1,835,925,504.(12) The embeddings, output projection, Mamba, attention, normalization, and routing parameters reconcile these figures with the measured 31.75B total and NVIDIA’s 3.31B active-parameter convention. 31 Gnani-Evon-v3.3-30B-A3BTechnical Report B Exploratory 64K Vocabulary Study mixv5 is the strongest completed fertility experiment. It uses an approximately 8B-token mixture, balanced across the eight target languages and spanning native, romanized, and transliterated text. It is a parallel study and does not alter the production CPT lineage. The 64K extension reduces mean Samanantar fertility by 40.8% relative to the upstream tokenizer. It improves further on the selected 32K tokenizer overall, but Malayalam is slightly worse. The per-language results are reported in Table 21. Figure 12: Fertility of the exploratory 64K mixv5 tokenizer against the upstream tokenizer. This tokenizer was not used by the production model. Table 21: Per-language fertility for mixv5, evaluated on 2,000 Samanantar samples per language with seed 42. Language Base Nemotron mixv5 64K Change Hindi2.05131.4924 −27.2% Bengali3.05982.0637 −32.6% Gujarati3.60861.7402 −51.8% Kannada3.86432.1830 −43.5% Malayalam4.96542.8548 −42.5% Marathi3.09811.8135 −41.5% Tamil3.77052.2906 −39.2% Telugu3.95952.3585 −40.4% Mean3.54722.0996 −40.8% The normalization variants are nearly indistinguishable by fertility. The shipped regex variant therefore reflects segmentation quality, not a measurable sequence-length advantage. 32 Gnani-Evon-v3.3-30B-A3BTechnical Report B.1 Production tokenizer chain of custody The production tokenizer descends from AdaptMix Round 09. Of its 32,000 extension entries, 31,999 remain byte-identical in the staged CPT tokenizer. The only intentional exception is ID 144891. Model preparation removes one degenerate merge and assigns ID 144891 to an unreachable placeholder. Before SFT export, the slot becomes <|file_sep|>; only its label changes. Production therefore uses the 163,072-entry AdaptMix Round-09 lineage. The 195,072-entry mixv5 result is a later parallel study and should never be described as the released tokenizer. An independent loader and sampler reproduces the native-script macro fertility within 0.006 of the reported result. A smaller off-protocol evaluation is retained only as a robustness check. C Training-Time and Throughput Diagnostics Iteration time was logged every tenth optimizer step. Table 22 reports the steady median after step 50 and the full logged range, including startup and recovery costs. The CPT phase-1 figures combine both segments of that stage. Its step time drifts upward across training, so the median reflects the settled value rather than a stationary rate; the other stages are stationary. Table 22: Optimizer-step timing across the production CPT and SFT stages. StageLogged n Medianp5–p95 Max / steady CPT phase 15,8086.055.63–6.1816.73 / 6.05 CPT annealing1,1615.845.81–5.9720.80 / 5.84 CPT long context37159.89 157.73–162.65 173.96 / 159.91 SFT-phase-120716.7115.43–18.0952.85 / 16.70 SFT-phase-238317.4016.55–18.3149.99 / 17.39 Evon-Instruct26617.5816.74–18.4260.07 / 17.57 These medians supply the throughput and utilization values in Table 8. Table 23 reports medians after step 50 from the recorded SFT telemetry. MFU is relative to the H200 BF16 peak of 989 TFLOP/s per GPU. Table 23: Measured steady SFT throughput by retained stage. StageTFLOP/s/GPU Aggregate TFLOP/s Tokens/s Tokens/s per GPU Seq./s MFU SFT-phase-1321.282.2k1.938M7,57014.8 32.5% SFT-phase-2308.479.0k1.894M7,39814.4 31.2% Evon-Instruct305.278.1k1.870M7,30414.3 30.9% 33 Gnani-Evon-v3.3-30B-A3BTechnical Report D CPT Ablations and Non-lineage Validation Runs The experiments in this section informed configuration and mixture choices but are not checkpoints in the released model’s ancestry. D.1 Operational validation runs A separate 5B-token smoke run validated 128K memory fit, NoPE execution, and cache construction. Smaller regression runs checked the topology. Neither is included in production quality or compute totals. D.2 The 120B mixing ablation The 55:45 English–Indic ablation showed that marginal value did not track data abundance. Kannada gained 14.4 points, while Hindi, the strongest starting language, declined slightly. A second, more Indic-heavy mixture did not improve the aggregate. Later planning therefore combined measured uplift with source quality and repetition constraints; Figure 13 and Table 24 give the full language breakdown. Figure 13: MILU accuracy before and after the 55:45 English–Indic 120B CPT ablation. Labels show percentage- point change. This run is not part of the released checkpoint ancestry. 34 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 24: Per-language MILU results from the 120B ablation. Displayed endpoints are rounded to one decimal place; deltas are computed from the underlying higher-precision scores. Language Base 120B CPT∆ Kannada32.847.2 +14.4 Tamil34.246.6 +12.4 Gujarati34.446.5 +12.2 Telugu35.847.9 +12.1 Malayalam 35.545.7 +10.1 Bengali39.447.7 +8.4 Marathi40.146.8 +6.7 Hindi53.151.4 −1.8 English65.663.9 −1.7 D.3 Uplift-weighted planning table Table 25 records the planning weights derived from measured uplift and high-quality data availability. Table 25: Language weights derived from 120B-ablation uplift and high-quality availability. MILU uplifts use the underlying higher-precision scores. Language MILU uplift HQ availability Weight Est. epochs Telugu+12.120B16%1.03 Tamil+12.412B15%1.60 Kannada+14.46B13%2.78 Gujarati+12.28B12%1.92 Malayalam+10.117B12%0.90 Bengali+8.419B12%0.81 Hindi−1.835B12%0.44 Marathi+6.79B8%1.14 35 Gnani-Evon-v3.3-30B-A3BTechnical Report E Detailed CPT Configuration Table 26: Selected broad-CPT configuration fields. FieldValue Initializationembedding-warmed 163,072-vocabulary checkpoint PrecisionBF16 Sequence length8,192 for broad CPT and annealing Broad-CPT global / micro batch 2,048 / 1 ParallelismTP=1, EP=8, P=1; data parallelism fills remaining ranks Optimizerdistributed AdamW Peak / minimum learning rate1.0× 10 −5 / 1.0× 10 −6 ScheduleWarmup–Stable–Decay with minus-square-root decay Warmup200 iterations in the locked production recipe Stable fraction0.8 MoE dispatcherflex + HybridEP Expert capacitycapacity factor 1.25 with pad-to-capacity Kernelsgrouped GEMM, selective recomputation, CUDA graph capture for attention and Mamba TrackingWeights & Biases; supplied production panels reproduced below E.1 Production CPT training traces The panels below reproduce the production training series under the public stage names. The sequence load-balancing curve is the MoE auxiliary objective, not the language-model loss. CPT phase 1 ran in two segments with a 123-iteration overlap. Their language-model losses agree to 3× 10 −5 in that interval; Figure 14 shows the combined trace. 36 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 14: CPT phase-1 training traces through iteration 58,084. Figure 15: CPT annealing traces. Learning-rate decay begins late while validation loss remains near 1.0. 37 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 16: CPT long-context traces through iteration 372; the early transition settles into stable training. E.2 CPT data composition The three continual-pretraining stages draw on separate blend manifests. Broad CPT uses a 974B English-majority mixture and annealing a 196B high-quality mixture, while the long-context stage uses the pool already reported in Table 11. The broad-CPT manifest retains 184 shard prefixes; 90 shards from its source manifest were dropped for absent per-shard caches and the remaining weights renormalized to unity. Two share conventions appear in the manifest audit block and are easy to conflate. A target share is a fraction within the English or Indic budget; a sampling share is a fraction of the whole mix. The same code allocation reads 53.2% one way and 29.1% the other. The figures below are realized sampling shares, computed as weight×974.5B from the per-source accounting, and are therefore comparable across the whole table. Blend weights are sampling probabilities rather than guarantees: realized per-source consumption also depends on shard exhaustion, and several sources in the annealing manifest are scheduled for less than one epoch. Table 27: Broad-CPT composition from the 184 retained shard weights. GroupTarget Realized Share∆ English536.0B 532.7B 54.66% −3.3B Indic (12 languages) 438.5B 436.6B 44.80% −1.9B Other multilingual–5.3B 0.54%– Total974.5B 974.5B 100% The largest shard is English code at 12.21% of the mix, followed by synthetic code question answering at 6.90%. No Indic shard exceeds 2.5%. 38 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 28: Realized Indic allocation in broad CPT, by language. Only the first eight languages received dedicated vocabulary expansion (Section 3); Odia, Punjabi, Urdu and Nepali receive pretraining data on a small budget without tokenizer support. Language Tokens Share Language Tokens Share Hindi132.4B 13.59% Marathi17.1B 1.76% Tamil69.3B 7.11% Odia4.4B 0.45% Telugu66.6B 6.84% Punjabi1.6B 0.17% Bengali53.8B 5.52% Urdu1.4B 0.14% Malayalam 33.5B 3.43% Nepali1.3B 0.14% Kannada28.7B 2.95% Gujarati26.3B 2.70% Table 29: Annealing composition by bucket from the manifest’s per-dataset allocations. BucketAllocated Share Reasoning62.70B 32.2% Knowledge retention34.20B 17.5% Mathematics and STEM31.52B 16.2% Code25.65B 13.2% Indic high-quality17.10B 8.8% Instruction10.10B 5.2% Multilingual5.70B 2.9% Web5.70B 2.9% The listed allocations are 1.14% below the stage budget because two dropped shards were redistributed across the survivors. Annealing contains 11.7% Indic data, down from 44.8% in broad CPT; its long- document complement supplied the long-context stage. That reduction should not be read as the cause of the further MMLU decline recorded for this stage in Table 30. The English majority in annealing is skill-shaped rather than knowledge-shaped: reasoning, mathematics and code together account for 61.5% of the stage against 17.5% of explicit knowledge retention. A stage that is 88% English can still cost English knowledge when little of that English is knowledge-dense, which is the more plausible mechanism for the observed change. F CPT Checkpoint Evaluation Series The matched series covers the base and each CPT endpoint. Table 30 reports all benchmarks, with percentage and bits-per-byte metrics kept separate. Deltas compare long context with the base and should be read within rows; ‡ marks a diagnostic, and ◦ an unevaluated slice. 39 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 30: Per-benchmark matched trajectory across the three CPT stages. BenchmarkMetricBase Phase 1 Annealing Long ctx.∆ Indic capability Indic macro (excl. BharatMath) accuracy41.0946.8346.6346.83+5.74 Indic macro (incl. BharatMath) accuracy39.5446.0345.6646.01+6.47 ARC-Challenge-Indicaccuracy35.2348.2048.2047.09+11.86 Belebeleaccuracy37.4942.1742.2742.54+5.05 BharatMathaccuracy29.9941.0639.7040.95+10.96 Global-MMLU-Liteaccuracy32.5036.1736.0035.75+3.25 IndicGLUE-selectedscore52.1257.8657.2257.97+5.85 IndicMMLU-Proaccuracy14.4420.0320.0219.44+5.00 IndicParamaccuracy28.0831.5931.5531.58+3.50 MILUaccuracy40.6353.5453.5253.24+12.61 Sanskritiaccuracy84.1580.6080.8280.85 −3.30 FLORES-IN BPB, uncond. ↓bits/byte1.09450.82720.81400.8172 −0.2773 FLORES-IN BPB, cond. ↓bits/byte0.59620.41070.41240.4181 −0.1781 Indic BPB overall ↓bits/byte0.84530.61890.61320.6177 −0.2276 English retention Overall, nine-bench meanmean70.6267.9868.4067.95 −2.67 ARC-Challengelikelihood acc67.6667.4967.9267.15 −0.51 GSM8K, strictexact match86.1384.4684.4684.00 −2.13 HellaSwaglikelihood acc85.5783.5283.6983.69 −1.88 MMLUlikelihood acc78.5075.2075.1175.28 −3.22 MMLU-ProCoT exact match 64.8354.7855.4354.76 −10.07 OpenBookQAlikelihood acc46.6043.6044.8044.20 −2.40 PIQAlikelihood acc84.4483.0383.4682.75 −1.69 RACElikelihood acc42.3941.7242.3941.53 −0.86 WinoGrandelikelihood acc79.4878.0678.3778.22 −1.26 HumanEval pass@1pass@131.7129.8834.1533.54+1.83 Diagnostic BharatBBQ acc_normacc_norm35.7335.0439.22‡◦– BharatMath uses precision-corrected accuracy because boxed-answer emission changes across the training ladder. Phase 1 and long-context accuracy are nearly identical, at 41.06 and 40.95, while the raw numeric- match score and response length move with answer format. This is the same capability-versus-convention pattern discussed in Section 7.3. Eight of nine Indic proportion metrics improve from the base to the final long-context checkpoint; Sanskriti is the exception. The English mean falls 2.67 points, almost entirely during phase 1, with MMLU-Pro contributing the largest decline. Long-context training adds little further English loss. HumanEval finishes above its base score, while several other tasks recover during annealing before easing again. 40 Gnani-Evon-v3.3-30B-A3BTechnical Report F.1 HELMET long-context evaluation HELMET uses base-model completion prompts, greedy decoding, and seed 42; values are on a 0–100 scale. Recall aggregates the harder retrieval slices and should not be compared directly with the broader RULER mean. Citation and re-ranking are omitted because their structured output format is not elicited by completion prompting. Indic slices were not evaluated at 128K. Table 31: HELMET evaluation of Evon-Base, the checkpoint that concludes long-context continual pretraining. Category8K 16K 32K 64K 128K Recall95.9 81.4 57.0 37.3 14.0 RAG62.7 60.2 56.6 49.4 37.2 ICL59.2 71.7 78.6 81.8 81.7 LongQA28.8 38.5 36.5 40.6 26.8 Indic-RAG57.1 59.1 54.2 46.8– Indic-Reader17.9 17.0 16.0 15.2– Indic-Abstention 99.7 99.6 99.0 95.6– G SFT Corpus Composition Across Retained Stages The SFT curriculum was rebuilt across three retained stages. This appendix reports materialized source rows, while Table 32 separately records packed and scheduled exposure. G.1 Selected checkpoint accounting The RL initialization is the iteration-2,400 export recorded in the export metadata. At global batch size 256 and sequence length 131,072, it represents 2,400× 256× 131,072 = 80,530,636,800(13) scheduled sequence positions, including padding. A direct mask measurement estimates that 18.2% are trainable, or roughly 15B loss-bearing tokens. Table 32 gives the exact sample and packing measurements. The prepared epoch contains 702,824 packed sequences. Sequential bucket boundaries produce 2,752 configured steps, with masked padding in each final partial batch and no wrap-around. The job reached iteration 2,665; RL uses the iteration-2,400 export. Table 32 records its loss, schedule, endpoint, and checkpoint accounting. 41 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 32: Training record for the SFT checkpoint used to initialize RL. SettingValue Public stageEvon-Instruct Export sourceselected export, verified from export metadata Scheduled sequence positions through export80.53B, including padding Approximate loss-bearing positions∼15B (18.2%; assistant-only loss mask) Trainable-rate measurement95,515 / 524,288 positions over four packed-parquet rows Intra-pack efficiency99.98% Prepared packed-sequence capacity92.12B positions across 702,824 sequences Fully scheduled capacity92.34B positions across 2,752 global batches Packed rows / scheduled documents702,824 / 15,160,245 Iterations: configured / recorded / RL export2,752 / 2,665 / 2,400 Last job-log line / last checkpoint written2,660 / 2,600 Epoch traversalsequential bucket blocks; no repeat or wrap-around Learning-rate schedulecosine, 5× 10 −7 peak to 1× 10 −7 minimum OptimizerAdam, β 2 = 0.95, ε = 10 −8 , weight decay 0.1 Materialized-row language balance64.0% English and 36.0% Indic RL-init loss0.614 Table 33: Stage totals for the retained SFT corpus chain. StageMaterialized rows Buckets English Key character SFT-phase-1∼12.5M1250.7% Era-1 phase-1 pool; instruction following, mathematics, and code dominate. SFT-phase-2∼14.7M1454.6%Era-1 phase-2 pool; code, science, and agentic shares increase. SFT-rebuild-3∼15.2M1564.0% Era-2 rebuild with a larger English share. 42 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 34: Materialized-row language mix across the retained SFT chain. Language SFT-phase-1 SFT-phase-2 SFT-rebuild-3 English50.7%54.6%64.0% Hindi16.5%15.2%11.1% Tamil6.5%6.0%4.8% Telugu6.5%6.0%4.8% Kannada6.5%6.0%4.8% Malayalam6.5%6.0%4.8% Bengali3.4%3.1%2.7% Gujarati1.7%1.6%1.5% Marathi1.7%1.6%1.5% G.2 Source-family evolution Table 35 groups the SFT sources into broad families and reports approximate within-stage shares of materialized rows, rounded to the nearest percentage point. English sources are drawn primarily from NVIDIA’s publicly released Nemotron post-training collections [1], alongside Gnani-built Indic dialogue and agentic-trace data. Indic rows in each family include structure-aware translations of the corresponding English sources (Section 6.3). Table 35: Approximate source-family composition across the retained SFT corpus chain (share of materialized rows). Source familySFT-phase-1 SFT-phase-2 SFT-rebuild-3 Mathematics (incl. proofs)∼29%∼28%∼15% Code (incl. competitive programming)∼24%∼29%∼13% Instruction following and chat∼28%∼15%∼19% Science and reasoning∼17%∼19%∼9% Agentic, tool use, and software engineering∼2%∼9%∼12% Indic dialogue–∼8% Safety and domain-specialized<1%∼1%∼1% Other curriculum buckets–∼23% The final SFT-rebuild-3 build is 64.0% English and 36.0% Indic at the materialized-row level. These are not token-level sampling or assistant-loss shares, which may differ from the row distribution. H SFT Training Traces The SFT-phase-1, SFT-phase-2, and SFT-rebuild-3 panels below show the retained SFT training stages. Step-local changes in language-model loss reflect changes in curriculum buckets as well as optimization; they should not be interpreted as a single stationary loss curve across the three stages. 43 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 17: Weights & Biases traces for SFT-phase-1. The panels report the MoE sequence load-balancing loss, gradient norm, cosine learning-rate schedule, and assistant-masked language-model loss. Loss plateaus and transitions align with changes in the sequential curriculum buckets. Figure 18: Weights & Biases traces for SFT-phase-2. The assistant-masked loss trends gradually downward under the cosine learning-rate schedule, while isolated gradient-norm and load-balancing spikes remain transient. 44 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 19: Final SFT-rebuild-3 SFT traces through iteration 2,665; Evon-Instruct lies within this run. 45 Gnani-Evon-v3.3-30B-A3BTechnical Report I Matched Evaluation of the Released Checkpoint Both checkpoints use one matched 35-benchmark campaign. Four extractive-QA rows were rescored with strict exact match after a harness update; F1 was unaffected and is used in the narrative. Deltas are RL minus SFT. Accuracy metrics are percentages, while translation metrics use native sacreBLEU units. BharatMath is omitted pending a harness revision; ↓ means lower is better. Table 36: Matched evaluation of Evon-Instruct and the released Evon-RL-Gen checkpoint. BenchmarkMetricFinal SFT ckpt RL ckpt step 40 ∆ (RL − SFT) Math & Reasoning MATHexact match48.6984.76+36.07 MATH-500exact match48.8084.00+35.20 GSM8Kexact match85.7594.31+8.56 MGSM-Englishexact match91.6098.40+6.80 MGSM-Indicexact match79.6084.60+5.00 AIME 2025pass@172.9274.38+1.46 AIME 2026pass@172.0874.58+2.50 Beyond AIMEpass@148.5053.56+5.06 HMMT February 2025 pass@159.5860.83+1.25 HMMT November 2025 pass@158.5463.33+4.79 GPQA Diamondpass@138.1965.03+26.84 Code LiveCodeBenchpass@151.3561.27+9.92 MBPP+pass@175.2478.78+3.54 HumanEval+pass@188.6688.54−0.12 Knowledge and MCQ MMLU (exact match) † exact match77.2186.68+9.47 MMLU-Proaccuracy66.6777.26+10.59 IndicMMLU-Proaccuracy47.0768.88+21.81 MILUaccuracy73.1678.33+5.17 ARC-Challengeexact match93.2695.90+2.64 AGIEval Englishexact match60.0273.47+13.45 BhashaBench-Multi accuracy42.0649.22+7.16 Instruction Following IFEvalprompt strict66.3682.81+16.45 IFEvalprompt loose67.6583.73+16.08 IFEval instruction strict73.2687.17+13.91 IFEvalinstruction loose74.2287.77+13.55 IFEval-Hindiprompt strict52.7150.59−2.12 IFEval-Hindiprompt loose53.4251.30−2.12 IFEval-Hindi instruction strict60.5159.64−0.87 IFEval-Hindiinstruction loose61.6260.27−1.35 Continued on next page 46 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 36: continued BenchmarkMetricFinal SFT ckpt RL ckpt step 40 ∆ (RL − SFT) IndicIFEval- translated prompt strict37.5149.87+12.36 IndicIFEval- translated prompt loose38.8850.87+11.99 IndicIFEval- translated instruction strict47.6861.65+13.97 IndicIFEval- translated instruction loose49.0462.43+13.39 IndicIFEval-ground prompt/ instruction strict 40.5233.05−7.47 IndicIFEval-ground prompt/ instruction loose 43.5933.43−10.16 IFBenchprompt strict21.6744.33+22.66 IndicParamaccuracy32.9439.28+6.34 QA and Translation XQuAD-English (EM) exact match46.9763.70+16.73 XQuAD-English (F1) F167.1578.86+11.71 XQuAD-Indic (EM) exact match18.8412.86−5.98 XQuAD-Indic (F1) F136.5117.19−19.32 XorQA-Indic (EM) exact match6.6735.24+28.57 XorQA-Indic (F1)F116.1644.09+27.93 IndicQA (EM)exact match12.7712.22−0.55 IndicQA (F1)F126.3617.72−8.64 FLORES-INchrF++14.5536.33+21.78 FLORES-INBLEU3.6414.94+11.30 CrossSum-IN (chrF)chrF7.0012.81+5.81 Bias diagnostic BBQaccuracy (disamb.) 94.1397.70+3.57 BBQaccuracy (amb.)92.2988.69−3.60 BBQbias score (disamb.) ↓ 0.990.60−0.39 BBQ bias score (amb.) ↓ 4.337.39+3.06 † MMLU here is generative exact match and is not comparable to the loglikelihood accuracy reported in Table 30. The MATH gain combines emission and conditional accuracy. In raw responses, boxed-answer emission rises from 2,911 to 4,612 of 5,000 problems, while boxed conditional accuracy rises from 86.4% to 92.2%. Emission explains 81–83% of the gain, depending on decomposition order. Section 7.3 instead partitions after reasoning is stripped, matching the live grader. These are two definitions applied to the same campaign and should not be mixed. MATH emission decomposition. The main text partitions responses after reasoning is removed. Partitioning the raw response instead finds 150 additional boxed SFT outputs without changing the 2,516 correct boxed answers. The resulting denominators describe different conventions and must not be mixed. 47 Gnani-Evon-v3.3-30B-A3BTechnical Report J Indic Extractive-QA Failure Taxonomy Section 7 attributes the Indic extractive-QA regression to an unrewarded output convention rather than to a loss of reading ability. This appendix records the item-level evidence. Each response is assigned to one bin, in order: B0, no answer; B1b, English where the gold span is Indic; B2, a gold-equivalent answer recovered only by prose-tolerant regrading; B3, a target-script answer that is neither gold-equivalent nor a verbatim passage span; B4, a wrong passage span; and OK otherwise. Table 37 reports the four largest bins, so rows do not sum to 100%. No response is empty, and the omitted bins contribute at most three percent per row. Shares are equal-weight macros over eight languages and should not be pooled across languages. This slice differs from the eleven-leaf scorecard macro in Table 36. Table 37: XQuAD-Indic failure taxonomy, equal-weight macro over eight languages. CheckpointMacro F1 B1b B2B3 OK Evon-Instruct (SFT initialization)40.38 8.7% 5.8% 62.6% 21.6% Released Evon-RL-Gen16.59 91.7% 0.1% 2.5% 2.8% Indic-aware sibling (Sec. K)53.43 15.3% 6.0% 45.4% 31.9% IndicIFEval also varies across evaluation states. The FP8-paired BF16 row below is the canonical release-protocol reading used in the main text; the earlier rows are retained as sensitivity checks. Table 38: Recorded IndicIFEval sensitivity across evaluation states. MeasurementGround strict Translated strict Ground in-script Initial matched evaluation28.6647.455.4% Dedicated IndicIFEval evaluation ∗ 63.88–54.2% FP8-paired BF16 baseline33.0549.87– ∗ The loose-criterion score is 65.62. IndicQA reproduces the language shift: B1b rises from 9.5% to 73.0%. A prose-tolerant regrade moves macro F1 by at most 0.24 points, so extraction alone does not explain the loss. The taxonomy has two limits. B3 is neither gold-equivalent nor verbatim, but it is not independently classified as ungrounded because incorrect Indic answers rarely reproduce a passage span exactly. Likewise, OK means that the response contains the gold span, not necessarily that it is only the gold span. These distinctions matter most for the sibling branch, whose answers are usually in the target script. Conditioning on target-script output separates emission rate from answer quality. Table 39 shows that the script rate collapses while conditional EM and F1 rise on both benchmarks. This is consistent with preserved reading ability and a changed output convention, although it is not a controlled estimate. Table 39: Target-script rate and conditional accuracy before and after released RL. Target-script rate Conditional EM Conditional F1 Benchmark SFTRL SFTRL SFTRL XQuAD-Indic 89.95.4 23.2251.50 37.6961.19 IndicQA86.924.3 20.2445.23 27.0050.54 48 Gnani-Evon-v3.3-30B-A3BTechnical Report Target-script answers form a self-selected subset, so the conditional rise may reflect selection. We use it as evidence against broad reading collapse, not as a capability estimate. The sibling study’s Indic-aware checkpoint restores F1 above the initialization while its own B1b share, at 15.3%, remains above the initialization’s 8.7%. Rewarding target-script output recovered the task score without fully restoring the convention. K Sibling Indic-Aware RL Study (Not Released) K.1 Branch definition and reward Evon-RL-Indic is a sibling research branch, not the public Evon v3.3 checkpoint and not a continuation of the released RL run. It initializes independently from the same Evon-Instruct iteration-2,400 checkpoint and adds a fifth environment, indic_if, designed for Indic instruction following. Its reward is R = R task + 0.3R lang − 0.0R len − 0.3R rep ,(14) Malformed or abandoned answers score−0.25. The branch also floors the advantage-normalization denominator at 0.05, preventing tiny reward differences from becoming large updates. R task is the fraction of constraints satisfied.R lang measures the share of answer words written in the row’s target script.R rep detects repeatedn-gram bursts and character runs across the full generation. The length term is disabled in the experimental run. The scopes are deliberate. Language compliance is computed only over the answer after the final reasoning block, because English internal reasoning with an Indic user-visible answer is acceptable. Repetition is computed over the entire generation, because looping reasoning is undesirable in any language. The reward is target-aware rather than “Indic is always good”. English examples target Latin script; Hindi targets Devanagari; Bengali targets Bengali script; and so on. A language-identification gate helps distinguish languages that share a script. Unknown language labels cause the language reward to be skipped rather than guessed. K.1.1 Why the experimental mix retains an English guardrail pool The final mix diverts 1,509 English rows, or 9.9% of the input, from the existing task environments into a shared guardrail pool. These rows are not an additional English corpus. This guardrail is essential. An Indic-only reward taught the model to answer English prompts in Indic scripts. The English rows instead make “use the requested language” learnable. Directive ablations show that the literal language name is the active cue: explicit Hindi directives rise from 38.4% to 90.5% in-script, while implicit forms remain near zero. The training prompts therefore vary wording and placement but retain explicit language supervision. 49 Gnani-Evon-v3.3-30B-A3BTechnical Report K.1.2 Experimental mixture and configuration The experimental run reinitializes from Evon-Instruct and trains for 40 steps over 15,300 prompts. Table 40 and the configuration table give the mixture and rollout details. The input proportions are cap-driven rather than specified as a target weight vector. Each rated environment is first filtered to a difficulty band and then capped; the final proportions are the row counts that survive those operations. Mathematics dominates because it has the largest eligible pool.indic_if and MCQA retain unrated rows and are therefore pool-size limited rather than difficulty selected. Logic environments are excluded from this experimental mix. Figure 20: Experimental Evon-RL-Indic training-row composition after difficulty filtering, environment caps, and English-row diversion. This mixture is not used by the released model. Table 40: Cap-driven composition of the experimental Evon-RL-Indic input mix. Component shares are rounded to one decimal place and therefore sum to 100.1%; row counts sum exactly to 15,300. PoolRows Final share Mathematics6,19240.5% Indic instruction following 3,17920.8% Structured outputs1,93212.6% Workplace assistant1,80411.8% English guardrail1,5099.9% MCQA6844.5% Total15,300100% These row shares are not effective gradient shares: dynamic sampling removes all-pass and all-fail groups, so environment contributions must be measured from the surviving rollouts. 50 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 41: Experimental Evon-RL-Indic configuration. SettingValue AlgorithmGRPO with leave-one-out baseline and whole-group standard-deviation normalization Inherited estimator flagsuse_leave_one_out_baseline=true; normalize_rewards=true Generations per prompt 8 Dynamic samplingenabled Sampling temperature 1.0 Training steps40 Training rows / validation rows 15,300 / 256 Recorded rollouts251,906 Maximum generated tokens 16,384 Maximum RL sequence length 49,152 Model context capabilityremains 131,072 Advantage standard-deviation floor 0.05 Checkpoint cadenceevery 5 steps The 49,152-token RL cap is an optimization limit for rollout collection; it does not alter the model’s trained or exported 131,072-token context capability. Aggregate validation reward rises from 0.320 at initialization to 0.649 at step 40. Indic instruction following and structured output contribute most of the gain; mathematics peaks at step 30 before easing. These branch-specific rewards are not directly comparable with the released run. 51 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 21: Validation rewards for the Indic-aware branch. These optimization diagnostics are not benchmark accuracies. K.1.3 Experimental matched results Evon-RL-Indic is the first experimental artifact in this programme to improve reasoning and Indic instruction following simultaneously. It is reported as evidence about reward design, not as a result for the released Evon v3.3 checkpoint. Both columns come from the campaign in Table 36, allowing direct comparison. FLORES-IN chrF++ uses native sacreBLEU units, not percentages. 52 Gnani-Evon-v3.3-30B-A3BTechnical Report Figure 22: Selected Evon-Instruct to experimental Evon-RL-Indic results. Deltas are percentage points; the checkpoint is not released. Table 42: Matched evaluation of the unreleased Indic-aware checkpoint against its SFT initialization. BenchmarkEvon-Instruct Evon-RL-Indic∆ IndicIFEval-ground, prompt strict40.5260.00+19.48 IndicIFEval-translated, prompt strict37.5163.27+25.76 XQuAD-Indic, F136.5150.44+13.93 MATH48.6986.76+38.07 GSM8K85.7591.58+5.83 IFEval, prompt strict66.3686.88+20.52 GPQA Diamond38.1960.04+21.85 FLORES-IN, chrF++14.5539.37+24.82 Native-Indic in-script rate † 99.998.7 −1.2 † In-script rate is a separate measurement and is not part of the benchmark campaign above. Relative to the released checkpoint, the largest gains are on IndicIFEval-ground (+26.95 prompt-strict) and XQuAD-Indic (+33.25 F1). The table above measures the same benchmark against Evon-Instruct rather than against the released checkpoint, which is why its figure differs. Figure 23 shows the remaining trade-offs. Native-script retention is not uniform across prompt types. On the translated IndicIFEval split, in-script output falls from 95.1% for Evon-Instruct to 84.6% for the experimental checkpoint, even as task accuracy improves. This branch is therefore evidence for the reward-design mechanism, not a claim that every language-control problem is solved. Step 40 is selected because it retains the step-30 mathematics gain while improving several other tracked tasks. Mathematics is well defended because it has the largest share of the mixture. 53 Gnani-Evon-v3.3-30B-A3BTechnical Report K.1.4 Direct comparison with the released checkpoint The two branches are compared under one matched configuration, with strict exact match on both sides. Their 33-metric means are effectively equal, but the equality hides opposite trade-offs. The Indic-aware branch makes large gains on the instruction-following rows it targets, while the released checkpoint has more numerous, smaller advantages elsewhere. Removing the five targeted instruction-following rows gives the released checkpoint a 0.0257 lead. Figure 23 shows the largest differences. Figure 23: Selected matched differences between experimental Evon-RL-Indic and released Evon-RL-Gen. Positive values favor the experimental checkpoint. Evon-RL-Indic leads on instruction following, translation, and several Indic tasks. The released checkpoint remains stronger on reasoning, code, and English knowledge. The same trade-off appears in agentic evaluation: the branch improves the TauBench tasks but falls on SWE-bench Verified (Table 43). Table 43: Additional released-to-experimental agentic comparison. Deltas are percentage points. BenchmarkEvon-RL-Gen Evon-RL-Indic∆ MultiChallenge0.49250.5000 +0.75 TauBench airline0.37330.4400 +6.67 TauBench retail0.15200.2807 +12.87 TauBench telecom0.09360.3626 +26.90 SWE-bench Verified0.22800.1220 −10.60 54 Gnani-Evon-v3.3-30B-A3BTechnical Report K.1.5 Experimental tool-use decomposition On BFCL v4, the overall 17-category score changes from 53.81 for SFT to 51.16 after RL. By itself, that suggests a general tool-use regression, but the category breakdown reveals a narrower issue. Table 44: BFCL v4 decomposition. CategoryEvon-Instruct Evon-RL-Indic Headline, renormalized over 17 categories53.8151.16 Parallel / parallel-multiple69.5 / 80.00.5 / 0.5 Live parallel / live parallel-multiple75.0 / 75.00.0 / 0.0 Simple Python / multiple86.8 / 81.094.5 / 93.0 Multi-turn base34.058.0 The experimental policy usually emits one tool call when the benchmark expects two in the same assistant message. Its syntax parses correctly, and it issues the missing call on a second turn in 95 of 96 examined cases. The model retains the functional ability but has learned a one-action-per-turn policy from an environment that always follows a call with an observation. L Reinforcement-Learning Configuration, Side by Side This appendix sets the released run beside the Indic-aware branch of Section 9. The two share an initialization, an optimizer, a length budget, and every KL and clipping setting. They differ in how prompts are batched, and in what the reward mixture contains. The saved branch configuration names 30 nodes, while the run itself used 16. It also carries three corrections absent from the released run. First, it clamps the advantage-normalization divisor atmax(σ,0.05) + 10 −6 . Without this floor, tiny reward differences can become full-sized updates. Second, multi-turn rows are reduced from 37% to 11% of the Indic instruction pool. Third, empty content is treated as malformed rather than merely unrewarded. M Reward Model and Distillation Study (Not Released) This appendix records the reward-model, distillation, and abandoned-run configurations behind Sec- tions 9.2 and 9.3. None belongs to the released lineage. M.1 Preference data Prompts combine native Indic instruction data with HelpSteer3-style English prompts translated into eight Indian languages, in a 70:30 translated-to-native ratio. Two models generate one candidate response each, and Gemini Flash labels the preference on a seven-point scale. HelpSteer3 contributes the format and protocol, not its data. Of 98,744 pairs in Hindi, Telugu, Tamil, and Malayalam, 99.6% were labeled. 55 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 45: Training configuration for the released run and the Indic-aware branch, both read at step 40. “Same” means the two runs carry an identical setting. SettingEvon-RL-Gen (released) Evon-RL-Indic (branch) Optimizer and learning rate OptimizerAdam, distributed, BF16, weight decay 0.01 Same Peak learning rate5× 10 −6 Same Minimum learning rate5× 10 −7 Same Schedule constant, 13-iteration warmup from the floor Same Gradient clipping1.0Same Learning rate logged at step 405× 10 −6 5× 10 −6 Batching Prompts per step128255 Generations per prompt168 Nominal rollouts per step2,0482,040 Training global batch2,0482,040 Training micro-batch44 Generation batch3232 Retained samples at step 402,048, with 238 discarded2,040, with 610 discarded Dynamic samplingon, up to 10 extra generation batches Same Cluster in the saved configuration32 nodes, 8 GPUs each30 nodes, 8 GPUs each KL, clipping, and advantage PPO ratio clip0.2 either sideSame KL penalty in the loss0Same KL penalty in the rewardoffSame KL estimatork 3 , input clamp 20, output clamp 10 Same Advantage GRPO, leave-one-out baseline, normalized Same KL penalty logged at step 400.00.0 Lengths Maximum total sequence49,152Same Maximum new tokens per generation16,384Same Rollout-engine context limit49,152Same Sequence-packing tile98,304 tokensSame Generated tokens at step 40, median2,5272,220 Generated tokens at step 40, 95th percentile 10,27611,781 56 Gnani-Evon-v3.3-30B-A3BTechnical Report 010203040 Share of training prompts (%) Mathematics, judged Indic instructions Structured output Workplace assistant English guardrail Multiple choice 22.5 40.5 absent 20.8 31.0 12.6 25.9 11.8 9.3 9.9 11.2 4.5 Released general mixture Indic-aware branch Figure 24: Nominal environment shares before dynamic sampling for the released and Indic-aware runs. M.2 The judge’s reward Initialized from Gemma 4 26B-A4B with its vision tower frozen, the judge emits two absolute helpfulness scores and a forced relative ranking. GRPO rewards the correct side, calibration, and agreement between the outputs; Table 46 gives the terms. Table 46: Reward terms used to train the generative judge. TermReward Correct relative ranking +4 Calibration−1.2H 1.5 (d r )− 0.6[H 1 (d h 1 ) + H 1 (d h 2 )] Absolute/relative inconsistency −0.5 Unparseable verdict −8 Parseable rangeclipped to [−6, 4], then adjusted for batch-level rank balance Here H δ is Huber loss, d h k = ˆ h k − h k , and d r = ˆr− r. The format penalty lies below the clip floor, so every parseable verdict beats every unparseable one. Calibration also preserves a gradient when all samples choose the same side, which occurs in 78% of groups. Advantages are therefore centered but not divided by group standard deviation, avoiding the amplification of small calibration differences. Training uses 262,834 examples, with each pair shown in both orders; 72.4% come from the Indic set. Calibration, rather than accuracy, continues to improve. The rank-balance term reduces total variation distance between predicted and target rank histograms from 0.327 to 0.083 without lowering accuracy. Step 100 is exported; later checkpoints regress on both measures. 57 Gnani-Evon-v3.3-30B-A3BTechnical Report RM-Bench uses pairwise comparisons, so its scores are not directly comparable with scalar reward-model scores. JudgeBench is a smaller capped-generation subset. The largest margin below is on the hard slice, where style and substance point to different responses. Table 47: The 26B-A4B judge against a 235B-parameter generative reward model. RM-Bench ModelChat Math Code Safety Easy Normal Hard Overall JudgeBench Qwen3-Nemotron-235B-A22B-GenRM 76.5 96.9 81.4 94.4 94.090.5 77.487.387.4 Ours, 26B-A4B76.1 93.9 84.4 95.7 94.1 92.0 86.4 90.888.3 M.3 Reinforcement learning against the judge Each prompt samplesG= 16 completions. Comparing neighbors around a ring requires 16 judge calls instead of all 120 pairs. Every completion appears once in each position, cancelling position bias, and the forced ranking prevents zero-variance groups. Reasoning is hidden from the judge. The policy reward also includes script consistency, repetition control, and a stochastic reasoning-length term. Table 48: Selected gains from policy optimization against the judge, relative to Evon-RL-Indic. BenchmarkChange LiveCodeBench+13.2 MMLU-Pro+8.8 IndicMMLU-Pro+6.3 MMLU+5.8 GPQA Diamond+5.8 AGIEval+5.5 Grounded Indic instruction following+4.4 Mathematics aggregate+2.7 Evaluation and distillation use step 20 to limit drift. Step 20 is not the highest-scoring judge checkpoint, and the same regressions remain visible later, so this choice does not create them. M.4 Length control Three designs were tested. Rewarding the shortest trace drove reasoning to zero, while a two-sided ramp reduced it by 60% and cost more than 25 AIME points. The adopted one-sided penalty preserves judge quality without attracting the policy to an artificial length target. M.5 The 70:30 mixture that failed The recipe was retrained with 30% of prompts scored by binary mathematics and multiple-choice verifiers. For a problem solved with probabilityp, a group ofG= 16 is degenerate with probability z(p) =p 16 +(1− p) 16 . Dynamic sampling discards such all-pass and all-fail groups. The measured degenerate rate was high enough to trigger a second generation batch on 23 of 29 steps, making each step 2.27 times slower than the judge-only baseline. 58 Gnani-Evon-v3.3-30B-A3BTechnical Report All judge groups survived, while 35–49% of verifier groups were discarded. The specified 70:30 split therefore became roughly 80:20 at the gradient, and the run was stopped after thirty steps. About 40% of the verifier pool contained problems the model almost always solved or almost always missed. Restricting the pool to intermediate pass rates would reduce the discard probability by more than an order of magnitude. The original monitor missed this because it inspected only the judge environment, where groups cannot degenerate. M.6 Distillation configuration The teacher is Evon-RL-Indic, and the student starts from judge-training step 20. At each student token, supervision uses the teacher’s top-64 next-token distribution under Equation 8, withw= 0.5. The 29,988 mathematics and multiple-choice prompts cover English and eight Indian languages, with source-problem splits preventing translated leakage into validation. Training runs for 100 steps at a constant learning rate of 10 −5 ; only step 20 is evaluated. N Matched BF16 and FP8 Benchmark Results 59 Gnani-Evon-v3.3-30B-A3BTechnical Report Table 49: All 32 paired BF16 and FP8 accuracy results. Deltas are FP8 minus BF16. BenchmarkBF16 FP8∆ AGIEval English0.7347 0.7440+0.0093 AIME 20250.7438 0.7500+0.0062 AIME 20260.7458 0.7729+0.0271 ARC-Challenge0.9590 0.9582 −0.0008 BBQ0.8869 0.8819 −0.0050 Beyond-AIME0.5356 0.5337 −0.0019 BharatMath0.7317 0.7370+0.0053 BhashaBench-Multi0.4922 0.4916 −0.0006 GPQA-Diamond0.6503 0.6566+0.0063 GSM8K0.9431 0.9378 −0.0053 HMMT February0.6083 0.6438+0.0355 HMMT November0.6333 0.6354+0.0021 HumanEval+0.8854 0.8756 −0.0098 IFEval0.8281 0.7930 −0.0351 IFEval-Hindi0.5059 0.4965 −0.0094 IndicIFEval grounded 0.3305 0.3290 −0.0015 IndicIFEval translated 0.4987 0.4997+0.0010 IndicMMLU-Pro0.6888 0.6882 −0.0006 IndicParam0.3928 0.3939+0.0011 IndicQA0.1222 0.1320+0.0098 LiveCodeBench v60.5774 0.5793+0.0019 MATH0.8476 0.8451 −0.0025 MATH-5000.8400 0.8380 −0.0020 MBPP+0.7878 0.7757 −0.0121 MGSM-English0.9840 0.9680 −0.0160 MGSM-Indic0.8460 0.8520+0.0060 MILU0.7833 0.7825 −0.0008 MMLU0.8668 0.8719+0.0051 MMLU-Pro0.7726 0.7739+0.0013 XorQA-Indic0.3524 0.3528+0.0004 XQuAD-English0.6370 0.6378+0.0008 XQuAD-Indic0.1286 0.1321+0.0035 60 Gnani-Evon-v3.3-30B-A3BTechnical Report O Glossary BPE Byte-pair encoding, an iterative method that learns frequently occurring token merges. CPT Continual pretraining, further next-token training of an existing base model. CP Context parallelism, which partitions a long sequence across devices. EP Expert parallelism, which partitions MoE experts across devices. Fertility Average tokenizer tokens produced per whitespace-delimited word. Constituent-mean initialization Initialization of a new token row from the old-token pieces that exactly compose it. GQA Grouped-query attention, in which multiple query heads share fewer key/value heads. GRPOGroup relative policy optimization, which estimates relative advantages from several completions of the same prompt. In-script rate Fraction of outputs written in the target writing system. MoE Mixture of experts, a sparse layer that routes each token through a subset of expert networks. NoPE No explicit positional embedding. SFT Supervised fine-tuning on prompt/response or conversational demonstrations. TP Tensor parallelism, which partitions individual tensors and matrix operations across devices. WSD Warmup–Stable–Decay learning-rate schedule. 61 Gnani-Evon-v3.3-30B-A3BTechnical Report References [1]NVIDIA. NVIDIA Nemotron 3 Nano Technical Report. 2025.https://research.nvidia.com/la bs/nemotron/files/NVIDIA-Nemotron-3-Nano-Technical-Report.pdf [2] The Soofi-Team. A Sovereign, Open-Source Foundation Model for German and English: Soofi S Pretraining Report v1.0. arXiv:2607.09424, 2026. [3]Jimmy T. H. Smith, Tarek Dakhran, Alberto Cabrera, et al. In-Place Tokenizer Expansion for Pre-trained LLMs. arXiv:2607.15232, 2026. [4] A. Thakur et al. The Art of Breaking Words: Rethinking Multilingual Tokenization and Adaptive Data Mixtures. arXiv:2508.06533, 2025. [5]Jeffrey Li, Alex Fang, Georgios Smyrnis, et al. DataComp-LM: In Search of the Next Generation of Training Sets for Language Models. arXiv:2406.11794, 2024. [6]Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300, 2024. [7] Tri Dao and Albert Gu. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. arXiv:2405.21060, 2024. [8] NVIDIA Corporation. NVIDIA Model Optimizer. 2024–2026.https://github.com/NVIDIA/Mode l-Optimizer [9] Gnani.AI LLM Team. Gnani-Evon-v3.3-30B-A3B Model Card and Evaluation. 2026.https: //huggingface.co/gnani/gnani-evon-v3.3-30B-A3B [10]Mohd Arshad Khan et al. IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-tuning Datasets for Indian Languages. ACL 2024. https://aclanthology.org/2024.acl-long.843/ [11]Christopher Toukmaji and Jeffrey Flanigan. A Systematic Study of Low-Resource Language Adapta- tion for Large Language Models. GEM Workshop, 2025.https://aclanthology.org/2025.gem-1 .61/ [12]Ahmet Üstün et al. Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model. ACL 2024. https://aclanthology.org/2024.acl-long.845/ [13]Niklas Muennighoff et al. Crosslingual Generalization through Multitask Finetuning. ACL 2023. https://aclanthology.org/2023.acl-long.891/ [14]Pinzhen Chen et al. Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca? Findings of EACL 2024. https://aclanthology.org/2024.findings-eacl.90/ [15]Yihao Li, Jiayi Xin, Miranda Muqing Miao, Qi Long, and Lyle Ungar. The Impact of Language Mixing on Bilingual LLM Reasoning. EMNLP 2025, pages 32531–32548.https://aclanthology.o rg/2025.emnlp-main.1654/ [16]Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. On the Cross-lingual Transferability of Monolingual Representations. ACL 2020, pages 4623–4637.https://aclanthology.org/2020.ac l-main.421/ 62 Gnani-Evon-v3.3-30B-A3BTechnical Report [17]Tarun Jayakumar, Mohammed Safi Ur Rahman Khan, Raj Dabre, Ratish Puduppully, and Anoop Kunchukuttan. IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages. arXiv:2602.22125, 2026. [18] Harman Singh et al. IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages. ACL 2024. https://aclanthology.org/2024.acl-long.595/ [19]Freda Shi, Mirac Suzgun, Markus Freitag, et al. Language Models are Multilingual Chain-of-Thought Reasoners. ICLR 2023. https://arxiv.org/abs/2210.03057 [20] Dan Hendrycks, Collin Burns, Steven Basart, et al. Measuring Massive Multitask Language Under- standing. ICLR 2021. https://arxiv.org/abs/2009.03300 [21]Dan Hendrycks, Collin Burns, Saurav Kadavath, et al. Measuring Mathematical Problem Solving with the MATH Dataset. NeurIPS Datasets and Benchmarks, 2021.https://arxiv.org/abs/2103.03874 [22]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al. Training Verifiers to Solve Math Word Problems. arXiv:2110.14168, 2021. [23]David Rein, Betty Li Hou, Asa Cooper Stickland, et al. GPQA: A Graduate-Level Google-Proof Q&A Benchmark. arXiv:2311.12022, 2023. [24] Mark Chen et al. Evaluating Large Language Models Trained on Code. arXiv:2107.03374, 2021. [25] Naman Jain, King Han, Alex Gu, et al. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv:2403.07974, 2024. [26]Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, et al. Instruction-Following Evaluation for Large Language Models. arXiv:2311.07911, 2023. 63