Paper deep dive
Enemray: Toward Capable Language Models for Hassaniya
Cheikh Ahmed
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We introduce Enemray, a Hassaniya-centric language model that enables general-purpose interaction in Hassaniya. Enemray is trained around a stability--plasticity objective: acquire strong Hassaniya linguistic and cultural competence while preserving the general reasoning, multilingual, instruction-following, and safety behaviors of a capable instruction-tuned model. The development pipeline separates language acquisition from behavioral specialization. A separately assembled continual-pretraining corpus provides broad exposure to natural Hassaniya and Mauritanian text; layer-selective continual pretraining learns a compact language-specific parameter update; that update is transferred into the instruction-tuned parameter space; and supervised post-training develops conversational, cultural, literary, task-oriented, and cross-lingual behavior. The supervised corpus integrates selected public Hassaniya and Mauritanian resources with a substantially larger body of newly collected, reconstructed, curated, and constructed instruction data, while policy-generated replay provides a retention signal from the reference model's own behavior distribution. The resulting collection is substantially larger and broader in purpose than existing Hassaniya text resources. In evaluation, Enemray achieves the strongest English to Hassaniya translation among the compared open and proprietary models and the highest overall score on Mauritanian translation error detection, while retaining most of the general capabilities of its instruction-tuned base model on mathematical reasoning, knowledge, code generation, and function calling. This report describes the motivation, data construction, model design, training methodology, and evaluation of Enemray.
Tags
Links
- Source: https://arxiv.org/abs/2609.14829v1
- Canonical: https://arxiv.org/abs/2609.14829v1
Trouble viewing inline? Open PDF directly →
Full Text
58,804 characters extracted from source content.
Expand or collapse full text
Enemray Enemray: Toward Capable Language Models for Hassaniya Cheikh Ahmed Abstract We introduce Enemray, a Hassaniya-centric language model that enables general-purpose interaction in Hassaniya. Enemray is trained around a stability–plasticity objective: acquire strong Hassaniya linguis- tic and cultural competence while preserving the general reasoning, multilingual, instruction-following, and safety behaviors of a capable instruction-tuned model. The development pipeline separates lan- guage acquisition from behavioral specialization. A separately assembled continual-pretraining corpus provides broad exposure to natural Hassaniya and Mauritanian text; layer-selective continual pretrain- ing learns a compact language-specific parameter update; that update is transferred into the instruction- tuned parameter space; and supervised post-training develops conversational, cultural, literary, task- oriented, and cross-lingual behavior. The supervised corpus integrates selected public Hassaniya and Mauritanian resources with a substantially larger body of newly collected, reconstructed, curated, and constructed instruction data, while policy-generated replay provides a retention signal from the refer- ence model’s own behavior distribution. The resulting collection is substantially larger and broader in purpose than existing Hassaniya text resources. In evaluation, Enemray achieves the strongest English →Hassaniya translation among the compared open and proprietary models and the highest overall score on Mauritanian translation error detection, while retaining most of the general capabilities of its instruction-tuned base model on mathematical reasoning, knowledge, code generation, and function calling. This report describes the motivation, data construction, model design, training methodology, and evaluation of Enemray. Keywords:Hassaniya Arabic; low-resource language modeling; continual pretraining; language adaptation. spBLEU chrF++ spBLEU chrF++ 0 20 40 60 English→HassaniyaHassaniya→English 8.8 26.1 21.9 42.9 8.6 26.1 27.0 48.8 5.3 21.6 28.3 49.9 8.3 25.3 20.6 44.2 12.6 29.4 29.5 49.0 Gemma 4 E4B-it GPT-5.6 Luna Gemini 3.5 Flash Lite Qwen 3.8 Flash Enemray ی س و م علم الواحد د كا ؟ ر الا مں علم ال را هو س عو؟ ر ع عو، ی س و م علم الواحد د ك ، ح ں ر ر الا مں علم ال .ه ح اس رك و اط ص ا دو ع عود ال . ه رل ك الا علم دورو ، ع ص وەكس وكو هادال واحد لا، م ؟ٔدا اس ك راها كو م، لهاسحو دروس، ها الی اٮ ط ال ل ر ٔدا ال .ه ح ا ود ه، را ك هار كل و ص حو ، هاد لاصه ال م ی عط و؟ ط حا الا ماهو علم، الی هم و ع عرڡ وكوماە .وحدو ورح د ك راو ك حاولو ، ه رل ك الا ال م را ك طر عں همو و ع ورح د ك وڡ س و كا وع، الموص س مں ر حٓا ال م را ك ا د وعاك همو، و .و رٯ ال Figure 1.Left: English↔Hassaniya translation on the Alexandria (Mauritania) test set, compared with open and proprietary models. Right: a three-turn conversation with Enemray in Hassaniya (unedited model output). arXiv:2609.14829v1 [cs.CL] 13 Sep 2026 1.Introduction 1.1Why Hassaniya Hassaniya (Ḥassāniyya) is a western Arabic variety spoken across a contiguous region of the western Sahara and Sahel, with its largest speaker population in Mauritania, where it functions broadly as a lingua franca. Its historical development reflects sustained contact with Classical and Modern Standard Arabic, Berber languages, and, in modern usage, French and other regional languages [1,2]. These contact dynamics are visible in Hassaniya phonology, morphology, lexicon, register, and patterns of code-switching, making it a linguistically distinct modeling target [1]. Hassaniya also carries a substantial oral and literary tradition. UNESCO’s documentation of the Moorish epic T’heydinn describes a body of dozens of poems in Hassaniya that preserves collective memory, social values, and historical narratives [3]. This cultural role motivates a model spanning everyday conversation and translation alongside local forms of expression, heritage knowledge, and poetic structure. Despite this linguistic and cultural breadth, Hassaniya remains sparsely represented in computa- tional resources. Earlier large-scale Arabic dialect identification work excluded Mauritania because sufficient dialectal Twitter data could not be collected [4]. More recent resources have expanded coverage through sentiment data, English–Hassaniya parallel corpora, community datasets, culturally grounded Arabic instruction, multi-country dialectal translation, task-oriented dialogue, proverb col- lections, OCR text, and speech [5–13]. The resulting ecosystem is valuable, but it is distributed across distinct tasks and modalities rather than organized as a broad training distribution for a general-purpose Hassaniya language model. The central objective of Enemray is to make Hassaniya a first-class interaction language for a general- purpose language model: a language in which the model can reason, explain, translate, converse, write, and engage with culturally specific knowledge and expressive forms. 1.2The adaptation problem Specializing a strong multilingual model to an underrepresented language creates a stability–plasticity problem. The model must acquire new linguistic structure from a comparatively concentrated target- language corpus, while retaining capabilities learned from much broader pretraining and post-training distributions. Naive continued training can improve target-language modeling while moving param- eters away from behaviors that support reasoning, instruction following, multilinguality, coding, and safety. This challenge is particularly acute for assistants that must combine target-language compe- tence with broad general-purpose reasoning and interaction capabilities. Enemray addresses this problem by separatinglanguage acquisitionfrombehavioral specialization. First, a language-specific update is learned from a base model through layer-selective continual pretrain- ing (CPT). Second, this update is transferred into the corresponding instruction-tuned model through parameter-space composition. Third, supervised post-training, implemented as supervised fine-tuning (SFT), binds the acquired Hassaniya competence to interaction behaviors. General-capability replay is included within the third stage to maintain exposure to broader assistant behavior during supervised post-training. Enemray implements this decomposition using Gemma 4 E4B. The design follows the broader observation that language addition, parameter-efficient adaptation, and post-training retention are related but distinct optimization problems [ 14–16]. 1.3Contributions The technical contributions of this report are fourfold: •Large-scale Hassaniya data construction.We construct two complementary data streams: a separately assembled continual-pretraining corpus for broad language acquisition, and a Hassaniya supervised corpus that integrates selected public resources with a substantially larger body of Enemray-constructed instruction data. Together they cover naturally occurring language, Mau- ritanian journalism and cultural writing, poetry, translation, grammar, role-play, task-oriented dia- 2 I LEARN I TRANSFERIII SPECIALIZE Gemma Base Selective CPT ∆ H Language update Hassaniya text Gemma Instruct + ×0.7 Graft Instruction weights SFT Supervision + replay Enemray Figure 2.Enemray training pipeline. Selective continual pretraining learns a language update from natural Hassaniya text. The scaled update is added to the instruction-tuned checkpoint, which is then specialized using supervised examples and replay. Solid arrows indicate the main data or model flow; the dashed arrow denotes parameter transfer. logue, and cultural knowledge. The resulting collection supports a broader training objective than the individual task-specific resources summarized in this report. •Layer-selective language acquisition.We use LayRA-style continual pretraining to place the primary language-learning update in selected early and late layers, reducing the number of trainable parameters while keeping intermediate-layer weights fixed [ 14]. •Language capability transfer.We represent the acquired Hassaniya competence as a parameter update and compose it with the corresponding instruction-tuned model. This follows the task-vector view that fine-tuning-induced parameter differences can encode transferable behavior in weight space [ 17]. •Replay-regularized post-training.We combine Hassaniya-centric supervised learning with of- fline general-capability replay. The replay component is motivated by recent evidence that training on model-generated, policy-aligned trajectories can reduce forgetting during continual supervised fine-tuning [ 16]. The full development path is summarized in Figure2. Each stage answers a different requirement: ac- quiring the language, transferring that acquisition to an interactive model, shaping Hassaniya behavior, and stabilizing broad capabilities during specialization. 2.Model Design 2.1Architectural foundation Enemray adapts the Gemma 4 E4B base and instruction-tuned checkpoints. The backbone is a dense multimodal model with approximately 8 billion parameters including embeddings and approximately 4.5 billion effective parameters. The distinction reflects per-layer embedding tables rather than mixture- of-experts activation [ 18]. The text decoder contains 42 layers, with hidden dimension 2,560, feed- forward dimension 10,240, and a vocabulary of 262,144 entries. The decoder repeats a block of five sliding-attention layers followed by one full-attention layer seven times, and the configured context capacity is 131,072 tokens [19]. The complete multimodal model is loaded, while adaptation is restricted to attention and feed- forward projections in the text decoder. Vision and audio encoders, multimodal projectors, per-layer embedding components, token embeddings, and the language-model head remain frozen. 3 Table 1.Inherited Gemma 4 E4B configuration [18,19]. PropertySpecification Parameter scaleApproximately 8B total; 4.5B effective Text-decoder layers42 Hidden / feed-forward dimension2,560 / 10,240 Vocabulary262,144 entries Attention heads / KV heads8 / 2 Attention patternSeven blocks of five sliding and one full-attention layer Attention-layer totals35 sliding and 7 full-attention layers Sliding window512 tokens Architectural context131,072 tokens Per-layer embedding contribution256 dimensions KV-state sharingFinal 18 text layers Inherited input modalitiesText, image, audio Adaptation modalityText only 2.2Layer-selective adaptation Following LayRA, which concentrates language adaptation in the early and late layers of a transformer while leaving its intermediate representations unchanged [14], Enemray restricts continual pretraining to the first ten and the final two layers of the text decoder: L=0,1,...,9∪40,41.(1) Continual pretraining therefore updates 12 of the 42 decoder layers, and the remaining 30 intermediate layers retain their pretrained weights. Within each selected layer, low-rank adapters are attached to the query, key, value, and output projections of the attention block and to the up- and down-projections of the feed-forward block, while the feed-forward gate projection is left frozen. All adapters use rank r= 16and scaling parameterα= 32[ 20]. Continual pretraining thus adapts 72 weight matrices in total. During supervised post-training, adaptation is extended to every layer of the decoder and to all attention and feed-forward projections, including the gate projection, yielding 294 adapted weight matrices. 2.3Language capability transfer Letθ base denote the base checkpoint,θ post its instruction-tuned counterpart, andθ base,H the base check- point after Hassaniya continual pretraining. For each adapted weight matrixW, continual pretraining learns a low-rank update ∆W H = α r BA,(2) whereAandBare the learned low-rank factors. Because all other parameters remain unchanged, the language update of the full model is ∆ H =θ base,H −θ base ,(3) which equals∆W H for each adapted matrix and zero elsewhere. This update is then added to the instruction-tuned checkpoint: θ H,post =θ post +γ∆ H ,γ= 0.7.(4) We refer to the resulting modelθ H,post as the grafted checkpoint. This formulation follows task arithmetic, which interprets differences between fine-tuned and pre- trained weights as reusable directions in parameter space [17], and is closely related to chat-vector 4 Table 2.Publicly documented Hassaniya or Mauritanian-Arabic resources identified during corpus development. Corpus-wide sizes are shown when a source does not publish a Hassaniya-only count. “SFT” indicates incorpo- ration into Enemray supervised post-training only. ResourceTypePublicly reported scaleEnemray use HASSANIYA [5]Sentiment2,000 labeled Facebook comments– HASSANIYA-DTCD [22]Classification1,851 labeled text records– SMART-9K-H-CORP [6]Parallel text9,000 English–Hassaniya sentence pairs– DAH [7]Parallel text3,002 English–Hassaniya rows in Arabic script and Latin transliteration SFT PALM [8]Cultural instruction1,298 Mauritania instructions, including 294 dialect-category instructions SFT Alexandria [9]Multi-turn translation / dialogue 107K samples overall across 13 Arab countries and 11 domains, including Mauritania SFT AI-for-RIM [10]Translation / dialogue About 4,430 translation pairs and 594 support turns SFT dialogue Amthal Hassaniya [11]Proverbs319 Hassaniya proverb examples with explanations SFT Hassaniya Stories OCR [ 12] OCR / cultural text552 image–text rows– Casablanca [13]SpeechAbout 48 hours overall across eight Arabic dialects, including Hassaniya – Hassaniya Speech Dataset [23] Speech294 audio–text rows– Peace Corps Hassaniya material [24] Language learningHassaniya–English instructional text and phrases – Basic English–Hassaniya Dictionary [ 25] Lexical referenceMore than 8,000 words, phrases, and sentences – Hassaniya scripture resources [ 26,27] Parallel textVerse-aligned Hassaniya scripture; edition-dependent SFT transfer, in which weight differences between base and instruction-tuned models are combined across checkpoints that share a common origin [21]. The transfer assumes thatθ post is derived fromθ base and shares its parameterization, so that an update learned relative to the base model remains meaningful in the neighborhood of the instruction-tuned weights. The coefficientγcontrols the strength of the transferred update; we useγ= 0.7as a fixed setting and leave its systematic selection to future work. Subsequent supervised training specializes the grafted checkpoint for Hassaniya interaction. 3.Training Data Data is the central constraint in Hassaniya language modeling. Public resources have expanded in recent years, but they remain distributed across narrow tasks such as sentiment classification, transla- tion, cultural instruction, dialogue, speech, proverbs, and language learning. Enemray therefore treats the public ecosystem as a starting point rather than as a complete language-model training distri- bution. The resulting data pipeline separates two objectives: broad language acquisition from inde- pendently assembled natural text during continual pretraining, and behavioral specialization from a larger Hassaniya supervised mixture that incorporates selected public resources together with addi- tional Enemray-constructed and curated data. 3.1Existing Hassaniya data landscape Table2summarizes publicly documented resources containing Hassaniya or explicitly labeled Maurita- nian Arabic that we identified during corpus development. The resources are not directly comparable— they differ in modality, task, annotation, and unit of measurement—but together they show both the growing value and the fragmentation of the available ecosystem. The final column indicates which public resources are incorporated into Enemray; all of them are used in supervised post-training. 5 The ecosystem is useful but task-specific. SMART-9K-H-CORP provides substantial translation su- pervision, Casablanca contributes speech coverage, and newer multidialectal efforts such as PALM and Alexandria include Mauritania within broader Arabic collections [6,8,9,13]. None of these resources alone supplies the broad natural-text and general-purpose conversational distribution needed for a Has- saniya generative model. Enemray therefore integrates selected public supervision while constructing a substantially larger training distribution around natural Hassaniya text and general assistant behavior. 3.2Building the Enemray training corpus The Enemray training data comprise two complementary streams that serve distinct objectives. The first is a continual-pretraining corpus assembled from naturally occurring and long-form Hassaniya text, including social narratives, reconstructed long-form narratives, Mauritanian news and articles, cultural essays, poetry, and language-learning material. Its purpose is to expose the model to the dis- tribution of the language itself. The second is a supervised corpus designed to shape assistant behavior. It integrates selected public resources from Table2with a larger body of Enemray-constructed data spanning cultural knowledge, dialogue, translation, linguistic refinement, practical interaction, and Hassaniya literary forms. Table3summarizes the size of each stream after cleaning. Because the two streams are organized around different training units, the continual-pretraining corpus is measured in documents and the supervised corpus in conversations, and both are additionally reported in tokens of the Gemma 4 to- kenizer. The supervised stage also includes a small general-capability replay set. Although it is not Hassaniya data, it is listed here to present the complete supervised mixture, and its role is discussed with the training methodology in Section4. Table 3.Size of the Enemray training data after cleaning. Token counts are computed with the Gemma 4 tokenizer over message content, excluding system prompts and conversational formatting. ComponentSize Gemma 4 tokens Continual-pretraining corpus11,629 documents20.9M Hassaniya supervised data46,259 conversations3.0M General-capability replay273 conversations0.4M Complete supervised mixture46,532 conversations3.4M The supervised mixture contains 46,532 conversations with 50,273 assistant turns. Since each as- sistant turn is treated as a separate training example, the number of assistant turns corresponds to the number of supervised examples used during post-training, whereas the number of conversations reflects the size of the underlying dialogue collection. Within this design, public datasets constitute one component of the supervised stage rather than its foundation: data constructed for Enemray account for roughly two thirds of the supervised response tokens, while public resources contribute the remaining third. In scale, the continual-pretraining corpus alone exceeds the individual public text resources in Table 2, which are typically measured in thousands of sentence pairs or instructions, by more than an order of magnitude. 3.3Data construction recipes Beyond integrating public resources, Enemray relies on several explicit construction recipes designed around the forms in which Hassaniya is naturally available. These recipes preserve the distinction be- tween naturally occurring text, machine-assisted derived supervision, and deliberately constructed nor- malization data. Figure3summarizes the main recipe families and how they feed either the continual- pretraining stream or the Hassaniya supervised mixture. Natural web text for continual pretraining.The continual-pretraining stream is built primarily through web collection from many Mauritanian online sources. We collect naturally occurring Has- 6 saniya posts, social narratives, stories, articles, news, and related text, then organize the retained mate- rial as document-level causal-language-modeling examples. This stream is intentionally kept separate from the supervised public datasets in Table2: its role is exposure to the language distribution itself rather than instruction following. Social posts and narratives form the largest component because they provide substantial naturally occurring colloquial Hassaniya, while journalism, long-form articles, cul- tural writing, and literary material broaden register coverage. Context-guided pseudo-parallel translation.Web posts and stories collected for continual pre- training are also used to build supervised translation data. Each document is split into short and medium Hassaniya segments, which become the source side of the translation pairs. A short segment taken on its own is often ambiguous: it may refer to people or events mentioned earlier, omit information that is clear from the surrounding text, or use a local expression whose meaning depends on the situation. For this reason, Gemini [28] does not translate segments in isolation. It receives the full source doc- ument together with the selected segment, and produces an English rendering of that segment only, using the document to resolve the narrative situation, referents, and dialectal expressions. Because the English side is produced by a model, the resulting pairs are treated asmachine-generated pseudo- translationsrather than human-authored gold translations. Before inclusion, each rendering is checked by a human against the Hassaniya segment and its context. Context-guided conversation construction.User–assistant conversations are built from the same natural Hassaniya documents. Generating dialogue from an isolated sentence tends to produce generic exchanges that are only loosely related to the source. Instead, Gemini receives the source document and writes conversational turns grounded in its content, so that the exchange reflects the situations, people, and topics that appear in the original text. As with the translation data, each candidate conversation is verified by a human before it enters the supervised mixture. A single source document can therefore contribute both pseudo-parallel translation pairs and contextual assistant conversations, while both types of supervision remain tied to naturally occurring Hassaniya. Correction and Hassaniya normalization.Correction data is constructed in the reverse direction: a clean Hassaniya sentence is kept as the assistant target, while a controlled noisy version is presented as the user input. The corruption pipeline includes rule-based character, spacing, punctuation, spelling, and local word-order noise; machine-translation (MT) and back-translation noise that can introduce translationese or less-natural phrasing; and conservative substitution of selected Hassaniya forms with Modern Standard Arabic (MSA) or Moroccan Darija alternatives. The goal is broader than spelling repair: the model is trained to normalize noisy, mixed, or model-like text back into natural Hassaniya while preserving meaning. Generated correction examples are built only from the training split of the clean translation targets; held-out translation examples and Alexandria translation evaluation data are excluded from this augmentation. Poetry and literary curation.Because poetry and oral expression are central to Hassaniya cultural and linguistic coverage, we also select substantial Hassaniya poetic and literary material from additional sources rather than relying only on generic Arabic corpora. Raw literary text contributes language ex- posure where appropriate, while structured examples are used to teach supervised literary and cultural behavior, including Hassaniya poetic forms such as Aagel, Lghna, and Tebraa. This provides lexical, stylistic, figurative, and culturally grounded patterns that are comparatively sparse in ordinary web prose. 3.4Continual-pretraining corpus The continual-pretraining corpus exposes the base model to natural text before assistant specialization. The clean corpus contains 11,629 documents totaling approximately 20.9 million Gemma 4 tokens (9.5 million words). Documents are typically long-form, with a median length of 581 tokens. Its source domains are summarized in Table 4. 7 SOURCESPREPARATION TRAINING DATA Natural Hassaniya Web / news / poetry Collect & clean CPT corpus Public resources Selected Hassaniya datasets Construct & curate SFT corpus Derived supervision Translation / dialogue / correction Replay Figure3.Two complementary training-data streams. Natural text supplies CPT and material for derived supervi- sion. Public resources join constructed examples through curation; replay enters the final SFT mixture separately. Dashed arrows identify derived supervision and replay inputs. Construction includes context-guided generation, human verification, normalization, and literary curation, as described in the text. Table 4.Source domains of the continual-pretraining corpus. Source domainContribution Standalone social-media narrativesColloquial writing and everyday discourse Reconstructed long-form narrativesExtended narrative and contextual continuity Mauritanian news mediaContemporary events and expository language Mauritanian long-form articlesLonger factual and topical discussion Cultural essaysLocal knowledge and cultural expression Poetry and language-learning textsLiterary forms and linguistic explanation The mixture is intentionally register-diverse. Social narratives provide everyday dialectal discourse, dialogue, interpersonal relations, narrative progression, colloquial lexical choice, and naturally varying orthography. Journalism and long-form articles extend coverage toward factual, expository, and more formal registers. Cultural essays and language-learning material add local knowledge and pedagogical structure, while poetry introduces expressive forms that are culturally salient in Hassaniya and difficult to recover from general Arabic corpora alone [ 3]. 3.5Hassaniya supervised corpus The Hassaniya supervised data comprise 46,259 conversations and 49,964 assistant turns, with approx- imately 3.0 million tokens of user and assistant content, of which 1.5 million tokens form the assistant responses used as training targets. Selected public datasets contribute high-value forms of supervision that are difficult to recover reliably from raw text alone. DAH contributes English–Hassaniya parallel examples; the Mauritanian portion of Alexandria adds multi-domain conversational translation; PALM contributes country-specific cultural and dialectal instructions; selected AI-for-RIM customer-support dialogues add practical multi-turn interaction; Amthal Hassaniya contributes proverb explanation and idiomatic grounding; and Hassaniya scripture parallels add further cross-lingual lexical and syntactic coverage [ 7–11,26,27]. Beyond these public sources, Enemray data extend supervision across five capability families:cul- tural and contextual grounding, including Mauritanian heritage, history, local knowledge, and ques- tion answering;poetry and oral tradition, including Aagel, Lghna, and Tebraa;situational conver- sation, including role-play and everyday social scenarios;cross-lingual communication, including English–Hassaniya dialogue and translation; andlinguistic refinement, including grammar, spelling, phrasing, lexical choice, correction, and stylistic naturalness. 8 3.6Curation, separation, and supervision The data pipeline is organized around the objective of each training stage. Natural text is used for continual pretraining, where it is presented as documents under a causal language-modeling objec- tive, whereas instruction, translation, proverb, dialogue, and parallel-text resources are reserved for supervised post-training. This separation distinguishes learning the distribution of the language from learning how to respond within an interaction. In the supervised stage, conversations are represented in the native conversational format of the model, and the training signal is restricted to assistant responses. Each assistant turn forms its own training example, with the preceding dialogue serving as context, so that the model learns to produce responses rather than to reproduce prompts. The general-capability replay set is kept separate from the Hassaniya data and is added only to the final supervised mixture, where it serves as a retention signal (Section 4). Cleaning is deliberately conservative. For the continual-pretraining corpus, it removes invisible for- matting characters while preserving every document, including dialectal spelling and informal punctu- ation. For the supervised corpus, it removes exact duplicate conversations and residual markup from generated responses. These steps ensure structural consistency across the data. Because supervision is built from the same body of Hassaniya literature, some poems appear both as documents in the continual-pretraining corpus and as responses in the supervised poetry data. The model therefore encounters these texts first as natural language and subsequently as targets of literary generation. 4.Training Methodology Enemray is trained in three stages that separate language acquisition, language transfer, and supervised post-training (Figure2). The configuration of each training stage is summarized in Table5. 4.1Stage I: Continual pretraining The base checkpoint is adapted to Hassaniya under the causal language-modeling objective, with low- rank adapters restricted to the layers and projections described in Section 2. Documents from the continual-pretraining corpus are separated by an end-of-sequence token and packed into sequences of 2,048 tokens. Training runs for two epochs, during which the model processes approximately 42 million tokens. Optimization uses 8-bit AdamW with a learning rate of5×10 −5 , linear decay after a short warmup, and an effective batch size of 16 sequences. 4.2Stage I: Language transfer The trained language adapter is transferred to the instruction-tuned checkpoint by scaling its update withγ= 0.7and merging it into the model weights, as defined in Equation 4. The merge is performed in full precision. On a reference input, the merged model reproduces the logits of the unmerged adapted model to within a maximum absolute difference of 1 . 7 × 10 −4 , confirming that merging preserves the behavior of the adapted model. 4.3Stage I: Supervised post-training The grafted checkpoint is then trained on the supervised mixture described in Section3.5, with adap- tation extended to all attention and feed-forward projections across the full depth of the decoder (Sec- tion 2). Each assistant turn forms a separate training example, in which the assistant response is the target and the preceding dialogue serves as context. LetT(x)denote the target token positions of an examplex. The training objective is L SFT (θ) =− 1 ∑ x |T(x)| ∑ x ∑ t∈T(x) logp θ (x t |x <t ),(5) so that prompt and padding tokens do not contribute to the loss. 9 Table 5.Training configuration of the continual-pretraining and supervised post-training stages. SettingContinual pretrainingSupervised post-training Maximum sequence length2,0488,192 Effective batch size1616 Gradient accumulation11 Epochs22 Learning rate5×10 −5 5×10 −5 Learning-rate scheduleLinearLinear Warmup fraction0.030.03 Weight decay0.010.01 Gradient clipping 1.0 1.0 Optimizer8-bit AdamWAdamW LoRA rank / scaling parameter16 / 3216 / 32 LoRA dropout00 Adapted weight matrices72 in 12 layers294 in 42 layers LossCausal language modelingAssistant responses Random seed4242 Conversations are rendered in the native Gemma 4 conversational format with the reasoning mode disabled, and sequences are limited to 8,192 tokens. When a dialogue exceeds this length, the earliest context is removed so that the target response is always retained in full. Training runs for two epochs with AdamW at a learning rate of5×10 −5 and an effective batch size of 16, with examples sampled in random order. The supervised mixture also contains the general-capability replay set, which covers reasoning, cod- ing, multilingual assistance, and instruction following. Replay accounts for approximately 17% of the supervised target tokens and maintains exposure to general assistant behavior while the model spe- cializes in Hassaniya, following evidence that replay of model-generated responses reduces forgetting during continual fine-tuning [ 16]. 4.4Training setup All stages are trained on a single NVIDIA H100 80 GB GPU. Model computation uses BF16 precision, while the adapter merge in Stage I is performed in FP32 before the grafted model is stored in BF16. The complete multimodal checkpoint is loaded throughout, and the native Gemma 4 processor and conversational format are retained [ 29]. 5.Evaluation We evaluate Enemray along three axes: translation between English and Hassaniya, detection and classification of errors in Hassaniya machine translation, and retention of general capabilities after adaptation. All metrics are computed automatically against fixed references or executable tests, without human or model-based judges. Table 6summarizes the experimental setup. 10 Table7.Translation results on the Alexandria (Mauritania) test set, 1,000 examples per direction. Amalaz denotes the direction-specific MarianMT models amalaz-en2ha-micro and amalaz-ha2en-micro. The best score in each row is shown in bold. Metric Amalaz Gemma 4 GPT-5.6 Gemini 3.5 Qwen 3.8 Enemray micro E4B-it Luna Flash Lite Flash English→Hassaniya spBLEU 5.748.838.615.328.3012.64 chrF++ 21.9626.0926.1421.5925.3429.42 Hassaniya→English spBLEU 13.2221.9426.9528.3120.6229.52 chrF++ 32.7642.9148.7649.9344.1848.99 Table 6.Evaluation setup. All samples are drawn once with a fixed random seed and shared across models. ExperimentDataSamples Metric TranslationAlexandria (Mauritania), English→Hassaniya and Hassaniya→English 2,000 spBLEU, chrF++ Error detectionAlexandriaX Subtask 3 (Mauritania), LQM-annotated error spans 274 Exact match F1, overlap F1, class F1 Math reasoningGSM8K [30]500 Exact match KnowledgeMMLU-Pro [31]500 Accuracy Code generation HumanEval [ 32] 164 pass@1 Function callingBFCL [33]500 Call accuracy Models and decoding.Enemray is compared with its underlying instruction-tuned model, Gemma 4 E4B-it, so that differences reflect the adaptation pipeline. For translation and error detection, we additionally include three proprietary and open models accessed through OpenRouter: GPT-5.6 Luna, Gemini 3.5 Flash Lite, and Qwen 3.8 Flash. For translation, we also report Amalaz, a pair of small direction-specific MarianMT models for English→Hassaniya and Hassaniya→English, which trans- late the source sentence directly with their default beam-search decoding. Gemma 4 E4B-it and En- emray are decoded greedily with the reasoning mode disabled. The external models are queried with a fixed seed, a low reasoning effort, and the provider’s default sampling configuration. Within each experiment, every model receives the same prompt. 5.1Translation Translation is evaluated on the Mauritanian portion of Alexandria [9], a human-translated and reviewed collection of multi-domain dialogues. From its test split, we sample 1,000 parallel dialogue turns cover- ing all 11 domains and evaluate each turn in both directions, yielding 1,000 English→Hassaniya and 1,000 Hassaniya→English examples. Each model receives a neutral translation instruction specifying the source and target languages and asking for the translation only. Outputs are scored against the reference translation with spBLEU, computed with the FLORES-200 SentencePiece tokenizer [ 34], and chrF++ [35], both implemented in sacreBLEU [36]. Results.Table7and Figure1report the translation results. Enemray achieves the highest English→ Hassaniya scores on both metrics, improving over Gemma 4 E4B-it by 3.8 spBLEU and 3.3 chrF++. In the Hassaniya→English direction, Enemray obtains the highest spBLEU, while Gemini 3.5 Flash Lite achieves the highest chrF++. 5.2Error detection The error-detection experiment measures whether a model can recognize what is wrong in a Hassaniya translation, rather than produce or repair one. It follows Subtask 3 of the AlexandriaX-2026 shared task 11 Gemma 4 E4B-it GPT-5.6 Luna Gemini 3.5 Flash Lite Qwen 3.8 Flash Enemray 0 5 10 15 20 8.01 16.18 9.61 13.82 17.04 Overall score Figure 4.Overall error-detection score on the Mauritanian portion of AlexandriaX Subtask 3, the average of overlap F1 and class F1 (multiplied by 100). Table 8.Error-detection results on the Mauritanian portion of AlexandriaX Subtask 3 (274 examples, 390 gold error spans). Scores are multiplied by 100, and the best score in each metric row is shown in bold. The lower rows describe prediction behavior; in the gold annotation, erroneous spans cover 19.5% of each translation on average. Gemma 4 GPT-5.6 Gemini 3.5 Qwen 3.8 Enemray E4B-it Luna Flash Lite Flash Overall8.0116.189.6113.8217.04 Exact match F1 3.39 9.97 6.37 10.03 2.79 Overlap F112.2122.1313.5520.3927.58 Class F13.8110.235.667.246.51 Examples reported as error-free227601747919 Translation marked as erroneous (%)6.923.18.216.246.9 [37], in which a system receives an English sentence and its machine translation into an Arabic dialect, identifies the erroneous spans in the translation, and assigns each span an error category. The under- lying data were built with the LQM framework [38]: English sentences derived from conversational di- alect transcripts were translated by six Arabic-aware large language models, and native-speaker expert annotators marked the erroneous spans and labeled them with one of six linguistically grounded cat- egories: sociolinguistics, pragmatics, semantics, morphosyntax, orthography and writing conventions, and graphetics. We use all 274 English → Mauritanian Hassaniya examples with released annotations, comprising the 244 training and 30 development examples of the shared task, which together contain 390 annotated error spans. Each model receives the English source, the translation, and the list of the six categories with their LQM definitions, and returns the erroneous spans with their categories in JSON format. Character offsets are obtained by locating each returned span in the translation. Predictions are scored with the official shared-task scorer, which reports exact match F1, where a predicted span must match the gold character offsets exactly; overlap F1, computed over the characters marked as erroneous; and class F1, where predicted and gold spans are paired by their overlap and a pair counts as correct when the categories agree. Following the shared task, the overall score is the average of overlap F1 and class F1. We report results on all 274 examples. Results.Table8and Figure4report the error-detection results. The task remains difficult for all mod- els. Adaptation markedly changes the behavior of the base model: Gemma 4 E4B-it reports no error for 227 of the 274 translations, all of which contain at least one annotated error, whereas Enemray reports errors for all but 19 and obtains the highest overall and overlap scores. Its spans are, however, con- 12 GSM8K (accuracy) MMLU-Pro (accuracy) HumanEval (pass@1) BFCL (call accuracy) 0 20 40 60 80 100 93.6 68.8 88.4 92.0 89.4 62.8 74.4 91.2 Score (%) Gemma 4 E4B-it Enemray Figure 5.Capability retention of Enemray compared with Gemma 4 E4B-it on GSM8K, MMLU-Pro, HumanEval, and BFCL. siderably broader than the gold annotations, covering 46.9% of each translation on average compared with 19.5%, which yields the lowest exact match F1. Category assignment is the main weakness across models: although sociolinguistic errors account for 81% of the gold spans, the models predominantly label errors as semantic or morphosyntactic. 5.3Capability retention Capability retention compares Enemray with Gemma 4 E4B-it on four established benchmarks, using the same prompts, no system prompt, and greedy decoding for both models. Mathematical reasoning.We sample 500 problems from the GSM8K test set [30]. The model rea- sons step by step and states its final answer in a fixed format, and a response is correct when the extracted number equals the reference answer. Knowledge and reasoning.We sample 500 questions from the MMLU-Pro test set [31], stratified by subject category. Each question has up to ten answer options; the model reasons step by step and concludes with the selected letter, which is extracted following the official MMLU-Pro evaluation. We report accuracy. Code generation.All 164 HumanEval problems [32] are used. The model completes each function from its signature and docstring, and the completed program is executed against the official unit tests. We report pass@1. Function calling.We sample 500 single-turn problems from the Berkeley Function Calling Leader- board [33]: 200 simple, 100 multiple-function, 100 parallel, and 100 irrelevance cases. Tool definitions are provided through the native Gemma 4 function-calling format, and the generated calls are verified with abstract-syntax-tree matching following the official BFCL criteria, which check the function name, required parameters, and parameter values. In the irrelevance category, a response is correct when the model makes no function call. Results.Figure5reports the capability retention results. Enemray retains most of the general capa- bilities of Gemma 4 E4B-it, with some reduction after adaptation. Function calling is largely unchanged, with BFCL accuracy of 91.2% compared with 92.0%. On GSM8K and MMLU-Pro, Enemray reaches 89.4% and 62.8%, retaining 95.5% and 91.3% of the base model’s accuracy. The largest reduction appears in code generation, where HumanEval pass@1 decreases from 88.4% to 74.4%. Enemray’s solutions tend 13 Gemma 4 E4B-it Enemray Enemray (reasoning) 0 20 40 60 80 100 88.4 74.4 87.8 HumanEval pass@1 (%) Figure 6.HumanEval pass@1 of Enemray with and without its reasoning mode, compared with Gemma 4 E4B-it. Without reasoning, both models are decoded greedily; with reasoning, Enemray is sampled at temperature 0.8 with up to 8,192 new tokens. to be shorter and to implement the function directly, whereas the base model more often works through the problem in code comments before implementing it. This suggests that much of the gap reflects how the model approaches the problem rather than a loss of coding knowledge. Consistent with this, when Enemray is evaluated with its reasoning mode enabled (sampling temperature 0.8, up to 8,192 new tokens), its HumanEval pass@1 rises to 87.8%, nearly matching the 88.4% of the base model without reasoning (Figure 6). Although replay accounts for approximately 17% of the supervised target tokens, it consists of only 273 conversations, and coding tasks form a small subset of them. Increasing the number and diversity of policy-generated replay conversations, with broader coverage of coding and reasoning tasks, is a direct way to further improve retention in future versions of Enemray. 6.Limitations and Release 6.1Limitations Enemray is trained on a collection that emphasizes natural narratives, social writing, Mauritanian jour- nalism, cultural material, and targeted assistant tasks. Although this collection spans a broad range of registers, coverage of specialized domains such as scientific, legal, and professional writing remains lim- ited, and competence in these domains is correspondingly weaker. Hassaniya is also primarily an oral variety with considerable variation in spelling, code-switching, and proximity to Standard Arabic [ 1]. Model outputs may therefore alternate between Hassaniya and Standard Arabic forms, and judgments of dialectal naturalness, cultural appropriateness, and poetic form ultimately require native-speaker expertise. The present adaptation is restricted to text. Although the underlying Gemma 4 E4B checkpoint supports image and audio input, the vision and audio components are not adapted, and the multimodal behavior of Enemray is outside the scope of this report. 6.2Release of the current checkpoint The Enemray checkpoint described in this report will not be publicly released. The base Gemma 4 E4B model is distributed under the Apache 2.0 license [ 18] and does not itself restrict the release of derived models. The constraint arises instead from the supervised training data. Several third-party resources incorporated into the supervised mixture are distributed under terms that limit their use and redistri- bution. Alexandria is released under a Creative Commons Attribution–NonCommercial license, with access conditioned on an agreement to non-commercial use [ 9], and PALM is likewise distributed un- der non-commercial terms [8]. Other resources in the mixture do not state explicit licensing terms, and their use in a publicly distributed model would require permission from the respective rights holders. Because model weights trained on these resources cannot be released without such permissions, the current checkpoint is not made publicly available. 14 6.3Toward a releasable Enemray We are actively working toward a version of Enemray that can be released openly. This effort follows two directions. First, resources whose licensing terms require permission for redistribution or deriva- tive use are being removed from the training mixture. Second, we are developing and curating locally sourced Hassaniya data to replace these resources and to further extend coverage of Hassaniya lan- guage, culture, and interaction. The resulting corpus will be used to train a more capable Enemray model, which we intend to release publicly. Beyond changes to the data, the next version will extend the training pipeline beyond supervised fine-tuning. We plan to incorporate reinforcement learning from human feedback, with preferences provided by native Hassaniya speakers, to better align the model’s responses with the expectations of Hassaniya users in terms of naturalness, cultural appropriateness, and helpfulness. We also plan to simulate native Hassaniya-speaking users, so that the model can be trained and assessed on realistic multi-turn interactions that reflect how Hassaniya speakers actually phrase requests and conduct con- versations. Finally, the next version is intended to extend Enemray beyond text by adapting both its audio and image capabilities, allowing the model to understand spoken Hassaniya and to engage with visual content in Hassaniya. 7.Additional Examples Table9presents additional examples of Enemray outputs covering everyday advice, English→Has- saniya translation, and poetry generation. Table 9.Additional examples of Enemray outputs. Outputs are unedited and were generated with greedy decod- ing and the reasoning mode disabled. TaskInputEnemray output Everyday advice System: general ر ع ، هلكرا ی و مں ی س ص ح عود دور .كمعل السلام ه؟ ی صح الی هو س . راں عود لوال واحل كامل هار ال كل ا و حد .طس امح ر ٔدا اصح .السلام كموعل . هلكرا وحدە ساعه ه ص وح العصر، صلاە عاك ك هار، Translation English → Hassaniya I waited for you near the market, but my phone ran out of battery. If you arrive before me tomorrow, wait at the same place. ح عود ا ںل .ه طار اٮ او ی و ل ر ع ، المرصه د ع حا . لاصهل س ڡ ی حا ح،الص ی كسا Translation English → Hassaniya Learning a little every day is often more useful than studying for many hours once a month. The important thing is to keep practicing, even when progress feels slow. اسرە لساعاٮ الدراسه مں دە ٔ ا ر كٔا عود ال ع هار كل وى س علم دم ال عود ںل ی ح ، ںمر ال مرار الاس هو المهم .هر س كل مرە .ء طی Poetry generation System: poetry هلؤا عں دع واحد عں ں لوا عرٔا مں دح اڡ ك ی اعط اڡ الك ی وعط ، وک الس ه كوں .لال هم عد ك د ك و .رح س لا وحدو وحدى هر الع ںٓا و == طول الال لال هلیٔا د ك ك ما == طولٔا و طولٔا لال و 8.Conclusion This report presented Enemray, a Hassaniya-centric language model built on Gemma 4 E4B. Enemray approaches the adaptation of a capable instruction-tuned model to an underrepresented language as a stability–plasticity problem and addresses it by separating language acquisition from behavioral special- ization. Layer-selective continual pretraining learns a compact Hassaniya update from natural text, this update is transferred into the instruction-tuned model through parameter-space composition, and su- pervised post-training with general-capability replay shapes Hassaniya interaction while maintaining broader assistant behavior. Supporting this pipeline, we assembled a continual-pretraining corpus of natural and long-form Hassaniya text and a supervised corpus that combines selected public resources with a larger body of Enemray-constructed data covering cultural knowledge, poetry, conversation, translation, and linguistic refinement. The evaluation indicates that this design yields clear gains in Hassaniya while largely preserving 15 general capabilities. Enemray achieves the strongest English→Hassaniya translation among the eval- uated models, including the proprietary systems, and the highest spBLEU in the Hassaniya→English direction. It also obtains the highest overall score on Mauritanian error detection, although precise span localization and error categorization remain difficult for all models. On general benchmarks, En- emray retains most of the capabilities of its instruction-tuned base model: function calling is largely unchanged, mathematical reasoning and knowledge show modest reductions, and the reduction in code generation is almost entirely recovered when the reasoning mode is enabled. Several directions remain open. The next version of Enemray will replace resources with restrictive licensing terms by locally sourced Hassaniya data so that the model can be released openly. It will also increase the amount and coverage of policy-generated replay, incorporate reinforcement learning from human feedback with native Hassaniya speakers and simulated Hassaniya-speaking users, and extend adaptation to audio and images. We hope that Enemray and the methodology documented in this report contribute to making Hassaniya a first-class language for modern language technology. References [1]Catherine Taine-Cheikh. Ḥassāniyya arabic. In Christopher Lucas and Stefano Manfredi, editors, Arabic and Contact-Induced Change, pages 245–263. Language Science Press, Berlin, 2020. doi: 10.5281/zenodo.3744523. URLhttps://langsci-press.org/catalog/book/235. [2]Harald Hammarström, Robert Forkel, Martin Haspelmath, and Sebastian Bank. Glottolog 5.3: Hassaniyya. Glottolog, 2025. URL https://glottolog.org/resource/languoid/id/hass1 238. [3]UNESCO. Moorish epic t’heydinn. Intangible Cultural Heritage, n.d. URLhttps://ich.unes co.org/en/USL/moorish-epic-t-heydinn-00524. [4]Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. Arabic dialect identification in the wild.arXiv preprint arXiv:2005.06557, 2020. URLhttps://arxiv.or g/abs/2005.06557. [5]Med El Moustapha El Arby. Hassaniya dataset. Mendeley Data, 2025. URLhttps://data.men deley.com/datasets/m2swkr2bhx/1. [6]Rahma Boughale, Aichetou Nemin, Mamadou Abdoulaye Aw, Mohamedou Abewa, and Cheikh Seyidi Dedaha Mohamed Jedou. A low-resource english–hassaniya parallel corpus with neural machine translation benchmarks. InArtificial Intelligence and its Practical Applications, volume 1916 ofLecture Notes in Networks and Systems, pages 187–200. Springer, 2026. doi: 10.1007/978-3 -032-22032-5_12. URL https://link.springer.com/chapter/10.1007/978-3-032-22032 -5_12 . [7]Ahlam Abdelkader, Emani Babe, and Oumoukelthoum Sidenna. Dah: Dataset hassaniya. Hugging Face dataset, 2026. URLhttps://huggingface.co/datasets/hassan-IA/dah. [8]Fakhraddin Alwajih, Abdellah El Mekki, Samar Mohamed Magdy, Abdelrahim A. Elmadany, Omer Nacar, El Moatez Billah Nagoudi, et al. Palm: A culturally inclusive and linguistically diverse dataset for arabic llms. InProceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025. URL https://aclanthology.org/2025.acl-long.1579/. [9]Abdellah El Mekki, Samar M. Magdy, Houdaifa Atou, Ruwa AbuHweidi, Baraah Qawasmeh, Omer Nacar, et al. Alexandria: A multi-domain dialectal arabic machine translation dataset for culturally inclusive and linguistically diverse llms.Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), 2026. URL https://arxiv.org/abs/2601.13099. arXiv:2601.13099. 16 [10]Emin009. Ai for rim: Hassaniya dialect datasets. Hugging Face dataset repository, 2026. URL https://huggingface.co/datasets/Emin009/AI-for-RIM. Repository contains separate Arabic–Hassaniya translation and Hassaniya customer-support resources. [11]ahmed02mk. Amthal hassaniya: Hassaniya proverb dataset. Hugging Face dataset, 2026. URL https://huggingface.co/datasets/ahmed02mk/amthal-hassaniya. 319 Hassaniya proverbs with explanations and usage-oriented supervision. [12]Hassan-IA. Hassaniya stories ocr. Hugging Face dataset, 2026. URLhttps://huggingface.co /datasets/hassan-IA/hassaniya-stories-ocr. [13]Bashar Talafha, Karima Kadaoui, Samar Mohamed Magdy, Mariem Habiboullah, Chafei Mohamed Chafei, Ahmed Oumar El-Shangiti, Hiba Zayed, Mohamedou Cheikh Tourad, Rahaf Alhamouri, Rwaa Assi, et al. Casablanca: Data and models for multidialectal arabic speech recognition. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21745–21758. Association for Computational Linguistics, 2024. URLhttps://aclanthology.o rg/2024.emnlp-main.1211/. [14]Abraham Toluwase Owodunni and Sachin Kumar. Continually adding new languages to multilin- gual language models.arXiv preprint arXiv:2509.11414, 2025. URLhttps://arxiv.org/abs/25 09.11414. [15]Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham. Lora learns less and forgets less.Transactions on Machine Learning Research, 2024. URLhttps://arxiv.org/abs/2405.09673. [16]Yan Chen, Taojie Zhu, Meng Zhang, Xin Chen, Jiaqi Huang, Dongyang Xu, and Yizhi Wang. On- policy replay for continual supervised fine-tuning.arXiv preprint arXiv:2605.29495, 2026. URL https://arxiv.org/abs/2605.29495. [17]Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022. URL https://arxiv.org/abs/2212.04089. [18]Google DeepMind. Gemma 4 E4B. Hugging Face model card, 2026. URLhttps://huggingfac e.co/google/gemma-4-E4B . Accessed 12 September 2026. [19]Google DeepMind. Gemma 4 E4B: Pinned model configuration. Hugging Face model configura- tion, 2026. URL https://huggingface.co/google/gemma-4-E4B/blob/411a17b749a9 52df1359d2dcea73917a544d9a/config.json . [20]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021. URL https://arxiv.org/abs/2106.09685. [21]Shih-Cheng Huang, Pin-Zu Li, Yu-Chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tzong-Han Tsai, and Hung-yi Lee. Chat vector: A simple approach to equip LLMs with instruction following and model alignment in new languages. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL https://arxiv.org/ abs/2310.04799. [22]Med El Moustapha El Arby. Hassaniya-dtcd: A new dataset for benchmarking text classification tasks on hassaniya dialect. Mendeley Data, 2025. URLhttps://data.mendeley.com/datase ts/r5k9ktwr4g/1 . [23]Mamadou-Aw. Hassaniya speech dataset. Hugging Face dataset, 2026. URLhttps://huggingf ace.co/datasets/Mamadou-Aw/Hassaniya-speech-dataset. 17 [24]Peace Corps and alvations. Peace corps language resources: Hassaniya. Hugging Face dataset aggregation of Peace Corps language-learning materials, 2026. URLhttps://huggingface.co /datasets/alvations/peace-corps. [25]Ahmed Salem. Basic english-hassaniya dictionary. Internet Archive, 2022. URLhttps://arch ive.org/details/eng-hass-dict-a-z. Public reference containing more than 8,000 words, phrases, and sentences. [26]ScriptureEarth. Arabic, hassaniya (mey): Scripture resources. Hassaniya scripture resource index, 2026. URLhttps://w.scriptureearth.org/00eng.php?iso=mey. [27]Christos Christodoulopoulos and Mark Steedman. A massively parallel corpus: The bible in 100 languages.Language Resources and Evaluation, 49:375–395, 2015. doi: 10.1007/s10579-014-9287-y. URLhttps://christos-c.com/bible/. [28]Gemini Team, Google. Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805 , 2023. URL https://arxiv.org/abs/2312.11805. [29]Hugging Face. Gemma4: Transformers documentation, 2026. URLhttps://huggingface.co /docs/transformers/model_doc/gemma4. Accessed 12 September 2026. [30]Karl Cobbe et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021. URLhttps://arxiv.org/abs/2110.14168. [31]Yubo Wang et al. MMLU-Pro: A more robust and challenging multi-task language understanding benchmark. InAdvances in Neural Information Processing Systems (Datasets and Benchmarks Track), 2024. URLhttps://arxiv.org/abs/2406.01574. [32]Mark Chen et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021. URLhttps://arxiv.org/abs/2107.03374. [33]Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The Berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. InProceedings of the 42nd International Conference on Machine Learning, 2025. URLhttps://proceedings.mlr.press/v267/patil25a.html. [34]NLLB Team, Marta R. Costa-jussà, et al. No language left behind: Scaling human-centered machine translation.arXiv preprint arXiv:2207.04672, 2022. URLhttps://arxiv.org/abs/2207.04672. [35]Maja Popović. chrF++: words helping character n-grams. InProceedings of the Second Conference on Machine Translation, 2017. URLhttps://aclanthology.org/W17-4770/. [36]Matt Post. A call for clarity in reporting BLEU scores. InProceedings of the Third Conference on Machine Translation, 2018. URLhttps://aclanthology.org/W18-6319/. [37]AlexandriaX-2026 Organizers. AlexandriaX-2026: Context-aware dialectal arabic MT and MT evaluation shared task. Shared task, Subtask 3: Dialectal Arabic MT Error Detection and Classifi- cation, 2026. URLhttps://alexandriax.dlnlp.ai/. [38]Samar M. Magdy, Fakhraddin Alwajih, Abdellah El Mekki, Wesam El Sayed, and Muhammad Abdul-Mageed. LQM: Linguistically motivated multidimensional quality metrics for machine translation. InFindings of the Association for Computational Linguistics: ACL 2026, 2026. URL https://aclanthology.org/2026.findings-acl.2012/. 18