Paper deep dive
Mitigating Gender Bias in English to Romanian Machine Translation
Ioana Grigore, Sergiu Nisioi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 8/12/2026, 2:06:51 AM
Summary
This paper addresses gender bias in English-to-Romanian machine translation by proposing a hybrid pipeline that combines Large Language Model (LLM)-based gender classification with Neural Machine Translation (NMT). The system uses a fine-tuned LLaMA model to detect gender context and insert inline tags, which guide a Transformer model to produce morphologically correct Romanian translations. The authors introduce three novel datasets (EnGen and EnRoGend) and demonstrate a significant improvement in gender accuracy on WinoMT and WinoGender benchmarks.
Entities (11)
Relation Signals (7)
Llama-3.2 → usedfor → LLM-based Gender Classification
confidence 98% · As such, we employ the pre-trained LLaMA 3.2 (1B parameters) model from Unsloth [18]... For this step, the main focus is to have a classifier that is fast and easily deployable in an MT system.
Hybrid Pipeline → mitigates → Gender Bias
confidence 97% · We propose a hybrid pipeline to mitigate this issue by combining large language model (LLM)-based gender classification with neural machine translation (NMT).
EnRoGend → usedforfinetuning → Helsinki-NLP/opus-mt-en-ro
confidence 96% · We fine-tune an English-to-Romanian pre-trained Transformer model Helsinki-NLP/opus-mt-en-ro... The dataset consists of 1,974 examples... This dataset contains only occupations and person-related nouns
EnGen → usedforfinetuning → Llama-3.2
confidence 96% · We build two datasets for fine-tuning LLMs to predict the gender of a target word in a given context... Dataset 1 consists of 11,472 examples.
Low-Rank Adaptation → appliedto → Llama-3.2
confidence 95% · We adopt Low-Rank Adaptation (LoRA) [4]. ... LoRA adapters are inserted into the query, key, value, output, gate, up, and down projection layers.
Hybrid Pipeline → improvesaccuracyon → WinoMT
confidence 95% · Our approach improves gender accuracy on the WinoMT and WinoGender benchmarks by over 40 percentage points compared to a baseline MT system.
Hybrid Pipeline → improvesaccuracyon →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Machine translation (MT) systems often fail to correctly translate gender, especially when converting from a gender-neutral language like English to a gendered target language such as Romanian. This bias results in translations that default to masculine forms or reinforce gender stereotypes. We propose a hybrid pipeline to mitigate this issue by combining large language model (LLM)-based gender classification with neural machine translation (NMT). Our system uses a fine-tuned LLM to detect the intended gender of target words in English sentences and insert inline gender hint tags. These tagged sentences are then passed to a Transformer model fine-tuned to generate morphologically correct Romanian translations. To support this, we introduce three novel datasets for gender disambiguation and translation. Our approach improves gender accuracy on the WinoMT and WinoGender benchmarks by over 40 percentage points compared to a baseline MT system. This is the first method to explicitly address and evaluate gender bias in English-Romanian MT using both LLM inference and tag-aware translation.
Tags
Links
- Source: https://arxiv.org/abs/2608.08606v1
- Canonical: https://arxiv.org/abs/2608.08606v1
Trouble viewing inline? Open PDF directly →
Full Text
39,919 characters extracted from source content.
Expand or collapse full text
Mitigating Gender Bias in English to Romanian Machine Translation Ioana Grigore [0009−0006−7370−8159] and Sergiu Nisioi [0000−0003−2247−4488] Human Language Technologies Research Center Faculty of Mathematics and Computer Science University of Bucharest ioanaagrigore28@gmail.com, sergiu.nisioi@unibuc.ro Abstract. Machine translation (MT) systems often fail to correctly translate gender, especially when converting from a gender-neutral lan- guage like English to a gendered target language such as Romanian. This bias results in translations that default to masculine forms or re- inforce gender stereotypes. We propose a hybrid pipeline to mitigate this issue by combining large language model (LLM)-based gender clas- sification with neural machine translation (NMT). Our system uses a fine-tuned LLM to detect the intended gender of target words in English sentences and insert inline gender hint tags. These tagged sentences are then passed to a Transformer model fine-tuned to generate morphologi- cally correct Romanian translations. To support this, we introduce three novel datasets for gender disambiguation and translation. Our approach improves gender accuracy on the WinoMT and WinoGender benchmarks by over 40 percentage points compared to a baseline MT system. This is the first method to explicitly address and evaluate gender bias in En- glish–Romanian MT using both LLM inference and tag-aware transla- tion. Keywords: Gender Bias· Neural Machine Translation· Large Lan- guage Models 1 Introduction Gender bias in machine translation (MT) remains a well-documented challenge, particularly when translating from languages like English—where gender is often implicit—to target languages such as Romanian, which require explicit gram- matical gender agreement. Most neural MT systems tend to default to mascu- line forms or fail to resolve gender correctly from context, resulting in biased or grammatically incorrect outputs. This issue is especially prominent in transla- tions involving professions, roles, or named entities referring to people. In this work, we introduce novel datasets and a hybrid pipeline that combines large language model (LLM)-based gender inference with a fine-tuned neural ma- chine translation (NMT) model. Specifically, we use a fine-tuned LLaMA model to classify the gender of target words in an English sentence and insert explicit in- line gender hint tags (e.g., <tgF>teacher</tgF>). A Transformer model is then arXiv:2608.08606v1 [cs.CL] 9 Aug 2026 2Grigore and Nisioi, 2026 trained to recognize and act on these tags, producing gender-aware Romanian translations. To support this system, we release 1 novel, high-quality corpora that support (i) gender-aware English classification and (i) gender-controlled EN→RO trans- lation. We evaluate our approach on multiple test suites including WinoMT and WinoGender [11, 16], achieving significant gains in gender translation accuracy over baseline MT. Our results demonstrate that combining LLM-based context understanding with targeted NMT adaptation can mitigate gender bias in low- resource language pairs. Direct LLM translation remains costly and difficult to control at scale; our pipeline offers explicit controllability, interpretability, and compatibility with existing MT systems. 2 Related Work Gender bias in machine translation (MT) has been previously documented, par- ticularly in language pairs where the source language (e.g., English) lacks overt gender markers, while the target language (e.g., German, French, Spanish) re- quires grammatical gender agreement. Early studies [11, 16] introduced diagnos- tic datasets such as WinoGender and WinoMT to systematically evaluate gender bias in translation. Recent work on gender-inclusive machine translation shows that state-of-the- art MT systems and LLMs continue to default to masculine forms, particularly in morphologically gendered languages, even when gender-neutral or gender- ambiguous translations are appropriate [1, 14]. Newly introduced multilingual benchmarks and evaluation datasets reveal that models struggle to exploit con- textual cues, extended discourse, and explicit instructions to reliably produce inclusive or neutral forms [9]. Cross-linguistic analyses further highlight persis- tent difficulties with gender ambiguity and non-binary constructions, motivating approaches that explicitly detect and control gender-relevant information prior to or during translation [2]. Related work has also explored sentence-level source-side gender tags, where the gender of the speaker is provided as an explicit signal to the translation model [19]. More recently, large language models (LLMs) have been explored as an alternative to traditional tagging. Instruction-tuned models have been shown to produce gender-controlled outputs via prompt engineering [12, 13]. These models can generate separate masculine and feminine translations by conditioning on contextual cues or examples. However, most of this work focuses on high-resource languages such as Spanish, French, German. Shared-task style evaluations such as the WMT 2020 Gender Coreference and Bias task further highlight persistent gender biases across many submitted MT systems and target languages [5]. 1 All data is released under C BY-NC 4.0 license at https://github.com/Ioannnnna/ EnRoGend. Gender Bias in Romanian MT3 To date, gender bias in English-Romanian MT remains unaddressed. Ro- manian presents special challenges due to its three-gender system and complex morphological agreement. 3 EnGen: The English Gender Disambiguation Dataset We build two datasets for fine-tuning LLMs to predict the gender of a target word in a given context. The process is semi-automatic - a native speaker of Romanian creates sentences, additional examples are generated by GPT through the OpenAI API [6] 2 and then the output is checked and filtered again by a native speaker. The datasets follow a two-stage curriculum learning approach, where training starts with simpler examples and increases in difficulty: 3.1 Dataset 1 - Single-Entity Phrases Fig. 1. Dataset 1 distributions. Left: gender is equally balanced across feminine, mas- culine, and ambiguous entities. Middle: category is equally balanced for nouns related to family, animals, roles, jobs, names. Right: the majority of phrases contain only one sentence, followed by two and three sentence context lengths. Dataset 1 consists of 11,472 examples. Each example contains one gender- able word, such as a job title, family member, animal, role, or proper name. The phrases may contain up to three sentences. We use ambiguous to denote cases where gender cannot be inferred from context; this category includes gender-neutral or non-binary references (e.g., ‘they’), without making assump- tions about the speaker’s gender identity. Statistics regarding the distribution of each categories in the dataset are visible in Figure 1. The data generation process begins with one-sentence contexts. For each semantic category, a separate prompt is used to create examples where the gender of the target word is clear or ambiguous. Once all categories are completed for the one-sentence context, the process repeats with two-sentence contexts, and then again with three-sentence contexts. Across all context lengths, the dataset is balanced to contain as close as possible the number of feminine, masculine, and ambiguous examples. 2 Full LLM prompt is provided in the official repository. 4Grigore and Nisioi, 2026 Table 1. Sample examples from Dataset 1. Each sentence includes a target word and its corresponding gender. SentenceTarget Word (Category) Gender Thepilot skillfully navigated the plane through turbulent weather. pilot (job)Ambiguous The engineer designed an innovative solution to the problem. Her technical skills were instrumental in the project’s success. The team appreciated her forward-thinking approach. engineer (job)Feminine The leader gave a rousing speech to the team. He inspired everyone to do their best. leader (role)Masculine Our cousin is an excellent cook.cousin (family)Ambiguous 3.2 Dataset 2 - Multi-Entity Phrases Dataset 2 contains 996 examples with unique target words. Every sentence con- tains exactly two genderable words: the target word, whose gender must be predicted, and a distractor word, which serves to increase task complexity. The context length is one or two sentences. The distractor entity has ambiguous gen- der and is placed in various positions within the sentence to reduce predictability. We ensure a balanced gender representation (feminine, masculine, ambigu- ous) while keeping the surrounding context highly similar across variants. All three gender labels appear for each target word and the context remains seman- tically and syntactically natural. The format is identical to dataset 1, consisting of an input (sentence, target word and category) and an output (gender label). Table 2. Triplet Sample from Dataset 2. Here the "librarian" represents the target word and "teacher", "student", "detective" are the other entities from the context that can also be gendered. SentenceTarget Word (Category) Gender Thelibrarian cataloged the new books, and the teacher borrowed a few from her. librarian (job)Feminine Thelibrarian organized the shelves as the student sought guidance from him. librarian (job)Masculine The librarian recommended a thriller, which the detective found thrilling. librarian (job)Ambiguous Gender Bias in Romanian MT5 3.3 Dataset Splitting Both datasets are split into training, validation, and test sets to support the two-stage fine-tuning procedure, with splits designed to preserve class balance and avoid leakage. For Dataset 1 (D1) we use an 80/10/10 split stratified by gender and category, ensuring each gender–category combination is proportionally represented. The resulting sizes are: train = 9,177, val = 1,147, test = 1,148. For Dataset 2 (D2) we use a 70/15/15 triplet-locked split: each (masculine, feminine, ambiguous) triplet receives a group ID and the whole group is assigned to exactly one partition (train, validation or test). This guarantees that no variant from the same triplet appears in different splits, including the test set. The resulting sizes are: train = 699, val = 150, test = 147. Training sets are used for parameter updates, validation sets for hyperparameter selection, and test sets are reserved strictly for final evaluation. We adopt a curriculum learning approach: dataset 1 contains single-entity phrases so the model can first learn the basic mapping from context to gender without interference. Dataset 2 raises the difficulty with two genderable words (one target and one distractor) varying in positions and cues. 4 EnRoGend: a Parallel English-Romanian Gender-Tagged Dataset The dataset consists of 1,974 examples, organized into pairs: two versions of the same sentence, each with the same target word marked for a masculine and feminine genders. Each example includes the English source, where the target word is surrounded by a gender tag <tgM>target word</tgM> or <tgF>target word</tgF>, and the corresponding Romanian translation, which has no tags. This dataset contains only occupations and person-related nouns having the purpose of teaching the MT system that it should use feminine when it sees <tgF></tgF> and masculine when it sees <tgM></tgM>. The construction of this dataset begins with a predefined list of 82 entities referring to jobs and roles. For each entity, we write between 10 and 15 sentences where the entity’s gender is elicited using the feminine pronoun (she), then we duplicate the sentence using the masculine pronoun; see for reference Table 3. All sentences are translated into Romanian and verified by two annotators. The dataset is balanced between gendered target words, with 987 masculine and 987 feminine instances, covering 82 distinct occupations. We split the machine–translation dataset with a similar pair-locked strategy to avoid near-duplicate leakage. Each example belongs to a 2-item minimal pair. Instead of shuffling individual rows, we shuffle pairs and keep both members together in the same partition. The splits are 80% train, 10% validation and 10% test. This guarantees that if the masculine variant of a sentence is in training, its feminine counterpart cannot appear in validation or test (and vice versa), including for the held-out test set. The result is a fair evaluation that is not inflated by near-duplicate examples. 6Grigore and Nisioi, 2026 Table 3. Sample Pair from the English-Romanian Gender Tagged Dataset. En The teacher thanked the<tgM>lawyer</tgM> sincehe had been gener- ous throughout the project. Ro Profesorul i-a mult , umit avocatului, deoarece fusese generos pe parcursul proiectului. En The <tgF>journalist</tgF> advised the painter on the task becauseshe was kind. Ro Jurnalista l-a sfătuit pe pictor pentru că era amabilă. 5 Methodology The datasets described in previous sections are the first ones to address gen- der bias for English-Romanian language pairs and can thus enable the creation of an end-to-end machine translation pipeline. The pipeline processes an in- put sentence through a sequence of analysis and generation stages designed to preserve intended gender information during translation. Candidate entities that may require gendered realization are first identified, after which a large language model infers the contextual gender of each entity based on discourse cues. The inferred gender information is then encoded using inline tags and merged back into the original sentence, forming an intermediate representation that makes gender constraints explicit. This tagged sentence is subsequently passed to the machine translation system, which uses the annotations to guide the selection of appropriate gendered forms in the target language, thereby reducing reliance on default or biased gender choices. The entire process is rendered in Figure 2. To enable gender-aware processing in realistic settings, the pipeline relies on an Entity Selector that used a predefined list of gendered entities that serve as candidate targets for gender inference. The list is compiled based on previous studies [3]. Each word in the input sentence is matched against this list to identify role nouns or occupations whose gender must be inferred. To improve robustness, fuzzy string matching is applied to account for misspellings, plural forms, and minor lexical variations. 5.1 LLM Gender Classification For this step, the main focus is to have a classifier that is fast and easily de- ployable in an MT system. As such, we employ the pre-trained LLaMA 3.2 (1B parameters) model from Unsloth [18]. To make fine-tuning feasible on limited hardware, the model is loaded in 4-bit NF4 quantization and fine-tuned with mixed precision. We adopt Low-Rank Adaptation (LoRA) [4]. Instead of updat- ing a full weight matrix W 0 ∈R d×k , LoRA learns a low-rank update: W = W 0 + ∆W, ∆W = α r AB, where A∈R d×r , B ∈R r×k , r ≪ min(d,k), and α scales the update. The frozen W 0 preserves pre-trained knowledge, while ∆W captures task-specific adjust- ments. LoRA adapters are inserted into the query, key, value, output, gate, up, Gender Bias in Romanian MT7 User Input Sentence The doctor congratulated the nurse because he did a good job. Entity Selector Identifies: doctor, nurse LLM Gender Classification Target word: doctor → ambiguous Target word: nurse → masculine Merged Tagged Sentence The doctor congratulated the<tgM>nurse</tgM> because he did a good job. Machine Translation (MT) Output: Doctorul l-a felicitat pe asistent pentru că a făcut o treabă bună. Fig. 2. End-to-end pipeline example. The system takes an English sentence, extracts candidate entities, classifies gender for each using the LLM, inserts tags for gendered entities, and translates the tagged sentence into Romanian. and down projection layers. Only LoRA parameters and the classification head are updated during training. We use AdamW with separate parameter groups, one for the LoRA adapter parameters and one for the language modeling head. This allows assigning distinct learning rates to each group, while weight decay is applied only where specified in the grid configuration. Training is performed with gradient accumulation and mixed-precision computation in bfloat16, on a single Nvidia L4 GPU in Colab Pro. Classification Strategy a Linear Classification Head is attached to the pre- trained LLaMA 3.2 (1B parameters ) model. After processing the input sequence through the language model, we extract the hidden state corresponding to the final token position in the sequence. This hidden representation, denoted h last , encodes the full context of the input, including the sentence and target word. A fully connected linear layer then transforms this hidden vector into a fixed- size output vector representing class logits for the three possible gender labels: feminine, masculine, and ambiguous. Formally, the prediction is computed as: h last = LLM(x)[−1],y = Wh last + b, where W ∈R 3×d and b ∈R 3 are the learned parameters of the classification head, and d is the dimensionality of the hidden state. Model training is guided by the cross-entropy loss. The Two-Stage Curriculum Learning is designed to gradually increase task complexity while training. In the first stage, the model is fine-tuned on Dataset 1, which contains sentences with a single named entity and well-balanced gender classes. These examples provide both clear and ambiguous contexts, allowing 8Grigore and Nisioi, 2026 the model to learn the core gender classification task with minimal distraction or noise. Once this base ability is acquired, the second stage uses the Dataset 2, which presents more realistic challenges, including sentences containing multiple named entities. This stage encourages the model to reason about context and resolve gender cues in more complex linguistic scenarios. Hyperparameter Search and Evaluation We explore hyperparameter con- figurations for both stages of curriculum training: – Stage 1 (initial fine-tuning on D1): LoRA rank r ∈ 8, 12, 16, 24, 32 with scaling α≈ 2r, dropout in [0.00, 0.06], learning rates for the LoRA adapters and classifier head (4× 10 −5 –1.6× 10 −4 ), weight decay in 0.0, 0.005, 0.01, and 2–3 epochs. – Stage 2 (continued fine-tuning on D2): LoRA ranks 8, 12, 16 with propor- tionally scaled α, dropout in [0.04, 0.07], smaller learning rates (4× 10 −5 – 1.0× 10 −4 ), weight decay in 0.005, 0.01, and 1–2 epochs. For hyperparameter tuning we use optuna search. It runs a fixed budget of trials (e.g., 12), guided by a Tree-structured Parzen Estimator (TPE) sampler. We observe that several configurations achieve high accuracy and F1 scores on Dataset 1, with the top-performing model reaching an F1 of 0.97 and accuracy of 97% on the test set. The performance on Dataset 2 varies more widely due to its complex, multi-entity structure. While the best model achieves a strong F1 score of 0.95, others drop as low as 0.66. Overall, the performance is sensitive to regularization and learning rate balance, and improves with moderate-to-deep LoRA ranks (e.g., 12–16) paired with well-scaled α values. Table 4 summarizes the generalization in both datasets. The best joint test F1 score reaches 0.96, confirming that high-quality performance can be maintained by generalizing across D1 and D2 simultaneously. In addition, we evaluate the final Stage 2 models both on D2 and again on D1, which allows us to quantify catastrophic forgetting via the Forgetting F1 metric (see Table 4). This approach makes the runs easy to compare and gives us a reliable way to measure the effect of curriculum learning. The joint metrics represent the model’s performance on the combined test set, which merges the test splits from both datasets, D1 and D2. The Forgetting F1 metric shows how much performance on D1 is lost after the second stage of training on D2. It is computed as the difference in F1 score on the D1 test set after and before Stage 2 fine-tuning, a negative value indicates forgetting (performance decreased), while a positive value implies improvement or recovery on D1 after further training. 5.2 Gender-Aware Machine Translation (En→Ro) We fine-tune an English-to-Romanian pre-trained Transformer model Helsinki- NLP/opus-mt-en-ro [17] and evaluate performance on both in-domain splits and external diagnostic sets. The tokenizer is extended with four special tokens: <tgM>, </tgM>, <tgF>, and </tgF> on the source side. If no padding token Gender Bias in Romanian MT9 Table 4. The Forgetting F1 metric quantifies the degree of catastrophic forgetting, with most configurations showing mild drops (e.g., -0.0121 or -0.0015), while a few show large negative values (e.g., -0.1820), indicating significant loss of earlier knowledge. D2 idx D1 idx Joint Acc Joint F1 Joint MCC Forgetting F1 D2-1 D1-196%0.960.940.0083 D2-2 D1-283%0.820.75-0.1040 D2-3 D1-395%0.950.93-0.0121 D2-4 D1-196%0.960.940.0072 D2-6 D1-280%0.800.70-0.0851 D2-8 D1-395%0.950.93-0.0015 D2-9 D1-273%0.740.60-0.0851 D2-10 D1-486%0.860.81-0.1820 D2-11 D1-566%0.660.580.1950 is defined, the EOS token is reused as PAD. Romanian references are kept tag- free to avoid leaking gender labels into the target side. Because the dataset contains many minimal pairs (i.e., identical English sen- tences differing only in masculine vs. feminine tags), a random split would risk leaking near-duplicate examples across train, validation, and test sets, leading to artificially inflated evaluation scores. To prevent this, we enforce a pair-locked split: – Sentences are grouped into pairs consecutively by file order: (0, 1), (2, 3),.... Each pair is assigned a unique integer pair_id defined as ⌊i/2⌋. We ensure an even number of rows; otherwise, the source file is adjusted. – Exact duplicates on the (English, Romanian) tuple are removed to avoid trivial matches at evaluation time. – The set of unique pair_ids is randomly shuffled with a fixed seed for repro- ducibility. – The first 80% of pair_ids are assigned to the training set, the next 10% to validation, and the remaining 10% to test. Both members of each minimal pair are always placed in the same split. – We verify that pair_id sets are disjoint across splits and report hashed over- laps on both source and target texts as a sanity check against accidental data leakage. Hyperparameter Tuning To determine the most effective fine-tuning strategy for Transformer with gender tags, we explore three adaptation regimes under identical training loops: – Full fine-tuning: all model parameters are updated, providing maximum flex- ibility but at a higher computational cost. – Partial fine-tuning: most encoder layers are frozen, and only the decoder, shared embeddings, language modeling head, and the last N encoder layers (N ∈ 1, 2, 3) are updated, reducing training cost while retaining adapt- ability. 10Grigore and Nisioi, 2026 – LoRA fine-tuning: a parameter-efficient strategy where low-rank adapters are injected into the attention layers, keeping the base model frozen and updating only the adapter parameters. Hyperparameter tuning is performed using Optuna with a grid-based search over the following space: learning rate ∈ 1e−5, 5e−5, 1e−4, batch size ∈ 8, 16, epochs ∈ 1, 2, and, for partial fine-tuning, the number of unfrozen encoder layers N unfreeze ∈ 1, 2, 3. Model selection is based on validation performance, and the best configurations are evaluated on the held-out test set using BLEU [8], chrF++ [7] 3 , TER [15], and COMET [10] metrics. The reported metrics reflect different aspects of translation quality: BLEU and chrF++ measure word and character n-gram overlap between system output and reference; TER reflects the number of edits needed to reach the reference, where lower is better; and COMET is a learned metric that evaluates adequacy and fluency based on source, hypothesis, and references. The surface-form metrics such as BLEU, chrf++, and TER tend to reflect good results due to the high overlap, however the gender mismatch is more nuanced semantic aspect, therefore we rely more on the COMET metric for the final judgments and keep the remaining metrics as evidence. Table 5. Hyperparameter optimization results on the validation set for fine-tuning Transformer with gender tags. The table compares full, partial, and LoRA fine-tuning modes across various learning rates, batch sizes, epochs, and (for partial mode) number of unfrozen encoder layers. Full fine-tuning with a learning rate of 1e−4, batch size 16, and 2 epochs achieved the best validation performance, with the highest BLEU (97.29), chrF (98.57), COMET (0.853), and the lowest TER (1.71). ModeLR Batch EpochsN unfreeze BLEU chrF++ TER COMET partial 1.00e-05 162163.69 80.31 24.88 0.7723 full5.00e-05 8193.03 96.73 3.55 0.8409 partial 1.00e-04 161192.94 96.40 3.90 0.8396 full1.00e-05 8167.57 83.28 21.50 0.7857 lora1.00e-04 8141.12 76.78 44.84 0.7263 full 1.00e-04 16 297.29 98.57 1.71 0.8530 lora1.00e-04 16241.71 67.35 43.98 0.7261 lora1.00e-05 8123.99 57.59 71.13 0.6467 lora5.00e-05 8133.08 61.98 53.70 0.7103 partial 1.00e-05 162369.30 84.06 20.13 0.7853 partial 5.00e-05 161389.64 94.95 5.61 0.8306 partial* 1.00e-04 81297.07 98.40 1.93 0.8511 The results in Table 5 show that full fine-tuning with a learning rate of 1e−4, batch size 16, and 2 epochs achieved the best overall performance (highlighted 3 chrF++signatureis nrefs:1|case:mixed|eff:yes|nc:6|nw:0|space: no|version:2.4.3 and BLEU signature is nrefs:1|case:mixed|eff:no|tok: 13a|smooth:exp|version:2.4.3. Gender Bias in Romanian MT11 in boldface). Partial fine-tuning with 2–3 unfrozen encoder layers performed competitively (bottom row marked with an *), achieving a COMET scores above 0.85 and comparable overall scores to the full-finetuning model. This represents a good compromise between efficiency and accuracy. The best hyperparameter configuration selected on the validation set based on the results of Table 5 is the fully finetuned model. This model achieves similar scores on the held-out test set: a COMET score of 0.846, a BLEU score of 96.92, a chrF++ score of 98.21, and a TER of 1.94. LoRA, however, underperformed in our experimental setting, suggesting that lightweight adapter tuning does not provide enough capacity for this task. A likely explanation is that this task requires more than light domain adapta- tion. The model must learn to use new source-side gender tags and realize them through correct Romanian morphology, including agreement on nouns, adjec- tives, and verbs. Full and partial fine-tuning are better suited to this controlled generation setting because they let more of the model adapt to the tag signal. By contrast, the LoRA setups tested may have been too limited to capture the link between explicit gender tags and downstream morphological realization. The large gap in performance therefore suggests that this form of parameter- efficient tuning is not sufficient for robust gender control in English–Romanian translation. Another possible factor is that Romanian morphological agreement is more demanding than the target-side changes required in some higher-resource settings. 6 Benchmark Evaluation To evaluate gender bias in machine translation, we rely on several benchmarks that are designed to probe whether a system can correctly resolve gender based on cues and whether it reflects correct grammatical agreement in gender-marked target languages. Although GPT-4o was used in the data creation process, all benchmark evaluations are conducted on external datasets that are not used during training, to ensure that there are no data leaks. In addition, we have compared all the sentences with our dataset to ensure that there is no data leakage. The WinoMT dataset [16] consists of (1,584 pro and 1,584 anti) English sen- tences in which gender must be inferred from context. Each sentence is presented in two forms: a pro-stereotypical variant, where the pronoun aligns with com- mon gender stereotypes and an anti-stereotypical variant, where the pronoun contradicts such stereotypes, e.g., pro-stereotypical sentence: The nurse helped the patient because she was kind. Anti-stereotypical sentence: The nurse helped the patient because he was kind. An unbiased system is expected to perform similarly on both pro-stereotypical and anti-stereotypical examples. The WinoGender dataset [11] is a pronoun resolution benchmark that tests the impact of gendered pronouns on translation, consisting of 720 sentences. Each 12Grigore and Nisioi, 2026 example is a minimal pair differing only in the pronoun (he, she or they) and is used to decide whether the system’s output reflects these distinctions correctly in the gendered target-language translation. All English test sentences are translated using three systems: 1. Raw MT - the base Transformer model, without any additional gender hint tags; and 2. Gender-Aware Pipeline - each sentence passed through our pipeline (as described in Figure 2); 3. GPT-5.2 using default system settings and zero-shot translation prompts; due to its proprietary nature, exact replication may not be possible. These test sets do not include gold-standard Romanian reference translations. Therefore, a manual evaluation was conducted by a native Romanian speaker and verified through spot checks. For each test case, we check whether the trans- lation preserves the correct grammatical gender of the target word, along with agreement (e.g., adjective inflection, verb conjugation). Table 6. The results presented in the table are the accuracies on benchmark test sets. It shows that the raw Transformer system performs better when the correct gender is masculine, as it often defaults to masculine forms during translation. While this leads to higher scores on Pro-stereotypical examples (where the gold label aligns with masculine bias), it harms performance on Anti-stereotypical and feminine cases. Similarly, a state- of-the-art model such as GPT-5.2 has a strong preference for stereotypical biases. The gender-aware pipeline significantly improves accuracy across all subsets by explicitly guiding the model toward the intended gender. Model / Metric WinoMT (Pro) WinoMT (Anti) WinoGender Raw Transformer59.05%50.13%50.28% Pipeline LLM-MT96.34%93.68%91.53% GPT-5.279.23%58.79%67.71% Some translations contain issues such as incorrect word choices or missing diacritics. However, as these errors occur in both raw and fine-tuned outputs, we consider a translation correct if the intended gender is correct and if it stays in agreement with the rest of the sentence. For example, if the MT translates sales- person for feminine gender as "vânzătora" instead of "vânzătoarea" we consider it correct. 7 Conclusion We investigate gender bias in English→Romanian machine translation, where ambiguous English inputs must be rendered with explicit grammatical gender in Romanian. Our results confirm that a standard Transformer MT baseline frequently defaults to masculine forms and exhibits stereotype sensitivity, per- forming substantially better on pro-stereotypical WinoMT examples than on anti-stereotypical ones. Furthermore, even proprietary state-of-the-art models such as GPT-5.2 have a bias towards stereotypical translations into Romanian, despite the fact that such models might have been exposed to the WinoMT and WinoGender datasets. To mitigate gender bias, we propose a hybrid pipeline that combines LLM- based contextual gender disambiguation with tag-aware neural machine trans- lation: a fine-tuned LLM predicts the intended gender of target entities in the English source and inserts inline gender hint tags, which a Transformer model learns to follow during translation. Across external diagnostic benchmarks, this approach yields large gains in gender correctness, improving accuracy by over 40 percentage points relative to the raw MT system and substantially reducing the pro/anti performance gap, indicating reduced reliance on stereotypical defaults. To support research in this low-resource setting, we introduced several datasets for (i) EnGen - English language gender disambiguation with a curriculum learn- ing setup and (i) EnRoGend - a controlled En→Ro translation with gender tags, using leakage-safe splitting strategies. We also found that full and par- tial fine-tuning of the MT model effectively leverage the gender tags, while the LoRA configurations we tested underperformed for this task, suggesting that tag-conditioned morphological control may require greater adaptation capacity. While our experiments focus on Romanian, the proposed pipeline is applica- ble to other morphologically gendered target languages. Acknowledgments This research is supported by InstRead: Research Instruments for the Text Com- plexity, Simplification and Readability Assessment CNCS - UEFISCDI project number PN-IV-P2-2.1-TE-2023-2007 and by the project “Romanian Hub for Ar- tificial Intelligence - HRIA”, Smart Growth, Digitization and Financial Instru- ments Program, 2021-2027, MySMIS no. 351416. Note This preprint has not undergone peer review or any post-submission improve- ments or corrections. The Version of Record of this contribution is published in Lecture Notes in Computer Science (LNCS, Springer), and is available online at https://doi.org/10.1007/978-3-032-29532-3_11 References [1] Hackenbuchner, J., Gkovedarou, E., Daems, J.: GENDEROUS: Machine translation and cross-linguistic evaluation of a gender-ambiguous dataset. In: Faleńska, A., Basta, C., Costa-jussà, M., Stańczak, K., Nozza, D. (eds.) Proceedings of the 6th Workshop on Gender Bias in Natural Language Pro- cessing (GeBNLP). p. 302–319. Association for Computational Linguistics, 14Grigore and Nisioi, 2026 Vienna, Austria (Aug 2025). https://doi.org/10.18653/v1/2025.gebnlp-1. 27, https://aclanthology.org/2025.gebnlp-1.27/ [2] Hackenbuchner, J., Tezcan, A., Daems, J.: Automatic detection of (poten- tial) factors in the source text leading to gender bias in machine translation. In: Scarton, C., Prescott, C., Bayliss, C., Oakley, C., Wright, J., Wrigley, S., Song, X., Gow-Smith, E., Forcada, M., Moniz, H. (eds.) Proceedings of the 25th Annual Conference of the European Association for Machine Transla- tion (Volume 2). p. 27–28. European Association for Machine Translation (EAMT), Sheffield, UK (Jun 2024), https://aclanthology.org/2024.eamt-2. 14/ [3] Hackenbuchner, J., Tezcan, A., Daems, J.: Gender bias and the role of context in human perception and machine translation. Computational Lin- guistics in the Netherlands Journal 14, 215–239 (2025), https://w. clinjournal.org/clinj/article/view/197 [4] Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022), https: //openreview.net/forum?id=nZeVKeeFYf9 [5] Kocmi, T., Limisiewicz, T., Stanovsky, G.: Gender coreference and bias eval- uation at WMT 2020. In: Barrault, L., Bojar, O., Bougares, F., Chatter- jee, R., Costa-jussà, M.R., Federmann, C., Fishel, M., Fraser, A., Graham, Y., Guzman, P., Haddow, B., Huck, M., Yepes, A.J., Koehn, P., Martins, A., Morishita, M., Monz, C., Nagata, M., Nakazawa, T., Negri, M. (eds.) Proceedings of the Fifth Conference on Machine Translation. p. 357–364. Association for Computational Linguistics, Online (Nov 2020). https://doi. org/10.18653/v1/2020.wmt-1.39, https://aclanthology.org/2020.wmt-1.39/ [6] OpenAI: GPT-4o: Openai’s multimodal language model. https://openai. com/index/ (2024), accessed: 29 August 2025 [7] Popović, M.: chrF++: words helping character n-grams. In: Proceedings of the Second Conference on Machine Translation. p. 612–618. Association for Computational Linguistics, Copenhagen, Denmark (Sep 2017). https: //doi.org/10.18653/v1/W17-4770, https://aclanthology.org/W17-4770/ [8] Post, M.: A call for clarity in reporting BLEU scores. In: Proceedings of the Third Conference on Machine Translation: Research Papers. p. 186–191. Association for Computational Linguistics, Belgium, Brussels (Oct 2018), https://w.aclweb.org/anthology/W18-6319 [9] Pranav, A., Hackenbuchner, J., Attanasio, G., Lardelli, M., Lauscher, A.: Glitter: A multi-sentence, multi-reference benchmark for gender-fair Ger- man machine translation. In: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2025. p. 18450–18477. Association for Computational Linguistics, Suzhou, China (Nov 2025). https://doi.org/10.18653/v1/2025. findings-emnlp.1002, https://aclanthology.org/2025.findings-emnlp.1002/ [10] Rei, R., Guerreiro, N.M., Pombal, J., van Stigt, D., Treviso, M., Coheur, L., C. de Souza, J.G., Martins, A.: Scaling up CometKiwi: Unbabel-IST 2023 submission for the quality estimation shared task. In: Koehn, P., Haddow, Gender Bias in Romanian MT15 B., Kocmi, T., Monz, C. (eds.) Proceedings of the Eighth Conference on Machine Translation. p. 841–848. Association for Computational Linguis- tics, Singapore (Dec 2023). https://doi.org/10.18653/v1/2023.wmt-1.73, https://aclanthology.org/2023.wmt-1.73/ [11] Rudinger, R., Naradowsky, J., Leonard, B., Van Durme, B.: Gender bias in coreference resolution. Association for Computational Linguistics, New Orleans, Louisiana (Jun 2018), https://aclanthology.org/N18-2002/ [12] Sánchez, E., Andrews, P., Stenetorp, P., Artetxe, M., Costa-jussà, M.R.: Gender-specific machine translation with large language models. In: Pro- ceedings of the Fourth Workshop on Multilingual Representation Learning (MRL 2024). p. 148–158 (2024), https://aclanthology.org/2024.mrl-1.10/ [13] Sant, A., Escolano, C., Mash, A., De Luca Fornaciari, F., Melero, M.: The power of prompts: Evaluating and mitigating gender bias in MT with LLMs. In: Proceedings of the 5th Workshop on Gender Bias in Natural Lan- guage Processing (GeBNLP). p. 94–139 (2024). https://doi.org/10.18653/ v1/2024.gebnlp-1.7, https://aclanthology.org/2024.gebnlp-1.7/ [14] Savoldi, B., Attanasio, G., Cupin, E., Gkovedarou, E., Hackenbuchner, J., Lauscher, A., Negri, M., Piergentili, A., Thind, M., Bentivogli, L.: Mind the inclusivity gap: Multilingual gender-neutral translation evaluation with mGeNTE. In: Christodoulopoulos, C., Chakraborty, T., Rose, C., Peng, V. (eds.) Proceedings of the 2025 Conference on Empirical Methods in Natu- ral Language Processing. p. 13698–13720. Association for Computational Linguistics, Suzhou, China (Nov 2025). https://doi.org/10.18653/v1/2025. emnlp-main.692, https://aclanthology.org/2025.emnlp-main.692/ [15] Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers. p. 223–231. Association for Machine Trans- lation in the Americas, Cambridge, Massachusetts, USA (Aug 8-12 2006), https://aclanthology.org/2006.amta-papers.25/ [16] Stanovsky, G., Smith, N.A., Zettlemoyer, L.: Evaluating gender bias in machine translation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. p. 1679–1684 (2019), https: //aclanthology.org/P19-1164/ [17] Unsloth AI: Helsinki-NLP/opus-mt-en-ro. https://huggingface.co/ Helsinki-NLP/opus-mt-en-ro (2025), accessed: 29 August 2025 [18] Unsloth AI: unsloth/Llama-3.2-1B-bnb-4bit. https://huggingface.co/ unsloth/Llama-3.2-1B-bnb-4bit (2025), accessed: 29 August 2025 [19] Vanmassenhove, E., Hardmeier, C., Way, A.: Getting gender right in neural machine translation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. p. 3003–3008 (2018), https:// aclanthology.org/D18-1334/