Paper deep dive
The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation
Michael Jungo, Aixiu An
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/25/2026, 1:29:10 AM
Summary
This paper investigates the cost-quality tradeoffs of using Reinforcement Learning with Verifiable Rewards (RLVR) for Neural Machine Translation (NMT), specifically in the legal domain. The authors evaluate the impact of including reasoning traces (thinking) during both training and inference phases using Qwen3.5 models on the SwiLTra-Bench dataset. Key findings indicate that enabling reasoning during inference improves translation quality (measured by COMET, chrF, METEOR) but significantly increases output tokens and inference costs. The optimal configuration balances quality and cost by training with reasoning to tame the thinking process, then enabling reasoning during inference, avoiding mismatched configurations that degrade quality or inflate costs disproportionately.
Entities (12)
Relation Signals (7)
Qwen3.5 → evaluatedon → SwiLTra-Bench
confidence 97% · We fine-tune Qwen3.5 ... on theSwiLTra-Benchdataset
RLVR → usedfor → NMT
confidence 95% · RLVR has been established as a viable paradigm for the post-training of Large Language Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT).
Reasoning Trace → affects → Translation Quality
confidence 94% · including the reasoning, specifically during inference, has a positive effect on the overall translation quality.
Reasoning Trace → increases → Inference Cost
confidence 93% · the reasoning leads to an increase in output tokens, hence we study the cost-quality tradeoff
Qwen3.5 → trainedwith → GRPO
confidence 92% · we train models with Group Relative Policy Optimisation (GRPO) [13]
Qwen3.5-9B → achievesbestresult → COMET
confidence 91% · The best result is achieved by Qwen3.5 9B [T:✓, I:✓], which incorporates thinking into both the training and inference, with a COMET score of 82.50.
Mismatched Configurations → degrades → Translation Quality
confidence 90% · Mismatched configurations should be avoided: disabling reasoning at inference after reasoning-based training degrades quality below the reasoning-free baseline
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Reinforcement learning with verifiable rewards (RLVR) has been established as a viable paradigm for the post-training of Large Language Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT). With the latest research indicating that RLVR could be the preferred training method for translating legal documents due to the induced reasoning capabilities, it raises the question whether it is really attributed to the reasoning or more generally to the training paradigm. We investigate the importance of including the model's reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases. Our experiments show that including the reasoning, specifically during inference, has a positive effect on the overall translation quality. Furthermore, we recognise that the reasoning leads to an increase in output tokens, hence we study the cost-quality tradeoff between the increased computational demands and the improved translation quality.
Tags
Links
- Source: https://arxiv.org/abs/2607.19226v1
- Canonical: https://arxiv.org/abs/2607.19226v1
Trouble viewing inline? Open PDF directly →
Full Text
22,717 characters extracted from source content.
Expand or collapse full text
The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation Michael Jungo ✉ michael.jungo@unifr.ch AIBEX, University of Fribourg Fribourg, Switzerland Aixiu An ✉ aixiu.an@hefr.ch iCoSys, University of Applied Sciences and Arts Western Switzerland Fribourg, Switzerland Abstract Reinforcement learning with verifiable rewards (RLVR) has been established as a viable paradigm for the post-training of Large Lan- guage Models (LLMs), including downstream tasks, such as Neural Machine Translation (NMT). With the latest research indicating that RLVR could be the preferred training method for translating legal documents due to the induced reasoning capabilities, it raises the question whether it is really attributed to the reasoning or more generally to the training paradigm. We investigate the importance of including the model’s reasoning trace in the generated responses during both training and inference by systematically omitting it from one of the phases. Our experiments show that including the reasoning, specifically during inference, has a positive effect on the overall translation quality. Furthermore, we recognise that the reasoning leads to an increase in output tokens, hence we study the cost-quality tradeoff between the increased computational demands and the improved translation quality. CCS Concepts • Computing methodologies→Machine translation; Rein- forcement learning;• Applied computing→ Law. ACM Reference Format: Michael Jungo and Aixiu An. 2026. The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine Translation. In Proceedings of the 2026 ACM Symposium on Document Engineering (DocEng ’26), August 25–28, 2026, Fribourg, Switzerland. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3820755.3833407 1 Introduction Legal work runs on language, and increasingly that language is generated, analysed, and translated by machines. Large language models now assist across the legal workflow—from research and contract analysis to drafting, summarisation, and translation. Trans- lation stands out among these applications: it is what makes legis- lation, judicial decisions, contracts, and regulatory texts accessible across language communities. The stakes are highest in multilingual jurisdictions such as Switzerland, where legal documents must be produced and interpreted with equal authority in German, French, and Italian. This work is licensed under a Creative Commons Attribution 4.0 International License. DocEng ’26, Fribourg, Switzerland © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2786-3/2026/08 https://doi.org/10.1145/3820755.3833407 Recent LLMs have markedly improved legal translation quality. On demanding benchmarks such asSwiLTra-Bench[10], the Swiss Legal Translation Benchmark, reinforcement learning with verifi- able rewards (RLVR) has been shown to push small open-weight models beyond what supervised fine-tuning (SFT) achieves [1]— though still short of the strongest frontier models. These gains, however, surface a practical tension. The largest reasoning models tend to deliver the best translations, but they also demand far more computation, inference time, and money. For organisations translating thousands or millions of documents, chasing maximal accuracy may not be economically defensible. The question is no longer which model is most accurate, but which model best balances legal reliability against inference cost. This question of cost–quality tradeoffs is becoming increasingly central to the field. Zhuang et al. [16]introduce a framework that provides a quantifiable decision-making tool for selecting the best AI technology within a limited budget, marking a key shift in the industry’s focus from the pursuit of ever-larger models toward the efficiency of engineered, measurable application deployment. Rea- soning exemplifies this tradeoff: while it has emerged as a promising approach for enhancing LLM capabilities across a wide range of tasks, generating explicit reasoning introduces additional output tokens, increasing inference cost and latency. In this work, we study the cost-quality tradeoff for legal machine translation, asking whether reasoning traces should be enabled during training and inference. Our main findings are: (1) Reasoning during both training and inference yields the best translation quality, at the cost of longer outputs and higher inference cost and latency. (2)Mismatched configurations should be avoided: disabling rea- soning at inference after reasoning-based training degrades quality below the reasoning-free baseline, while enabling it without reasoning-based training triples the inference cost. 2 Related Work 2.1 RLVR for Legal Translation Several recent studies have investigated the impact of RL on ma- chine translation [6,14]. Recently, [1] demonstrated that RLVR consistently outperforms SFT across different model sizes and lan- guage pairs for legal translation on theSwiLTra-Benchbenchmark. They keep the rewards to a minimum, by using a structural reward to ensure that the thinking and the translation are inside the ap- propriate XML tags to facilitate their extraction from the response. The reward for the translation quality uses chrF [11], which can be arXiv:2607.19226v1 [cs.CL] 21 Jul 2026 DocEng ’26, August 25–28, 2026, Fribourg, SwitzerlandJungo et al. Qwen3.5 4B [I:✓] Qwen3.5 4B [I:✗] Qwen3.5 9B [I:✓] Qwen3.5 9B [I:✗] 5001 0001 5002 0002 500 76 78 80 82 Number of Training Samples Mean COMET Score Trained with Thinking [T:✓] 5001 0001 5002 0002 500 76 78 80 82 Number of Training Samples Trained without Thinking [T:✗] Figure 1: Translation quality across number of training samples. The COMET score for the models evaluated on a subset of 4 000 validation samples in intervals of 100 seen training samples. (left) models trained with thinking, marked as [T:✓], and (right) trained without thinking [T:✗]. Solid lines correspond to performing inference with thinking enabled [I:✓], whereas dashed lines indicate that thinking has been disable during the inference [I:✗]. computed efficiently and provides continuous values in the range of [0,1]to have a nuanced distinction between the sampled responses. 2.2 The Economics of RLVR Given that RLVR improves translation quality, a natural question is how to apply it most economically—in particular, whether ex- plicit reasoning is genuinely necessary. Chen et al. [3]argue that RLVR does not endow models with fundamentally new reasoning capabilities. Instead, the observed gains arise from more effectively sampling reasoning trajectories that are already present in the base model’s distribution: improvements in benchmark performance reflect better exploitation of existing capabilities rather than the acquisition of new ones. If reasoning traces merely resurface what the base model already knows, their cost deserves scrutiny. From an economic point of view, this motivates two questions: 1) is the reasoning step necessary during RL training, given that generating responses with reasoning increases training time and cost; and 2) is reasoning necessary at inference time, where the additional output tokens directly inflate latency and cost? 2.3 Energy and Inference Cost Estimation For frontier models accessed through commercial APIs, inference cost is straightforward to compute: providers charge per token, with separate rates for input and output tokens. For locally deployed models, however, no such unified pricing ex- ists, and the true cost depends on numerous factors, including hard- ware, energy consumption, and infrastructure overhead. Zhuang et al. [16]propose a practical method for translating task execution time into hardware cost. They first estimate an hourly GPU cost as the sum of three components—depreciation, power consump- tion, and maintenance—parametrised by the GPU purchase price, depreciation period, utilisation rate, average power draw, data cen- ter power usage effectiveness, electricity price, and maintenance rate. According to Zhuang et al. [16], the total cost of an inference workload is then obtained by multiplying this hourly rate by the total execution time. 퐶 GPU,hour ≈ 퐶 depreciation +퐶 power +퐶 maintenance (1) 3 Experiments To investigate the effectiveness of the reasoning in RLVR, we train models with Group Relative Policy Optimisation (GRPO) [13], with and without reasoning traces. This is achieved by providing a re- ward for the structure of the response where the thinking process must be put inside the<think></think>tag while the final trans- lation is within the<translation></translation>tag, such that it can be extracted easily. Disabling the reasoning is enforced by requiring the thinking tag to be empty, and for efficiency reasons, the empty tag is prefilled when sampling responses for the GRPO objective during training. The reward for the translation quality uses the chrF score after being extracted from the translation tag. We fine-tune Qwen3.5 [15], specifically the 4B and 9B parameter variants, on theSwiLTra-Benchdataset using QLoRA [5] with a group size of퐺=4, i.e. four different responses are sampled for each input sentence. As the cost effectiveness is one of our topics of interest, we save a checkpoint of the model after every 100 training samples in order to assess the impact of the amount of training data, and therefore training time, in relation to the translation quality. In order to minimise training and inference times we resort to optimised libraries, namelyunsloth 1 for training and vLLM [9] for inference. 2 1 https://github.com/unslothai/unsloth 2 The code used for the experiments is available in the GitHub repository https://github.com/aixiuxiuxiu/Legal-MT-SFT-RL The Price of Reasoning: Cost-Quality Tradeoffs in Reinforcement Learning for Neural Machine TranslationDocEng ’26, August 25–28, 2026, Fribourg, Switzerland Table 1: Result for the best models on the SwiLTra-Bench test set. Each model has been trained with (✓) and without (✗) thinking and subsequently evaluated by enabling or disabling thinking during inference. The estimated costs for the training and the inference of the complete test set are given alongside the number of generated tokens to make the overhead of the thinking quantifiable. ThinkingMetricsCostOutput ModelTrainingInferencechrF↑COMET↑METEOR↑MetricX↓TrainingInferenceTokens Qwen3.5 4B ✗53.79± 0.1079.09± 0.0750.11± 0.124.94± 0.02 $5.46 $0.07 1.08M ✗✓55.54± 0.1081.05± 0.0752.35± 0.124.18± 0.02$0.8619.85M ✓✗52.69± 0.1178.61± 0.0848.14± 0.135.01± 0.02 $12.56 $0.101.90M ✓56.33± 0.1082.05± 0.0652.13± 0.123.91± 0.02$0.248.08M Qwen3.5 9B ✗55.81± 0.1081.66± 0.0653.42± 0.124.03± 0.02 $4.54 $0.091.05M ✗✓56.90± 0.1282.32± 0.0854.48± 0.14 3.73± 0.02$1.0320.67M ✓✗54.60± 0.1080.80± 0.0750.96± 0.134.19± 0.02 $11.61 $0.192.00M ✓57.51± 0.10 82.50± 0.06 54.61± 0.123.78± 0.02$0.286.27M 4 Results The translation quality is evaluated with the following four metrics: chrF [11], METEOR [2], COMET [12], and MetricX [8]. We put most weight behind the COMET score, as An et al. [1]have found the strongest correlation with the judgement of human experts on the SwiLTra-Bench dataset. Since the thinking can be turned on and off during training or inference independently of each other, we compare all possible permutations and indicate within brackets whether thinking was used during training (T) and during inference (I) with a check mark (✓) if it was enabled or a cross (✗) if disabled. For example,[T:✓, I:✗] means, the model was trained with thinking but the inference was performed without thinking. 4.1 Number of Training Samples Figure 1 depicts the COMET score on a subset of 4 000 validation samples for the models trained in increments of 100 training sam- ples. The left plot contains the models that have been trained with thinking enabled, but instead of evaluating them only in the same setting, the inference has equally been run with the thinking turned off (indicated by the dashed lines). Similarly, the right plot evaluates both scenarios but for the models that have been trained without thinking. It can be observed that enabling thinking during inference has a positive effect on translation quality, regardless of whether the model has been trained with thinking. The difference is much more noticeable for the smaller, 4B parameter model. Importantly, train- ing without thinking, which enforces an empty<think></think>, does not destroy the thinking ability that the base model exhibits, which is consistent with the effects observed in SFT vs. RL compar- isons [4, 7]. For this particular dataset, between 1 000 to 1 500 translation pairs are adequate to reach a plateau. While increasing the number of training samples beyond the depicted range could improve the overall results, it is unlikely that the improvements are dispropor- tionally greater than the additional computational demands. The same can be said for the extra tokens that need to be generated for the reasoning during training, hence it is important to take the costs into account. 4.2 Cost-Quality Tradeoff Following Zhuang et al. [16], we estimate the running cost of the models based on the total execution time with a standardised cost of $0.79/hour. While the real cost depends on the specific hardware as well as the electricity cost, it gives a reasonable approximation. For each of the trained models, we select the best checkpoints and run the inference on theSwiLTra-Benchtest set, consisting of roughly 18.1k sentence pairs. The results are summarised in Table 1 including the estimated costs for the training and inference. When it comes to cost, the inference of Qwen3.5 9B is more expensive, as expected due to the higher parameter count, with differences ranging from $0.02 up to $0.17. The gap widens as the number of output tokens increases, which reveals the primary inference bottleneck of being limited by the memory bandwidth in the autoregressive generation process. A considerable difference can be observed in the generated out- put tokens. When the model is trained with thinking, [T:✓], the reasoning makes the responses 2−3×longer, although it also boosts the translation quality, such as the COMET score by 2−4 percent- age points. The starkest difference however, is during inference with thinking enabled. When the models are trained without think- ing, the output tokens shoot up from 8.08M to 19.85M (+145%) for Qwen3.5 4B, and from 6.27M to 20.67M (+230%) for Qwen3.5 9B, compared to the same inference if they were trained with thinking. These additional reasoning tokens do not result in a better trans- lation and are in fact off by slightly over one percentage point in regards to the COMET score. Therefore, the substantial increase in inference cost is not justified. The best result is achieved by Qwen3.5 9B [T:✓, I:✓], which incorporates thinking into both the training and inference, with a COMET score of 82.50. Enabling thinking during inference con- sistently improves the translation quality, regardless of whether the model has been trained to produce thinking traces. Training with thinking leads to marginal improvements at increased training cost. However, the aforementioned difference in output tokens tips the scale firmly in favour of training with thinking, whose most important effect seems to be taming the thinking process. Training is often considered a one-time fixed cost, as the same model is expected to be used for many more inference requests DocEng ’26, August 25–28, 2026, Fribourg, SwitzerlandJungo et al. across a longer time span, which surpasses the training cost fairly quickly, hence the inference cost becomes the deciding factor. Fig- ure 2 shows the COMET scores of all models in relation to the inference cost, including a comparison with commercial models with the pricing for their API requests on the same test set. The models on the Pareto frontier represent the best tradeoff between inference cost and translation quality. 0.1110100 78 80 82 84 86 Qwen3.5 4B [T:✓, I:✓] Qwen3.5 4B [T:✓, I:✗] Qwen3.5 4B [T:✗, I:✓] Qwen3.5 4B [T:✗, I:✗] Qwen3.5 9B [T:✓, I:✓] Qwen3.5 9B [T:✓, I:✗] Qwen3.5 9B [T:✗, I:✓] Qwen3.5 9B [T:✗, I:✗] OpenAI o3 OpenAI 4o OpenAI o3-mini OpenAI o4-mini DeepSeek-R1 Mistral-Small-2503 Inference Cost (US$) Mean COMET Score Local Models Pareto Frontier Commercial Models Figure 2: Reasoning Pareto frontier. Each point depicts the model’s translation quality in relation to its inference cost. The rea- soning configuration is specified as[T:✓, I:✓], for whether think- ing was employed during training and inference, respectively. The marker size is proportional to the parameter count of the model. 5 Conclusion The experiments conducted in this paper showed that including the model’s reasoning in the output has a positive effect on the over- all translation quality. By adding the reasoning to the output, the model produces significantly more tokens, which result in longer generation times and therefore increased operating costs. To min- imise the excess of tokens, the thinking needs to be integrated into the training. Even though we did not impose any restrictions on the reasoning traces, letting the model generate responses with reasoning for the GRPO objective keeps them more compact, with a reduction in tokens of up to 70% for Qwen3.5. The cost-quality tradeoff favours the models that preserve the thinking mode be- tween training and inference, i.e. either no thinking at all or always thinking. While the training costs can be reduced by disabling the thinking mode during training, the inference costs quickly surpasses the total costs if the thinking is going to be enabled after the fact. A much more impactful lever is the number of training samples used during the training. Within the context of machine translation in the legal domain, or more specifically for theSwiLTra-Benchdataset, limiting the training to include around 1 500 training samples is sufficient to reach the plateau after which every small improvement incurs a disproportionally large cost. Therefore it would likely be more beneficial to curate a set of high quality training samples. Acknowledgements The authors would like to thank the Hasler Foundation and the Mercator Foundation Switzerland for their financial support. References [1]Aixiu An, Michael Jungo, Eloi Eynard, Mark Drenhaus, Andreas Fischer, Jean Hennebert, and Sébastien Rumley. 2026. Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoning. In AI for Law Workshop @ ICML 2026. https://openreview.net/forum?id=822yKEGxe2 [2] Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. 65–72. [3] Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. 2026. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? Advances in Neural Information Processing Systems 38 (2026), 57654–57689. [4] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Sergey Levine, and Yi Ma. 2025. SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training. In The Second Conference on Parsi- mony and Learning (Recent Spotlight Track). https://openreview.net/forum?id= d3E3LWmTar [5] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems 36 (2023), 10088–10115. [6]Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, et al.2025. R1-t1: Fully incentivizing translation capability in llms via reasoning learning. arXiv preprint arXiv:2502.19735 (2025). [7] Michael Jungo and Andreas Fischer. 2025. Rule-Based Reinforcement Learning for Document Image Classification with Vision Language Models. In International Conference on Document Analysis and Recognition. Springer, 292–309. [8]Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task. In Proceedings of the Ninth Conference on Machine Translation, Barry Haddow, Tom Kocmi, Philipp Koehn, and Christof Monz (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 492–504. https://aclanthology.org/2024.wmt- 1.35 [9] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (Koblenz, Germany) (SOSP ’23). Association for Computing Machinery, New York, NY, USA, 611–626. doi:10.1145/3600006.3613165 [10]Joel Niklaus, Jakob Merane, Luka Nenadic, Sina Ahmadi, Yingqiang Gao, Cyrill AH Chevalley, Claude Humbel, Christophe Gösken, Lorenzo Tanzi, Thomas Lüthi, et al.2025. Swiltra-bench: The swiss legal translation benchmark. arXiv preprint arXiv:2503.01372 (2025). [11]Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the tenth workshop on statistical machine translation. 392–395. [12]Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A Neural Framework for MT Evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, 2685–2702. doi:10.18653/v1/2020.emnlp-main.213 [13]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al.2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024). [14]Jiaan Wang, Fandong Meng, and Jie Zhou. 2026. Deeptrans: Deep reasoning translation via reinforcement learning. Transactions of the Association for Com- putational Linguistics 14 (2026), 47–63. [15]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al.2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025). [16]Boqin Zhuang, Jiacheng Qiao, Mingqian Liu, Mingxing Yu, Ping Hong, Rui Li, Xiaoxia Song, Xiangjun Xu, Xu Chen, Yaoyao Ma, et al.2025. Beyond Benchmarks: The Economics of AI Inference. arXiv preprint arXiv:2510.26136 (2025). Received 8 July 2026; accepted 20 July 2026