Paper deep dive
Query-aware routing for Cross-lingual performance gains in Encoders
Akshay Jain, Edward Kim
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multilingual encoders can exhibit reduced retrieval effectiveness when queries and relevant documents differ in language, despite strong same-language performance. We investigate whether Finnish and Swedish cross-lingual retrieval can improve while preserving an encoder's existing same-language performance and document index. We combine a query-only low-rank adapter, trained against frozen document embeddings, with deterministic routing based on query and index languages. Cross-language queries use the adapter, while same-language queries use the original encoder. SampoTron, our fine-tuned low-rank (LoRA) adapter alongwith the Nemotron-3-Embed-1B model, improves average retrieval quality across six English, Finnish, and Swedish directions from 0.241 to 0.291 in normalized discounted cumulative gain (nDCG) at rank ten, a 20.9% relative gain on a sampled financial benchmark. All six cross-lingual directions improve, and routing preserves the original same-language performance, including two full-corpus Finnish evaluations. The approach enables selective cross-language specialization with reusable document embedding vectors.
Tags
Links
- Source: https://arxiv.org/abs/2610.02875v1
- Canonical: https://arxiv.org/abs/2610.02875v1
Trouble viewing inline? Open PDF directly →
Full Text
34,631 characters extracted from source content.
Expand or collapse full text
Query-aware routing for Cross-lingual performance gains in Encoders Akshay Jain Affiliation: ConfidentialMind Edward Kim Affiliation: ConfidentialMind Abstract Multilingual encoders can exhibit reduced retrieval effectiveness when queries and relevant documents differ in language, despite strong same-language performance. We investigate whether Finnish and Swedish cross-lingual retrieval can improve while preserving an encoder’s existing same-language performance and document index. We combine a query-only low-rank adapter, trained against frozen document embeddings, with deterministic routing based on query and index languages. Cross-language queries use the adapter, while same-language queries use the original encoder. SampoTron11 1 https://huggingface.co/ConfidentialMind/SampoTron, our fine-tuned low-rank (LoRA) adapter alongwith the Nemotron-3-Embed-1B model, improves average retrieval quality across six English, Finnish, and Swedish directions from 0.241 to 0.291 in normalized discounted cumulative gain (nDCG) at rank ten, a 20.9% relative gain on a sampled financial benchmark. All six cross-lingual directions improve, and routing preserves the original same-language performance, including two full-corpus Finnish evaluations. The approach enables selective cross-language specialization with reusable document embedding vectors. 1 Introduction Multilingual organizations and public information services require access to relevant documents independently of the language used to formulate a query. For example, Finnish or Swedish queries may target English technical documentation, while English queries may target local-language public information. Cross-lingual retrieval addresses this query–document language mismatch by ranking documents according to relevance across language boundaries. In contrast, multi-monolingual retrieval uses a shared model across languages while restricting each query to documents in the same language. Dense retrievers encode queries and passages as vector representations, or embeddings, and rank passages by their similarity to the query embedding [6]. In retrieval-augmented generation, the retrieved passages provide evidence for response generation. Reranking is restricted to the retrieved candidate set, making initial retrieval quality a constraint on the evidence available downstream. Improved cross-lingual retrieval can support access to organizational and public-service documents without requiring users to reformulate queries in a second language. Multilingual pretraining and retrieval fine-tuning have substantially expanded language coverage [15, 19]. However, this does not ensure equally effective retrieval across language boundaries. Recent work documents cross-lingual misalignment and develops training objectives that improve it while limiting monolingual regressions [3, 7]. An addtional constraint that arises in deployed systems is that changing the document encoder requires computing the stored corpus vectors again. The practical problem is thus to improve cross-language retrieval while retaining both a useful same-language retriever and its existing document index. We address this problem through query-only adaptation and language-routed selection at inference. A low-rank adapter introduces trainable parameter updates into the frozen encoder and learns query representations against document embeddings produced by the original model. The proposed query-aware routing policy uses explicit query and index language metadata to activate the adapter when the languages differ. Consequently, a single cross-language adapter is able to serve all evaluated language pairs. Our principal implementation to achieve this, SampoTron, adapts Nemotron-3-Embed-1B [11]. We evaluate its performance over English, Finnish, and Swedish financial question-answer retrieval and full-corpus Finnish benchmarks. Our contributions can be outlined as follows: 1. A language-routed retrieval architecture that combines a query-only adapter with the original encoder and reuses the original document index. Its same-language score preservation follows directly from retaining the original embedding path. 2. A mathematical analysis of how contrastive learning directs adapted queries toward relevant, fixed document embeddings relative to distractors, together with the conditions under which routing preserves original rankings. 3. A performance evaluation of SampoTron demonstrating 20.9% mean cross-language gain on the sampled FIQA corpus, with all six language pairs improving and same-language scores preserved. Additionally, SampoHarrier applies the same architecture to microsoft/harrier-oss-v1-0.6b with a separately trained query adapter and also improves cross-language retrieval. These results support the architecture’s generalizability across encoder models through model-specific query-adapter training. Notation. We write q for a query, d for a document, and ℓq,ℓD _q, _D for query and index languages. Bold letters denote vectors. All retrieval vectors have unit Euclidean norm, so their inner product equals cosine similarity. The upright subscripts basebase and adaptadapt label quantities associated with the original and adapted encoders, respectively. The subscript RR denotes the routed system. Parameter sets are denoted separately by θ for frozen base parameters and ϕφ for trainable adapter parameters. Section 2 positions the proposed approach against the state-of-the-art. Section 3 presents the architecture and geometry. Sections 4 and 5 describe the experiments and results. Section 6 concludes the paper. 2 State of the art Multilingual representations and cross-lingual alignment. Multilingual E5 uses large-scale contrastive pretraining followed by supervised fine-tuning [15]. Qwen3 Embedding adapts pretrained Qwen3 language models for embedding and reranking tasks [19]. Its training pipeline also incorporates synthetic multilingual data. Both model families provide representations for multiple tasks and languages. Beyond multilingual coverage, targeted methods address the balance between monolingual and cross-lingual retrieval. Hybrid batch training combines both types of query–document pairs to improve performance in each setting [1]. Furthermore, Hong et al. [3] combine distributional alignment with contrastive retrieval training. Their evaluation also identifies language bias in mixed-language document pools. CLEAR uses English passages as a bridge between languages [7]. It combines reverse-direction contrastive training with a distribution-matching objective to improve cross-lingual retrieval while limiting English degradation. These methods seek to preserve monolingual performance through training. In contrast, our method preserves the original encoder and document vectors for same-language requests at inference. Adapters and fixed document indices. Low-rank adaptation (LoRA) freezes a pretrained model and learns small matrix updates [4]. Its use in multilingual retrieval is well established in literature. Yang et al. [16] study language and retrieval adapters for zero-shot cross-language transfer. They show that changing language adapters at inference can create alignment problems. Additionally, Jina-embeddings-v3 provides task-specific LoRA adapters, including separate adapters for query and passage encoding [12]. Query-only adaptation is also an explored area in the state-of-the-art. Specifically, Jain et al. [5] adapt query encoders while keeping document indices fixed. They evaluate this approach on enterprise retrieval and public benchmarks. Moreover, their multi-tenant system selects tenant-specific adapters over a shared base model. Contribution relative to this work. Our contribution combines these components through an explicit language-routing rule. We train one cross-language query adapter against document embeddings from a frozen encoder. The router selects this adapter when the query and index languages differ. Otherwise, it retains the vanilla encoder. Same-language preservation therefore follows from the inference policy, rather than from the adapted model’s ability to retain monolingual performance. Our evaluation covers Finnish and Swedish with English, including both Finnish–Swedish directions. Consequently, the results demonstrate gains over the selected base models on translated financial retrieval dataset, while maintaining base model performance on monolingual retrieval for Wikipedia dataset in Finnish. 3 Architecture and embedding alignment 3.1 A fixed document encoder and an adaptable query path Let FbaseF_base denote the original text encoder with frozen parameters θ. It includes a pooling operation that aggregates token representations into a single embedding. For SampoTron, PmP_m selects the first m=1024m=1024 coordinates of its 2048-dimensional output. The normalized representation is Ebase(x)=PmFbase(x)‖PmFbase(x)‖2,base(d)=Ebase(TD(d)),base(q)=Ebase(TQ(q)),E_base(x)= P_mF_base(x) P_mF_base(x) _2, _base(d)=E_base(T_D(d)), _base(q)=E_base(T_Q(q)), (1) where TQT_Q and TDT_D denote query and document preprocessing, respectively. Since, document encoding uses frozen weights, our query adaptation approach does not require rebuilding the inference index. Concretely, for a frozen weight matrix W∈ℝa×bW ^a× b, LoRA uses Wadapt=W+αrBA,A∈ℝr×b,B∈ℝa×r.W_adapt=W+ αrBA, A ^r× b, B ^a× r. (2) Only the parameter matrices A and B receive updates. The parameter set ϕφ collects these matrices across all adapted projections. The rank r controls each update’s information depth. We apply the updates to the attention and feed-forward projections. The resulting normalized query vector, adapt(q)q_adapt(q), depends on the frozen parameters θ and trainable parameters ϕφ. In contrast, documents always use the original weights θ. Figure 1 shows how training and inference share the frozen document encoder. Training Query encoder + LoRA Trainable adapter Document encoder Frozen base weights Contrastive loss Retrieval scores Positive vectors Negative vectors Frozen cache Inference Query language + index language Deterministic language rule Same language Base encoder Different languages Base + LoRA Similarity search Original document index The frozen encoder anchors document geometry. Only query representations are adapted. Figure 1: Training and inference for language-routed query adaptation. Training updates only the query adapter using positive and negative document vectors from the frozen encoder. The dashed arrow denotes adapter updates. At inference, query and index language labels select the original or adapted query path. Both paths share one base model and search the same original document index. 3.2 Contrastive objective and query–document alignment The training objective promotes higher query similarity to relevant documents than to competing documents. For query i, let i+d_i^+ denote its positive document vector. The candidate set iC_i contains the batch’s positive and hard-negative vectors. Here, hard negatives receive high lexical-ranking scores but are not labeled relevant. For a batch of N queries, inverse temperature β=20β=20 controls the concentration of the softmax distribution. The objective function, as defined in [14], ℒL is expressed as: ℒ=1N∑i=1Nℒi,ℒi=−logexp(βi⊤i+)∑j∈iexp(βi⊤j),i=adapt(qi).L= 1N _i=1^NL_i, _i=- ( _i d_i^+) _d_j _i ( _i d_j), _i=q_adapt(q_i). (3) A full batch contains 256 positive and 256 explicit negative vectors. Additionally, other rows’ positives serve as negatives for each query. Gradient caching computes this large-batch loss and replays smaller query batches for backpropagation [2]. For unit vectors, ‖i−j‖22=2−2i⊤j _i-d_j _2^2=2-2q_i d_j. Consequently, the same loss can be written as ℒi=log∑j∈iexp[β2(‖i−i+‖22−‖i−j‖22)].L_i= \! _d_j _i \! [ β2 ( _i-d_i^+ _2^2- _i-d_j _2^2 ) ]. (4) Reducing this objective favors a smaller distance to the relevant document relative to competing documents. Thus, relevance supervision can align a Finnish query with an English answer without requiring literal translations. The gradient makes the direction of the training signal explicit. Let pijp_ij be the softmax weight assigned to candidate j: pij=exp(βi⊤j)∑kexp(βi⊤k),−∇iℒi=β(i+−∑jpijj).p_ij= ( _i d_j) _k ( _i d_k), - _q_iL_i=β (d_i^+- _jp_ijd_j ). (5) The positive vector contributes attraction, partly offset by its own softmax term. Other candidates contribute repulsion, with higher-scoring distractors receiving larger weights. To isolate the attractive component, consider the normalized step i(ϵ)=(i+ϵi+)/‖i+ϵi+‖2q_i(ε)=(q_i+ _i^+)/ _i+ _i^+ _2. Its local effect is dϵ‖i(ϵ)−i+‖22|ϵ=0=−2[1−(i⊤i+)2]≤0. . ddε _i(ε)-d_i^+ _2^2 |_ε=0=-2 [1-(q_i d_i^+)^2 ]≤ 0. (6) This local result describes the attractive component in embedding space. However, the complete training update also depends on candidate repulsion and the network’s parameterization. Note that, the frozen document vectors provide stationary targets throughout training. Geometrical understanding through the document subspace. For a given candidate set, define i=spanj:j∈iS_i=span\d_j:d_j _i\. This span contains all linear combinations of the candidate document vectors. The gradient in Equation 5 lies in this span. Normalization projects the correction onto the unit sphere’s tangent space at iq_i and scales it by the inverse pre-normalization norm. The projected direction changes orientation while preserving unit norm to first order. Let ih_i be the unnormalized query vector, with i=i/‖i‖2q_i=h_i/ _i _2. Then ∇iℒi=β‖i‖2(I−ii⊤)(∑jpijj−i+). _h_iL_i= β _i _2(I-q_iq_i ) ( _jp_ijd_j-d_i^+ ). (7) The loss therefore induces an angular correction determined by the relevant document and competing candidates. Backpropagation maps this correction to the trainable low-rank parameters. This gradient explains how the objective promotes alignment with document geometry. However, a language’s corpus need not occupy a distinct linear subspace. Its full span can be all of ℝmR^m. Moreover, the objective does not guarantee that every update reduces the distance to every positive document. Candidate repulsion, shared parameters, and limited adapter capacity affect the actual update. Subspace alignment, from a training perspective, is indicative of geometric agreement and not an absolute measure of convergence. 3.3 Language routing and same-language preservation The routing policy determines when to use the adapted query representation. It receives explicit query and index language labels and selects g(ℓq,ℓD)=[ℓq≠ℓD],R=(1−g)base+gadapt,sR(q,d)=R⊤base(d).g( _q, _D)=1[ _q≠ _D], _R=(1-g)q_base+gq_adapt, s_R(q,d)=q_R d_base(d). (8) The binary gate g selects either baseq_base or adaptq_adapt without interpolation. The base scoring function is sbase(q,d)=base(q)⊤base(d)s_base(q,d)=q_base(q) d_base(d). For same-language requests, ℓq=ℓD⟹sR(q,d)=sbase(q,d)for every indexed d. _q= _D\ \ s_R(q,d)=s_base(q,d) every indexed d. (9) This identity preserves the original ranking when preprocessing, document vectors, search settings, and tie-breaking remain unchanged. Consequently, every metric computed from that ranking is also preserved. The result is independent of the learned adapter parameters. However, it requires correct language metadata and faithful execution of the selected encoder. It does not imply that the original model is equally accurate across languages. The proposed approach requires one copy of the base model and an unmerged adapter. During serving, the adapter is enabled or disabled for each route. If requests share the adapter state, selection and encoding must form one uninterrupted operation. Furthermore, the evaluated policy assumes one known language per index. Code-switching and mixed-language document pools require a separate routing policy and evaluation, and are beyond the scope of this paper. 4 Experimental setup 4.1 SampoTron: Query - Routed LoRA Adapted Nemotron-3-1B-BF16 Embedder We disclose in this paper a rank-64 LoRA adapter called SampoTron. It uses NVIDIA’s Nemotron-3-Embed-1B-BF16 embedder as the base model22 2 Model: nvidia/Nemotron-3-Embed-1B-BF16. Full revision: a5e0f804b9e90a1ca6784ecbf6e41595774fc834.. The model uses average pooling and supports a normalized 1024-dimensional prefix of its native output [11]. We compare the original model, the adapter applied to every query (ungated), and the routed system. All three configurations use the same original document vectors. This comparison also assists in separating the adapter’s effect from the routing policy’s own contribution. Training data. The training mixture contains 113,043 query–positive–negative rows. These comprise 76,377 FIQA rows, 25,674 MIRACL-derived rows, and 10,992 Mr. TyDi-derived rows [8, 13, 18, 17]. The FIQA examples come from its training split. Each row contains one relevant passage and one explicit hard negative. The FIQA portion has 12,729 rows in each of five cross-language directions and 12,732 in Swedish-to-English. However, the full mixture is not balanced across directions. The Finnish datasets add proportionally more English-to-Finnish examples. The final counts are 28,041 (en→ ), 16,977 (en→ ), 16,914 (fi→ ), 16,956 (fi→ ), 17,076 (sv→ ), and 17,079 (sv→ ). Cross-language training rows include machine-translated Finnish and Swedish text. For retrieval against English passages, we reused FIQA training examples containing Finnish and Swedish queries translated using NLLB-200 [10]. For additional translations a draft-and-refinement pipeline was utilized, wherein the drafting and refinement models were glm-5-3-flash and inferact-glm53-nvfp4, respectively. These models were deployed on an on-premises cluster consisting of NVIDIA B300 GPUs. Further, we also mine new FIQA hard negatives with BM25 lexical ranking in the document language. A translated query retrieves the top-50 candidates, and deterministic cycling selects eligible negatives. Positive–negative text duplicates are filtered. Additionally, rows sharing a query can contain different relevant passages. During training-data preparation, we exclude any FIQA training example whose query identifier or query text matches a query in the FIQA test set. MIRACL-derived rows originate from the training split. Although evaluation queries are excluded from training, some passages in the evaluation corpus may also appear in the training data as passages that are positives or negatives for a given query in training might be utilized as positive or negative for another query in evaluation phase. Training setup. Table 1 presents the training settings utilized for developing the LoRA adapter. The training workload used one NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB physical memory. The run imposed an 18 GiB allocator cap. Note that, while the training settings only mention LoRA rank 6464, in the actual training runs other LoRA ranks were also used. However, for the sake of brevity, training parameters and results for the most optimal SampoTron settings are reported in this paper. The target projections are the internal linear layers receiving LoRA updates: q, k, v, and o denote the attention query, key, value, and output projections. Additionally, gate, up, and down denote the feed-forward gating, expansion, and contraction projections. Early stopping monitors the trailing 20-step mean training loss from epoch 2.2. Training stops at a checkpoint boundary after 60 optimizer steps without at least a 1% relative decrease from the monitor’s best recorded mean. Table 1: SampoTron training configuration. Only LoRA parameters are trainable. The document cache contains 90,490 unique texts encoded by the frozen base model. Setting Value LoRA rank / scaling / dropout 6464 / α=128α=128 / 0.050.05 Target projections q, k, v, o, gate, up, down Optimizer / learning rate / weight decay AdamW / 2×10−52× 10^-5 / 0.010.01 Effective / replay batch size 256 / 16 Inverse temperature / served dimensions 20 / 1024 Maximum sequence length / random seed 1024 tokens / 42 Schedule / warmup / epoch budget Cosine / 10% / 3 Embedder weight precision bfloat16 Early stopping 20-step mean, 1% improvement, 60-step patience Monitoring start / stopping boundary Epoch 2.2 / checkpoint boundary 4.2 Evaluation methodology and metrics Cross-lingual Financial retrieval. We evaluate 194 questions selected from FIQA’s 648 test questions by a random shuffle with a fixed seed, corresponding to approximately 30% of the test set. Each question is available in English, Finnish, and Swedish. Queries in each language search passages in the same language and in each of the other two languages. This gives three same-language and six cross-language tests, each using the same 194 questions. Each test searches a collection of 14,330 passages drawn from FIQA’s 57,638-passage corpus. The collection contains all 1,706 passages labeled relevant across the full set of 648 test questions, plus 12,624 additional distractor passages. These distractors were selected using BM25 lexical ranking, pooling the top-30 non-relevant passages per test question and removing duplicates. They provide challenging alternatives with lexical similarity to the questions. The same passage identifiers are used for the English, Finnish, and Swedish versions of this collection. Finnish-only retrieval. We evaluate Finnish-only retrieval using questions originally written in Finnish from the MIRACL-fi and Mr. TyDi-fi datasets. MIRACL-fi uses 1,271 development questions to search 1,883,509 Finnish passages. Mr. TyDi-fi uses 1,738 development questions to search 1,908,757 Finnish passages. Both evaluations search the complete passage collections rather than sampled subsets. Scoring. We rank documents by the dot product of normalized embeddings. The main metric is normalized discounted cumulative gain at rank ten (nDCG@10). It rewards relevant passages near the top of the list and normalizes the score against an ideal ranking. We treat each passage as either relevant or not relevant to a query. For each query-language and document-language pair, we report the average score across all evaluated queries. We then report equal-weight means across the six cross-language or three same-language directions. 5 Results and discussion 5.1 Cross-language gains in the final system Figure 2 reports the complete language matrix. SampoTron increases the six-direction cross-lingual mean from 0.2406 to 0.2908 nDCG@10. This corresponds to an absolute gain of 0.0502 and a relative gain of 20.9%. Moreover, every cross-language direction improves. Finnish-to-English rises from 0.285 to 0.382, while Finnish-to-Swedish rises from 0.146 to 0.209. Swedish-to-English remains the highest-scoring cross-language direction, increasing from 0.370 to 0.436. English-to-Swedish has the smallest absolute gain, from 0.303 to 0.315. Figure 2: Final SampoTron evaluation on 194 query identities per direction and a 14,330-passage FIQA corpus sample. Cells show nDCG@10. Rows indicate query language and columns indicate index language. The routed matrix follows the language rule: off-diagonal cells use measured adapter scores, while diagonal cells retain original scores. The right panel gives routed-minus-base differences. All six cross-language directions improve. en: English, fi: Finnish, sv: Swedish. These results show improved cross-language ranking of financial passages. Additionally, we evaluated the same query-routed architecture on microsoft/harrier-oss-v1-0.6b [9]. This implementation uses a separately trained rank-32 adapter. Its six-direction cross-lingual financial retrieval (FIQA dataset) mean rises from 0.278 to 0.334, approximately 20%. Language routing retains the original same-language FIQA mean of 0.384. It also retains the Finnish monolingual performance scores of 0.723 on MIRACL and 0.709 on Mr. TyDi. Together, the SampoTron and SampoHarrier results support the generalizability of the architecture across encoder models, i.e., separately trained query adapters improve cross-language retrieval for both evaluated models. 5.2 Preserving multi-monolingual performance through routing Table 2 separates the effects of query adaptation from those of language routing. Applying the adapter to every query slightly increases mean same-language FIQA performance. However, this mean combines an English-to-English decrease with Finnish-to-Finnish and Swedish-to-Swedish improvements. Larger decreases occur on the native Finnish benchmarks. Ungated MIRACL-fi falls from 0.756 to 0.675, and Mr. TyDi-fi falls from 0.728 to 0.665. Thus, query-only training does not by itself preserve native Finnish performance. Table 2: nDCG@10 for the original model, the final adapter applied to all queries, and SampoTron with language routing. FIQA values average six cross-language or three same-language directions on the sampled corpus. The Finnish development evaluations search the full corpora. Routed same-language values follow the original scoring path and are inherited from the base measurements. Configuration FIQA cross FIQA same MIRACL-fi Mr. TyDi-fi Original Nemotron-3-Embed-1B 0.2406 0.3303 0.7556 0.7284 Query adapter, ungated 0.2908 0.3361 0.6746 0.6648 SampoTron, routed 0.2908 0.3303 0.7556 0.7284 In contrast, routing preserves every original diagonal score and both native-Finnish scores. Same-language requests select the original encoder, as established in Equation 9. Cross-language requests retain the adapted-query gains. The routed system therefore combines improved cross-language retrieval with unchanged same-language rankings on these benchmarks. 5.3 Deployment implications and limitations The SampoTron adapter occupies approximately 84 MB. It reuses the original document index, avoiding corpus re-encoding and duplicate storage for the adapted route. The architecture therefore adds a specialized query path without duplicating document representations. The evaluation covers Finnish and Swedish with English, not other Nordic languages. Additionally, native Swedish has no dedicated benchmark in this study. Translated FIQA queries and passages may share systematic translation artifacts. As a result, the gains may not fully represent naturally authored queries. Moreover, in this paper we do not measure the sensitivity to incorrect language labels, downstream answer quality, organizational productivity, or public-service outcomes. The demonstrated benefit is cross-lingual retrieval improvement with preserved same-language behavior and reusable document vectors. 6 Conclusions Query-aware routing supports cross-language specialization while preserving the original same-language retrieval path. A query-only low-rank adapter learns relevance against fixed document representations. A deterministic language rule then selects when to use the adapter. On the sampled FIQA benchmark, SampoTron increases mean cross-language nDCG@10 from 0.241 to 0.291. All six English–Finnish–Swedish directions improve, while the original same-language and native-Finnish scores remain unchanged. Additionally, SampoHarrier shows that the architecture can be applied to another base model with a separately trained query adapter to obtain cross-language gains. The approach also reuses the base model and its document index. The reported cross-language gains are demonstrated on the sampled FIQA financial-retrieval benchmark. Future work should evaluate naturally authored Swedish queries, additional Nordic languages, broader domains, and mixed-language corpora. Future work could combine the proposed routing architecture with other cross-lingual training objectives to assess whether they provide additional retrieval gains while preserving same-language performance. Acknowledgments and Disclosure of Funding We acknowledge the contributions of Lucas Björklund, Dan Kozytskyi, Henri Dahl, Nico Roos and Sakari Tanskanen to this work. References [1] Adel Elmahdy, Sheng-Chieh Lin, and Amin Ahmad. Synergistic approach for simultaneous optimization of monolingual, cross-lingual, and multilingual information retrieval. arXiv:2408.10536, 2024. [2] Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan. Scaling deep contrastive learning batch size under memory limited setup. In Proceedings of the Sixth Workshop on Representation Learning for NLP, 2021. URL https://arxiv.org/abs/2101.06983. [3] Seongtae Hong, Youngjoon Jang, Jungseob Lee, Hyeonseok Moon, and Heuiseok Lim. Improving semantic proximity in information retrieval through cross-lingual alignment. In International Conference on Learning Representations, 2026. URL https://proceedings.iclr.c/paper_files/paper/2026/hash/4e5432baa6f217e24f9a4b5921cad30b-Abstract-Conference.html. [4] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://arxiv.org/abs/2106.09685. [5] Prateek Jain, Shabari S Nair, Ritesh Goru, Prakhar Agarwal, Ajay Yadav, Yoga Sri Varshan Varadharajan, and Constantine Caramanis. Succeeding at scale: Enterprise retrieval benchmark construction and index-preserving query adaptation for multi-tenant search. In ICML Workshop on Connecting Low-rank Representations in AI, 2026. doi: 10.48550/arXiv.2601.04646. arXiv:2601.04646v4. [6] Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of EMNLP, pages 6769–6781, 2020. doi: 10.18653/v1/2020.emnlp-main.550. [7] Seungyoon Lee, Minhyuk Kim, Seongtae Hong, Youngjoon Jang, Dongsuk Oh, and Heuiseok Lim. CLEAR: Cross-lingual enhancement in retrieval via reverse-training. In Proceedings of ACL (Volume 1: Long Papers), pages 347–362, 2026. doi: 10.18653/v1/2026.acl-long.13. [8] Macedo Maia, Siegfried Handschuh, André Freitas, Brian Davis, Ross McDermott, Manel Zarrouk, and Alexandra Balahur. W’18 open challenge: Financial opinion mining and question answering. In Companion Proceedings of The Web Conference, pages 1941–1942, 2018. doi: 10.1145/3184558.3192301. [9] Microsoft. harrier-oss-v1-0.6b model card. Hugging Face, 2026. URL https://huggingface.co/microsoft/harrier-oss-v1-0.6b. Accessed September 15, 2026. [10] NLLB Team. No language left behind: Scaling human-centered machine translation. arXiv:2207.04672, 2022. [11] NVIDIA. Nemotron-3-Embed-1B-BF16 model card. Hugging Face, 2026. URL https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16. Accessed September 15, 2026. Experiments use the pinned revision specified in Section 4. [12] Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao. jina-embeddings-v3: Multilingual embeddings with task LoRA. arXiv:2409.10173, 2024. [13] Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In NeurIPS Datasets and Benchmarks, 2021. URL https://arxiv.org/abs/2104.08663. [14] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv:1807.03748, 2018. [15] Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilingual E5 text embeddings: A technical report. arXiv:2402.05672, 2024. [16] Eugene Yang, Suraj Nair, Dawn Lawrie, James Mayfield, and Douglas W. Oard. Parameter-efficient zero-shot transfer for cross-language dense retrieval with adapters. arXiv:2212.10448, 2022. [17] Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. Mr. TyDi: A multi-lingual benchmark for dense retrieval. In Proceedings of the First Workshop on Multilingual Representation Learning, 2021. URL https://arxiv.org/abs/2108.08787. [18] Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. MIRACL: A multilingual retrieval dataset covering 18 diverse languages. Transactions of the Association for Computational Linguistics, 11:1114–1131, 2023. doi: 10.1162/tacl_a_00595. [19] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. Qwen3 Embedding: Advancing text embedding and reranking through foundation models. arXiv:2506.05176, 2025.