Paper deep dive
Resolving the Robustness-Precision Trade-off in Financial RAG through Hybrid Document-Routed Retrieval
Zhiyuan Cheng, Longying Lai, Yue Liu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/31/2026, 1:33:50 AM
Summary
The paper introduces Hybrid Document-Routed Retrieval (HDRR), a two-stage RAG architecture designed to resolve the robustness-precision trade-off in financial document question answering. By combining Semantic File Routing (SFR) to identify relevant documents with scoped chunk-based retrieval (CBR), HDRR eliminates cross-document chunk confusion while maintaining high precision, outperforming both baseline methods on the FinDER benchmark.
Entities (5)
Relation Signals (3)
HDRR → usesasfilter → SFR
confidence 100% · HDRR, a two-stage architecture that uses SFR as a document filter
HDRR → resolvestradeoffbetween → SFR and CBR
confidence 95% · HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision
FinDER → evaluates → HDRR
confidence 90% · Experimental results demonstrate that HDRR achieves the best performance on every metric
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Retrieval-Augmented Generation (RAG) systems for financial document question answering typically follow a chunk-based paradigm: documents are split into fragments, embedded into vector space, and retrieved via similarity search. While effective in general settings, this approach suffers from cross-document chunk confusion in structurally homogeneous corpora such as regulatory filings. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices the precision of targeted chunk retrieval. We identify this robustness-precision trade-off through controlled evaluation on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document confusion while preserving targeted chunk precision. Experimental results demonstrate that HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a failure rate of only 6.4%, a correctness rate of 67.7% (+18.7 pp over CBR), and a perfect-answer rate of 20.1% (+6.3 pp over CBR, +11.6 pp over SFR). HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision across all five experimental groups.
Tags
Links
- Source: https://arxiv.org/abs/2603.26815v1
- Canonical: https://arxiv.org/abs/2603.26815v1
Trouble viewing inline? Open PDF directly →
Full Text
73,935 characters extracted from source content.
Expand or collapse full text
Resolving the Robustness-Precision Trade-off in Financial RAG through Hybrid Document-Routed Retrieval Zhiyuan Cheng a,∗ , Longying Lai b and Yue Liu c a School of Engineering, Stanford University, Stanford, CA, USA b Simon Business School, University of Rochester, Rochester, NY, USA c Accounting & Information Systems, Rutgers University, Newark, NJ, USA A R T I C L E I N F O Keywords: Retrieval-Augmented Generation Financial Document Analysis Question Answering Hybrid Retrieval Document Routing 10-K Reports A B S T R A C T Retrieval-Augmented Generation (RAG) systems for financial document question answering typically follow a chunk-based paradigm: documents are split into fragments, embedded into vector space, and retrieved via similarity search. While effective in general settings, this approach suffers from cross-document chunk confusion in structurally homogeneous corpora such as regulatory filings. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices the precision of targeted chunk retrieval. We identify this robustness-precision trade-off through controlled evaluation on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document confusion while preserving targeted chunk precision. Experimental results demonstrate that HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a failure rate of only 6.4%, a correctness rate of 67.7% (+18.7 p over CBR), and a perfect-answer rate of 20.1% (+6.3 p over CBR, +11.6 p over SFR). HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision across all five experimental groups. 1. Introduction 1.1. Background and Motivation Financial 10-K reports are comprehensive annual filings mandated by the U.S. Securities and Exchange Commis- sion (SEC) for all publicly traded companies. A typical 10-K report for a large corporation spans 100–300 pages and encompasses audited financial statements, management discussion and analysis (MD&A), risk factor disclosures, and supplementary schedules. For the S&P 500 index alone, these filings collectively represent tens of thousands of pages of dense, domain-specific text that analysts, investors, and regulators must navigate to extract actionable information. Recent advances in generative AI have broadened the scope of automated analysis across such complex, real-world do- mains (Yao, Zhan, Cao, Li, Lin, Shao, Wang, Wang, Zhang, Zhang et al., 2025). Retrieval-Augmented Generation (RAG) (Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, tau Yih, Rocktäschel, Riedel and Kiela, 2020) has emerged as a lead- ing paradigm for building question-answering (QA) systems over such large document collections. The standard RAG pipeline operates in two phases: an offline indexing phase that splits documents into chunks, computes embeddings, and stores them in a vector database; and an online query phase that retrieves the most relevant chunks for a given question and feeds them as context to a large language ∗ Corresponding author zhycheng@stanford.edu (Z. Cheng) ORCID(s): 0009-0003-2845-5322 (Z. Cheng) model (LLM) for answer generation. This chunk-based ap- proach has proven effective across many domains (Gao, Xiong, Gao, Jia, Pan, Bi, Dai, Sun and Wang, 2023; Izac- ard, Lewis, Lomeli, Hosseini, Petroni, Schick, Dwivedi-Yu, Joulin, Riedel and Grave, 2023; Siriwardhana, Weerasekera, Wen, Kaluarachchi, Rana and Nanayakkara, 2023). However, applying chunk-based RAG to corpora of structurally homogeneous documents reveals a class of failure modes that are underexplored in the literature. Fi- nancial regulatory filings such as 10-K reports share stan- dardized section headings, boilerplate language, and tabular formats mandated by SEC regulations. When hundreds of such documents are chunked and indexed together, struc- turally similar sections from different companies become nearly indistinguishable in embedding space. A query about “Apple’s revenue recognition policy” may retrieve chunks from Microsoft’s or Google’s 10-K report because all three companies use similar regulatory language in their revenue recognition disclosures. We term this phenomenon cross- document chunk confusion, and it represents a fundamental limitation of chunk-level retrieval in homogeneous corpora. 1.2. Proposed Approach This paper first introduces Semantic File Routing (SFR), an alternative RAG paradigm designed for structurally reg- ular document corpora. Instead of chunking and indexing documents, SFR leverages the ability of modern LLMs to produce structured output (OpenAI, 2024a) to resolve document identity directly from the natural language query. Given a query, the LLM extracts document-identifying Z. Cheng et al.: Preprint submitted to ElsevierPage 1 of 18 arXiv:2603.26815v1 [cs.CL] 26 Mar 2026 Hybrid Document-Routed Retrieval for Financial RAG metadata (in our financial domain, the company ticker symbol and fiscal year) as a structured JSON object. These metadata fields are then mapped to a file path in a pre- organized repository (e.g., year/ticker.pdf), and the entire document is provided as context to the generation model. SFR eliminates cross-document confusion by construc- tion, but our comparative evaluation reveals a fundamental trade-off: SFR reduces catastrophic failures while chunk- based retrieval (CBR) achieves higher precision when it retrieves correctly. To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR’s document routing as a first- stage filter, followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document chunk confusion (the primary source of CBR failures) while preserving the precision advantage of targeted chunk re- trieval (which SFR’s full-document context dilutes). With a graceful fallback to full-corpus search when routing fails, HDRR combines the robustness of document-level routing with the precision of chunk-level retrieval. 1.3. Contributions This work makes the following contributions: 1. Trade-off Identification: Through controlled evalua- tion of CBR and SFR on 1,500 queries across five in- dependent groups from the FinDER benchmark (Lin- qAlpha, 2025), we identify and quantify a fundamen- tal robustness-precision trade-off between chunk-level and document-level retrieval paradigms. 2. Hybrid Document-Routed Retrieval: We propose HDRR, a two-stage architecture that uses LLM-based document routing to scope chunk-based retrieval, re- solving the identified trade-off. HDRR achieves the best performance on all four evaluation metrics, si- multaneously attaining the lowest failure rate and the highest answer precision. 3. Three-Paradigm Comparative Evaluation: We con- duct a rigorous three-way comparison of CBR, SFR, and HDRR using identical query sets, providing sta- tistically robust evidence that the hybrid approach dominates both baseline paradigms. 4. Generalizability Analysis: We formalize the corpus properties (naming regularity and structural homo- geneity) that determine when document-level routing is beneficial, and discuss applicability to structured- document domains beyond finance. 1.4. Paper Organization The remainder of this paper is organized as follows. Section 2 surveys related work on RAG systems, finan- cial document analysis, and LLM structured output. Sec- tion 3 formulates the problem and defines the three retrieval paradigms. Section 4 describes the architectures of all three systems in detail. Section 5 presents the experimental setup and evaluation methodology. Section 6 reports quantitative results. Section 7 provides in-depth analysis and discussion. Section 8 concludes with a summary and future research directions. 2. Related Work 2.1. Retrieval-Augmented Generation Retrieval-Augmented Generation (RAG), introduced by Lewis et al. (2020), augments language model generation with relevant passages retrieved from an external knowledge base. The core architecture comprises a retriever that iden- tifies relevant documents given a query and a generator that produces responses conditioned on both the query and the retrieved context. This paradigm has become a standard ap- proach for knowledge-intensive NLP tasks, with numerous variants proposed to improve retrieval quality, generation fidelity, and end-to-end performance (Gao et al., 2023). A comprehensive survey by Gao et al. (2023) catego- rizes RAG systems along three dimensions: indexing strate- gies (how documents are preprocessed and stored), retrieval strategies (how relevant passages are identified), and gen- eration strategies (how the LLM produces answers from retrieved context). Our work contributes to this taxonomy by introducing a fourth dimension: retrieval granularity, con- trasting chunk-level retrieval with document-level routing. 2.2. Hybrid Retrieval and Reranking Modern RAG systems frequently combine multiple re- trieval strategies to improve recall. Hybrid search approaches merge sparse retrieval methods such as BM25 (Robertson and Zaragoza, 2009) or full-text search with dense semantic retrieval using learned embeddings (Karpukhin, Oguz, Min, Lewis, Wu, Edunov, Chen and tau Yih, 2020; Wang, Yang, Huang, Yang, Majumder and Wei, 2024). Reciprocal Rank Fusion (RRF) (Cormack, Clarke and Büttcher, 2009) pro- vides an unsupervised method for combining ranked lists from different retrievers, assigning each document a fused score based on its ranks across retrieval methods. Neural reranking further refines retrieval quality by re- scoring candidates with cross-encoder models that jointly encode query-document pairs (Nogueira and Cho, 2019; Reimers and Gurevych, 2019). Cross-encoders capture fine- grained interaction patterns between queries and passages that bi-encoder models cannot, at the cost of increased com- putational overhead. Models such as Jina Reranker v2 (Jina AI, 2024) and ColBERT (Khattab and Zaharia, 2020) repre- sent the current state of the art in neural reranking. Our prior work (Cheng, Lai, Liu, Cheng and Qi, 2026) demonstrated that neural reranking yields a 15.5 percentage point improve- ment in answer correctness for financial RAG, establishing it as a critical component of the chunk-based pipeline that serves as the baseline in this study. The effectiveness of transformer-based dense embeddings for short-text retrieval has also been empirically examined across domains includ- ing e-commerce (Lai, Cheng, Cheng and Qi, 2026), pro- viding foundational validation for embedding-based chunk retrieval in specialized corpora. Z. Cheng et al.: Preprint submitted to ElsevierPage 2 of 18 Hybrid Document-Routed Retrieval for Financial RAG 2.3. Financial Document Analysis Financial document analysis has attracted significant attention from both the NLP and financial technology com- munities (Chen, Li and Chen, 2024; Loukas, Stogiannidis, Malakasiotis and Vassos, 2023). The FinDER dataset (Lin- qAlpha, 2025), introduced by LinqAlpha, provides 5,703 query-evidence-answer triplets derived from real-world fi- nancial inquiries across 10-K filings. The dataset reflects realistic analyst workflows with ambiguous, concise queries featuring domain-specific abbreviations and jargon. Its char- acteristics, including 84.5% qualitative questions, 15.5% quantitative questions (with half requiring multi-step reason- ing), and evidence often scattered across multiple document sections, make it a challenging benchmark for RAG systems. Prior work on FinDER reports that advanced models such as GPT-4-Turbo achieve only 9% accuracy on closed- book questions, demonstrating the necessity of retrieval- augmented approaches (LinqAlpha, 2025). Complementing FinDER, QuantEval (Kang, Gong, Hu, Yin, Jiang, Fang, He, Meng, Fu, Chen et al., 2026) provides a benchmark evaluat- ing LLMs across three dimensions of quantitative finance— knowledge-based QA, mathematical reasoning, and strategy coding—revealing substantial gaps between frontier models and human experts particularly in reasoning tasks. On the generative side, FinSentLLM (Zhang, Fu, He, Shen, Wang, Du, You, Shi and Fong, 2025) demonstrates that multi-LLM ensemble approaches can achieve consistent gains in finan- cial sentiment analysis, with structured semantic signals providing econometric evidence linking model predictions to market dynamics. Multi-hop reasoning in RAG systems has been explored through graph-based and tree-based re- trieval (Shi, Sun, Liu, Yang, Fang, Sun and Gu, 2026), and context pruning techniques such as AttentionRAG (Fang, Sun, Shi and Gu, 2025) aim to select only the most relevant passages from retrieved candidates. 2.4. LLM Structured Output A recent development in LLM capabilities is the abil- ity to generate outputs conforming to a specified schema, commonly referred to as structured output or constrained decoding (OpenAI, 2024a). By providing a JSON schema, developers can instruct the model to produce outputs that are guaranteed to parse correctly, enabling reliable extraction of structured information from unstructured text. Structured output has been applied to information ex- traction, data normalization, and tool use, but its application to retrieval, specifically to resolve document identity as a routing mechanism, has not been systematically studied. More broadly, prompt-engineered LLMs demonstrate strong few-shot classification capability across domain-specific tasks (Liu and Shi, 2024), achieving reliable extraction without task-specific retraining—a property directly ex- ploited by our SFR component. As frontier LLMs are increasingly deployed in structured real-world settings, their evaluation in realistic operational contexts has attracted growing attention (Wang, Tang, Qian and Zhao, 2026); our study contributes such an evaluation in the financial document QA domain, where structured output extraction directly determines retrieval quality. Our SFR approach leverages structured output as the core retrieval mechanism: the LLM extracts document-identifying metadata (ticker symbols and fiscal years) from natural language queries, effectively converting a retrieval problem into a structured classification problem. 2.5. Long-Context LLMs and Document-Level Reasoning The emergence of LLMs with extended context win- dows (GPT-4 supports 128K tokens (OpenAI, 2023), and newer models push even further) has reopened the ques- tion of whether chunk-based retrieval is always necessary. Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni and Liang (2024) demonstrated that LLMs struggle to utilize informa- tion in the middle of long contexts (the “lost in the middle” phenomenon), while Chen, Wong, Chen and Tian (2023) explored positional interpolation techniques for extending context windows. Jiang, Ma and Chen (2024) proposed LongRAG, which retrieves longer document segments rather than short chunks, demonstrating that coarser retrieval granularity can improve performance when combined with long-context models. Our SFR approach takes this concept to its logical extreme: retrieving the entire document, relying on the LLM’s long- context capabilities to process full financial filings (typically 50K–200K tokens after conversion). 2.6. Document Routing and Classification Document routing, defined as the task of directing a query to the most relevant document or subset of documents, has been studied in the information retrieval literature, tradi- tionally in the context of distributed search (Cormack et al., 2009). In a distributed search system, a routing algorithm determines which document shards are most likely to contain relevant results, avoiding exhaustive search across all shards. SFR adapts this concept by treating each document in the corpus as a “shard” and using LLM structured output as the routing function. Unlike traditional routing methods that rely on statistical models of shard contents, SFR ex- ploits the LLM’s world knowledge to map query entities (company names, ticker symbols) to file-system locations. This approach is most effective when the mapping between query entities and documents is predictable and consistent, a condition naturally satisfied by regulatory filing corpora. Routing mechanisms have also been studied in other AI- driven systems: for example, risk-aware dynamic routing in smart logistics (Xue, Zhao, Qi, Zeng and Yu, 2026) similarly dispatches tasks to appropriate processing paths based on predicted network conditions, demonstrating the generality of query-driven routing as a systems design principle across domains. 2.7. Multi-Stage and Cascaded Retrieval Multi-stage retrieval architectures decompose the re- trieval process into successive refinement steps. The clas- sic two-stage retrieve-then-rerank paradigm (Nogueira and Z. Cheng et al.: Preprint submitted to ElsevierPage 3 of 18 Hybrid Document-Routed Retrieval for Financial RAG Cho, 2019) uses a fast first-stage retriever to generate can- didates, followed by a computationally expensive cross- encoder reranker. More recent work extends this to three or more stages, incorporating document filtering, passage retrieval, and answer extraction as distinct pipeline steps. In distributed information retrieval, resource selection algorithms first identify the most relevant document collec- tions (“shards”) before performing within-shard retrieval (Cor- mack et al., 2009). This shard-selection approach is concep- tually related to our hybrid architecture, where document routing serves as a shard selector that restricts subsequent chunk retrieval to the identified document(s). However, traditional resource selection relies on statistical models of collection contents, whereas our approach leverages LLM structured output for zero-shot document identification based on query semantics. The idea of combining document-level and chunk-level retrieval has been explored in enterprise search systems, but systematic evaluation of such hybrid architectures for RAG, particularly with LLM-based routing as the first stage, remains limited. Recent work in LLM-based agentic systems has explored difficulty-aware orchestration (Su, Lan, Xia, Sun, Tian, Shi, Song, He and Jingsong, 2026), where pre- dicted query complexity dynamically determines workflow structure and which model handles each step. HDRR follows a related principle at the retrieval level: the routing stage classifies each query by whether document identity can be resolved, adapting the retrieval scope accordingly and falling back gracefully when routing confidence is low. 2.8. Gap Analysis While individual components of our approach (RAG, structured output, long-context LLMs, document routing, and multi-stage retrieval) have been studied independently, their integration into a coherent hybrid retrieval paradigm for structured-document corpora has not been explored. Existing RAG research overwhelmingly focuses on chunk- level retrieval and implicitly assumes that document-level context is either unnecessary or impractical. Our work chal- lenges this assumption by: (1) demonstrating that document- level routing outperforms chunk-level retrieval on aggregate metrics while introducing a robustness-precision trade-off, and (2) proposing a hybrid architecture that resolves this trade-off by combining document routing with scoped chunk retrieval. 3. Problem Formulation 3.1. Task Definition We address the task of financial document question answering: given a natural language query 푞 about a specific company’s financial performance, operations, or disclo- sures, and a corpus of financial documents = 푑 1 ,푑 2 ,...,푑 푁 , the system must produce a natural language answer 푎 that accurately and completely addresses the query based on information contained in . In our setting, the corpus consists of 10-K annual reports filed by S&P 500 companies. Each document 푑 푖 is associated with a company identifier (ticker symbol) 푡 푖 and a fiscal year 푦 푖 , yielding a bijective mapping: 푓 ∶ (푡,푦)↦ 푑 ∈ (1) where each (ticker, year) pair uniquely identifies a document. This structural regularity is a defining characteristic of the corpus and a prerequisite for the SFR paradigm. 3.2. Retrieval Paradigms We formally define and compare three retrieval paradigms that differ in their retrieval unit, mechanism, and scope. 3.2.1. Chunk-Based Retrieval (CBR) The chunk-based paradigm operates on sub-document units. During an offline indexing phase, each document 푑 푖 is partitioned into overlapping text segments: 푑 푖 → 푐 푖,1 ,푐 푖,2 ,...,푐 푖,푀 푖 (2) where 푐 푖,푗 denotes the 푗-th chunk of document 푑 푖 and 푀 푖 is the number of chunks. Each chunk is embedded into a vector spaceℝ 푛 via an embedding function 휙: 퐯 푖,푗 = 휙(푐 푖,푗 ) ∈ℝ 푛 (3) At query time, the system retrieves a set of top-푘 chunks via similarity search: CBR (푞) = top-k 푐 푖,푗 ∶ sim(휙(푞 ′ ),퐯 푖,푗 ) (4) where 푞 ′ is a rewritten version of query 푞 and sim(⋅,⋅) denotes a similarity function. The answer is then generated as: 푎 = LLM ( 푞, CBR (푞) ) (5) In our implementation, retrieval is further enhanced with full-text search, rank fusion, and neural reranking (detailed in Section 4.1). 3.2.2. Semantic File Routing (SFR) The SFR paradigm operates on whole documents and bypasses offline indexing entirely. At query time, an LLM with structured output extracts document-identifying meta- data from the query: LLM parse (푞)→ (푡 1 ,푦 1 ),(푡 2 ,푦 2 ),...,(푡 퐾 ,푦 퐾 ) (6) where each (푡 푘 ,푦 푘 ) is a (ticker, year) pair. These pairs are resolved to file paths via the mapping in Eq. (1): SFR (푞) = 푓(푡 푘 ,푦 푘 ) ∶ 푘 = 1,...,퐾 (7) The answer is generated with the full document(s) as context: 푎 = LLM ( 푞, SFR (푞) ) (8) Z. Cheng et al.: Preprint submitted to ElsevierPage 4 of 18 Hybrid Document-Routed Retrieval for Financial RAG 3.2.3. Hybrid Document-Routed Retrieval (HDRR) The hybrid paradigm combines document-level routing with chunk-level retrieval. It uses SFR’s metadata extraction (Eq. 6) to identify the target document(s), then restricts CBR’s chunk-based retrieval (Eq. 4) to only the chunks belonging to those documents: HDRR (푞) = top-k 푐 푖,푗 ∶ 푑 푖 ∈ SFR (푞), sim(휙(푞 ′ ),퐯 푖,푗 ) (9) When document routing fails (no documents resolved), HDRR falls back to unrestricted chunk retrieval over the full corpus, equivalent to standard CBR: fallback HDRR (푞) = CBR (푞)(10) The answer is generated from the scoped chunk context: 푎 = LLM ( 푞, HDRR (푞) ) (11) HDRR inherits SFR’s immunity to cross-document chunk confusion (by restricting retrieval to the correct document) while preserving CBR’s ability to surface precisely relevant text fragments (by performing chunk-level retrieval within the scoped document). 3.3. Corpus Structure Assumption SFR relies on two structural properties of the corpus: Definition 1 (Naming Regularity). A document corpus exhibits naming regularity if there exists a deterministic mapping 푔 ∶ → Paths from a metadata space to file-system paths, such that 푔 is injective and the metadata 푚 푖 ∈ for each document푑 푖 can be reliably extracted from natural language queries about 푑 푖 . Definition 2 (Structural Homogeneity). A document cor- pus exhibits structural homogeneity if the documents share a common organizational template, such that analo- gous sections across different documents contain semanti- cally similar but entity-distinct content. Financial regulatory filings naturally satisfy both prop- erties: 10-K reports follow SEC-mandated structures (nam- ing regularity via ticker and year), and all reports contain standardized sections such as “Risk Factors,” “MD&A,” and “Financial Statements” (structural homogeneity). Our hypothesis is that structural homogeneity creates retrieval confusion for chunk-based systems while naming regularity enables SFR to bypass this confusion entirely. 3.4. Evaluation Framework We evaluate both paradigms using an LLM-as-judge approach (Zheng, Chiang, Sheng, Zhuang, Wu, Zhuang, Lin, Li, Li, Xing, Zhang, Gonzalez and Stoica, 2023), where a separate LLM instance scores the generated answer푎 against a ground-truth reference answer 푎 ∗ on a scale of 1 to 10: 푠 = LLM eval (푞,푎,푎 ∗ ) ∈ 1,2,...,10(12) We report four complementary metrics: • Average Score: ̄푠 = 1 || ∑ 푞∈ 푠 푞 , reflecting overall answer quality. • Failure Rate: 푃(푠 = 1), measuring the proportion of completely incorrect answers. • Correctness Rate: 푃(푠 ≥ 8), measuring the propor- tion of substantively correct answers. • Perfect Rate:푃(푠 = 10), measuring the proportion of fully correct and complete answers. These metrics capture different aspects of system behav- ior: average score reflects central tendency, failure rate mea- sures worst-case performance, correctness rate measures practical utility, and perfect rate measures ceiling perfor- mance. 4. System Architectures This section describes the three RAG architectures com- pared in our study. All systems share the same generation model (GPT-4.1) and evaluation pipeline, differing only in how they retrieve relevant context for a given query. The first paradigm, Chunk-Based Retrieval (CBR), follows the canonical RAG design: documents are split into overlap- ping fragments, embedded into a shared vector space, and retrieved via hybrid search and neural reranking. The second paradigm, Semantic File Routing (SFR), bypasses chunking entirely by using LLM structured output to resolve document identity from the query and providing the whole document as context. The third paradigm, Hybrid Document-Routed Retrieval (HDRR), combines both: it uses SFR’s routing step to identify the target document, then applies CBR’s chunk- level retrieval scoped to that document. This progression— from unconstrained to document-scoped chunk retrieval— directly addresses the cross-document confusion problem identified in Section 1. 4.1. Chunk-Based RAG (CBR) The chunk-based system implements a full retrieval pipeline with offline indexing and online query processing, as shown in Fig. 1. It serves as the primary baseline in our study and represents the dominant paradigm for RAG over large document collections (Lewis et al., 2020; Gao et al., 2023). The pipeline is divided into two phases: an offline indexing phase that preprocesses the entire corpus once, and an online query phase that retrieves and generates answers at inference time. 4.1.1. Offline Indexing Pipeline Document Conversion. The 500 S&P 500 10-K reports are obtained in HTML format from company filings and converted to PDF using Playwright (Microsoft Corporation, 2024), a browser automation tool that renders HTML with proper formatting and layout preservation. This conversion ensures consistent text extraction across documents. Text Extraction and Chunking. PDF documents are parsed using PyMuPDF to extract textual content while pre- serving document structure. We employ fixed-size chunking Z. Cheng et al.: Preprint submitted to ElsevierPage 5 of 18 Hybrid Document-Routed Retrieval for Financial RAG with a chunk size of 2,500 characters and 50% overlap (1,250 characters). Fixed-size chunking was selected over seman- tic or recursive strategies for reproducibility and computa- tional efficiency across a 500-document corpus. The chunk size of 2,500 characters (approximately 500–600 tokens) is calibrated to fit comfortably within the input limit of the embedding model while retaining sufficient local context for meaningful semantic representation (Lai et al., 2026). The 50% overlap ensures that information spanning chunk boundaries—such as multi-paragraph tables, footnoted fi- nancial figures, or management commentary that continues across page breaks—is captured in at least one chunk with- out duplication artifacts. Dual-Index Storage. Extracted chunks are stored in two complementary index structures: • SQLite FTS5 (SQLite, 2024): A full-text search index supporting Boolean queries, phrase matching, and BM25-based relevance ranking for keyword retrieval. • FAISS (Johnson, Douze and Jégou, 2021): A vector in- dex storing 1,024-dimensional embeddings generated by OpenAI’s text-embedding-3-small model (Ope- nAI, 2024b), enabling nearest neighbor search for semantic retrieval. The FAISS index uses an exact inner-product search over L2-normalised vectors (IndexFlatIP), trading computational efficiency for retrieval precision on a corpus of manage- able scale (∼500K chunks). This avoids the quantisation errors introduced by approximate methods such as IVF or HNSW, which are better suited for billion-scale corpora. To- gether, the FTS5 and FAISS indices provide complementary retrieval coverage: BM25-based full-text search excels at precise entity matching—ticker symbols, monetary figures, and SEC section identifiers—while semantic search captures paraphrase and intent variations that keyword lookup cannot handle. 4.1.2. Online Query Pipeline Query Rewriting. Raw user queries are processed by GPT-4.1 to produce two outputs: (1) a clarified query for semantic search and (2) a list of extracted keywords for keyword-based retrieval. This preprocessing step addresses query ambiguity, corrects grammatical issues, and improves retrieval recall. Hybrid Retrieval. Two parallel retrieval paths are exe- cuted: • Full-Text Search (FTS): The extracted keywords are used to query the SQLite FTS5 index, returning the top-20 most relevant chunks. • Semantic Search: The clarified query is embedded and used for approximate nearest neighbor search in the FAISS index, returning the top-30 semantically similar chunks with a distance threshold of 2.0. Reciprocal Rank Fusion. Results from both retrieval methods are combined using RRF (Cormack et al., 2009) with 푘 = 60: RRFscore(푑) = 1 60 + 푟 FTS (푑) + 1 60 + 푟 sem (푑) (13) where 푟 FTS (푑) and 푟 sem (푑) are the ranks of chunk 푑 in the FTS and semantic result lists, respectively. Neural Reranking. The fused candidate list (up to 30 chunks) is re-scored by Jina Reranker v2 (Jina AI, 2024), a 278M-parameter cross-encoder model. Two adaptive cutoff strategies select the final context: • Cumulative Probability Threshold: Chunks are re- tained until the cumulative normalized score reaches 55%. • Score Cliff Detection: If a chunk’s score drops by more than 0.15 from the top score, it and all lower-ranked chunks are excluded. This typically yields 5–10 high-quality chunks for genera- tion. Answer Generation. The selected chunks are concate- nated into a context string and provided to GPT-4.1 with a system prompt instructing the model to answer based solely on the provided context. 4.1.3. Design Rationale and Limitations The CBR pipeline reflects established best practices for dense retrieval over large document collections (Karpukhin et al., 2020; Gao et al., 2023). The dual-index architecture (FTS5 + FAISS) captures complementary retrieval signals: keyword matching excels at entity names and financial iden- tifiers such as ticker symbols, while semantic search handles paraphrase and intent variations. RRF fusion is unsupervised and rank-based, making it robust to score scale differences between the two retrievers (Cormack et al., 2009). Adaptive cutoff reranking ensures that the generation model receives a compact, high-precision context window rather than a fixed- size candidate set. However, CBR’s core assumption—that relevant chunks can be identified from a global embedding space shared by all documents—breaks down in structurally homogeneous corpora. When hundreds of 10-K filings are indexed to- gether, semantically near-identical sections from different companies occupy adjacent regions of the embedding space. This cross-document chunk confusion is the primary failure mode analyzed in Section 7 and the principal motivation for the routing-based architectures introduced below. Z. Cheng et al.: Preprint submitted to ElsevierPage 6 of 18 Hybrid Document-Routed Retrieval for Financial RAG Figure 1: Chunk-Based RAG (CBR) pipeline. The offline phase indexes documents into dual stores (FTS5 for keyword search, FAISS for semantic search). The online phase retrieves, fuses, reranks, and generates. Z. Cheng et al.: Preprint submitted to ElsevierPage 7 of 18 Hybrid Document-Routed Retrieval for Financial RAG 4.2. Semantic File Routing (SFR) The SFR system eliminates the offline indexing phase entirely. It requires only that documents be organized in a directory structure following a predictable naming conven- tion. Fig. 2 illustrates the pipeline. Figure 2: Semantic File Routing (SFR) pipeline. No offline indexing is required. The query is parsed into structured metadata, resolved to file paths, and the full document is provided to the generation model. 4.2.1. Corpus Organization Documents are stored in a hierarchical directory follow- ing the pattern: data/year/ticker.ext where year is the fiscal year, ticker is the company’s stock ticker symbol, and ext is the file extension (pdf or txt). For the S&P 500 corpus, this results in a flat hierarchy with approximately 500 files per year. 4.2.2. Online Query Pipeline Metadata Extraction via Structured Output. The query 푞 is sent to GPT-4.1 configured with JSON structured output mode (OpenAI, 2024a). The model is instructed to extract: • tickers: an array of company ticker symbols refer- enced in the query. • years: an array of fiscal years referenced in the query, or a default year if none is specified. Algorithm 1 Semantic File Routing: Query Processing Require: Query 푞, data directory 퐷, default year 푦 0 Ensure: Answer 푎 1: metadata← LLM parse (푞) ⊳ JSON structured output 2: tickers← metadata.tickers 3: years← metadata.years 4: if years = ∅ then 5: years← [푦 0 ] 6: end if 7: files← ∅ 8: for each (푡,푦) in tickers × years do 9: 푝← 퐷∕푦∕(푡 + ".pdf") 10: if exists(푝) then 11:files← files ∪ 푝 12: else 13:푝← 퐷∕푦∕(푡 + ".txt") 14:if exists(푝) then files← files ∪ 푝 15:end if 16: end if 17: end for 18: if files = ∅ and years ≠ [푦 0 ] then 19: Retry with years← [푦 0 ]⊳ Fallback 20: end if 21: context← encode_files(files)⊳ Base64 encoding 22: 푎← LLM gen (푞, context) 23: return 푎 The structured output guarantee ensures that the re- sponse always conforms to the expected JSON schema, eliminating parsing failures. An example extraction is shown in Algorithm 1. File Resolution and Fallback. For each extracted (ticker, year) pair, the system constructs a file path and checks for existence, trying PDF first and falling back to TXT. If no files are found for the extracted years, the system retries with a configurable default year (2023 in our experiments). This fallback mechanism handles queries that do not specify a fiscal year explicitly. Document Encoding. Located files are encoded in base64 format and attached to the API request as file inputs, leveraging the multimodal file-reading capabilities of modern LLM APIs. This enables the model to process PDF documents directly, including formatted text, tables, and layout information that would be lost in text-only extraction. Answer Generation. The generation model (GPT-4.1) receives the original query along with the full document file(s) and produces an answer. Unlike the CBR system where context is limited to selected chunks, SFR provides the complete document, allowing the model to reason over the full scope of the filing. 4.3. Hybrid Document-Routed Retrieval (HDRR) The HDRR system combines SFR’s document routing with CBR’s chunk-level retrieval. It reuses the same offline index as CBR (no additional indexing required) and adds a Z. Cheng et al.: Preprint submitted to ElsevierPage 8 of 18 Hybrid Document-Routed Retrieval for Financial RAG Figure 3: Hybrid Document-Routed Retrieval (HDRR) pipeline. Stage 1 routes the query to the correct document via LLM structured output. Stage 2 performs scoped hybrid search (FTS + semantic + RRF + reranking) restricted to the routed document’s chunks. If routing fails, the system falls back to full-corpus retrieval. Z. Cheng et al.: Preprint submitted to ElsevierPage 9 of 18 Hybrid Document-Routed Retrieval for Financial RAG Algorithm 2 Hybrid Document-Routed Retrieval: Query Processing Require: Query 푞, data directory 퐷, default year 푦 0 , chunk index Ensure: Answer 푎 1: metadata← LLM parse (푞)⊳ Stage 1: Document Routing 2: tickers← metadata.tickers 3: years← metadata.years 4: if years = ∅ then 5: years← [푦 0 ] 6: end if 7: doc_ids← ∅ 8: for each (푡,푦) in tickers × years do 9: 푑← lookup_document(퐷,푦,푡) 10: if 푑 ≠ null then 11:doc_ids← doc_ids ∪ id(푑) 12: end if 13: end for 14: if doc_ids = ∅ then 15: scope← ⊳ Fallback: full-corpus search 16: else 17: scope← 푐 ∈ ∶ doc(푐) ∈ doc_ids ⊳ Scoped index 18: end if 19: 푞 ′ ← LLM rewrite (푞) ⊳ Stage 2: Scoped Chunk Retrieval 20: 푅 fts ← FTS(푞 ′ , scope) 21: 푅 sem ← SemSearch(푞 ′ , scope) 22: 푅 fused ← RRF(푅 fts ,푅 sem ) 23: 푅 final ← Rerank(푞 ′ ,푅 fused ) 24: 푎← LLM gen (푞,푅 final ) ⊳ Stage 3: Answer Generation 25: return 푎 document-routing stage before retrieval. Fig. 3 illustrates the pipeline and Algorithm 2 provides the pseudocode. 4.3.1. Stage 1: Document Routing Identical to SFR, the query is processed by GPT-4.1 with structured output to extract ticker symbols and fiscal years. The extracted metadata is resolved to document IDs in the existing chunk index database. If exact-path lookup fails, the system falls back to filename-based matching. If no documents can be identified at all, the system degrades gracefully to full-corpus retrieval (equivalent to standard CBR), ensuring that HDRR never performs worse than CBR due to routing failures. 4.3.2. Stage 2: Scoped Chunk Retrieval The full CBR retrieval pipeline, comprising query rewrit- ing, hybrid search (FTS + semantic), RRF fusion, and neural reranking, is executed with one critical modification: all retrieval operations are restricted to chunks belonging to the routed document(s). This scoping is implemented by filtering the chunk ID space before both FTS and FAISS queries, requiring no changes to the underlying search algorithms. By restricting the search space to a single document (typ- ically 50–200 chunks), HDRR eliminates cross-document chunk confusion by construction. The reranker operates on candidates that all originate from the correct document, enabling it to focus on selecting the most query-relevant passages rather than filtering out cross-document noise. 4.3.3. Stage 3: Answer Generation The reranked chunks are provided to GPT-4.1 with the same prompt template as CBR. Because the chunks are drawn from the correct document and selected by the reranker for relevance, the generation model receives compact, precisely targeted context that combines SFR’s document-level accuracy with CBR’s chunk-level precision. 4.4. Architectural Comparison Table 1 summarizes the key architectural differences between the three paradigms. The key insight is that HDRR inherits the strengths of both paradigms while avoiding their weaknesses: it elimi- nates cross-document confusion (like SFR) and avoids con- text dilution (like CBR). The cost is an additional LLM API call for document routing and the requirement that the corpus exhibit naming regularity. 5. Experimental Setup 5.1. Dataset We evaluate all three systems using the FinDER bench- mark (LinqAlpha, 2025), a dataset specifically designed for financial question answering and RAG evaluation. Dataset Characteristics. FinDER contains 5,703 query- evidence-answer triplets derived from real-world financial inquiries across 10-K filings of S&P 500 companies. The dataset exhibits several properties that make it a challenging and realistic benchmark: • Question type distribution: 84.5% qualitative ques- tions (requiring narrative or explanatory answers) and 15.5% quantitative questions (requiring numerical an- swers, with half involving multi-step reasoning). • Query characteristics: Concise, analyst-style queries with domain-specific abbreviations, jargon, and fre- quently ambiguous references. • Evidence complexity: Relevant evidence is often scattered across multiple non-adjacent sections within a filing, requiring synthesis and cross-referencing. 5.2. Sampling Strategy Following the methodology of our prior work (Cheng et al., 2026), we employ stratified random sampling to gen- erate five independent test groups. Each group contains 300 queries, sampled uniformly at random (approximately 5%) from the full FinDER dataset of 5,703 queries. This yields a total evaluation set of 1,500 queries. Critically, the same five sample groups are used for all three system evaluations (CBR, SFR, and HDRR). This Z. Cheng et al.: Preprint submitted to ElsevierPage 10 of 18 Hybrid Document-Routed Retrieval for Financial RAG Table 1 Architectural comparison of CBR, SFR, and HDRR paradigms. PropertyCBRSFRHDRR Offline indexingRequiredNot requiredRequired (shared with CBR) Retrieval unitChunkWhole documentDocument→ Chunk Retrieval methodHybrid + rerankLLM structured outputRouting + scoped hybrid Context scope5–10 chunksFull document(s)5–10 scoped chunks Vector databaseRequiredNot requiredRequired (shared with CBR) Corpus assumptionNoneNaming regularityNaming regularity Cross-doc confusionPossibleEliminatedEliminated Context dilutionNoneSignificantNone Generation tokens∼5K–15K∼50K–200K∼5K–15K paired experimental design ensures that observed perfor- mance differences are attributable to the retrieval paradigm rather than sampling variation. The multi-group design also enables assessment of inter-group consistency and estima- tion of result variance. 5.3. System Configurations Chunk-Based RAG (CBR). The CBR system is con- figured identically to the “with reranking” configuration from our prior study (Cheng et al., 2026), representing the strongest variant of the chunk-based pipeline. The complete configuration is provided in Appendix A, Table A1. Semantic File Routing (SFR). The SFR system uses GPT-4.1 for both query parsing (structured output) and answer generation. The file repository contains 10-K re- ports organized as year/ticker.pdf, with a default year of 2023. The complete configuration is provided in Ap- pendix A, Table A2. Hybrid Document-Routed Retrieval (HDRR). The HDRR system combines SFR’s document routing (Stage 1) with CBR’s full retrieval pipeline (Stage 2). It uses the same chunk index as CBR (no additional indexing) and the same structured output parsing as SFR. When document routing fails, HDRR falls back to full-corpus chunk retrieval. The complete configuration is provided in Appendix A, Table A3. All three systems use GPT-4.1 as the generation model with temperature set to 0.0 for deterministic output, ensuring that differences in answer quality are attributable to the retrieval mechanism rather than generation stochasticity. 5.4. Evaluation Methodology 5.4.1. LLM-as-Judge Scoring We employ an LLM-as-judge approach (Zheng et al., 2023) using GPT-4.1 as the evaluation model. For each query, the evaluator receives three inputs: (1) the original query, (2) the ground-truth answer from FinDER, and (3) the system-generated answer. The evaluator assigns a score on a 1–10 integer scale using the following rubric: • Score 1: Completely incorrect or unrelated answer. • Score 5: Partially correct with significant missing information. • Score 8: Substantively correct with minor omissions or imprecisions. • Score 10: Fully correct and complete answer. The evaluator is configured with temperature 0.0 and produces structured JSON output containing the score and a brief justification. This ensures deterministic, reproducible evaluation. 5.4.2. Evaluation Metrics We report four metrics designed to capture different aspects of system behavior: 1. Average Score (̄푠): The arithmetic mean of all evalu- ation scores within a group, reflecting overall answer quality on the 1–10 scale. 2. Failure Rate (푃(푠 = 1)): The percentage of queries receiving the minimum score, indicating completely incorrect answers. This metric measures worst-case behavior and system reliability. 3. Correctness Rate (푃(푠 ≥ 8)): The percentage of queries receiving a score of 8 or above, indicating sub- stantively correct answers. This metric reflects prac- tical utility, measured as the proportion of queries where the system provides a usable answer. 4. Perfect Rate (푃(푠 = 10)): The percentage of queries receiving the maximum score, indicating fully correct and complete answers. This metric measures ceiling performance. 5.4.3. Statistical Controls The five-group experimental design serves multiple pur- poses: • Variance estimation: Inter-group variance provides a measure of result stability under different query samples. • Paired comparison: Using the same groups for both systems enables paired statistical analysis, reducing the influence of query difficulty on comparative con- clusions. • Reproducibility: Multiple independent groups demon- strate that findings are not artifacts of a particular query selection. Z. Cheng et al.: Preprint submitted to ElsevierPage 11 of 18 Hybrid Document-Routed Retrieval for Financial RAG 6. Results 6.1. Overall Performance Comparison Table 2 presents the complete results for all three systems across the five experimental groups. Fig. 4 provides a side- by-side overview of the CBR and SFR paradigms whose trade-off motivates the hybrid approach. Figure 4: Side-by-side comparison of the CBR and SFR paradigms for the same query. CBR retrieves targeted chunks (∼10K tokens) while SFR provides the full document (∼100K+ tokens), illustrating the precision-vs-robustness trade-off that HDRR resolves. 6.2. Key Findings The results reveal a clear performance hierarchy, with HDRR dominating both baseline paradigms on every metric. Finding 1: The CBR–SFR robustness-precision trade- off. Comparing CBR and SFR confirms the trade-off iden- tified in our analysis: SFR achieves higher average scores (6.45 vs. 6.02) and far fewer catastrophic failures (푠=1: 10.3% vs. 22.5%), but CBR produces more high-quality answers (푠≥8: 49.0% vs. 44.0%) and more perfect answers (푠=10: 13.8% vs. 8.5%). Neither paradigm dominates the other across all metrics. Finding 2: HDRR resolves the trade-off. HDRR achieves the best performance on all four metrics simultaneously. It attains the highest average score (7.54), the lowest failure rate (6.4%), the highest correctness rate (67.7%), and the highest perfect rate (20.1%). This result holds in every individual group, not just on average. Finding 3: HDRR’s improvement over CBR is sub- stantial. Compared to CBR, HDRR improves average score by 1.52 points (+25.2%), reduces failures by 16.1 percent- age points (−71.6% relative), increases correctness rate by 18.7 p (+38.2%), and increases perfect rate by 6.3 p (+45.7%). Finding 4: HDRR’s improvement over SFR is equally substantial. Compared to SFR, HDRR improves average score by 1.09 points (+16.9%), reduces failures by 3.9 p (−37.9% relative), increases correctness rate by 23.7 p (+53.9%), and increases perfect rate by 11.6 p (+136.5%). Finding 5: All three systems show consistent inter- group patterns. The standard deviation of average scores across the five groups is 0.24 for CBR, 0.22 for SFR, and 0.18 for HDRR. HDRR shows the lowest variance, indicat- ing the most stable performance. The performance ranking (HDRR > SFR > CBR on average score; HDRR > CBR > SFR on perfect rate) is preserved across all five groups. 6.3. Score Distribution Analysis Table 3 summarizes the distributional shift across the three systems using three score bands. The distributional pattern reveals the mechanism behind each system’s performance profile. SFR shifts probability mass from the low band to the medium band (reducing failures but not achieving high precision), while the medium band swells by 18.0 p. HDRR, in contrast, shifts mass directly from both the low and medium bands into the high band. Compared to CBR, HDRR reduces the low band by 15.5 p and the medium band by 3.3 p, channeling all of this mass (+18.7 p) into the high band. This pattern demon- strates that HDRR avoids the “context dilution” problem that causes SFR’s mass to accumulate in the medium range. 6.4. Paired Group Analysis Table 4 shows the per-group performance differences between HDRR and CBR. HDRR improves over CBR on every metric in every group. The average score improvement ranges from +1.18 (Group 05) to+1.75 (Group 03), with low standard deviation (0.21), indicating a highly consistent improvement. The failure rate reduction is also consistent (mean −16.2 p, std 2.2 p), as is the correctness rate improvement (mean +18.7 p, std 1.9 p). Table 5 shows the per-group differences between HDRR and SFR. HDRR’s improvements over SFR are even more consis- tent (Avg Score std = 0.14, 푠=10 std = 1.1). The largest gains are in correctness rate (+23.7 p mean) and perfect Z. Cheng et al.: Preprint submitted to ElsevierPage 12 of 18 Hybrid Document-Routed Retrieval for Financial RAG Table 2 Performance comparison across five experimental groups. CBR = Chunk-Based RAG; SFR = Semantic File Routing; HDRR = Hybrid Document-Routed Retrieval. Best values per metric are bolded. GroupSystemAvg Score푠=1 (%)푠≥8 (%)푠=10 (%) 01 CBR5.8524.347.712.0 SFR6.539.344.17.7 HDRR7.447.064.720.0 02 CBR6.1721.751.616.0 SFR6.619.346.48.7 HDRR7.765.072.719.0 03 CBR5.6425.744.014.0 SFR6.1112.036.710.7 HDRR7.397.363.021.7 04 CBR6.2121.051.314.7 SFR6.679.750.610.0 HDRR7.725.371.021.3 05 CBR6.2120.050.612.3 SFR6.3211.042.35.3 HDRR7.397.367.318.3 Mean CBR6.0222.549.013.8 SFR6.4510.344.08.5 HDRR7.546.467.720.1 Table 3 Score distribution comparison across score bands (mean per- centages across five groups). System Low (푠 ≤ 3) Medium (4 ≤ 푠 ≤ 7) High (푠 ≥ 8) CBR27.823.249.0 SFR14.841.244.0 HDRR12.319.967.7 Table 4 Per-group performance differences (HDRR − CBR). Positive values on Avg Score and 푠≥8/푠=10 favor HDRR; negative values on 푠=1 favor HDRR. GroupΔAvgΔ푠=1Δ푠≥8Δ푠=10 01+1.59−17.3+17.0+8.0 02+1.59−16.7+21.1+3.0 03+1.75−18.4+19.0+7.7 04+1.51−15.7+19.7+6.6 05+1.18−12.7+16.7+6.0 Mean+1.52−16.2+18.7+6.3 Std0.212.21.92.0 rate (+11.6 p mean), confirming that HDRR recovers the precision that SFR sacrifices through context dilution. 7. Discussion 7.1. The CBR–SFR Trade-off: Root Causes The comparison of CBR and SFR reveals a fundamental trade-off between retrieval robustness and answer precision. Table 5 Per-group performance differences (HDRR − SFR). Positive values on Avg Score and 푠≥8/푠=10 favor HDRR; negative values on 푠=1 favor HDRR. GroupΔAvgΔ푠=1Δ푠≥8Δ푠=10 01+0.91−2.3+20.6+12.3 02+1.15−4.3+26.3+10.3 03+1.28−4.7+26.3+11.0 04+1.05−4.4+20.4+11.3 05+1.07−3.7+25.0+13.0 Mean+1.09−3.9+23.7+11.6 Std0.140.93.01.1 Understanding the mechanisms behind this trade-off is es- sential for appreciating why HDRR resolves it. 7.1.1. Why CBR Suffers Catastrophic Failures CBR’s high failure rate (푠=1: 22.5%) stems from cross- document chunk confusion. In the S&P 500 10-K corpus, documents share standardized section structures mandated by SEC regulations. Sections such as “Risk Factors,” “Man- agement’s Discussion and Analysis,” and “Financial State- ments” appear in virtually every filing with similar headings, boilerplate language, and tabular formats. When these docu- ments are chunked and indexed together, a query about Com- pany A’s risk factors may retrieve chunks from Company B’s risk factors because the semantic similarity between analo- gous sections of different filings is high. The reranking stage mitigates this by re-scoring can- didates using cross-encoder attention (Jina AI, 2024), but Z. Cheng et al.: Preprint submitted to ElsevierPage 13 of 18 Hybrid Document-Routed Retrieval for Financial RAG it can only reorder candidates that were retrieved in the first stage. If the correct chunks were not in the initial can- didate pool, reranking cannot recover them. Furthermore, reranking operates on individual chunks without awareness of which document they originate from, so it cannot enforce the constraint that retrieved chunks should come from the same company. 7.1.2. Why SFR Sacrifices Precision SFR eliminates cross-document confusion by routing to the correct document, but it introduces context dilu- tion. A typical 10-K report contains 50,000–200,000 tokens. The relevant information for any given query may occupy only a small fraction of this context. Research on long- context LLMs has demonstrated the “lost in the middle” phenomenon (Liu et al., 2024): models struggle to attend uniformly to all positions in very long inputs. When SFR provides the entire document, the generation model must locate the answer-relevant content within a vast context, reducing precision compared to CBR’s compact, curated chunk context. The net effect is a distributional shift: SFR moves proba- bility mass from the tails (both high and low scores) toward the middle of the score distribution. It raises the floor by eliminating chunk confusion but lowers the ceiling by in- troducing context dilution. 7.2. How HDRR Resolves the Trade-off HDRR eliminates both failure modes simultaneously: 1. No cross-document confusion: By routing to the correct document before retrieval, HDRR ensures that all candidate chunks originate from the target com- pany’s filing. The reranker then operates exclusively on within-document candidates, where it can focus on identifying the most query-relevant passages rather than filtering out cross-document noise. 2. No context dilution: Unlike SFR, HDRR does not send the full document to the generation model. In- stead, it provides only the top reranked chunks (typi- cally 5–10 passages), giving the model compact, pre- cisely targeted context identical in format to CBR but drawn exclusively from the correct document. The result is additive: HDRR inherits SFR’s low fail- ure rate (6.4% vs. 10.3% for SFR, 22.5% for CBR) while achieving even higher precision than CBR (푠=10: 20.1% vs. 13.8% for CBR, 8.5% for SFR). The improvement in precision beyond CBR is attributable to the elimination of cross-document noise from the reranker’s input: when all candidates come from the correct document, the reranker can make finer-grained relevance distinctions, surfacing the most precisely relevant passages. 7.3. Error Analysis of Remaining HDRR Failures HDRR’s residual failure rate of 6.4% (푠=1) is lower than both SFR (10.3%) and CBR (22.5%). We identify three categories of remaining failures. Table 6 Qualitative cost comparison between CBR, SFR, and HDRR paradigms. Cost DimensionCBRSFRHDRR Offline indexingHigh (one-time) NoneHigh (shared with CBR) Storage (index)ModerateNoneModerate (shared) API calls per query 3–424–5 Gen. tokens/query 5K–15K50K–200K5K–15K Reranking cost/query ModerateNoneModerate 7.3.1. Ticker Symbol Mismatch with Successful Fallback Some failures stem from ticker mismatches (e.g., PEAK → DOC due to corporate rebranding), identical to SFR’s failure mode. However, HDRR’s fallback mechanism par- tially mitigates this: when document routing fails, HDRR degrades to full-corpus retrieval. This means that ticker mismatches in HDRR result in CBR-equivalent retrieval rather than complete failure, explaining why HDRR’s failure rate (6.4%) is lower than SFR’s (10.3%). 7.3.2. Inherent Query Difficulty A subset of failures is shared across all three systems, representing queries that are inherently difficult regardless of retrieval strategy. Examples include queries requiring information genuinely absent from the corpus, and queries whose answers require multi-hop reasoning across sections that even targeted retrieval cannot fully capture. 7.3.3. Routing to Correct Document but Insufficient Chunk Relevance In rare cases, HDRR correctly routes to the target doc- ument but the chunk-level retrieval within that document fails to surface sufficiently relevant passages. This can occur when the relevant information is spread across many small fragments that individually score low in both FTS and se- mantic search. 7.4. Economic Cost Analysis Table 6 compares the computational costs of the three paradigms. HDRR adds one API call per query (for document rout- ing) compared to CBR but maintains the same generation token cost (∼5K–15K), avoiding SFR’s order-of-magnitude token overhead. This makes HDRR only marginally more expensive than CBR per query while achieving substantially better results. Compared to SFR, HDRR is significantly cheaper per query due to the elimination of full-document context. For production deployments, HDRR offers the best cost- performance ratio: it achieves the highest answer quality at a per-query cost close to CBR, making it practical for Z. Cheng et al.: Preprint submitted to ElsevierPage 14 of 18 Hybrid Document-Routed Retrieval for Financial RAG high-volume workloads where SFR’s token costs would be prohibitive. 7.5. Scalability and Maintenance HDRR inherits the scalability properties of both parent paradigms: Adding new documents. Like CBR, adding a new doc- ument requires re-indexing (parsing, chunking, embedding). However, the document routing stage automatically includes new documents once they are placed in the correct directory and indexed, requiring no changes to the routing logic. Corpus expansion. Unlike CBR, whose retrieval quality may degrade as the corpus grows (due to increased cross- document confusion in the vector space), HDRR’s routing stage isolates the retrieval to the correct document regardless of corpus size. This means HDRR’s quality is invariant to corpus expansion for queries where routing succeeds, and degrades gracefully to CBR quality when routing fails. Multi-year corpora. HDRR naturally handles multi- year corpora through the (ticker, year) metadata pair, avoid- ing the compounded confusion that CBR faces when struc- turally similar sections appear across both companies and years. 7.6. Generalizability to Other Domains HDRR is applicable to any domain where the corpus ex- hibits naming regularity (Definition 1) and structural homo- geneity (Definition 2). In such domains, document routing eliminates cross-document confusion while chunk retrieval provides precision. Regulatory filings. Legal and regulatory documents follow standardized formats with predictable naming con- ventions (case numbers, filing dates). HDRR can route to the correct filing and retrieve relevant sections. Medical records. Electronic health records organized by patient identifier and encounter date provide natural routing metadata. HDRR could route to the correct patient record and retrieve relevant clinical notes. Technical documentation. Software documentation or- ganized by product name and version enables routing fol- lowed by targeted section retrieval. Heterogeneous corpora. For corpora without naming regularity, HDRR’s routing stage would frequently fail, causing fallback to full-corpus retrieval. In such settings, the hybrid approach reduces to standard CBR, offering no disadvantage but also no advantage over the baseline. More broadly, the principle of adapter-based and domain- scoped retrieval architectures extends across specialized domains—from malware detection, where flow-adapter mod- els translate binary code for unsupervised classification (Hu, Wang, Zhao, Zeng and Luo, 2025), to financial document QA as demonstrated in this work—underscoring the versatil- ity of constrained retrieval as a design strategy for structured corpora. 8. Conclusion and Future Work 8.1. Summary of Findings This paper presented a three-paradigm comparative study of RAG architectures for financial document question answering. We first identified a fundamental robustness- precision trade-off between Chunk-Based Retrieval (CBR) and Semantic File Routing (SFR), then proposed Hybrid Document-Routed Retrieval (HDRR) to resolve it. Through controlled evaluation on 1,500 queries from the FinDER benchmark across five independent experimental groups, we demonstrated: 1. CBR and SFR exhibit a robustness-precision trade- off. SFR achieves higher average scores (6.45 vs. 6.02) and fewer catastrophic failures (10.3% vs. 22.5%), while CBR achieves more perfect answers (13.8% vs. 8.5%). Cross-document chunk confusion causes CBR’s failures; context dilution limits SFR’s preci- sion. 2. HDRR resolves the trade-off, achieving the best per- formance on all four metrics simultaneously: average score 7.54 (+25.2% over CBR), failure rate 6.4% (−71.6% relative to CBR), correctness rate 67.7% (+18.7 p over CBR), and perfect rate 20.1% (+6.3 p over CBR, +11.6 p over SFR). 3. HDRR dominates both baselines in every exper- imental group, with low inter-group variance (std = 0.18 for average score), demonstrating robust and consistent improvements. 4. The hybrid architecture is cost-effective, main- taining CBR’s generation token cost (∼5K–15K per query) while avoiding SFR’s order-of-magnitude to- ken overhead (∼50K–200K per query), at the cost of one additional API call for document routing. 5. Structural regularity of the corpus enables docu- ment routing to eliminate cross-document confusion, which is the primary mechanism behind HDRR’s improvements. 8.2. Contributions This work contributes to the RAG and financial NLP communities through: 1. Trade-off identification: We identify and quantify the robustness-precision trade-off between chunk- level and document-level retrieval, attributing it to cross-document chunk confusion (in CBR) and con- text dilution (in SFR). 2. Hybrid Document-Routed Retrieval: We propose HDRR, a two-stage architecture that uses LLM-based document routing to scope chunk-based retrieval, achieving the strengths of both paradigms without their weaknesses. 3. Rigorous three-paradigm evaluation: A controlled, paired experimental design with five independent groups and 1,500 total queries provides statistically robust evidence that HDRR dominates both baseline paradigms. Z. Cheng et al.: Preprint submitted to ElsevierPage 15 of 18 Hybrid Document-Routed Retrieval for Financial RAG 4. Generalizability framework: We formalize the cor- pus properties (naming regularity and structural ho- mogeneity) that determine when document routing is beneficial, extending the analysis beyond the financial domain. 8.3. Future Directions Several promising directions emerge from this work: Improving document routing. HDRR’s routing stage inherits SFR’s sensitivity to ticker symbol mismatches. In- corporating a lookup table mapping alternative names, for- mer tickers, and subsidiaries to canonical file names could further reduce the residual failure rate. LLM-based entity resolution or external knowledge bases could provide addi- tional robustness. Comparative and multi-document queries. Our evalu- ation focuses on single-company queries. An important class of financial queries involves cross-company comparisons (e.g., “How does Apple’s revenue compare to Google’s?”). HDRR can route to multiple documents simultaneously and retrieve relevant chunks from each, potentially yielding even larger advantages over CBR for comparative queries. Domain extension. Evaluating HDRR on other structured- document domains (legal filings, medical records, and tech- nical documentation) would test the generalizability of our findings. Each domain presents unique routing chal- lenges that may require domain-specific metadata extraction schemas. Adaptive routing confidence. A promising refinement is to incorporate a routing confidence score: when the LLM is uncertain about the document identity, the system could expand the retrieval scope to include related documents or fall back to full-corpus search. This adaptive approach could further reduce the residual failure rate while maintaining precision for high-confidence routes. Cost optimization. The document routing call could be combined with query rewriting into a single LLM call, reducing the API overhead from one additional call to zero. Alternatively, a smaller, cheaper model could be used for routing while retaining the full model for generation. Acknowledgments The authors thank OpenAI for API access used in exper- iments and LinqAlpha for providing the FinDER benchmark dataset. Code Availability The source code for all three RAG systems (CBR, SFR, and HDRR), including the indexing pipeline, retrieval components, and evaluation scripts, is publicly available at https://github.com/zhycheng614/financial-RAG-pipeline. A. System Hyperparameters Table A1 Complete configuration for the Chunk-Based RAG system. ComponentParameterValue ChunkingChunk size2,500 chars Overlap1,250 chars EmbeddingModeltext-embed.-3-small Dimension1,024 Batch size32 FTS RetrievalTop-푘20 Semantic Search Top-푘30 Dist. threshold2.0 RRF Fusion푘 constant60 RerankingModeljina-reranker-v2 Max candidates30 Cumul. threshold0.45 Cliff threshold0.15 GenerationModelGPT-4.1 Temperature0.0 Context limit10 chunks Table A2 Complete configuration for the Semantic File Routing system. ComponentParameterValue Query ParsingModelGPT-4.1 Output formatJSON (structured) Temperature0.0 File LookupDir. pattern yr/tick.pdf Fallback fmts.pdf, .txt Default year2023 File EncodingMethodBase64 Supported typesPDF, TXT GenerationModelGPT-4.1 Temperature0.0 ContextFull document(s) Z. Cheng et al.: Preprint submitted to ElsevierPage 16 of 18 Hybrid Document-Routed Retrieval for Financial RAG Table A3 Complete configuration for the Hybrid Document-Routed Re- trieval system. ComponentParameterValue Stage 1: Document Routing Query ParsingModelGPT-4.1 Output formatJSON (structured) Temperature0.0 File LookupDir. pattern yr/tick.pdf Fallback fmts.pdf, .txt Default year2023 Routing fallbackFull-corpus search Stage 2: Scoped Chunk Retrieval ChunkingChunk size2,500 chars Overlap1,250 chars EmbeddingModeltext-embed.-3-small Dimension1,024 FTS RetrievalTop-푘20 Semantic Search Top-푘30 Dist. threshold2.0 RRF Fusion푘 constant60 RerankingModeljina-reranker-v2 Max candidates30 Cumul. threshold0.45 Cliff threshold0.15 GenerationModelGPT-4.1 Temperature0.0 ContextReranked chunks CRediT authorship contribution statement Zhiyuan Cheng: Conceptualization, Methodology, Soft- ware, Writing – Original draft, Visualization. Longying Lai: Data curation, Validation, Writing – Review & Editing. Yue Liu: Validation, Writing – Review & Editing. References Chen, J., Li, Z., Chen, K., 2024. Financial document analysis using large language models: A survey. arXiv preprint arXiv:2409.15066 . Chen, S., Wong, S., Chen, L., Tian, Y., 2023. Extending context window of large language models via positional interpolation, in: arXiv preprint arXiv:2306.15595. Cheng, Z., Lai, L., Liu, Y., Cheng, K., Qi, X., 2026. Enhancing financial report question-answering: A retrieval-augmented generation system with reranking analysis. arXiv preprint arXiv:2603.16877 URL: https: //doi.org/10.48550/arXiv.2603.16877, doi:10.48550/arXiv.2603.16877. Cormack, G.V., Clarke, C.L.A., Büttcher, S., 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods, in: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 758–759. Fang, Y., Sun, T., Shi, Y., Gu, X., 2025. AttentionRAG: Attention-guided context pruning in retrieval-augmented generation. arXiv preprint arXiv:2503.10720 . Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., 2023. Retrieval-augmented generation for large language models: A survey, in: arXiv preprint arXiv:2312.10997. Hu, M., Wang, J., Zhao, W., Zeng, Q., Luo, L., 2025. FlowMalTrans: Unsupervised binary code translation for malware detection using flow- adapter architecture. arXiv preprint arXiv:2508.20212 . Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., Grave, E., 2023. Atlas: Few- shot learning with retrieval augmented language models, in: Journal of Machine Learning Research, p. 1–43. Jiang, Z., Ma, X., Chen, W., 2024. LongRAG: Enhancing retrieval- augmented generation with long-context LLMs. arXiv preprint arXiv:2406.15319 . Jina AI, 2024.Jina reranker v2: Multilingual cross- encoder for document reranking. https://jina.ai/models/ jina-reranker-v2-base-multilingual/. Johnson, J., Douze, M., Jégou, H., 2021. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data 7, 535–547. Kang, Z., Gong, J., Hu, W., Yin, S., Jiang, K., Fang, Z., He, Y., Meng, C., Fu, R., Chen, D., et al., 2026. QuantEval: A benchmark for financial quantitative tasks in large language models. arXiv preprint arXiv:2601.08689 . Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., tau Yih, W., 2020. Dense passage retrieval for open-domain question an- swering, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 6769–6781. Khattab, O., Zaharia, M., 2020. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , 39–48. Lai, L., Cheng, Z., Cheng, K., Qi, X., 2026. Do transformers always win? An empirical study of semantic embeddings for short-text E-commerce reviews. Research Square Preprint doi:10.21203/rs.3.rs-9163424/v1. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., tau Yih, W., Rocktäschel, T., Riedel, S., Kiela, D., 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks, in: Advances in Neural Information Processing Systems (NeurIPS), p. 9459–9474. LinqAlpha, 2025. FinDER: Financial dataset for question answer- ing and evaluating retrieval-augmented generation. arXiv preprint arXiv:2504.15800 . Liu, M., Shi, G., 2024. Enhancing LLM-based text classification in political science: Automatic prompt optimization and dynamic exemplar selection for few-shot learning. arXiv preprint arXiv:2409.01466 . Liu, N.F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., Liang, P., 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics 12, 157–173. Loukas, L., Stogiannidis, I., Malakasiotis, P., Vassos, S., 2023. Making LLMs worth every penny: Resource-limited text classification in bank- ing. Proceedings of the Fourth ACM International Conference on AI in Finance , 392–400. Microsoft Corporation, 2024. Playwright: Fast and reliable end-to-end testing for modern web apps. https://playwright.dev/. Nogueira, R., Cho, K., 2019. Passage re-ranking with BERT, in: arXiv preprint arXiv:1901.04085. OpenAI, 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774 . OpenAI, 2024a. Introducing structured outputs in the API. https://openai. com/index/introducing-structured-outputs-in-the-api/. OpenAI, 2024b. New embedding models and API updates. https://openai. com/blog/new-embedding-models-and-api-updates. Reimers, N., Gurevych, I., 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 3982–3992. Robertson, S., Zaragoza, H., 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval 3, 333–389. Shi, Y., Sun, M., Liu, Z., Yang, M., Fang, Y., Sun, T., Gu, X., 2026. Reasoning in trees: Improving retrieval-augmented generation for multi- hop question answering. arXiv preprint arXiv:2601.11255 . Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., Nanayakkara, S., 2023. Improving the domain adaptation of retrieval Z. Cheng et al.: Preprint submitted to ElsevierPage 17 of 18 Hybrid Document-Routed Retrieval for Financial RAG augmented generation (RAG) models for open domain question answer- ing. Transactions of the Association for Computational Linguistics 11, 1–17. SQLite, 2024. SQLite FTS5 extension. https://w.sqlite.org/fts5.html. Su, J., Lan, Q., Xia, Y., Sun, L., Tian, W., Shi, T., Song, X., He, L., Jingsong, Y., 2026. Difficulty-aware agentic orchestration for query-specific multi- agent workflows, in: Proceedings of the ACM Web Conference 2026 (W ’26). Wang, L., Yang, N., Huang, X., Yang, L., Majumder, R., Wei, F., 2024. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368 . Wang, Y., Tang, Y., Qian, Y., Zhao, C., 2026. VisualLeakBench: Auditing the fragility of large vision-language models against PII leakage and social engineering. arXiv preprint arXiv:2603.13385 . Xue, Z., Zhao, S., Qi, Y., Zeng, X., Yu, Z., 2026. Resilient routing: Risk-aware dynamic routing in smart logistics via spatiotemporal graph learning. arXiv preprint arXiv:2601.13632 . Yao, C., Zhan, Q., Cao, Z., Li, D., Lin, Y., Shao, Y., Wang, L., Wang, Z., Zhang, J., Zhang, Y., et al., 2025. Generative AI for simulating real world dynamics: Applications and challenges. Authorea Preprints . Zhang, Z., Fu, R., He, Y., Shen, X., Wang, Y., Du, X., You, H., Shi, J., Fong, S., 2025. FinSentLLM: Multi-LLM and structured semantic signals for enhanced financial sentiment forecasting. arXiv preprint arXiv:2509.12638 . Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I., 2023. Judging LLM-as-a-judge with MT-Bench and chatbot arena, in: Advances in Neural Information Processing Systems (NeurIPS). Z. Cheng et al.: Preprint submitted to ElsevierPage 18 of 18