Paper deep dive
Hierarchical Reranking for Scalable Financial RAG System
Joohyun Lee, Sungwoo Hong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/1/2026, 1:42:00 AM
Summary
The paper introduces 'Hierarchical Reranker', a Retrieval-Augmented Generation (RAG) framework designed for scalable financial document analysis. It addresses challenges in processing hybrid text-table structures and large-scale datasets through three innovations: Pre-Retrieval Optimization (normalization, keyword expansion, table-to-JSON conversion), a two-stage Hierarchical Reranker Architecture (lightweight filtering followed by high-capacity semantic reranking), and Long-Context Management (adaptive partitioning and fusion for inputs exceeding 64k tokens). The system achieved an NDCG@20 score of 0.7918 on benchmarks like FinQA and FinanceBench, securing second place in the ACM-ICAIF '24 FinanceRAG Challenge.
Entities (10)
Relation Signals (9)
Hierarchical Reranker → achievedscoreon → NDCG@20
confidence 98% · the proposed system achieved an NDCG@20 score of 0.7918
Hierarchical Reranker → evaluatedon → FinanceBench
confidence 95% · Across multiple benchmarks, including FinQA, FinanceBench... the proposed system achieved an NDCG@20 score of 0.7918
Hierarchical Reranker → evaluatedon → ConvFinQA
confidence 95% · Across multiple benchmarks, including FinQA, FinanceBench, and ConvFinQA, the proposed system achieved an NDCG@20 score of 0.7918
Hierarchical Reranker → evaluatedon → FinQA
confidence 95% · Across multiple benchmarks, including FinQA... the proposed system achieved an NDCG@20 score of 0.7918
Hierarchical Reranker → placedin → ACM-ICAIF '24 FinanceRAG Challenge
confidence 95% · Its robustness was further validated by achieving second place in the ACM-ICAIF '24 FinanceRAG Challenge.
Hierarchical Reranker → uses → Pre-Retrieval Optimization
confidence 95% · The system integrates three key innovations: Pre-Retrieval Optimization... Hierarchical Reranker Architecture... and Long-Context Management
Hierarchical Reranker → uses → jina-reranker-v3
confidence 95% · In the first stage, a fast, low-complexity jina-reranker-v3... is used to eliminate low-relevance or noisy candidates.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Analyzing financial documents such as 10-K filings, tabular disclosures, and macroeconomic reports demands expert reasoning and extensive time. However, existing Retrieval-Augmented Generation systems often struggle to process hybrid text-table structures or the massive scale of financial documents. To address these challenges, we propose Hierarchical Reranker, a RAG framework designed to improve retrieval performance and generative reliability across large-scale financial datasets. The system integrates three key innovations: Pre-Retrieval Optimization, enhancing query clarity and search efficiency through normalization, keyword expansion, and table transformation; Hierarchical Reranker Architecture, improving retrieval precision through a two-stage ranking mechanism; and Long-Context Management, preserving reasoning accuracy through adaptive input partitioning and fusion under extensive contexts. Across multiple benchmarks, including FinQA, FinanceBench, and ConvFinQA, the proposed system achieved an NDCG@20 score of 0.7918 and demonstrated superior factual consistency. Its robustness was further validated by achieving second place in the ACM-ICAIF '24 FinanceRAG Challenge. This work presents a deployable, domain-optimized RAG pipeline that enhances both the accuracy and scalability of financial reasoning, paving the way for automated audit reporting and quantitative investment analysis. The source code will be made publicly available on GitHub upon acceptance.
Tags
Links
- Source: https://arxiv.org/abs/2607.27523v1
- Canonical: https://arxiv.org/abs/2607.27523v1
Trouble viewing inline? Open PDF directly →
Full Text
35,722 characters extracted from source content.
Expand or collapse full text
Hierarchical Reranking for Scalable Financial RAG System Joohyun Lee1 Sungwoo Hong2 1Financial Security Institute 2Hanyang University dlee110600@gmail.com, toggiya0701@gmail.com Abstract Analyzing financial documents such as 10-K filings, tabular disclosures, and macroeconomic reports demands expert reasoning and extensive time. However, existing Retrieval-Augmented Generation systems often struggle to process hybrid text–table structures or massive scale of financial documents. To address these challenges, we propose Hierarchical Reranker, a RAG framework designed to improve retrieval performance and generative reliability across large-scale financial datasets. The system integrates three key innovations: Pre-Retrieval Optimization, enhancing query clarity and search efficiency through normalization, keyword expansion, and table transformation; Hierarchical Reranker Architecture, improving retrieval precision through a two-stage ranking mechanism; and Long-Context Management, preserving reasoning accuracy through adaptive input partitioning and fusion under extensive contexts. Across multiple benchmarks, including FinQA, FinanceBench, and ConvFinQA, the proposed system achieved an NDCG@20 score of 0.7918 and demonstrated superior factual consistency. Its robustness was further validated by achieving second place in the ACM-ICAIF ’24 FinanceRAG Challenge. This work presents a deployable, domain-optimized RAG pipeline that enhances both the accuracy and scalability of financial reasoning, paving the way for automated audit reporting and quantitative investment analysis. The source code will be made publicly available on GitHub upon acceptance. 1 Introduction Interpreting 10-K filings, reconciling tabular disclosures, and contextualizing results against shifting macroeconomic conditions remain among the most labor-intensive activities in the financial industry. These tasks resist automation because they demand both expert domain judgment and exhaustive cross-referencing across long, heterogeneous documents, resulting in high operational cost and limited scalability Lee and Han (2025). As financial institutions begin to deploy autonomous LLM-based agents for auditing, quantitative research, and portfolio analysis Wu et al. (2023); Papasotiriou et al. (2024), the bottleneck has shifted from whether LLMs can read financial text to how reliably and economically they can ground their reasoning in the underlying evidence. Retrieval-Augmented Generation (RAG) is the natural substrate for this grounding Yang et al. (2023), yet off-the-shelf RAG pipelines exhibit three persistent failure modes when transplanted into finance Yu et al. (2025); Databricks (2024): (1) domain-specific jargon, units, and abbreviations cause query–corpus embedding drift, degrading recall; (2) hybrid text–table evidence is poorly aligned by retrievers trained on prose, leading to numerically incorrect generations; and (3) reasoning quality collapses on long inputs, even on models that nominally support 100k+ token contexts. Each failure mode interacts with the others: a financial 10-K can simultaneously be long, dense in jargon, and dominated by tables. We address these failures with the Hierarchical Reranker, a finance-specific RAG framework engineered for deployable, large-scale use. Rather than scaling a single monolithic retriever or generator, the framework decomposes the problem across cooperating components, each addressing one failure mode. The two-stage retrieval design in particular instantiates a small-and-large model collaboration: a lightweight reranker rapidly prunes the candidate pool, and a high-capacity reranker performs fine-grained semantic adjudication on the survivors. This decomposition keeps end-to-end latency and token cost bounded while concentrating expensive computation where it matters — a property we view as essential for any RAG system that is to operate at institutional scale. Our contributions are threefold: • Pre-Retrieval Optimization. We normalize finance-specific units and abbreviations, augment queries with domain keywords, and deterministically convert Markdown tables to JSON so that numeric values stay bound to their headers — producing measurable gains in retrieval quality without introducing LLM-induced hallucinations during preprocessing. • Hierarchical Reranker. A two-stage cascade pairs a fast first-stage filter with a high-capacity second-stage reranker, capturing cross-sentence financial dependencies and table–text consistency while keeping inference cost bounded. • Long-Context Management. For inputs beyond a 64k-token threshold, we partition evidence into semantically coherent chunks and fuse intermediate answers with an evidence-aware merger that explicitly handles ambiguous or contradictory partials. Ablations isolate the contribution of each component. The framework was validated through extensive ablation studies and benchmark evaluations, has been deployed in institutional auditing and investment workflows, and secured 2nd place in the ACM-ICAIF ’24 FinanceRAG Challenge Choi et al. (2024), demonstrating both robustness and industry-level competitiveness. Figure 1: Hierarchical Reranker Framework, which integrates query expansion, corpus compression, and a two-stage reranking pipeline to enhance retrieval performance. The retrieved corpora are dynamically managed under a long-context fusion mechanism, ensuring efficient and accurate response generation even for inputs exceeding 64k tokens. 2 Related Work The integration of artificial intelligence in the financial domain has rapidly advanced through the emergence of retrieval-based benchmarks and reasoning datasets Lee and Han (2025). Early datasets such as FinQABench LighthouzAI (2024) and FinanceBench Islam et al. (2023) focused primarily on factual grounding in financial documents like 10-K filings, evaluating models’ ability to reduce hallucinations and improve factual correctness. While these benchmarks improved retrieval–generation alignment, they often assumed relatively short contexts and ignored multi-step numerical reasoning. Subsequently, TATQA Zhu et al. (2021), FinQA Chen et al. (2022a), and ConvFinQA Chen et al. (2022b) expanded the task scope to hybrid textual–tabular reasoning, requiring models to perform arithmetic and comparative analysis. These datasets highlighted the limitations of general-purpose retrievers and LLMs in understanding structured quantitative data, motivating research into more domain-aware retrieval architectures. From the retrieval perspective, query expansion Jagerman et al. (2023) and reranking Ma et al. (2023) have been widely studied to enhance search accuracy. Methods such as HyDE Gao et al. (2022) and query rewriting Liu and Mozafari (2024) improved retrieval recall through lexical and semantic enrichment, while rerankers refined document relevance post-retrieval. However, most existing studies Fan et al. (2024) have been optimized for horizontal tasks, focusing on either pre-retrieval or post-retrieval, with few demonstrating performance improvements in domain-specific (vertical) applications. Long-context reasoning represents another key research stream. Despite advances in models like Claude-Opus 4.6 Anthropic (2026), GPT-5.4 OpenAI (2026), Grok-4.20 xAI (2026) and Gemini-3.0-Pro Google (2026), several studies Databricks (2024); An et al. (2024) have observed performance degradation beyond 64k tokens, particularly in financial reasoning tasks with dense numerical references. Existing work Li et al. (2023) offers few solutions for dynamically constraining and fusing contexts while preserving reasoning consistency. By integrating pre-retrieval optimization, hierarchical reranker refinement, and context-length management into a unified finance RAG pipeline, our study aims to build a practically deployable RAG system for real-world financial applications. 3 Tasks and Dataset To construct a finance-specific RAG system, we define two interdependent tasks that jointly determine the overall pipeline design: Document Retrieval and Answer Generation. 3.1 Task 1: Document Retrieval Given a user query, the objective is to identify the top 20 most relevant passages from a large corpus of financial documents. Unlike general-purpose retrieval tasks, financial corpora contain both textual and numerical data, making semantic and quantitative alignment equally important. We thus formulate retrieval as a two-stage ranking problem embedding-based coarse retrieval followed by fine-grained reranking to balance efficiency and accuracy. Retrieval performance is evaluated using Normalized Discounted Cumulative Gain (NDCG@20), which captures both ranking quality and semantic relevance. This metric is chosen for its robustness in evaluating graded relevance rather than binary correctness, which aligns well with real-world financial information retrieval. 3.2 Task 2: Answer Generation Once the top-ranked corpora are retrieved, the goal is to generate a factual, numerically grounded answer that directly references the evidence within the selected documents. This task extends beyond text summarization by requiring precise interpretation of tables, ratios, and cross-document dependencies. To assess generation quality, we adopt the LLM-as-a-Judge framework Gu et al. (2025), which compares model outputs with ground-truth answers while evaluating factual consistency, logical reasoning, and numerical precision. This was feasible because the benchmark answers are short and clear (mainly numbers or simple words), allowing evaluation through straightforward accuracy metrics. Although the evaluation relied solely on LLM-as-a-Judge, the deterministic and fact-based nature of the benchmark tasks minimizes heuristic bias, making statistical significance testing less critical. 3.3 Datasets The proposed system is benchmarked on multiple finance-oriented datasets, each emphasizing distinct aspects of retrieval and reasoning. Each dataset sample consists of a natural-language query, a set of chunked corpora, the corresponding ground truth, and document sources. Collectively, these datasets provide a comprehensive benchmark suite for assessing both retrieval effectiveness and numerically grounded generation in real-world financial scenarios. • FinQABench: Based on 10-K filings; focuses on detecting hallucinations in generated answers and ensuring factual correctness. • FinQA: Derived from earnings reports; evaluates multi-step numerical reasoning using both tabular and textual data. • ConvFinQA: Also based on earnings reports; assesses model performance on conversational financial queries. • FinanceBench: Built on 10-K filings; measures a system’s ability to handle real-world financial questions with domain precision. • TATQA: Composed of financial reports; tests arithmetic, comparative, and logical reasoning over hybrid tabular–text data. 4 Method The proposed financial RAG system is composed of three sequential stages. The first two stages correspond to Task 1 (Retrieval), while the third stage corresponds to Task 2 (Generation). Overall, the pipeline is divided into three core phases: Pre-Retrieval, Retrieval, and Generation. As illustrated in Figure 1, the blue components represent the retrieval process, while the green components correspond to generation. 4.1 Pre-Retrieval The Pre-Retrieval phase is designed to enhance the interpretability of financial queries and the consistency of the document corpus before embedding-based retrieval. Financial texts often contain abbreviations, implicit relations, and domain-specific terminology, which often lead to semantic mismatches between queries and document embeddings. To mitigate these issues, we propose a four-stage pre-processing pipeline that improves both query clarity and corpus normalization. 4.1.1 Normalization All input queries are first normalized to reduce lexical ambiguity. This step includes lowercase, typographical correction, and expansion of financial abbreviations (e.g., EPS: Earnings per share, YoY: Year-over-Year). Measurement units such as “K” or “M” are standardized into consistent numeric expressions, and missing contextual terms are restored based on document metadata. This ensures that semantically equivalent expressions share a uniform representation within the embedding space. The corpus is normalized likewise for consistency. 4.1.2 Keyword Extraction Financial-specific keywords are extracted and appended to each query to strengthen the alignment between query and corpus embeddings. This increases the density of financial terminology within the input, allowing the retriever to better capture hybrid text–table semantics commonly found in financial documents. 4.1.3 Paraphrasing Each normalized query is semantically expanded through paraphrasing. This step generates multiple linguistically diverse but semantically equivalent variants, enabling the retriever to generalize across different formulations of financial questions (e.g., “What is Apple’s revenue in 2023?” vs. “Report Apple’s 2023 revenue”). This expansion broadens the search space without introducing significant computational overhead. 4.1.4 Hypothetical Document Generation (HyDE) Inspired by HyDE-based methods, a synthetic pseudo-document is generated for each query to represent a plausible context where the answer could appear. These hypothetical passages act as semantic anchors that enhance embedding alignment between the abstract query and the domain-specific corpus. 4.1.5 Table-to-json During the corpus pre-processing stage, it is crucial to maintain semantic coherence across documents while ensuring the accuracy of numerical information. Accordingly, each document was semantically chunked at the sentence level to prevent contextual fragmentation. In addition, since large-scale tables in financial documents contain key quantitative information, Markdown-style tables were converted into JSON structures (table-to-json) using a rule-based script, rather than relying on LLM-based conversion, to prevent potential hallucinations. This process explicitly preserves the relationships between numerical values and their headers, thereby strengthening the semantic alignment between textual and numerical data during the embedding and retrieval stages. The effectiveness of this transformation was validated through comparative experiments between the Markdown format (original tables) and the JSON format (table-to-json). 4.1.6 Summary When the corpus size is extremely large (over 10k tokens), utilizing all documents as-is may be inefficient and could lower retrieval performance. To address this, each document was replaced through summarization substitution, preserving only the essential financial indicators, results, and contextual information. This compression removes unnecessary narrative sentences while prioritizing semantically central statements, thereby improving both the efficiency and accuracy of the retrieval stage. Most Pre-Retrieval steps were performed using Claude-Opus-4.6 Anthropic (2026), while certain quantitative tasks, such as table conversion, were executed through rule-based scripts. Through this bidirectional normalization and structuring process applied to both queries and corpora, the system maximizes semantic compatibility between complex textual–numerical data in the financial domain and enables more precise and reliable retrieval in subsequent stages. 4.2 Retrieval The Retrieval stage identifies the most semantically and numerically relevant passages from a large-scale financial corpus, bridging the preprocessed query and the downstream generator. The design objective is twofold: maximize retrieval precision on hybrid text–table evidence, and bound inference cost so that the system remains deployable at institutional scale. We therefore avoid a single monolithic reranker and instead employ a two-stage hierarchical reranker in which a small, fast model and a large, accurate model cooperate. Each model is specialized for the regime where it dominates: the small model handles coarse lexical pruning across the full candidate pool, while the large model performs deep semantic adjudication on a much smaller surviving set. This division of labor decouples breadth from depth and concentrates expensive computation where it has the highest marginal value. 4.2.1 Stage 1: Lightweight Filtering In the first stage, a fast, low-complexity jina-reranker-v3 Wang et al. (2025) is used to eliminate low-relevance or noisy candidates. This model is optimized for lexical and shallow semantic similarity, leveraging extended context support up to 131k tokens. By narrowing the candidate pool to the top 100 passages, it significantly reduces downstream computational load without sacrificing recall. 4.2.2 Stage 2: Fine-Grained Semantic Reranking The second stage applies a high-capacity reranker to re-evaluate the top 100 candidates and extract the final top 20 passages. This reranker focuses on fine-grained contextual relationships, such as cross-sentence financial dependencies and numerical consistency across tables and text. Through this hierarchical refinement, the system effectively captures both semantic and quantitative correspondence, which is crucial for hybrid financial corpora. This architecture achieves an optimal trade-off between precision and efficiency. The lightweight first stage prevents unnecessary computation on irrelevant candidates, while the second stage provides the semantic depth needed to identify financially meaningful evidence. This separation of lexical filtering and contextual reasoning enables robust retrieval performance even under high-volume workloads, making the approach scalable for institutional use. Financial documents often combine narrative text and tabular disclosures, requiring models to align textual descriptions with structured numerical information. To handle this, the retrieval module leverages both the normalized corpus (from the Pre-Retrieval phase) and the JSON-formatted tabular data, allowing the reranker to compute cross-modal similarity between natural language and numeric fields. This design improves factual consistency and ensures that retrieved contexts are suitable for quantitative reasoning. The final output consists of the top 20 ranked passages, which collectively form a compact yet information-rich context for the generation stage. This cap balances semantic coverage and token efficiency, ensuring that subsequent long-context generation operates within model input limits while retaining all essential evidence. Algorithm 1 Proposed Framework Input: Query Q, Corpus C Output: Response R 1: Q′←Normalization(Q)∪Keywords-Extraction(Q)Q (Q) -Extraction(Q) 2: C′←Normalization(C)∪Table2Json(C)C (C) 2Json(C) 3: C100←Rerank1(Q′,C′)C_100 _1(Q ,\,C ) 4: C20←Rerank2(Q′,C100)C_20 _2(Q ,\,C_100) 5: if tokens(Q′∪C20)≤64ktokens(Q ∪ C_20)≤ 64k then 6: R←LLM(Q′,C20)R (Q ,\,C_20) 7: else 8: R1←LLM(Q′,C1:10)R_1 (Q ,\,C_1:10) 9: R2←LLM(Q′,C11:20)R_2 (Q ,\,C_11:20) 10: R←Fusion(R1,R2)R (R_1,\,R_2) 11: end if 12: return R Table 1: Ablation study of Pre-Retrieval components: Norm denotes normalization of queries and corpora, including abbreviation expansion, unit standardization, and typo or grammar correction. HyDE represents Hypothetical Document Embedding, where a pseudo-document is generated to enhance semantic alignment between the query and the corpus. Query Corpus NDCG@20 Original Norm Keywords Paraphrased HyDE Original Norm Table-to-json Summary Extraction ∘ - - - - ∘ - - - 0.7323 - ∘ - - - ∘ - - - 0.7446 - ∘ ∘ - - ∘ - - - 0.7542 - ∘ - ∘ - ∘ - - - 0.7347 - ∘ - - ∘ ∘ - - - 0.7347 ∘ - - - - - ∘ - - 0.7211 - ∘ - - - - ∘ - - 0.7333 - ∘ ∘ - - - ∘ - - 0.7503 - ∘ - ∘ - - ∘ - - 0.7446 - ∘ - - ∘ - ∘ - - 0.6759 ∘ - - - - - ∘ ∘ - 0.7301 - ∘ - - - - ∘ ∘ - 0.7529 - ∘ ∘ - - - ∘ ∘ - 0.7918 - ∘ - ∘ - - ∘ ∘ - 0.7677 - ∘ - - ∘ - ∘ ∘ - 0.6843 ∘ - - - - - - - ∘ 0.6544 - ∘ - - - - - - ∘ 0.6501 - ∘ ∘ - - - - - ∘ 0.6579 - ∘ - ∘ - - - - ∘ 0.6542 - ∘ - - ∘ - - - ∘ 0.5853 Table 2: Comparison of Hierarchical Reranker Combinations 1st 2nd NDCG@20 Reranker Reranker - 0.7260 Linq-Embed-Mistral 0.7378 jina-reranker-v3 gte-Qwen2-7B-instruct 0.7567 Qwen3-Reranker-4B 0.7763 Qwen3-Reranker-8B 0.7918 4.3 Generation Although recent LLMs nominally accept extremely long inputs OpenAI (2026); Anthropic (2026), multiple studies Databricks (2024); Jin et al. (2024); Paulsen (2025) report that response quality, and in particular numerical fidelity, degrades well before the advertised context limit. The gap between nominal and effective context length is especially consequential in finance, where 10-K filings routinely exceed 100k tokens and where a single misread cell can invalidate downstream reasoning. The generation stage must therefore decide not how much context to feed the model, but how to feed it in a way that preserves accuracy. 4.3.1 Context Size Management To empirically identify a reliable operational threshold, we conducted ablation studies using several state-of-the-art LLMs under long-context and numerically intensive conditions. We observed a noticeable decline in performance when input size exceeded 64k tokens. Based on this observation, we set 64k as the context threshold for all subsequent experiments. 4.3.2 Fusion When the combined input exceeds 64k tokens, the top-20 corpora are divided into two semantically coherent subsets, producing interim answers R1R_1 and R2R_2. The two outputs are then merged through a conditional fusion process: the system first identifies whether each interim response contains a definitive answer; if only one does, that answer is directly adopted; if both contain valid answers, the one with the higher confidence value is selected; and if neither provides a clear answer, the model outputs an explicit “unknown” response. This adaptive fusion ensures consistent and interpretable generation under long-context conditions while preventing hallucinated synthesis across partitions. Table 3: Comparison of LLM Performance with/without Context Management LLM Context Accuracy Management Gemini 3.0 Pro - 0.7593 ∘ 0.7610 GPT-5.4 - 0.7786 ∘ 0.7794 Grok-4.20 - 0.7901 ∘ 0.7938 Claude-4.6 Opus - 0.8103 ∘ 0.8152 5 Results This section presents the experimental results of the proposed framework, including analyses of Pre-Retrieval design, Hierarchical Reranker architecture, and Long-Context Management. 5.1 Pre-Retrieval Design Table 1 summarizes the retrieval performance under various combinations of query and corpus preprocessing methods. The results clearly show that normalization yields the highest performance gain. Combining Query Normalization, Keyword Extraction, and Table-to-json Conversion achieved the best score (NDCG@20 = 0.7918), outperforming the baseline (no Pre-Retrieval) by +5.9%. This confirms that pre-retrieval normalization significantly improves semantic coherence and retrieval performance within financial documents. 5.2 Impact of Hierarchical Reranker Table 2 compares different reranking combinations. Using jina-reranker-v3 as the lightweight first-stage model followed by Qwen3-Reranker-8B Zhang et al. (2025) as the second-stage model achieved the best performance (NDCG@20 = 0.7918). This hierarchical reranking improved relevance by +6.5% compared to a single-reranker setup. By separating a lightweight model for fast filtering (Stage1) and a larger model for fine-grained reranking (Stage2), the approach aimed to mitigate the inference time limitation while maintaining retrieval performance. 5.3 Long-Context Management Table 3 presents the impact of the proposed context segmentation and fusion strategy. Across all tested LLMs, applying the split-and-fusion mechanism resulted in a slight accuracy improvement ranging from 0.08% to 0.49%. Claude-4.6 Opus achieved the highest performance (Accuracy = 0.8152) under the 64k-token threshold, indicating that context segmentation had only a marginal effect on overall accuracy. 5.4 Summary Overall, the proposed Hierarchical Reranker framework showed consistent and statistically stable improvements in both retrieval and generation performance. These results can be attributed to three key components: (1) optimized pre-retrieval algorithms, (2) a hierarchical reranker architecture that enhances retrieval precision, and (3) long-context management for stable reasoning across extended inputs. The system provides a reliable, scalable, and domain-adaptive solution, supporting its applicability to financial document analysis. 6 Discussion This section discusses the strengths, limitations, and future directions of the proposed finance-specific RAG system. We first highlight how each component contributes to the system’s effectiveness in real-world financial tasks, then outline key computational and scalability limitations, and finally present potential avenues for improvement and research extension. 6.1 Contributions Three complementary components — Pre-Retrieval Optimization, the Hierarchical Reranker, and Long-Context Management — jointly produced consistent gains across retrieval precision, factual consistency, and reasoning stability. The Pre-Retrieval phase is the simplest yet highest-leverage stage: deterministic normalization, keyword augmentation, and Markdown-to-JSON table conversion lifted NDCG@20 by +5.9% over the no-preprocessing baseline, and did so without introducing LLM-induced hallucinations during preprocessing. This kind of conservative, rule-based engineering is what makes the system reproducible enough for regulated environments such as auditing and investment research. The hierarchical reranker complements this by realizing a small-and-large model collaboration: the lightweight first-stage model bounds compute, while the high-capacity second-stage model concentrates effort on the candidates most likely to matter. The two-stage cascade improved NDCG@20 by +6.5% over a single-reranker baseline while keeping per-query inference cost compatible with institutional throughput. We see this pattern — specialize small models for breadth and large models for depth — as broadly applicable to other vertical RAG settings. 6.2 Limitations Despite its effectiveness, the system entails several limitations that warrant attention. First, the hierarchical reranking architecture, while improving precision, introduces additional computational overhead compared to single-stage rerankers. This cost can be mitigated through user-guided search constraints — for instance, extracting company names from the user query and using them to substantially narrow the scope of candidate documents before reranking is invoked. Second, the reliance on a fixed 64k-token context threshold, though empirically validated, restricts scalability when processing extremely long financial reports or multi-document reasoning tasks. Finally, while the framework was evaluated on multiple financial benchmarks, further validation across multilingual or real-time financial streams remains an open challenge for industrial deployment. 6.3 Future Work We see three natural extensions. First, dynamic context prioritization Ikram et al. (2025) would replace the current fixed 64k threshold with importance-weighted allocation of model attention, reducing information loss during segmentation and fusion on very long filings. Second, we plan to push the small-and-large model collaboration toward adaptive, query-conditional reranking: invoke the heavy second-stage reranker only when the first stage exhibits low confidence, further compressing token spend on routine queries while preserving accuracy on hard ones. Third, embedding the pipeline inside a broader agentic workflow — with expert-in-the-loop feedback signals and tool use for numerical verification — could close the remaining gap between retrieval-grounded answers and analyst-grade financial reasoning, moving the system from a single-turn RAG pipeline toward an autonomous financial-analysis agent. 7 Conclusion Financial document analysis sits at the intersection of high economic value and high expertise cost, making it one of the most natural targets for LLM- and RAG-based automation. Global financial institutions Wu et al. (2023); Papasotiriou et al. (2024) have already begun integrating retrieval-grounded LLMs into their operational workflows, but the gap between research-grade RAG and institutional-grade deployment remains substantial. This work narrows that gap. We presented Hierarchical Reranker, a finance-specific RAG framework that couples deterministic Pre-Retrieval Optimization, a small-and-large model Hierarchical Reranker, and adaptive Long-Context Management into a single deployable pipeline. Comprehensive ablations show that each component contributes measurable gains, and the system as a whole achieved NDCG@20 = 0.7918 and secured 2nd place in the ACM-ICAIF ’24 FinanceRAG Challenge. Just as importantly, the framework is now running inside real auditing and investment workflows, supporting the claim that careful engineering — not only scale — is what carries RAG into production. We hope this study offers a concrete reference point for vertical RAG design in finance and contributes to the broader trajectory of LLM-based, agentic systems for high-stakes financial reasoning. References C. An, J. Zhang, M. Zhong, L. Li, S. Gong, Y. Luo, J. Xu, and L. Kong (2024) Why does the effective context length of LLMs fall short?. Note: arXiv:2410.18745 External Links: 2410.18745, Link Cited by: §2. Anthropic (2026) Introducing Claude 4.6. Note: https://w.anthropic.com/claude/opusOnline documentation. Accessed: 2026-02-05 Cited by: §2, §4.1.6, §4.3. Z. Chen, W. Chen, C. Smiley, S. Shah, I. Borova, D. Langdon, R. Moussa, M. Beane, T. Huang, B. Routledge, and W. Y. Wang (2022a) FinQA: a dataset of numerical reasoning over financial data. Note: arXiv:2109.00122 External Links: 2109.00122, Link Cited by: §2. Z. Chen, S. Li, C. Smiley, Z. Ma, S. Shah, and W. Y. Wang (2022b) ConvFinQA: exploring the chain of numerical reasoning in conversational finance question answering. Note: arXiv:2210.03849 External Links: 2210.03849, Link Cited by: §2. C. Choi, J. Sohn, Y. Lee, S. Pang, J. Ha, H. Ryoo, Y. Kim, H. Choi, and J. Kwon (2024) ACM-ICAIF ’24 FinanceRAG challenge. Note: https://kaggle.com/competitions/icaif-24-finance-rag-challengeKaggle Cited by: §1. Databricks (2024) The long context RAG capabilities of OpenAI o1 and Google Gemini. Note: https://w.databricks.com/blog/long-context-rag-capabilities-openai-o1-and-google-geminiBlog post. Accessed: 2024-10-27 Cited by: §1, §2, §4.3. W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T. Chua, and Q. Li (2024) A survey on RAG meeting LLMs: towards retrieval-augmented large language models. Note: arXiv:2405.06211 External Links: 2405.06211, Link Cited by: §2. L. Gao, X. Ma, J. Lin, and J. Callan (2022) Precise zero-shot dense retrieval without relevance labels. Note: arXiv:2212.10496 External Links: 2212.10496, Link Cited by: §2. Google (2026) Introducing Gemini 3.0. Note: https://deepmind.google/models/gemini/Online documentation. Accessed: 2025-11-19 Cited by: §2. J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liu, S. Wang, K. Zhang, Y. Wang, W. Gao, L. Ni, and J. Guo (2025) A survey on LLM-as-a-judge. Note: arXiv:2411.15594 External Links: 2411.15594, Link Cited by: §3.2. A. Ikram, X. Li, S. Elnikety, and S. Bagchi (2025) Ascendra: dynamic request prioritization for efficient LLM serving. Note: arXiv:2504.20828 External Links: 2504.20828, Link Cited by: §6.3. P. Islam, A. Kannappan, D. Kiela, R. Qian, N. Scherrer, and B. Vidgen (2023) FinanceBench: a new benchmark for financial question answering. Note: arXiv:2311.11944 External Links: 2311.11944, Link Cited by: §2. R. Jagerman, H. Zhuang, Z. Qin, X. Wang, and M. Bendersky (2023) Query expansion by prompting large language models. Note: arXiv:2305.03653 External Links: 2305.03653, Link Cited by: §2. B. Jin, J. Yoon, J. Han, and S. O. Arik (2024) Long-context LLMs meet RAG: overcoming challenges for long inputs in RAG. arXiv preprint arXiv:2410.05983. Cited by: §4.3. S. N. Lee and S. C. Han (2025) Large language models in finance (FinLLMs). Neural Computing and Applications 37 (30), p. 24853–24867. External Links: ISSN 1433-3058, Document, Link Cited by: §1, §2. Y. Li, B. Dong, C. Lin, and F. Guerin (2023) Compressing context to enhance inference efficiency of large language models. Note: arXiv:2310.06201 External Links: 2310.06201, Link Cited by: §2. LighthouzAI (2024) FinQABench: a new QA benchmark for finance applications. Note: https://huggingface.co/datasets/lighthouzai/finqabench Cited by: §2. J. Liu and B. Mozafari (2024) Query rewriting via large language models. Note: arXiv:2403.09060 External Links: 2403.09060, Link Cited by: §2. X. Ma, X. Zhang, R. Pradeep, and J. Lin (2023) Zero-shot listwise document reranking with a large language model. Note: arXiv:2305.02156 External Links: 2305.02156, Link Cited by: §2. OpenAI (2026) Introducing GPT-5.4. Note: https://openai.com/index/introducing-gpt-5-4Online documentation. Accessed: 2026-03-05 Cited by: §2, §4.3. K. Papasotiriou, S. Sood, S. Reynolds, and T. Balch (2024) AI in investment analysis: LLMs for equity stock ratings. In Proceedings of the 5th ACM International Conference on AI in Finance, ICAIF ’24, p. 419–427. External Links: Document, Link Cited by: §1, §7. N. Paulsen (2025) Context is what you need: the maximum effective context window for real world limits of LLMs. Note: arXiv:2509.21361 External Links: 2509.21361, Link Cited by: §4.3. F. Wang, Y. Li, and H. Xiao (2025) Jina-reranker-v3: last but not late interaction for listwise document reranking. Note: arXiv:2509.25085 External Links: 2509.25085, Link Cited by: §4.2.1. S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann (2023) BloombergGPT: a large language model for finance. Note: arXiv:2303.17564 External Links: 2303.17564, Link Cited by: §1, §7. xAI (2026) Grok 4.20. Note: https://x.ai/grokOnline documentation. Accessed: 2026-02-17 Cited by: §2. Y. Yang, Y. Tang, and K. Y. Tam (2023) InvestLM: a large language model for investment using financial domain instruction tuning. Note: arXiv:2309.13064 External Links: 2309.13064, Link Cited by: §1. X. Yu, P. Jian, and C. Chen (2025) TableRAG: a retrieval augmented generation framework for heterogeneous document reasoning. Note: arXiv:2506.10380 External Links: 2506.10380, Link Cited by: §1. Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, F. Huang, and J. Zhou (2025) Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §5.2. F. Zhu, W. Lei, Y. Huang, C. Wang, S. Zhang, J. Lv, F. Feng, and T. Chua (2021) TAT-QA: a question answering benchmark on a hybrid of tabular and textual content in finance. Note: arXiv:2105.07624 External Links: 2105.07624, Link Cited by: §2.