Paper deep dive
Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines
Akshay Gulati, Kanha Singhania, Tushar Banga, Parth Arora, Anshul Verma, Vaibhav Kumar Singh, Agyapal Digra, Jayant Singh Bisht, Danish Sharma, Varun Singla, Shubh Garg
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/13/2026, 12:55:08 AM
Summary
The paper introduces the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional framework designed to evaluate the financial reasoning, factual accuracy, and analytical completeness of large language models (LLMs) in professional investment research contexts. By testing five AI systems—GPT, Gemini, Perplexity, Claude, and SuperInvesting—against 95+ structured financial queries, the study identifies that SuperInvesting outperforms others in aggregate accuracy and completeness, while retrieval-oriented models like Perplexity excel in data recency but lag in analytical synthesis.
Entities (6)
Relation Signals (3)
AFIB → evaluates → SuperInvesting
confidence 100% · We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions
AFIB → measures → Financial Intelligence
confidence 95% · The AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities
SuperInvesting → performsbetterthan → Perplexity
confidence 90% · SuperInvesting achieves the highest aggregate performance... Retrieval-oriented systems such as Perplexity... exhibit weaker analytical synthesis
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models are increasingly used for financial analysis and investment research, yet systematic evaluation of their financial reasoning capabilities remains limited. In this work, we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities across five dimensions: factual accuracy, analytical completeness, data recency, model consistency, and failure patterns. We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions derived from real-world equity research tasks. The results reveal substantial differences in performance across models. Within this benchmark setting, SuperInvesting achieves the highest aggregate performance, with an average factual accuracy score of 8.96/10 and the highest completeness score of 56.65/70, while also demonstrating the lowest hallucination rate among evaluated systems. Retrieval-oriented systems such as Perplexity perform strongly on data recency tasks due to live information access but exhibit weaker analytical synthesis and consistency. Overall, the results highlight that financial intelligence in large language models is inherently multi-dimensional, and systems that combine structured financial data access with analytical reasoning capabilities provide the most reliable performance for complex investment research workflows.
Tags
Links
- Source: https://arxiv.org/abs/2603.08704v1
- Canonical: https://arxiv.org/abs/2603.08704v1
Trouble viewing inline? Open PDF directly →
Full Text
59,857 characters extracted from source content.
Expand or collapse full text
Evaluating Financial Intelligence in Large Language Models: Benchmarking SuperInvesting AI with LLM Engines Akshay Gulati ∗ , Kanha Singhania ∗ , Tushar Banga ∗ , Parth Arora ∗ , Anshul Verma ∗ , Vaibhav Kumar Singh ∗ , Agyapal Digra ∗ , Jayant Singh Bisht ∗ , Danish Sharma ∗ , Varun ∗ , and Shubh Garg ∗ ∗ The Future University Abstract—Large language models are increasingly used for financial analysis and investment research, yet systematic evalu- ation of their financial reasoning capabilities remains limited. In this work, we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess financial analysis capabilities across five dimensions: factual accuracy, analytical completeness, data recency, model consistency, and failure patterns. We evaluate five AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, using a dataset of 95+ structured financial analysis questions derived from real-world equity research tasks. The results reveal sub- stantial differences in performance across models. Within this benchmark setting, SuperInvesting achieves the highest aggregate performance, with an average factual accuracy score of 8.96/10 and the highest completeness score of 56.65/70, while also demonstrating the lowest hallucination rate among evaluated systems. Retrieval-oriented systems such as Perplexity perform strongly on data recency tasks due to live information access but exhibit weaker analytical synthesis and consistency. Overall, the results highlight that financial intelligence in large language models is inherently multi-dimensional, and systems that com- bine structured financial data access with analytical reasoning capabilities provide the most reliable performance for complex investment research workflows. I. INTRODUCTION Large language models (LLMs) have rapidly advanced the state of natural language understanding and reasoning. Recent models such as GPT-3[1], PaLM [2],and nd GPT- 4 [3]demonstrate strong performance across a wide range of language and reasoning tasks, enabling applications in domains including software engineering, scientific research, and professional decision support. Their capabilities are com- monly evaluated using large-scale benchmark suites such as MMLU[4] and BIG-Bench[5], while broader evaluation frameworks such as HELM[6] attempt to assess language models across multiple capability and risk dimensions. These evaluation efforts have provided valuable insights into general reasoning ability, knowledge coverage, and safety considera- tions in large-scale language models. At the same time, recent work has highlighted important limitations in existing evaluation frameworks. Research on hallucination and factual reliability in language generation demonstrates that LLMs can produce fluent but factually in- correct outputs[7], [8], [9].Benchmarks such as TruthfulQA[8] attempt to quantify models’ tendency to generate mislead- ing information, while interactive evaluation platforms such as Chatbot Arena[10] measure model performance through human preference comparisons. However, these evaluations primarily focus on general knowledge or conversational tasks and do not capture the domain-specific requirements of high- stakes analytical workflows. One domain where these limitations are particularly im- portant is financial analysis. LLMs are increasingly used by investors, analysts, and financial institutions to retrieve financial metrics, interpret corporate disclosures, summarize macroeconomic developments, and generate investment theses. Financial reasoning differs from many standard NLP tasks in that it requires precise numerical accuracy, integration of multiple financial statements, awareness of recent market developments, and coherent analytical synthesis across het- erogeneous data sources. Errors in such analyses can directly influence investment decisions, making reliable evaluation of AI systems in this domain especially important. Recent research has begun exploring the application of language models in finance. Domain-adapted models such as FinBERT[11] and BloombergGPT[12] demonstrate that train- ing language models on financial corpora can improve per- formance on financial NLP tasks such as sentiment analysis, document classification, and question answering. Similarly, emerging benchmarks such as FinanceBench[13] evaluate the ability of LLMs to answer questions based on corporate financial disclosures. While these efforts represent important progress, most existing evaluations focus on isolated informa- tion retrieval or document understanding tasks, rather than the full analytical workflow required in professional investment research. This gap motivates the need for evaluation frameworks that measure financial intelligence in AI systems more directly. In this paper we introduce the AI Financial Intelligence Benchmark (AFIB), a multi-dimensional evaluation framework designed to assess the capability of AI systems to perform financial analysis tasks relevant to capital markets. Rather than focusing on a single capability, AFIB evaluates models across five dimensions that reflect the practical requirements of financial research workflows: factual accuracy and hallucina- arXiv:2603.08704v1 [cs.AI] 9 Mar 2026 tion resistance, analytical completeness, data recency and news integration, model consistency, and real-world failure patterns. The benchmark evaluates five widely used AI systems: GPT, Gemini, Perplexity, Claude, and SuperInvesting, across 71 financial analysis queries covering corporate financial metrics, macroeconomic developments, and complex conglomerate analysis in the Indian equity market. In addition to controlled benchmark tasks, we incorporate a dataset of 432 negatively rated assistant responses collected from a production financial AI deployment, enabling the analysis of real-world failure modes that are rarely captured in conventional benchmarks. Our empirical results reveal substantial capability differ- ences across models. Retrieval-based systems demonstrate advantages in incorporating recent financial events, while domain-specialized models show stronger performance in nu- merical accuracy, analytical completeness, and cross-statement reasoning. These findings highlight that financial analysis performance in AI systems is multi-dimensional, and that architectures combining structured financial data access with analytical reasoning capabilities appear better suited for in- vestment research tasks. This work makes three primary contributions: • A multi-dimensional benchmark for evaluating finan- cial intelligence in AI systems, capturing five capability dimensions relevant to professional investment analysis. • A curated evaluation dataset spanning 71 financial analysis queries and a deployment dataset of 432 real- world failure cases, enabling both controlled benchmark- ing and empirical failure analysis. • A comparative evaluation of leading AI models, identifying systematic strengths and limitations across accuracy, completeness, recency, consistency, and hallu- cination resistance. Together, these results provide a structured framework for evaluating the reliability of AI systems used in financial analysis and highlight key challenges in deploying language models in high-stakes decision-making environments. I. RELATED WORK The evaluation of large language models (LLMs) has be- come a central research problem as these systems are increas- ingly deployed in real-world decision-support settings. Prior work relevant to this study spans four main areas: general- purpose LLM benchmarking, evaluation frameworks and auto- mated judging methods, reliability and hallucination analysis, and financial domain language models and benchmarks. While these lines of work have significantly advanced the evaluation of language models, existing approaches only partially capture the analytical requirements of professional financial research. While these lines of work have significantly advanced our understanding of LLM capabilities, they leave several aspects of financial analysis evaluation underexplored. A. Benchmarking Large Language Models A large body of research has focused on developing bench- marks to measure the reasoning and knowledge capabilities of LLMs. One of the most widely used evaluation datasets is MMLU[4], which measures model performance across 57 academic disciplines. Similarly, BIG-Bench[5]aggregates hun- dreds of tasks designed to evaluate reasoning, commonsense knowledge, and linguistic competence.Several works have attempted to extend these benchmarks into more comprehen- sive evaluation frameworks. HELM[6] introduced a holistic evaluation framework that assesses language models across multiple axes including accuracy, calibration, robustness, and fairness. Building on this idea, Chatbot Arena and MT-Bench [10] evaluate conversational models using large-scale human preference comparisons, providing dynamic leaderboards for frontier models.More recently, several benchmarks have begun to explore domain-specific evaluation of LLM capabilities. FinBen [16] introduces a large-scale financial benchmark containing dozens of datasets spanning tasks such as financial information extraction, reasoning, and decision support. Fi- nanceQA [17] evaluates LLMs on complex financial analysis questions derived from investment research scenarios, demon- strating that many models struggle with multi-step financial reasoning. Other recent benchmarks such as FinAuditing [18] and Fin-RATE [19] extend evaluation to structured financial documents and longitudinal financial analysis across multiple filings. These efforts highlight the growing interest in eval- uating LLMs for financial applications, but typically focus on isolated tasks rather than the broader analytical workflows involved in investment research. More recent research has explored automated and scalable evaluation approaches. AlpacaEval[20] and its improved ver- sion AlpacaEval 2.0[21] propose automated evaluation frame- works calibrated against human judgments. Similarly,Arena- Hard[22]introduces a challenging benchmark designed to stress-test frontier language models.Another emerging direc- tion is the use of LLM-as-a-judge evaluation frameworks, where language models themselves are used to evaluate outputs of other models [23], [10].While these approaches significantly improve the scalability of evaluation, they remain primarily focused on general reasoning and conversational tasks rather than domain-specific analytical workflows. B. Reliability and Hallucination in LLM Systems The reliability of language model outputs has emerged as a major concern in recent research. Early work by Maynez et al. (2020) [7]demonstrated that neural text generation systems fre- quently produce outputs that are fluent but factually incorrect. Building on this insight, TruthfulQA[8]introduced a bench- mark designed to measure whether language models repro- duce common misconceptions.Subsequent work has explored systematic analysis of hallucination behavior in language models.Ji et al. (2023) [9] provide a comprehensive survey of hallucination phenomena in natural language generation. More recent work has introduced benchmarks specifically designed to evaluate hallucination in complex domains. PHANTOM [24] evaluates hallucination detection in long-context finan- cial question answering tasks, while FAITH [25] analyzes hallucination behavior in tabular financial data extraction. TABLE I COMPARISON OF AFIB WITH EXISTING FINANCIAL LLM BENCHMARKS. BenchmarkYearFATNRMDERFDFARE FinQA[14]2021✗✓✗✓✗ FinanceBench [15]2023✗✓✗✓✗ FinBen [16]2024✓✗✓✗ FinanceQA [17]2025✓✗✓✗ FinAuditing [18]2025✓✗✓✗ Fin-RATE [19]2026✓✗ AFIB (Ours)2026✓ FAT: Financial Analysis Tasks, NR: Numerical Reasoning, MDE: Multi-Dimensional Evaluation, RFD: Real Financial Data, FA: Failure Analysis, RE: Recency Evaluation. These studies highlight the particular challenges associated with numerical reasoning and factual grounding in financial contexts.Additional studies have examined calibration and uncertainty in LLM outputs [26] as well as emergent risks associated with large-scale language models [27]. More recent evaluation frameworks have focused on trust- worthiness and safety. TrustLLM[28] proposes a multi- dimensional framework for evaluating trustworthiness in LLM systems, while Holistic LLM evaluation frameworks[6], [23] emphasize the importance of evaluating models across mul- tiple reliability dimensions.However, these benchmarks typi- cally evaluate hallucination and factuality in general knowl- edge tasks, rather than in contexts where numerical precision and temporal accuracy are critical, such as financial analysis. C. Financial NLP and Financial Language Models Parallel to developments in general LLM evaluation, re- searchers have explored the use of language models in finan- cial applications. One of the earliest domain-adapted models is FinBERT[11],which demonstrated improved performance on financial sentiment analysis tasks.More recently, large-scale financial language models have emerged. BloombergGPT[12] is a domain-specific LLM trained on financial data and pro- prietary financial documents. Similarly, FinGPT [29], [30] proposes an open-source framework for building financial AI systems that integrate LLMs with financial datasets and market information. Financial reasoning datasets have also been proposed. FinQA[14] evaluates models on numerical reasoning tasks involving financial tables, while ConvFinQA[31] extends this dataset to conversational financial reasoning tasks. More re- cently, FinanceBench[15] evaluates LLMs on question an- swering tasks derived from corporate financial reports.More recent benchmarks have explored financial decision-making and agent-based evaluation. InvestorBench [32] evaluates LLM agents on investment strategy tasks such as portfolio con- struction and market trading simulations. Similarly, the Agent Market Arena benchmark [33] studies the behavior of LLM- based trading agents operating in simulated financial markets. These studies illustrate the growing interest in applying LLMs to financial decision-support systems.Recent surveys of LLM applications in finance[34], [35]highlight the growing use of language models for tasks such as earnings call anal- ysis, automated financial reporting, and investment research support.Despite these advances, most existing financial NLP benchmarks focus on document understanding or question answering tasks, rather than the broader analytical workflows required in professional investment research. D. Gap in Existing Benchmarks Taken together, existing work reveals an important gap in the evaluation of AI systems for financial analysis. General- purpose benchmarks such as MMLU, BIG-Bench, and HELM evaluate broad reasoning capabilities but do not capture the domain-specific analytical processes required in professional financial research. Conversely, financial benchmarks such as FinQA, FinanceBench, FinBen, and FinanceQA primarily fo- cus on document-level question answering or isolated rea- soning tasks rather than evaluating the complete analytical workflow used by financial analysts. In practice, investment research requires the integration of multiple financial statements, interpretation of financial metrics within business context, incorporation of recent market developments, and the construction of coherent analytical narratives.To address this gap, we introduce the AI Financial Intelligence Benchmark (AFIB), a framework designed to evaluate AI systems across multiple dimensions of financial intelligence, including numerical accuracy, analytical com- pleteness, data recency awareness, reasoning consistency, and real-world failure patterns observed in deployed financial AI systems.Table I compares AFIB with recent benchmarks for financial language models. I. METHODOLOGY A. Benchmark Overview The AI Financial Intelligence Benchmark (AFIB) was de- veloped to evaluate the capability of modern AI systems to perform financial analysis tasks comparable to those under- taken by professional investment analysts. Unlike conventional language model benchmarks that focus primarily on general reasoning or factual recall, financial analysis requires the integration of numerical accuracy, financial statement inter- pretation, macroeconomic awareness, and structured analytical reasoning. The benchmark therefore evaluates models across multiple complementary dimensions that together capture the analytical workflow of institutional equity research.Figure 1 illustrates the overall AFIB evaluation pipeline, including the Fig. 1. Overview of the AI Financial Intelligence Benchmark (AFIB) evaluation pipeline. Financial analysis queries are constructed from multi-sector datasets and executed across multiple AI models. Model outputs are evaluated using a multi-dimensional scoring framework capturing accuracy, analytical depth, completeness, recency, consistency, and failure patterns. construction of financial query datasets, the parallel execu- tion of queries across multiple AI models, and the multi- dimensional evaluation framework used to compute the final benchmark scores. The evaluation framework was designed around five capabil- ity dimensions: factual accuracy and hallucination resistance, analytical depth and valuation logic, completeness of financial analysis, data recency and news integration, and consistency across repeated queries. Each dimension corresponds to a distinct analytical capability required in real-world financial research environments. These dimensions were implemented as independent benchmark modules that were applied consis- tently across all evaluated models. The study evaluates five AI systems : GPT, Gemini, Per- plexity, Claude, and SuperInvesting across structured financial analysis questions covering Indian listed equities. The dataset includes companies from multiple sectors of the Indian econ- omy, including large-cap conglomerates, banking institutions, information technology services firms, consumer goods com- panies, and emerging industries such as renewable energy and defence manufacturing. This sectoral diversity ensures that models are tested across different business models, regulatory environments, and capital structures. B. Evaluation Dimensions The benchmark evaluates model performance across five capability dimensions representing the core analytical require- ments of institutional financial research. Each dimension mea- sures a distinct aspect of model behaviour, ranging from nu- merical reliability to the stability of reasoning across repeated queries. Together, these dimensions provide a comprehensive view of financial intelligence in AI systems.Rather than evalu- ating models on a single metric, the benchmark treats financial analysis as a multi-dimensional capability in which accurate data retrieval, analytical reasoning, contextual interpretation, and response stability must operate simultaneously. Each di- mension therefore contributes to the overall evaluation of a model’s usefulness in financial research workflows. 1) Factual Accuracy and Hallucination Resistance: The first evaluation dimension measures the factual correctness of model outputs relative to verified financial ground truth. Financial analysis requires high numerical precision because incorrect figures relating to revenue, earnings, capital expendi- ture, or financial ratios can materially distort investment con- clusions. In this context, hallucinations are defined as instances where a model generates a specific factual claim particularly , a numerical value that cannot be verified against authoritative financial sources.To evaluate accuracy, all numerical claims produced by the models were cross-checked against verified financial data from company filings and regulatory disclosures. Let N denote the number of verifiable numerical claims produced by a model and C denote the subset of those claims that match the ground truth. The factual accuracy score is therefore defined as: Accuracy = C N (1) The hallucination rate represents the complement of this measure and captures the proportion of incorrect or fabricated numerical statements. In addition to numerical fabrication, hallucinations were categorized according to the nature of the error, including incorrect financial period attribution, misap- plied metric definitions, and unsupported factual assertions. Ground truth data was sourced exclusively from official financial documents including SEBI-regulated filings, stock exchange disclosures, and company annual reports for the relevant financial year. 2) Analytical Depth and Valuation Logic: Financial anal- ysis requires the synthesis of multiple variables rather than the retrieval of isolated financial metrics. The analytical depth dimension therefore evaluates whether models demonstrate structured reasoning when interpreting financial data. Specifi- cally, this dimension assesses whether models integrate inter- actions between financial variables, apply appropriate valua- tion frameworks, and demonstrate awareness of sector-specific dynamics. Questions in this module required models to analyse fi- nancial relationships such as return on capital trends, margin compression, capital expenditure cycles, and valuation mul- tiples. Responses were evaluated according to the depth of reasoning demonstrated in the analytical narrative. Models received higher scores when their responses reflected multi- variable reasoning linking financial metrics to business drivers, sector structure, and macroeconomic context.This evaluation distinguishes models that simply retrieve financial figures from those capable of constructing coherent investment reasoning comparable to that produced by human analysts. 3) Completeness of Financial Analysis: The completeness dimension evaluates whether model responses address all iden- tifiable components of complex financial analysis questions. Many financial queries require the integration of multiple analytical elements, including financial metrics, contextual explanations, comparative analysis, and the interpretation of financial statements. A response that addresses only a subset of these elements may be directionally correct but remains ana- lytically incomplete.To quantify completeness, responses were evaluated across a structured rubric measuring coverage of requested metrics, contextual interpretation, reasoning depth, narrative coherence, and evidence referencing. Let k q denote the number of analytical components required for question q, and let k r,q denote the number of components addressed in the model’s response for that question. The completeness score for a single response is therefore defined as: Completeness q = k r,q k q (2) To obtain an overall completeness score across the bench- mark dataset containing Q questions, the per-question com- pleteness scores are averaged: Completeness model = 1 Q Q X q=1 Completeness q (3) Questions in this module focused primarily on structurally complex companies such as Reliance Industries and ITC Ltd., whose diversified business segments require multi-statement analysis and segment-level reasoning. These companies there- fore provide a particularly demanding test of analytical com- pleteness. 4) Data Recency and News Integration: Financial markets are inherently forward-looking and highly sensitive to recent developments such as earnings announcements, regulatory changes, and macroeconomic policy decisions. Analytical out- puts based on outdated information can therefore produce misleading conclusions even when the underlying reasoning is correct. The data recency dimension evaluates whether models incorporate current financial information into their analytical outputs. The benchmark includes questions requiring knowledge of events occurring within the most recent financial reporting period, including quarterly earnings releases, Reserve Bank of India monetary policy decisions, sector-specific developments, and capital market activity. Responses were evaluated accord- ing to whether the model correctly identified the relevant financial period, incorporated recent events into its analysis, and interpreted their significance within the investment the- sis.This dimension therefore captures the ability of AI systems to integrate real-time financial information into analytical reasoning rather than relying solely on static knowledge. 5) Consistency Across Repeated Queries: The final evalua- tion dimension measures the stability of model responses when identical or semantically equivalent questions are presented across multiple independent sessions. In financial research environments, analysts frequently revisit the same analytical queries over time, and reliable analytical tools must therefore produce stable outputs across repeated interactions.To evaluate response consistency, a subset of benchmark questions was submitted to each model across multiple independent sessions. Variations in numerical outputs, analytical reasoning struc- tures, and final conclusions were recorded and analyzed. Let R i denote the response generated during the i-th run of a repeated query, and let V (R i ) represent the extracted numeri- cal output or analytical verdict from that response.Consistency is measured by examining the variance of these outputs across repeated runs. Formally, the consistency score is defined as: Consistency = 1− Var V (R i ) (4) where Var(·) denotes the statistical variance across all repeated responses for the same query. Higher consistency scores correspond to lower variance across repeated responses, indicating more stable and reproducible analytical behavior. C. Dataset Construction The benchmark dataset was constructed by domain spe- cialists with experience in institutional equity research. Ques- tions were designed to reflect the types of analytical tasks typically performed by professional investment analysts. The construction process followed three guiding principles: verifia- bility, representativeness, and analytical difficulty.Verifiability ensures that all questions involving financial metrics can be evaluated against authoritative financial data sources. Repre- sentativeness ensures that the dataset reflects realistic analyt- ical tasks including financial metric retrieval, valuation rea- soning, macroeconomic interpretation, and investment thesis development. Analytical difficulty ensures that the benchmark differentiates meaningfully between models with superficial retrieval capabilities and those capable of deeper financial reasoning. The dataset spans multiple sectors of the Indian capital markets including banking, information technology services, consumer goods, conglomerates, and renewable energy com- panies. This diversity ensures that models are tested across dif- ferent financial structures and industry dynamics.Ground truth data was obtained from publicly available financial documents including company annual reports, quarterly earnings releases, stock exchange disclosures, and macroeconomic publications issued by regulatory authorities. D. Evaluation Procedure All models were evaluated using a standardized evaluation protocol designed to ensure comparability across systems. Each benchmark question was submitted to every model using identical prompts without providing additional con- textual information about the evaluation process. Responses were recorded in full and evaluated without modification.The evaluation process consisted of four stages. First, benchmark questions were submitted to each model under controlled conditions. Second, responses were extracted and stored for analysis. Third, numerical claims were verified against authorita- tive financial sources. Finally, responses were scored using structured scoring rubrics corresponding to the evaluation dimensions described earlier.For consistency testing, selected questions were repeated across multiple independent sessions without shared conversation history. This ensured that ob- served differences in model responses reflect inherent vari- ability in model behaviour rather than contextual information from previous interactions. E. Scoring Framework The benchmark scoring framework aggregates performance across the five evaluation dimensions to produce a unified composite score. Each dimension produces a normalized score in the range [0, 100], enabling comparisons across different evaluation modules.Let S i denote the score of a model on evaluation dimension i, and let w i represent the weight as- signed to that dimension. The overall benchmark score for a model is computed as a weighted aggregation: Score model = 5 X i=1 w i S i (5) In the present benchmark design, all five evaluation dimen- sions are assigned equal weights such that w i = 0.20 ∀i∈1, 2, 3, 4, 5.(6) This ensures that no single dimension dominates the overall evaluation outcome.Additional adjustments were applied in cases where models declined to answer a question or produced partially complete responses. Refusal to provide an answer resulted in zero scores for the relevant analytical dimensions while preserving hallucination resistance where appropriate. Partially complete responses were scored proportionally ac- cording to the fraction of requested analytical components addressed. F. Evaluation Setup All evaluations were conducted during the FY2025–26 financial reporting period. Models were accessed through their publicly available interfaces under default configuration set- tings to replicate the typical working environment of financial practitioners using AI tools for research.Financial ground truth data was obtained from authoritative sources including SEBI- regulated filings, stock exchange disclosures, company annual reports, and regulatory publications issued by the Reserve Bank of India and the Ministry of Finance. All financial values are reported in Indian Rupees unless otherwise specified.The evaluation framework was applied consistently across all benchmark modules to ensure compa- rability of results. Detailed scoring rubrics, benchmark ques- tions, and raw model responses supporting the results of the study are available as supplementary materials. IV. RESULTS AND DISCUSSION A. Overall Benchmark Performance The overall performance of the evaluated models is sum- marized in Figure 2, which presents the composite benchmark scores aggregated across all evaluation dimensions. The re- sults reveal clear separation between model capabilities, with SuperInvesting achieving the highest overall score, followed by Gemini and Perplexity. GPT and Claude rank lower due to limitations in accuracy, completeness, and real-time data access. The performance gap suggests that financial analysis tasks require a combination of numerical accuracy, struc- tured reasoning, and contextual financial knowledge. Models optimized primarily for general reasoning or conversational capabilities appear less suited for institutional financial re- search workflows. The leaderboard therefore highlights the importance of domain specialization in financial AI systems. B. Dimension-Level Capability Analysis To understand the sources of these performance differ- ences, Figure 3 presents a var graph comparison of model capability across the six evaluation dimensions used in the benchmark. SuperInvesting demonstrates consistently strong performance across most dimensions, particularly in accuracy, completeness, and analytical depth. Gemini shows competitive performance in reasoning-heavy tasks but exhibits weaker results in recency-dependent questions. Perplexity achieves the highest score in data recency but performs comparatively weaker in analytical reasoning and completeness. GPT and Fig. 2. Overall Benchmark Leaderboard Claude display narrower capability profiles, with GPT demon- strating moderate reasoning ability but weaker accuracy, while Claude shows strong hallucination resistance but limited cov- erage across financial tasks. The radar comparison highlights that financial intelligence in AI systems is inherently multi- dimensional and cannot be captured by a single metric. C. Cross-Benchmark Leaderboard Comparison Table I provides the per-benchmark ranking of models across the five evaluation modules used in the study. SuperIn- vesting ranks first in four out of five benchmarks, indicating consistent performance across independent evaluation frame- works. Gemini consistently ranks second or third, reflecting strong analytical reasoning capabilities among general-purpose models. Perplexity leads the data recency benchmark due to its live retrieval architecture but performs weaker in reasoning- intensive evaluations such as valuation depth. GPT and Claude frequently appear in lower ranks due to their respective limi- tations: hallucination frequency in GPT and systematic refusal behavior in Claude. These rankings confirm that the composite benchmark scores are not driven by isolated outcomes but instead reflect stable performance patterns across multiple evaluation methodologies. D. Trade-Off Between Recency and Analytical Depth One of the most notable findings of the benchmark is the apparent trade-off between data recency capability and analytical reasoning depth. This relationship is illustrated in Figure 4, which plots model performance on the recency benchmark against analytical depth scores. Retrieval-oriented systems such as Perplexity demonstrate strong recency perfor- mance but comparatively weaker analytical reasoning. Con- versely, reasoning-oriented models such as Gemini achieve high analytical depth but lack access to real-time financial information. SuperInvesting represents a partial exception to this pattern by achieving relatively strong performance on both dimensions. This observation suggests that hybrid archi- tectures combining structured data pipelines with reasoning- oriented model design may provide a more effective approach for financial AI systems. E. Hallucination Risk in Financial Analysis Hallucination behavior represents a critical reliability con- cern for financial AI systems. Figure 5 shows the frequency of hallucinated numerical financial values generated by each model during the accuracy benchmark evaluation. GPT ex- hibits the highest hallucination frequency, producing sub- stantially more fabricated values than other models. Gem- ini and Perplexity show moderate hallucination rates, while Claude and SuperInvesting demonstrate stronger hallucination resistance. However, Claude achieves this primarily through conservative refusal strategies rather than improved numerical reasoning. From a financial risk perspective, confident numer- ical fabrication poses greater danger than refusal, as fabricated values may influence investment decisions if not independently verified. F. Multi-Dimensional Performance Patterns The combined performance patterns across all evaluation dimensions are summarized in the heatmap shown in Fig- ure 6. This visualization provides a compact view of how each model performs across the benchmark’s six evaluation dimensions. SuperInvesting demonstrates consistently high scores across most dimensions, while Gemini exhibits strong analytical reasoning but weaker recency capability. Perplexity displays the opposite pattern, performing well in recency but weaker in reasoning and completeness. Claude achieves strong hallucination resistance but scores lower on completeness and recency, while GPT shows moderate reasoning ability but weak accuracy performance. The heatmap highlights that financial AI capability is not dominated by a single dimension but instead reflects the interaction between retrieval capability, reasoning depth, and numerical reliability. G. Implications for Financial AI Systems Taken together, the benchmark results suggest that no sin- gle architectural paradigm currently dominates all financial analysis tasks. Retrieval-based systems excel at integrating current financial information but may lack analytical synthesis capabilities. Reasoning-oriented models demonstrate stronger conceptual financial analysis but struggle with current-market data. Hybrid architectures that integrate structured financial data pipelines with domain-aware reasoning appear to pro- duce the most reliable outputs for complex financial anal- ysis workflows. These findings highlight the importance of specialized design in financial AI systems and suggest that domain-specific benchmarks are essential for evaluating AI performance in high-stakes decision-making environments. H. Robustness and Statistical Stability of Results To assess the robustness of the benchmark findings, we examined the stability of model performance across the five in- dependent evaluation modules. The composite ranking remains largely consistent across benchmarks, as shown in Table I, where SuperInvesting ranks first in four out of five evaluation categories. Gemini consistently occupies the second position in reasoning-oriented benchmarks, while Perplexity leads only Fig. 3. Model Capability Comparison Across Financial Benchmark Dimensions TABLE I PER-BENCHMARK LEADERBOARD ACROSS EVALUATION DIMENSIONS Benchmark1st2nd3rd4th5th Valuation DepthSuperInvestingGeminiGPTClaudePerplexity ConsistencySuperInvestingClaudeGeminiGPTPerplexity Accuracy / HallucinationSuperInvestingGeminiPerplexityGPTClaude Data RecencyPerplexitySuperInvestingGeminiGPTClaude CompletenessSuperInvestingGeminiPerplexityClaudeGPT Fig. 4. Trade-off Between Data Recency and Analytical Depth the recency benchmark due to its live retrieval architecture. This consistency across independent evaluation tasks suggests that the final ranking is not driven by a single benchmark Fig. 5. Hallucination Frequency Across Models dataset but instead reflects stable capability differences be- tween models. In addition to cross-benchmark agreement, the distribution of scores across evaluation dimensions further confirms the stability of the results. As illustrated in the heatmap in Fig- ure 6, model performance patterns remain consistent across Fig. 6. Heatmap of model performance across benchmark evaluation dimen- sions. Higher scores (green) indicate stronger performance. dimensions, with each model exhibiting a distinct capabil- ity profile. SuperInvesting demonstrates strong performance across most dimensions, Gemini excels in analytical reasoning, and Perplexity performs best in recency-based tasks. GPT and Claude show more limited capability profiles due to accuracy and coverage limitations respectively. Finally, repeated-question experiments were conducted as part of the consistency benchmark to evaluate response sta- bility. These tests revealed significant differences in output variance across models. SuperInvesting and Claude produced the most stable responses across repeated prompts, while Perplexity showed greater variability due to fluctuations in retrieved information. These observations reinforce the impor- tance of consistency as a benchmark dimension for real-world financial workflows. Taken together, the agreement across multiple benchmark modules, the stable dimension-level performance patterns shown in Figure 3 and Figure 6, and the response reproducibil- ity results indicate that the benchmark findings are robust and not driven by isolated evaluation artifacts.Table I is showing some benchmakrking prompts and model responses in AIFB. V. LIMITATIONS While the proposed benchmark provides a structured eval- uation of AI systems for financial analysis, several limitations should be acknowledged. First, the evaluation is conducted within the context of Indian equity markets and focuses on a selected set of companies spanning sectors such as banking, information technology, consumer goods, and conglomerates. Although this sectoral diversity captures a wide range of financial structures, the findings may not generalize directly to other financial domains such as derivatives pricing, fixed income analysis, or global equity markets. Additionally, the benchmark primarily evaluates models through structured fi- nancial analysis questions rather than full end-to-end invest- ment workflows. In practice, professional financial research of- ten involves multi-step processes including document retrieval, spreadsheet modelling, and iterative hypothesis testing. Conse- quently, while the benchmark captures important components of financial reasoning and data interpretation, it does not fully replicate the complexity of institutional research environments. A second limitation arises from the rapidly evolving nature of large language models and financial AI systems. The bench- mark reflects model capabilities during a specific evaluation period and model versions, and performance characteristics may change as models receive architectural updates or ex- panded data integration capabilities. In particular, improve- ments in real-time retrieval systems and domain-specialized training may significantly alter model rankings in future eval- uations. For this reason, the results should be interpreted as a snapshot of current capabilities rather than a definitive long- term ranking of financial AI systems. VI. FUTURE WORK Several directions for future work emerge from the findings of this study. One important extension would be expanding the benchmark to cover a broader set of financial tasks, including portfolio construction, credit risk analysis, derivatives pric- ing, and macroeconomic forecasting. These domains require different forms of reasoning and quantitative modelling and would provide a more comprehensive assessment of financial intelligence in AI systems. Additionally, future benchmarks could incorporate longer-form financial documents such as earnings call transcripts, regulatory filings, and analyst re- search reports. Evaluating model performance on document- level reasoning tasks would better reflect real-world investment workflows where analysts must interpret large volumes of financial information. Another promising direction involves exploring hybrid ar- chitectures that combine retrieval-based data access with structured financial reasoning capabilities. The results of this benchmark suggest that models optimized solely for retrieval or reasoning tend to exhibit complementary strengths and weaknesses. Systems capable of integrating real-time financial data pipelines with domain-aware reasoning frameworks may therefore achieve superior performance across multiple dimen- sions of financial analysis. Future research could investigate such hybrid approaches and develop benchmarks that evaluate the interaction between retrieval, reasoning, and numerical reliability. VII. CONCLUSION This paper introduces the AI Financial Intelligence Bench- mark, a multi-dimensional evaluation framework designed to assess the capability of AI systems to perform financial analysis tasks relevant to institutional investment research. By evaluating five widely used AI models across multiple analyt- ical dimensions including analytical depth, factual accuracy, hallucination resistance, completeness, consistency, and data recency, the benchmark provides a systematic comparison of model performance in complex financial reasoning scenar- ios. The results demonstrate substantial capability differences across models and highlight that financial intelligence in AI systems cannot be adequately captured through single-metric evaluations. TABLE I EXAMPLE BENCHMARK PROMPTS AND MODEL RESPONSES USED IN THE AI FINANCIAL INTELLIGENCE BENCHMARK (AFIB). RESPONSES ARE SUMMARIZED FOR READABILITY. PromptGPTGeminiPerplexityClaudeSuperInvesting Q1: Reliance EBITDA Contribu- tion What percentage of Reliance’s con- solidated EBITDA comes from Oil-to-Chemicals, Jio, and Retail in the latest financial year? Providedsegment percentages(43%, 28%,29%)but incorrectly concluded thethreesegments represent100%of EBITDA. Estimatedsegment contributionsand explainedstructural shift toward consumer businesses. Combined estimate around 88%. Attempted calculation usingannualreport figures but produced inconsistenttotals duetosegment aggregation issues. Provided historical es- timates from earlier fi- nancial periods with- out latest-year verifica- tion. Providedstructured segmentanalysis withcontextual explanationand estimatedcombined contributionaround 88–90%. Q2: SBI vs ICICI NPA Compar- ison Compare the gross NPA and net NPA ratios of SBI and ICICI Bank for the most recent quarter and ex- plain the role of provision coverage ratio (PCR). Providedillustrative values and conceptual explanation of PCR butacknowledged figureswere approximate. Used recent financial results with structured explanation of GNPA, NNPA, and provision- ing impact. Retrieved figures from financial sources and presented a compara- tive table with PCR explanation. Used older financial period data but cor- rectly explained the PCR mechanism link- ing GNPA and NNPA. Providedstructured comparison of GNPA, NNPA, and PCR with explanation of asset qualitydifferences andprovisioning strategies. Q3: L&T Weighted Fundamen- tal Score Construct a weighted fundamental score (out of 100) for Larsen & Toubro using: 40% order book-to-bill ratio 30% working capital days 30% EPC EBITDA margins. Show exact calculation. Assumedsector benchmarksand normalizedmetrics to compute a score of60/100.Used hypothetical values. Appliedfinancial benchmarksand calculatedweighted score of 83.7/100 with detailed reasoning. Estimated financial in- puts from disclosures and derived a score of 79.3/100. Used sector averages and derived a score of 82/100 with emphasis on working capital ef- ficiency. Calculated normalized benchmark scores and produced a weighted score of 87.2/100 with detailed breakdown. Responses are summarized for brevity. Full model outputs used for scoring are available in the supplementary material. More broadly, the findings reveal a structural trade-off between retrieval capability and analytical reasoning among current AI architectures. Retrieval-oriented systems excel at incorporating real-time financial information, while reasoning- oriented models demonstrate stronger performance in analyt- ical interpretation and structured financial analysis. Hybrid systems that combine domain-specific data pipelines with financial reasoning frameworks appear to provide the most reliable performance across tasks. As AI systems continue to evolve, structured benchmarks such as the one presented in this study will be essential for understanding their strengths, limitations, and practical role in supporting financial decision- making workflows. REFERENCES [1] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [2] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language mod- eling with pathways. Journal of machine learning research, 24(240):1– 113, 2023. [3] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [4] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multi- task language understanding. arXiv preprint arXiv:2009.03300, 2020. [5] Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri ` a Garriga-Alonso, et al.Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research, 2023. [6] Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022. [7] Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 1906–1919, Online, July 2020. Association for Computational Linguistics. [8] Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. [9] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1–38, 2023. [10] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhang- hao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623, 2023. [11] Dogu Araci.Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063, 2019. [12] Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023. [13] Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance, 2023. URL https://arxiv. org/abs/2303.17564, 2024. [14] Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan R Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3697–3711, 2021. [15] Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen.Financebench: A new benchmark for financial question answering. arXiv preprint arXiv:2311.11944, 2023. [16] Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. Finben: A holistic financial benchmark for large language models. Advances in Neural Information Processing Systems, 37:95716–95743, 2024. [17] Spencer Mateega, Carlos Georgescu, and Danny Tang.Financeqa: a benchmark for evaluating financial analysis capabilities of large language models. arXiv preprint arXiv:2501.18062, 2025. [18] Yan Wang, Keyi Wang, Shanshan Yang, Jaisal Patel, Jeff Zhao, Fen- gran Mo, Xueqing Peng, Lingfei Qian, Jimin Huang, Guojun Xiong, et al.Finauditing: A financial taxonomy-structured multi-document benchmark for evaluating llms. arXiv preprint arXiv:2510.08886, 2025. [19] Yidong Jiang, Junrong Chen, Eftychia Makri, Jialin Chen, Peiwen Li, Ali Maatouk, Leandros Tassiulas, Eliot Brenner, Bing Xiang, and Rex Ying. Fin-rate: A real-world financial analytics and tracking evaluation benchmark for llms on sec filings. arXiv preprint arXiv:2602.07294, 2026. [20] Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback, 2023. [21] Yann Dubois, Bal ́ azs Galambosi, Percy Liang, and Tatsunori B Hashimoto.Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475, 2024. [22] Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. arXiv preprint arXiv:2406.11939, 2024. [23] Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas An- gelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, et al. Chatbot arena: An open platform for evaluating llms by human preference. In Forty-first International Conference on Machine Learning, 2024. [24] Lanlan Ji, Dominic Seyler, Gunkirat Kaur, Manjunath Hegde, Koustuv Dasgupta, and Bing Xiang. Phantom: A benchmark for hallucination detection in financial long-context qa.In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [25] Mengao Zhang, Jiayu Fu, Tanya Warrier, Yuwen Wang, Tianhui Tan, and Ke-wei Huang. Faith: A framework for assessing intrinsic tabular hallucinations in finance. In Proceedings of the 6th ACM International Conference on AI in Finance, pages 159–167, 2025. [26] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova Das- Sarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022. [27] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, et al. Predictability and surprise in large generative models. In Proceedings of the 2022 ACM conference on fairness, accountability, and transparency, pages 1747–1764, 2022. [28] Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. Trustllm: Trustworthiness in large language models.arXiv preprint arXiv:2401.05561, 2024. [29] Xiao-Yang Liu, Guoxuan Wang, Hongyang Yang, and Daochen Zha. Fingpt: Democratizing internet-scale data for financial large language models. arXiv preprint arXiv:2307.10485, 2023. [30] Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang.Fin- gpt: Open-source financial large language models.arXiv preprint arXiv:2306.06031, 2023. [31] Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering. In Proceedings of the 2022 conference on empirical methods in natural language processing, pages 6279–6292, 2022. [32] Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, Kp Subbalakshmi, Jimin Huang, et al. Investorbench: A benchmark for financial decision- making tasks with llm-based agent. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2509–2525, 2025. [33] Lingfei Qian, Xueqing Peng, Yan Wang, Vincent Jim Zhang, Huan He, Hanley Smith, Yi Han, Yueru He, Haohang Li, Yupeng Cao, et al. When agents trade: Live multi-market trading benchmark for llm agents. arXiv preprint arXiv:2510.11695, 2025. [34] Yuqi Nie, Yaxuan Kong, Xiaowen Dong, John M Mulvey, H Vincent Poor, Qingsong Wen, and Stefan Zohren. A survey of large language models for financial applications: Progress, prospects and challenges. arXiv preprint arXiv:2406.11903, 2024. [35] Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. Large language models in finance: A survey.In Proceedings of the fourth ACM international conference on AI in finance, pages 374–382, 2023. APPENDIX This appendix presents representative prompts from the AI Financial Intelligence Benchmark (AFIB) along with re- sponses generated by the evaluated models. These examples illustrate typical model behaviour observed during evaluation, including differences in numerical reasoning, analytical com- pleteness, and factual reliability. A. Prompt 1: Reliance Industries Segment EBITDA Contribu- tion Prompt: What percentage of Reliance’s consolidated EBITDA comes from Oil-to-Chemicals, Jio, and Retail in the latest financial year? B. Prompt 2: SBI vs ICICI Bank Asset Quality Comparison Prompt: Compare the gross NPA and net NPA ratios of SBI and ICICI Bank for the most recent quarter. Explain how their provision coverage ratios (PCR) account for the difference. C. Prompt 3: Asian Paints ROIC vs WACC Analysis Prompt: Asian Paints has historically commanded a high PE premium. Analyze whether this premium is justified by comparing its Return on Invested Capital (ROIC) and Weighted Average Cost of Capital (WACC) over the last five years. ModelResponse (Excerpt) GPTReported segment contributions as Oil-to-Chemicals 43%, Jio 28%, and Retail 29%, and incorrectly concluded that these segments together account for 100% of consolidated EBITDA. GeminiProvided a structured financial analysis estimating segment contributions around O2C ∼43%, Jio∼32%, and Retail∼13%, concluding that these three segments contribute approximately 88% of consolidated EBITDA. PerplexityAttempted a calculation using annual report figures but produced inconsistent totals due to incorrect aggregation of segment EBITDA values and reporting overlaps. ClaudeProvided historical segment estimates from earlier financial periods and explained the role of segment diversification, but used outdated financial data. SuperInvestingProduced a structured breakdown with contextual explanation of segment contributions (O2C∼42–44%, Jio∼31–33%, Retail∼13–15%), concluding that these segments contribute approximately 88–90% of consolidated EBITDA. TABLE IV EXAMPLE RESPONSES FOR RELIANCE INDUSTRIES EBITDA COMPOSITION PROMPT. ModelResponse (Excerpt) GPTProvided approximate illustrative values for GNPA and NNPA and explained the concept of provision coverage ratio (PCR), but acknowledged that the figures were not based on the latest financial results. GeminiUsed recent financial data to compare asset quality metrics and explained the relationship between GNPA, NNPA, and PCR in determining bank balance sheet strength. PerplexityRetrieved financial ratios from web sources and presented a comparison table of GNPA and NNPA values, linking the difference to provisioning behaviour. ClaudeUsed financial data from an earlier reporting period but provided a clear explanation of the relationship between PCR and the GNPA–NNPA gap. SuperInvestingProduced a structured comparison including GNPA, NNPA, and PCR metrics, ex- plaining how higher provisioning buffers reduce net NPA exposure and improve asset quality resilience. TABLE V EXAMPLE RESPONSES FOR SBI VS ICICI BANK ASSET QUALITY ANALYSIS PROMPT. ModelResponse (Excerpt) GPTOutlined the methodology for computing ROIC and WACC and provided illustrative estimates showing ROIC consistently exceeding WACC by roughly 9–11 percentage points. GeminiDeclined to provide a detailed calculation due to insufficient data availability and instead discussed valuation context using ROE and ROCE indicators. PerplexityPresented a detailed financial analysis including approximate historical ROIC and WACC values and discussed the implications of a narrowing value spread. ClaudeEstimated historical ROIC and WACC values and concluded that Asian Paints main- tains a significant positive ROIC–WACC spread supporting its premium valuation. SuperInvestingProduced a structured capital efficiency analysis showing ROIC significantly exceed- ing WACC but highlighted a declining spread due to increasing competition and margin pressure. TABLE VI EXAMPLE RESPONSES FOR ASIAN PAINTS ROIC–WACC VALUATION ANALYSIS PROMPT.