Paper deep dive
When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability
Russlan Ramdowar
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/12/2026, 5:44:10 AM
Summary
This study evaluates whether agreement among seven AI investment advisors (using Gemini models) indicates forecast reliability or merely reflects low asset risk. Analyzing 928 forecasts for 232 assets, the paper finds that while low disagreement correlates with lower error, this relationship is largely driven by historical volatility. Selecting forecasts based on low historical volatility yields lower error than selecting based on low disagreement. However, directional agreement (consensus on up/down/flat) is positively associated with directional accuracy, distinguishing it from return magnitude precision.
Entities (8)
Relation Signals (6)
Russlan Ramdowar → authored → When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability
confidence 99% · When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability RusslanRamdowar Future Edge Group FZE
iPulse AI → uses → Gemini
confidence 95% · Ten historical Gemini configurations are grouped into seven investment frameworks; their archived forecasts are compared with later market outcomes... of iPulse AI.
Directional Agreement → correlateswith → Directional Accuracy
confidence 93% · When at least six of seven frameworks share a direction, directional accuracy is higher
Historical Volatility → predicts → Mean Absolute Error
confidence 92% · Retaining the quarter of forecasts with lowest historical volatility produces lower error than retaining those with least disagreement: 7.97 versus 10.31 percentage points.
Elon Musk → ispartof → iPulse AI
confidence 90% · Framework label Historical modes retained Configurations Elon Musk Researcher 1
Warren Buffett → ispartof → iPulse AI
confidence 90% · Warren Buffett Researcher; thinker 2
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:If seven AI advisors give similar forecasts, should we trust them more? Agreement can indicate useful shared evidence, but it can also reflect an asset that was easier to forecast in the first place. We examine this distinction using 928 completed first-quarter forecasts for 232 assets across four production batches of iPulse AI. Ten historical Gemini configurations are grouped into seven investment frameworks; their archived forecasts are compared with later market outcomes, without generating new reports. Mean absolute return error rises from 10.31 percentage points in the lowest-disagreement quartile to 21.05 in the highest. Yet a forecast of no price change shows almost the same increase, from 9.89 to 21.15. The relationship between disagreement and additional forecast error is not resolved after accounting for visible asset risk. Retaining the quarter of forecasts with lowest historical volatility produces lower error than retaining those with least disagreement: 7.97 versus 10.31 percentage points. Agreement about direction behaves differently. When at least six of seven frameworks share a direction, directional accuracy is higher, but return-magnitude error is also larger. These results distinguish three questions often collapsed into one confidence score: how risky the asset is, whether advisors agree on its direction, and how precisely they forecast its return. For this historical council, agreement is informative in specific ways; it is not a general certificate of forecast reliability.
Tags
Links
- Source: https://arxiv.org/abs/2610.13431v1
- Canonical: https://arxiv.org/abs/2610.13431v1
Trouble viewing inline? Open PDF directly →
Full Text
38,003 characters extracted from source content.
Expand or collapse full text
When AI Advisors Agree: Distinguishing Forecast Risk from Forecast Reliability RusslanRamdowar Future Edge Group FZE, United Arab Emirates iPulse AI russlan@ipulseai.com ORCID:0009-0004-9311-6957 9October2026 Abstract If seven AI advisors give similar forecasts, should we trust them more? Agreement can indicate useful shared evidence, but it can also reflect an asset that was easier to forecast in the first place. We examine this distinction using 928 completed first-quarter forecasts for 232 assets across four production batches of iPulse AI. Ten historical Gemini configurations are grouped into seven investment frameworks; their archived forecasts are compared with later market outcomes, without generating new reports. Mean absolute return error rises from 10.31 percentage points in the lowest-disagreement quartile to 21.05 in the highest. Yet a forecast of no price change shows almost the same increase, from 9.89 to 21.15. The relationship between disagreement and additional forecast error is not resolved after accounting for visible asset risk. Retaining the quarter of forecasts with lowest historical volatility produces lower error than retaining those with least disagreement: 7.97 versus 10.31 percentage points. Agreement about direction behaves differently. When at least six of seven frameworks share a direction, directional accuracy is higher, but return-magnitude error is also larger. These results distinguish three questions often collapsed into one confidence score: how risky the asset is, whether advisors agree on its direction, and how precisely they forecast its return. For this historical council, agreement is informative in specific ways; it is not a general certificate of forecast reliability. 1 Introduction An AI council can produce a persuasive form of evidence: several advisors, different reasoning frameworks, and a common conclusion. Agreement is visible immediately; correctness arrives later. This gap makes consensus attractive as a confidence signal, especially in forecasting tasks where the answer does not yet exist. It also makes agreement easy to overinterpret. Consider two assets. One has a stable price history, and seven advisors predict modest returns close to one another. The other is volatile, and their predictions differ widely. If the first council later has smaller errors, agreement appears to predict reliability. But a forecast that ignores the council and simply predicts no price change might have smaller errors on the first asset too. The council could be identifyingforecast difficulty, rather than identifying when its own judgment adds value. 1 arXiv:2610.13431v1 [cs.MA] 9 Oct 2026 This paper tests that distinction in the ordinary production archive of iPulse AI, an Open Agentic Investment Research Platform. Its research pipeline generates reports through configured invest- ment frameworks and modes, followed by a synthesis layer. The present study isolates a transparent, equally weighted council from historical individual forecasts. It does not evaluate the evidence- weighted synthesizer, rank current provider models, or measure the profitability of an investment strategy. A separate study of this platform examines how model disclosure affects synthesis weights [8]; here the question concerns subsequent realized outcomes. We analyze four batches issued between April and July 2026. The central comparison asks whether forecast dispersion predicts errorbeyondwhat is already visible in an asset’s price vari- ability and in a no-change baseline. We then distinguish agreement about numerical returns from agreement about up, flat, or down. These are related signals, but they answer different questions. The contribution is an auditable outcome study with three connected results. First, a strong raw relationship between council disagreement and return error is largely mirrored by a simple baseline. Second, selecting forecasts by historical volatility performs better than selecting them by raw disagreement in this sample. Third, directional agreement has a positive association with directional accuracy while failing to identify more precise return estimates. Together, these findings show why a single agreement score can hide materially different notions of reliability. 2 Related work The distinction between disagreement and uncertainty predates language models. Lahiri and Sheng [ 1] show, under their forecasting model, that dispersion among forecasters does not capture all uncertainty associated with future common shocks. Glas [2] documents that the disagreement– uncertainty relationship depends on the forecast setting and how the quantities are measured. Our study brings this concern into an operational AI council: a forecast-spread signal should be tested against ordinary, observable task risk before it is interpreted as model-specific confidence. In machine learning, deep ensembles offer a practical route to uncertainty estimation [ 3]. Those re- sults do not establish that prompt frameworks sharing a model backbone behave like independently trained ensemble members. For language models, Xiao et al. [4] formalize and test relationships between generation consistency and correctness across several tasks. Their work motivates exam- ining consistency empirically rather than assuming that it is a universal confidence measure. Our task differs in an important respect: answers are later market returns, rather than labels already available when generations are evaluated. ForecastBench [ 5] emphasizes forecasts about genuinely future events. We likewise preserve the distinction between the generation date and the outcome date, but use archived production returns rather than a purpose-built question benchmark. This observational setting provides breadth at the cost of weaker control over the information supplied to each historical run. Selective prediction studies the tradeoff between retaining more predictions and accepting more error [ 6]. We use retention curves descriptively: how much error remains when a rule keeps only a specified fraction of forecasts? We do not transfer selective-classification guarantees to market returns. Finally, point-forecast evaluation requires attention to the forecast functional and scoring rule [ 7]. Our arithmetic council mean is evaluated under absolute error as a practical decision loss; it is not asserted to be an optimal conditional median. A median-aggregation sensitivity check addresses this choice. 2 3 Study setting and cohort 3.1 A fixed historical council The frozen evaluation archive has an outcome cutoff of 8 October 2026. We use native first-quarter forecasts from Batches 3–6, rather than daily refreshes, partially completed paths, or longer horizons. Batch 7, issued in September, is excluded because its native first-quarter targets had not matured at the cutoff. Earlier batches are excluded from the primary panel because their coverage and horizon representations are not comparable to the selected four-batch council. The eligible set contains ten configurations representing seven frameworks (Table1). These are platform persona labels for configured AI research perspectives, not forecasts made or endorsed by the named individuals. Four frameworks have a researcher mode; the other three have researcher and thinker modes. We first average modes within each framework, then average the seven frame- work forecasts. This gives each framework equal weight and prevents those with two modes from receiving twice the influence. Framework labelHistorical modes retainedConfigurations Elon MuskResearcher1 Michael BurryResearcher1 Niccolo Machiavelli Researcher1 Sherlock HolmesResearcher1 Ray DalioResearcher; thinker2 SuperintelligenceResearcher; thinker2 Warren BuffettResearcher; thinker2 TotalSeven framework perspectives10 Table 1:Fixed historical council. All configurations use archived Gemini model versions. Frame- work diversity is not provider diversity or statistical independence. The model-version identifiers change between the earlier and later pairs of batches; they are recorded in the companion data. Thus, “fixed council” means a fixed set of framework–mode configurations, not a constant model binary. Provider, prompt, and model-version effects are not separately identified. This study should not be read as a benchmark of the newest models or of all multi-agent architectures. Of 1,242 candidate asset–batch date groups, 15 lack the complete ten-configuration set. The remaining 1,227 matched outcomes cover 376 assets. Requiring an asset in all four batches yields 233 assets and 932 outcomes. One asset lacks valid historical-risk coverage across all four batches, leaving the primary balanced panel of232 assets and 928 outcomes. The panel includes 206 equities, 14 crypto assets, five commodities, five foreign-exchange assets, and two funds. Each contributes four outcomes. Selection is based on completeness, not observed forecast accuracy, but requiring future coverage still introduces a retrospective survivorship constraint. 3 Source forecasts 10 configurations 7 frameworks Matched panel 232 assets x 4 batches 928 completed quarters Future outcomes Native-currency returns Matched dates and splits What the council says Mean return; forecast dispersion Up / flat / down agreement What was already visible Pre-generation daily volatility Momentum and asset class Does disagreement add information? Forecast error versus no-change error Risk adjustment; selective evaluation A retrospective audit of future outcomes; no forecasts were regenerated. Figure 1:Study design. Seven framework-level forecasts are compared with later outcomes and with risk measures available from prices dated before generation. The analysis distinguishes raw error, error relative to a baseline, and selective evaluation. 3.2 The information clock matters Figure 2shows the forecast and outcome windows. Much of Batch 3 had not matured when Batch 6 was generated, and the following batches had still less completed history available. Within the primary panel, only one earlier first-quarter outcome had closed before the Batch 6 issue date; none had closed before the Batch 4 or Batch 5 issue dates. Fitting an error predictor to all four batches and presenting it as a policy that could have been trained at each issuance would therefore be misleading. Our retention rules require no fitted outcome labels. They rank assets within a batch using council dispersion, historical volatility, or their ratio. The later regression is explicitly aretrospective association analysis, not a prospectively trained confidence model. Rules, cutoffs, and analyses were developed after an initial exploratory inspection; this study was not preregistered. Forecast evaluation starts at the archive’s first saved close after generation, rather than an as- sumed executable price at the exact generation time. Predicted target values and realized target prices are expressed relative to that common entry close. Consequently, these rebased scores be- come available at the entry close, not necessarily at the generation instant. The price-history risk measures use dates strictly before generation. This construction aligns forecasts with a common evaluable return, without implying that a live trade was placed. 4 Apr 2026May 2026Jun 2026Jul 2026Aug 2026Sep 2026Oct 2026 Calendar date Batch 3 Batch 4 Batch 5 Batch 6 11 Apr 10 Jul 03 May 02 Aug 05 Jun 05 Sep 05 Jul 05 Oct Generation and outcome windowTarget-close range Figure 2:Overlapping first-quarter outcome windows. Blue circles mark generation dates; gold segments mark the range of target-close observations across assets. Most preceding-quarter labels were unavailable at the next generation. The figure explains why adjusted models are treated as retrospective diagnostics. 4 Measures and analysis 4.1 Agreement, error, and the baseline For assetain batchb, letp abj be the predicted percentage return of frameworkj, after averaging its retained modes, and lety ab be the realized percentage return. WithJ= 7, the council return and numerical disagreement are ̄p ab = 1 J J ∑ j=1 p abj ,D ab = √ √ √ √ 1 J J ∑ j=1 (p abj − ̄p ab ) 2 .(1) The population standard deviation describes the spread of this specified council. It is not a pre- dictive standard error, an estimate based on seven independent draws, or a probability interval for the future return. Absolute return error, no-change error, and their difference are E ab =| ̄p ab −y ab |,E 0 ab =|y ab |,∆E ab =E ab −E 0 ab .(2) All are measured in percentage points (p). Positive∆Emeans the council does worse than predicting a zero return; negative values mean it does better. Subtracting this simple baseline does not remove every source of forecast difficulty. It does show whether a raw error pattern also occurs without using the advisors’ predicted return. 5 Directional agreement is measured separately. A return above+0.5p is up, below−0.5p is down, and otherwise flat. Lets(p)denote this classification. Then A ab = max c∈−1,0,1 1 J J ∑ j=1 1s(p abj ) =c.(3) High directional agreement meansA≥6/7. Directional accuracy evaluates the sign of thecouncil mean, not the modal framework sign. An extreme forecast can therefore make the two disagree. We compare directional accuracy with an always-up baseline; predicting no change is the appropriate separate baseline for return error. 4.2 Visible asset risk Historical volatility is computed from the most recent 60 observed daily log returns before genera- tion, requiring at least 45 observations. Recorded active splits are incorporated into the historical returns. A conservative check excludes histories with an unsupported large split adjustment; 20- and 90-observation windows provide sensitivity comparisons. To place volatility on the approximate horizon scale, define V ab = 100 ˆσ ab √ h ab n a /365,(4) wherehis the number of calendar days from entry to target, andn a is 365 for crypto assets and 252 otherwise. This square-root scaling is a conventional descriptive risk measure, not a claim that returns are Gaussian or independent. Historical momentum is the cumulative return over the same 60-observation window. All outcomes are native-currency price returns; they do not include dividend reinvestment or a common-currency portfolio conversion. 4.3 Comparisons and uncertainty We assign disagreement quartileswithin each batch, so a change in the overall forecast scale cannot alone create the pooled gradient. Each primary quartile contains 58 assets per batch, or 232 out- comes. We also form five within-batch volatility groups and compare lower and higher disagreement within each group. The adjusted model relateslog(1 +E)to historical volatility, absolute forecast magnitude, ab- solute historical momentum, and disagreement, with batch and asset-class indicators. Continuous predictors are transformed and standardized; exact definitions appear in Appendix A. A second model uses∆Eas the response. These controls test conditional associations in this sample. They do not identify a causal effect of disagreement, remove all risk, or establish the absence of a small useful signal. For selective evaluation, we retain the lowest 25%, 50%, 75%, or 100% of scores within each batch and measure subsequent error among those retained. The selectors areD,V, andD/V. These are descriptive coverage comparisons, not an investment backtest: they contain no position sizing, execution, transaction costs, or utility objective. We report pointwise 95% percentile intervals from 5,000 asset-cluster bootstrap draws, resampling entire assets with their four outcomes together. Regression intervals use 2,000 draws. Paired contrasts use the same cluster draws for both groups or selectors. Intervals condition on the 6 archived observations and fixed rank membership. They account for repeated assets, but not fully for common market shocks across assets or the four overlapping periods. They are neither simultaneous intervals nor prospective performance guarantees. 5 Results 5.1 The apparent confidence signal mirrors a simple baseline Numerical disagreement strongly separates later absolute error. Mean error rises from 10.31 p in the lowest-disagreement quartile to 21.05 p in the highest (Figure3). The high-minus-low gap is 10.74 p, with a 95% interval of [7.11, 14.78]. Viewed alone, this looks like a useful confidence signal. The no-change baseline, however, rises almost identically: 9.89 p to 21.15 p. Its high-minus- low gap is 11.26 p [7.82, 15.10]. The corresponding gap in errorbeyond the baselineis−0.52p [−1.74, 0.69]. Thus, the raw gradient does not establish that low disagreement identifies greater council skill. Much of the separation also describes how far prices subsequently move from their starting values. Disagreement quartile Council MAE No-change MAE Difference MeanV Lowest10.319.89+0.4114.31 Second13.1413.09+0.0516.91 Third16.1015.81+0.2921.21 Highest21.0521.15−0.1030.03 Table 2:Within-batch disagreement quartiles, pooled across four batches; 232 outcomes per row. Return errors and horizon-scaled volatilityVare in p. Rounded entries may not subtract exactly. Across the full panel, council MAE is 15.15 p versus 14.99 p for no change. The paired difference is +0.16 p [−0.23, 0.56]. Council directional accuracy is 48.38%, compared with 50.75% for always up. These aggregate comparisons provide no clear evidence of an overall advantage under the chosen losses. They do not prove that the systems are equivalent, nor do they evaluate the quality of their qualitative research. 5.2 Disagreement tracks risk visible before generation The raw gradient has a plausible, observable explanation. Disagreement and historical volatility have within-batch Spearman correlations between 0.56 and 0.69 (Figure4). Historical volatility also correlates with subsequent council error, more strongly than disagreement does in each of the four batches. Advisors tend to spread their forecasts more widely on assets whose recent price behavior already signals greater variability. In the adjusted model, the standardized log-disagreement coefficient forlog(1 +E)is 0.006 [−0.072, 0.079], while the historical-volatility coefficient is 0.358 [0.259, 0.465]. The disagreement coefficient in the excess-error model is−0.08p [−0.63, 0.42]. These estimates do not resolve an additional association after the specified controls; they are not an equivalence test. The coarse, less model-dependent comparison gives a similar result. Within volatility groups, higher-disagreement forecasts have a mean error 1.02 p lower than lower-disagreement forecasts, 7 LowestQ2Q3Highest Within-batch disagreement quartile 0 5 10 15 20 25 30 Mean absolute return error (p) A. Absolute error Council forecast No change LowestQ2Q3Highest Within-batch disagreement quartile −1.0 −0.5 0.0 0.5 1.0 Council error minus no-change error (p) B. Error beyond the baseline Figure 3:The raw disagreement–error gradient is also present for no change. A: absolute errors for the council and baseline. B: their paired difference within each quartile. Bars are pointwise asset-cluster 95% intervals. 10 0 10 1 Historical volatility at horizon scale (p) 10 0 10 1 Council disagreement (p) A. Disagreement tracks visible risk B3B4B5B6 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Within-batch Spearman correlation B. Rank relationships by batch Disagreement vs risk Risk vs error Disagreement vs error Figure 4:Visible asset risk and council dispersion. A: 928 outcomes plotted on logarithmic axes. B: within-batch rank correlations. The council’s spread is closely related to historical volatility, which is more strongly related to absolute forecast error than spread is in each batch. with an interval spanning both directions [−3.99, 1.79]. Their excess-error difference is−0.31 p [−1.04, 0.41]. A strong unconditional association is therefore insufficient to justify treating dispersion as a distinct reliability measure. 8 0.00.20.4 Coefficient per 1 SD of transformed predictor Historical volatility Forecast magnitude Historical momentum Council disagreement A. Adjusted association with log(1 + error) Lower spreadHigher spread Compared within volatility groups 0 5 10 15 20 Mean absolute return error (p) B. Similar-risk comparison Council No change Figure 5:Adjusted and similar-risk comparisons. A: coefficients for standardized transformed pre- dictors in the model oflog(1 +E), controlling for batch and asset class. B: lower and higher disagreement within five volatility groups per batch. Bars show pointwise asset-cluster 95% inter- vals. These are retrospective diagnostics. 5.3 A cheaper observable ranks low-error forecasts better Selecting the least-disagreeing quarter of forecasts reduces MAE from 15.15 p to 10.31 p. Se- lecting the lowest-volatility quarter reduces it further, to 7.97 p (Figure6). The paired difference between the volatility and disagreement selectors is−2.34p [−4.26,−0.80]. Both rules retain 58 assets per batch, but need not retain the same assets. This improvement is about selecting easier cases, not demonstrating greater forecasting skill. On the lowest-volatility quarter, no change has MAE 8.00 p; the council-minus-baseline difference is −0.03p [−0.43, 0.37]. On the least-disagreeing quarter, the council’s difference is +0.41 p. A selection rule can substantially reduce error while leaving the council’s advantage over the baseline unresolved. Dividing disagreement by volatility does not automatically produce a better confidence signal. Keeping the lowestD/Vquarter yields MAE 18.45 p. A large denominator can make a volatile asset appear to have small relative disagreement. This failed shortcut is useful: controlling for risk in an analysis and constructing a reliable selection score are different tasks. 5.4 Agreement about direction is a different signal High directional agreement occurs in 436 outcomes; the other 492 have fewer than six frameworks sharing a direction. Council directional accuracy is 53.21% in the high-agreement group versus 44.11% in the other group (Figure7). The difference is 9.11 percentage points [3.03, 15.22]. The corresponding contrast in accuracy relative to always up is 11.39 points [1.95, 20.96]. The descriptive association is therefore not explained simply by there being more upward outcomes in the high- agreement group. 9 255075100 Forecasts retained within each batch (%) 0 5 10 15 20 Mean absolute return error (p) A. Lower error among retained forecasts 255075100 Forecasts retained within each batch (%) −0.5 0.0 0.5 1.0 1.5 Council error minus no-change error (p) B. Does selection beat the baseline? Least disagreement Lowest historical volatility Lowest disagreement / volatility Figure 6:Selective evaluation at equal within-batch coverage. A: absolute error among retained forecasts. B: paired error relative to no change, with pointwise cluster intervals. At 100% all rules retain the same panel. The rules require no fitted outcome labels; their evaluation is retrospective. Return precision moves in the opposite direction. The high-agreement group has MAE 16.84 p, compared with 13.65 p in the other group: a difference of +3.19 p [0.73, 5.80]. Six advisors can agree that a price will rise while disagreeing substantially about how much, or while jointly misjudging its magnitude. Numerical dispersion and directional concordance should not be used interchangeably. This is a secondary, exploratory finding under an explicit threshold. The unadjusted association may reflect asset composition and trend conditions. It supports further testing of direction-specific confidence, not a claim that 53.21% is a calibrated probability or that the rule is profitable. It has not been validated on an untouched subsequent batch. 10 Fewer than 6 of 7At least 6 of 7 Frameworks sharing a direction 0 10 20 30 40 50 60 70 Correct up / fl at / down forecasts (%) A. Directional agreement Always up Council Fewer than 6 of 7At least 6 of 7 Frameworks sharing a direction 0 5 10 15 20 25 Mean absolute return error (p) B. Agreement is not return precision No change Council Figure 7:Directional concordance does not imply numerical precision. A: council directional accu- racy and the always-up baseline. B: return MAE and no-change MAE for the same groups. Council error bars are pointwise asset-cluster 95% intervals; baseline markers show estimates. 5.5 Robustness and variation across batches The central excess-error contrast remains unresolved when we expand coverage to 1,221 outcomes, restrict to equities, require original published-cohort membership, remove the largest 1% of errors, replace the council mean with its median, or score against the original forecast anchor (Figure 8). Every corresponding interval spans zero. Alternative spread measures also give unresolved high- minus-low excess-error gaps: +0.11 p [−0.92, 1.17] for the interquartile range and−0.42p [−1.46, 0.61] for median absolute deviation. The lowest-volatility quarter remains a low-error subset with 20- and 90-observation windows: MAE is 7.60 and 7.50 p, respectively. Their council-minus-no-change differences are small, with intervals spanning zero. Root mean squared errors rise across disagreement quartiles for both the council and baseline as well (Appendix B). Thus, the visible raw gradient is not peculiar to a single absolute-error calculation. The pooled council–baseline comparison also varies by batch. Batch 3 has council MAE 18.88 p versus 16.97 p for no change; the other three batches have small mean advantages for the council (Table3). This variation cautions against interpreting the pooled near-zero difference as a stable characteristic of the platform. The available record contains only four overlapping evaluation periods. 11 −2.0−1.5−1.0−0.50.00.51.0 High-minus-low disagreement gap in error beyond no change (p) Primary balanced panel Expanded coverage Equities only Published-cohort membership Remove largest 1% of errors Median council forecast Original anchor scoring Figure 8:Sensitivity of the high-minus-low disagreement contrast in excess error. Negative values indicate a smaller council-minus-baseline error in the highest-disagreement quartile. Pointwise asset-cluster 95% intervals span zero across the listed specifications. 6 What an agreement score should communicate The results suggest separating three concepts in an AI forecasting system.Asset riskconcerns how variable the outcome may be.Directional concordanceconcerns whether the advisors share an up, flat, or down view.Return precisionconcerns how close their numerical prediction is to the eventual return. A score that blends these concepts can appear predictive while obscuring what it actually predicts. For this council, historical volatility is a strong first reference for forecast difficulty. Council dispersion adds no clearly resolved relationship with excess error under the tested comparisons. Directional agreement, however, has a positive association with directional accuracy. The conclusion is therefore more specific than “agreement is useless”: different forms of agreement carry different information, and none should be advertised as universal confidence without further validation. A sensible next test would freeze the feature definitions and thresholds before a new batch, register the available training labels, and evaluate direction and return losses separately after the native targets mature. If an error predictor is learned, validation must respect both the issue date and the maturity date of every label. A model trained on future-resolved labels cannot be presented as a real-time confidence layer merely because the forecasts themselves were made earlier. This archive does not contain the consistent predictive distributions needed to establish proba- bilistic calibration. A seven-framework spread cannot be labeled an 80% or 95% confidence interval without an additional, validated mapping to outcome coverage. Similarly, being more accurate on a retained subset does not establish an economic advantage. Such claims require a defined decision rule and appropriate costs, baselines, and out-of-time testing. 12 7 Limitations One system and few periods.The council shares historical Gemini model versions, and the four forecast periods overlap. Framework diversity is neither independence nor diversity across providers. Asset-cluster intervals preserve repeated assets but cannot eliminate dependence induced by common market shocks. Generalization to other models, markets, horizons, or regimes remains untested. Retrospective coverage and data vintage.Assets were curated for a production platform and then required to have complete forecasts and history. They are not a random sample of the investable universe. The evaluation uses a frozen October archive, not an immutable contempora- neous market-data vintage for every past day. Revised prices, corporate actions, and reconstructed context can differ from what was available during the original run. The analysis respects dates in the archive but cannot independently certify the full historical information set of every prompt. Scoring and operational choices.Native-currency price returns exclude dividend reinvestment, currency conversion, and execution costs. Evaluation is rebased to a saved post-generation close, rather than an assumed trade at the generation instant. The council mean, volatility scaling, sign threshold, and retention fractions are choices; the reported sensitivities address some, not all, alternatives. Qualitative report quality and synthesized portfolio recommendations are outside the scope. Exploration and unresolved effects.The analyses follow an exploratory audit rather than a preregistration. Multiple secondary comparisons are not multiplicity-adjusted. Intervals over- lapping zero do not demonstrate equivalence or rule out small useful effects. In particular, the directional-agreement result merits prospective confirmation before it is used as a confidence rule. 8 Conclusion In this historical AI council, wider numerical disagreement predicts larger return errors, but nearly the same pattern occurs for a no-change forecast and is strongly associated with visible asset volatil- ity. A simple volatility rule selects lower-error forecasts more effectively than raw disagreement. Agreement about direction has a different association: better directional accuracy, accompanied by larger numerical errors. The practical lesson is to evaluate agreement against the question it is meant to answer. Before calling consensus a confidence signal, establish whether it measures ordinary outcome risk, shared directional judgment, or an actual improvement in forecast reliability. Data, code, and author disclosure TheZenodo research deposit(DOI:10.5281/zenodo.23262216) andpublic research repositorycon- tain individual configuration forecasts, cohort metadata, aggregate results, deterministic analysis code, and vector figures. The frozen source-archive fingerprint and extraction rules are documented. The full market-derived outcome panel, raw market-history files, credentials, and operational meta- data are excluded from the public release. Recomputing outcome-dependent results requires com- patible market inputs obtained under appropriate terms or authorized access to the author-side 13 panel. Historical values come from the iPulse AI market-history archive; no alternative source attribution is implied. Original manuscript, figures, forecast data, and research summaries use C BY 4.0; code uses MIT. Third-party material retains its applicable terms. The author is the founder of Future Edge Group FZE, which develops iPulse AI, and has a commercial interest in the platform studied. AI systems assisted with research operations, coding, audit, and manuscript preparation. Authorship and responsibility for the final submission remain with the named human author. References [1]K. Lahiri and X. Sheng. Measuring forecast uncertainty by disagreement: The missing link. Journal of Applied Econometrics, 25:514–538, 2010.doi:10.1002/jae.1167. [2]A. Glas. Five dimensions of the uncertainty–disagreement linkage.International Journal of Forecasting, 36(2):607–627, 2020.doi:10.1016/j.ijforecast.2019.07.010. [3]B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles.Advances in Neural Information Processing Systems, 30, 2017. NeurIPS proceedings. [4]Q. Xiao, D. Bhattacharjya, B. Ganesan, R. Marinescu, K. Mirylenka, N. H. Pham, M. Glass, and J. Lee. The consistency hypothesis in uncertainty quantification for large language mod- els.Proceedings of the Forty-first Conference on Uncertainty in Artificial Intelligence, PMLR 286:4636–4651, 2025. PMLR proceedings. [5]E. Karger, H. Bastani, Chen Yueh-Han, Z. Jacobs, D. Halawi, F. Zhang, and P. E. Tetlock. ForecastBench: A dynamic benchmark of AI forecasting capabilities.arXiv:2409.19839, 2024, revised 2025. https://arxiv.org/abs/2409.19839. [6]R. El-Yaniv and Y. Wiener. On the foundations of noise-free selective classification.Journal of Machine Learning Research, 11:1605–1641, 2010.JMLR article. [7]T. Gneiting. Making and evaluating point forecasts.Journal of the American Statistical Asso- ciation, 106(494):746–762, 2011.doi:10.1198/jasa.2011.r10138. [8]R. Ramdowar. When models judge models: How source identity and disagreement shape multi- agent financial forecasts. iPulse AI research artifacts, 2026. doi:10.5281/zenodo.23083356. 14 A Reproduction and cohort details The analysis cutoff is 8 October 2026; the manuscript date is 9 October. The input is identified by SHA-256 1 . Source extraction selects completed native first-quarter observations in Batches 3–6 with the eligible ten configurations. Matching requires one observation per configuration, identical entry and target dates, a common issue date, finite predictions, and realized returns agreeing to tolerance10 −6 . No forecast is regenerated. The expanded matched panel has 242, 307, 303, and 375 outcomes in Batches 3, 4, 5, and 6. The primary panel has 232 in each. The one balanced asset excluded for risk validity contributes four outcomes; its omission is not based on its forecast error. The author-side derived panel preserves exclusion flags and coverage variables for auditing; the public cohort file identifies matched records and primary membership. Daily historical returns use positive finite closes before the issue date. Between consecutive observations, split ratios with recorded active ex-dates are applied. Histories are quarantined when a split adjustment produces an absolute daily log move above 0.4 while the unadjusted move is below 0.2. This heuristic conservatively avoids a possible double adjustment; it is not a general corporate-action validator. Momentum uses the last 60 available log returns. Historical volatility uses the sample standard deviation and at least 75% of the requested observation window, with a minimum of 15 observations. For the adjusted model, continuous variables arelogV,log(1 +|momentum|),log(1 +| ̄ p|), and log(1+D). Each is standardized using the primary panel’s mean and population standard deviation. Ordinary least squares includes batch and asset-class indicators; bootstrap refits preserve each asset’s complete cluster. The random seed is 20261009. No fitted model is claimed to be deployable at historical issue dates. A numerical tolerance of10 −10 p at the±0.5p sign boundaries prevents floating-point classi- fication artifacts. For retained fractions, assets are sorted within each batch by score, breaking ties with the stable asset identifier; the retained count is rounded upward. The bootstrap holds this selection mem- bership fixed. For the within-risk contrast, assets are sorted into five approximately equal-sized volatility bins per batch, then split into lower and higher disagreement halves within each bin. These finite-sample comparisons cannot eliminate all continuous-risk differences. B Additional numerical results Batch Council MAE No-change MAE Difference Direction Always up 318.8816.97+1.9138.36%52.16% 413.7014.21−0.5150.43%51.72% 513.4714.08−0.6253.88%59.91% 614.5414.68−0.1450.86%39.22% Table 3:Batch-level estimates in the primary panel, 232 outcomes per row. Return errors are in p. Direction uses the three-category sign rule; always up is a separate directional baseline. 1 f4f9510fe0ae2900f124b12e638f06eac073f0397c5646e6dfea62a278555be1. 15 Disagreement quartile Council RMSE (p) No-change RMSE (p) Lowest17.6916.55 Second21.0120.07 Third23.3922.79 Highest31.1430.63 Table 4:Squared-error sensitivity. Both series exhibit increasing error across disagreement quartiles. This table supplies a second loss function, not a fitted probabilistic forecast evaluation. Volatility window Retained outcomes Council MAE (p) Excess MAE (p) 20 observations 232 7.60 +0.08 [ − 0 . 31 , 0.45] 60 observations2327.97−0.03[−0.43, 0.37] 90 observations2327.50 +0.02 [−0.36, 0.41] Table 5:Lowest-volatility quarter under alternative historical windows. Brackets are pointwise asset-cluster 95% intervals for the paired error difference relative to no change. The public CSVs permit checking forecast grouping and cohort membership.prepare_panel.py documents extraction from the private frozen archive;analyze.pyandmake_figures.pyrequire the full derived author-side panel to recompute outcome-dependent tables and figures. That panel is retained privately, together with an executed notebook and independent validation checks. No production service calls or new model generations are required when compatible inputs are available. The public release therefore exposes the forecasts, methods, code, and aggregate findings, but is not a fully self-contained outcome-replication dataset. The arXiv source bundle contains only the manuscript, bibliography, and referenced figures. 16