Paper deep dive
Topological Signatures of Context-Level Reliability in TabPFN
James Hu, Mahdi Ghelichi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/21/2026, 6:03:22 AM
Summary
This paper investigates the internal representation geometry of TabPFN, a transformer-based foundation model for tabular prediction, using zigzag persistent homology. By treating layer representations as evolving point clouds, the authors construct a synthetic benchmark with varied topologies (warped circles, tori, spheres, etc.) to study how topological features correlate with reliability metrics. Key findings indicate that H0 fragmentation and H1 loop activity serve as topological signatures for dataset-level reliability, with increased fragmentation and loop activity correlating with higher error rates and overconfidence. The study suggests that zigzag persistence can diagnose when TabPFN operates in topologically stressed, unreliable regimes.
Entities (12)
Relation Signals (7)
Zigzag Persistent Homology → analyzes → TabPFN
confidence 95% · We study this behavior using zigzag persistent homology, treating TabPFN layer representations as evolving point clouds.
H0 Fragmentation → correlateswith → Mean Absolute Residual
confidence 95% · the zeroth homology group H0 fragmentation count correlates positively with mean absolute residual across controlled tasks
H0 Fragmentation → indicates → Reliability
confidence 92% · Our results suggest that zigzag persistence diagnoses the reliability of the inferred in-context task geometry
H1 Loop Activity → correlateswith → Bayes Error
confidence 90% · These descriptors correlate with Bayes error, mean absolute residuals, and overconfidence.
H1 Loop Activity → increaseswith → Geometric Difficulty
confidence 88% · Harder geometries induce a dual topological signature: increased H1 loop activity
H0 Fragmentation → increaseswith → Geometric Difficulty
confidence 88% · Harder geometries induce a dual topological signature: ... increased H0 fragmentation
Warped Circle → usedin → TabPFN
confidence 85% · We construct a controlled benchmark of synthetic tabular tasks... including warped circles... to study this behavior
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:TabPFN is a transformer-based foundation model for tabular prediction that performs inference without task-specific training by conditioning on a support set and query inputs. Despite its strong empirical performance, its internal behavior on structurally difficult tabular geometries remains poorly understood. We study this behavior using zigzag persistent homology, treating TabPFN layer representations as evolving point clouds. We construct a controlled benchmark of synthetic tabular tasks with known true probabilities and varied intrinsic topology, including warped circles, tori, spheres, Hopf links, trefoil knots, and Swiss rolls. Across these tasks, we find that the topology of TabPFN's internal representation geometry is strongly associated with dataset-level reliability; for example, the zeroth homology group $H_0$ fragmentation count correlates positively with mean absolute residual across controlled tasks, and this association strengthens in a high-resolution warped circle case study at large sample size. Harder geometries induce a dual topological signature: increased $H_1$ loop activity and increased $H_0$ fragmentation, while the $H_1$ persistence becomes shorter-lived. These descriptors correlate with Bayes error, mean absolute residuals, and overconfidence. Our results suggest that zigzag persistence diagnoses the reliability of the inferred in-context task geometry and provides a context-level view of when TabPFN operates in topologically stressed regimes.
Tags
Links
- Source: https://arxiv.org/abs/2607.17962v1
- Canonical: https://arxiv.org/abs/2607.17962v1
PDF not stored locally. Use the link above to view on the source site.
Full Text
66,233 characters extracted from source content.
Expand or collapse full text
Topological Signatures of Context-Level Reliability in TabPFN James Hu, Mahdi Ghelichi Model Development Innovation, Risk Management TD Bank, Toronto, Canada james.hu, mahdi.ghelichi@td.com Abstract TabPFN is a transformer-based foundation model for tabular prediction that performs inference without task-specific training by conditioning on a support set and query inputs. Despite its strong empirical performance, its internal behavior on structurally difficult tabular geometries remains poorly understood. We study this behavior using zigzag persistent homology, treating TabPFN layer representations as evolving point clouds. We construct a controlled benchmark of synthetic tabular tasks with known true probabilities and varied intrinsic topology, including warped circles, tori, spheres, Hopf links, trefoil knots, and Swiss rolls. Across these tasks, we find that the topology of TabPFN’s internal representation geometry is strongly associated with dataset-level reliability; for example, the zeroth homology group H0H_0 fragmentation count correlates positively with mean absolute residual across controlled tasks, and this association strengthens in a high-resolution warped circle case study at large sample size. Harder geometries induce a dual topological signature: increased H1H_1 loop activity and increased H0H_0 fragmentation, while the H1H_1 persistence becomes shorter-lived. These descriptors correlate with Bayes error, mean absolute residuals, and overconfidence. Our results suggest that zigzag persistence diagnoses the reliability of the inferred in-context task geometry and provides a context-level view of when TabPFN operates in topologically stressed regimes. 1 Introduction Foundation models for tabular data have recently become a promising alternative to task-specific model training. TabPFN, in particular, performs tabular prediction by conditioning on a set of labeled context rows and unlabeled query rows, using a transformer architecture trained on large collections of synthetic tabular tasks [19, 20, 14, 15]. This makes TabPFN an in-context learner for tabular data: at inference time, the model does not update its weights, but instead infers a task from the provided support set. Recent studies have begun to examine TabPFN’s robustness under controlled tabular perturbations [38, 21], but its inference mechanism raises a different interpretability question. When TabPFN works, what kind of internal task geometry does it construct? When TabPFN becomes unreliable, how does the geometry of the entire in-context task change inside the transformer? Additional work has been studying the combination of topological data analysis (TDA) with TabPFN in downstream prediction settings. For example, persistent-homology features extracted from medical imaging data have been successfully used as inputs to TabPFN models, with loop and cavity based topological descriptors contributing substantially to predictive performance [18]. These results suggest that topology can provide meaningful information for tabular foundation models, but leave open the complementary question we study here: whether the topology of TabPFN’s own internal representations is itself informative. Recent work on large language models has shown that the geometry of hidden representations can reveal how transformers reorganize inputs across layers [12]. In particular, zigzag persistence has been proposed as a way to track topological features as point clouds evolve through model depth, rather than analyzing each layer independently [4]. A related line of work, HalluZig, models layerwise LLM attention matrices as evolving graphs and uses zigzag persistence to study reliability-relevant internal dynamics [32]. These studies suggest that topological summaries of internal transformer dynamics can expose structure that is not visible from output probabilities alone. In this paper, we extend the representation-topology perspective to TabPFN. Unlike standard language prompts, TabPFN inputs define an in-context tabular task: support rows, support labels, and query rows jointly determine the prediction problem. We therefore ask whether the topology of TabPFN’s internal representations reflects the reliability of this inferred task geometry. We focus on the first two homology dimensions. The zeroth homology group, H0H_0, captures connected components; in TabPFN’s layerwise representation graph, it measures how the in-context task separates into or merges across coarse groups. The first homology group, H1H_1, captures one-dimensional cycles or loops; here it measures cyclic or entangled structure in the representation graph as TabPFN processes more difficult tabular tasks. In this paper, we make the following contributions: 1. We introduce a zigzag-persistence framework for analyzing TabPFN hidden representations across layers. 2. We construct a controlled topology benchmark for tabular prediction with known true probabilities and varied intrinsic topology. 3. We show that TabPFN representation topology predicts dataset-level reliability metrics, including observed error, Bayes error, mean absolute residuals, and overconfidence. 4. We identify an H0/H1H_0/H_1 “scissors” pattern—named for two topological quantities that move in opposite directions as difficulty rises: H1H_1 loop activity increases while the durable persistence of H0H_0 connected components decreases (with the H0H_0 event count rising alongside). 5. We connect these findings to prior topological analyses of transformer representations and interpret TabPFN topology as the topology of an inferred in-context task geometry. The organization of the paper is as follows. Section 2 discusses related work on topological analysis and zigzag persistence and their applications on transformers. In Section 3, we describe the experiment setups including zigzag construction and metrics definitions. Section 4 describes the synthetic datasets with different topology families used for the experiments. In Section 5, we present the experiment results. Further implications from the results, limitations, and future work are discussed in Section 6. Section 7 concludes the paper. 2 Related Work 2.1 Tabular foundation models and in-context learning Classical tabular prediction has long been dominated by tree-based ensembles such as random forests and gradient-boosted trees [3, 6, 30], which remain strong baselines and often match or exceed early tabular deep-learning models [16, 13, 1]. TabPFN reframed the problem as in-context learning: a transformer [36] pretrained on large collections of synthetic tasks predicts on a new dataset in a single forward pass, without per-dataset training [19, 20]. This mechanism connects to a broader literature on how transformers learn in context [8, 37]. A fast-growing line of work extends the paradigm by scaling it to larger and real-world tables [25, 14, 15, 2], benchmarking it [11], probing what it internalizes [38, 33], characterizing its robustness [28, 21], and adapting it through fine-tuning [34]. The prospect of a single pretrained model that transfers across tables has made tabular foundation models a research priority [35], with direct relevance to high-stakes tabular domains such as biomedical prediction [22] and financial risk management [17]. Our contribution is complementary to this line: rather than improving or benchmarking TabPFN, we ask what its internal representation geometry reveals about when its predictions can be trusted. 2.2 Topology of neural representations Topological Data Analysis (TDA) provides tools for studying the shape of high-dimensional point clouds. Persistent homology tracks the birth and death of topological features across a filtration, producing barcodes or persistence diagrams that record connected components, loops, voids, and higher-dimensional structures [10, 39, 5, 9]. Prior works such as [31, 27] on neural-network topology has used Betti numbers, persistence diagrams, and topological descriptors to study how internal representations change through depth of the networks. These studies motivate our use of H0H_0 and H1H_1 as interpretable summaries of TabPFN representation topology. 2.3 Zigzag persistence for transformer representations The closest methodological precedent is the recent zigzag persistence framework for large language models [12]. That work treats last-token representations from many prompts as point clouds that evolve through the layers of decoder-only transformers. For each layer, the authors construct a k-nearest-neighbor graph (with default setup k=4k=4), expand it into a simplicial complex, insert intersection complexes between adjacent layers, and compute zigzag persistence. Their descriptors reveal layerwise processing phases and support a layer-pruning criterion. Our work adopts the same dynamic view, but applies it to a different object. In TabPFN, the evolving point cloud represents an in-context tabular task built from context and query rows. This changes the interpretation of the topology: the descriptors are not only summaries of hidden-state shape, but diagnostics of the reliability of the inferred task geometry. 2.4 Topology and reliability in transformer models HalluZig demonstrates that zigzag persistence can be applied to transformer attention graphs for reliability analysis [32]. The method converts layerwise attention matrices into graphs, applies zigzag persistence, and uses the resulting signature to characterize internal attention dynamics. Our work shares the broad reliability motivation, but studies hidden representation topology in TabPFN rather than attention topology in language models. The common theme is that evolving transformer-internal graphs or point clouds can reveal reliability-relevant structure. 3 Experimental Method 3.1 Extracting TabPFN layer representations TabPFN processes a tabular task by conditioning on labeled context rows and unlabeled query rows. We extract hidden representations from each of the 12 transformer layers of TabPFN v2 [20]. For each layer, the selected representations form a point cloud in hidden space. Depending on the experiment, the point cloud may consist of query representations, support representations, or other embedding scopes. Our main experiments use query label-token representations as the primary view of the internal task geometry, with additional robustness checks across support feature-token and support label-token representations. 3.2 Constructing a zigzag filtration For each layer ℓ , we construct a k-nearest-neighbor graph from the representation point cloud and expand this graph into a simplicial complex KℓK_ by filling cliques up to a fixed maximum simplex dimension. Following prior zigzag representation work [12], we set k=4k=4 as our default value. Consecutive layer complexes are not nested, because nearest-neighbor relations can appear and disappear as representations move through the transformer. As used in [23], we therefore insert intersection complexes between adjacent layers: K0←K0∩K1→K1←K1∩K2→⋯←KL−2∩KL−1→KL−1.K_0← K_0∩ K_1→ K_1← K_1∩ K_2→·s← K_L-2∩ K_L-1→ K_L-1. (1) For TabPFN v2 [20] with L=12L=12 transformer layers, this produces 23 zigzag positions: 12 model-layer complexes and 11 intermediate transition complexes. The output of zigzag persistence is a multiset of intervals [b,d][b,d], where each interval represents a topological feature born at position b and dying at position d. Following prior work on LLM zigzag persistence, we map raw zigzag intervals to effective model-layer intervals to avoid artifacts induced by alternating model and intermediate intersection layers [4, 12]. 3.3 Topological descriptors We compute descriptors in homology dimensions H0H_0 and H1H_1. Here, H0H_0 reflects coarse grouping, fragmentation, and connectedness of the representation graph. H1H_1 reflects higher-order relational complexity or entanglement in the representation topology. Intuitively, H0H_0 and H1H_1 summarize two different ways TabPFN can arrange the in-context examples in its hidden space. H0H_0 counts connected components, so it tracks how the support and query rows group together: when the task is easy and aligned with the model’s prior, the rows coalesce into a few clean, long-lived clusters, which is the geometry of a confidently separable decision problem. As difficulty rises, those clusters break into many small, short-lived fragments—the model can no longer hold the examples in a single coherent grouping, so the H0H_0 event count grows while the lifetime of each component shrinks. H1H_1 counts one-dimensional loops, which appear when the model wraps the examples back on themselves instead of laying them out along a simple separating direction; a loop is the signature of an entangled, non-linearly-separable arrangement, and more or longer-lived loops indicate a more convoluted internal decision surface. Read together, H0H_0 reports whether the model still perceives clean groups and H1H_1 reports whether it must bend the representation into cycles to accommodate the labels, so their joint movement (Section 5.4) describes how the inferred task geometry deforms as the input becomes harder. We organize descriptors into three families: Activity metrics. These measure how much topology is generated across the layer trajectory: H1area=∑[b,d]∈H1(d−b),H_1\,area= _[b,d]∈ H_1(d-b), (2) with analogous total-persistence summaries for H0H_0. We also compute H0H_0 and H1H_1 bar counts and Betti-curve summaries. Stability metrics. These compute how long topological features survive, including mean persistence, maximum persistence, and durable-feature fractions. Timing metrics. These evaluate where features appear or resolve across model depth. We use early/middle/late persistence mass, birth-persistence heatmaps, and a persistence-weighted histogram πhist(α) _hist(α) analogous to the births’ relative frequency descriptor used in prior LLM zigzag work [12]. Unless otherwise noted, we report α=1α=1, which weights features by their effective persistence. 3.4 True-probability reliability metrics One limitation of using real datasets for our experiments is that the dataset’s true conditional probability P∗(y∣x)P^*(y x) is unknown. To study calibration and overconfidence precisely, we construct synthetic datasets for classification problems with known true probabilities. For each test row x, we compute p^pred=p^(y^∣x), p_pred= p( y x), (3) where y y is TabPFN’s predicted class. We also compute the true probability of that predicted class: ppred∗=P∗(y^∣x).p^*_pred=P^*( y x). (4) The signed residual is r(x)=p^pred−ppred∗.r(x)= p_pred-p^*_pred. (5) Positive residuals indicate overconfidence relative to the truth. The main calibration quantity used in the results is the dataset-level mean absolute residual (MAR), MAR=1n∑i=1n|r(xi)|,MAR= 1n _i=1^n|r(x_i)|, (6) which measures the typical gap between TabPFN’s confidence and the true probability of the predicted class. We also report metrics such as observed error, Bayes error, overconfidence rate, and wrong-overconfidence rate. Observed error is the fraction of test points for which TabPFN’s predicted label differs from the sampled test label. We use Bayes error as shorthand for Bayes-label error: the fraction of test points for which TabPFN’s predicted label differs from the Bayes-optimal label [P∗(y=1∣x)≥1/2]1[P^*(y=1 x)≥ 1/2]. Thus Bayes error measures disagreement with the true decision rule, not the irreducible Bayes risk of the data-generating process. The overconfidence rate is the fraction of test points with r(x)>0.10r(x)>0.10, and the wrong-overconfidence rate further requires the prediction to be incorrect. Statistical analysis. Unless otherwise noted, we quantify monotonic associations between a topological descriptor and a reliability metric with the Spearman rank-correlation coefficient ρ, which ranges from −1-1 for a perfectly decreasing monotone relationship to +1+1 for a perfectly increasing one. Hard-versus-easy contrasts are reported as Cohen’s d, a standardized mean difference. Significance is assessed with the Mann–Whitney U test for group comparisons and the Wilcoxon signed-rank test for paired base-versus-fine-tuned comparisons. We use the conventional thresholds p∗<0.05^*p<0.05, p∗<0.01^**p<0.01, and p∗∗<0.001^***p<0.001, and write “n.s.” for p≥0.05p≥ 0.05. 4 Controlled Topology Benchmark We construct six synthetic topology families designed to stress TabPFN across different geometric and topological regimes. 4.1 Topology families Table 1 summarizes the six families and Figure 1 visualizes them. The warped circle is a one-loop S1S^1 baseline; the torus is a two-loop T2T^2 structure; the sphere is a surface topology included to test whether H1H_1-focused descriptors are sufficient; the Hopf link consists of two interlinked circles; the trefoil knot is a knotted one-dimensional manifold; and the Swiss roll is a geometrically curved but topologically loop-free surface. The Swiss roll serves as a negative H1H_1 control where H1H_1 growth reflects spurious or representation-induced cycles rather than true input topology. Each family is generated at multiple difficulty levels by increasing noise, nuisance dimensions, warping, curvature, or reducing separation. Label sharpness is held fixed to avoid conflating geometric difficulty with changing Bayes noise. Table 1: The six controlled topology families. Label sharpness is held fixed so that geometric difficulty is decoupled from Bayes noise. Family Intrinsic topology Expected H1H_1 Role Warped circle S1S^1 1 loop Baseline Torus T2T^2 2 loops Intermediate Sphere S2S^2 0 (surface) H1H_1-insufficiency test Hopf link two linked S1S^1 2 linked loops Linked loops Trefoil knot knotted S1S^1 persistent H1H_1 Entangled knot Swiss roll ℝ2R^2 sheet ≈0≈ 0 Negative H1H_1 control (a) Warped circle (S1S^1) (b) Torus (T2T^2) (c) Sphere (S2S^2) (d) Hopf link (e) Trefoil knot (f) Swiss roll Figure 1: The six controlled topology families, shown as the ambient point cloud of a representative level 0 (lowest difficulty level) test set colored by the true probability P(y=1∣x)P(y=1 x). The warped circle is planar; the remaining families are embedded in ℝ3R^3. 4.2 Experimental scale The main topology-difficulty suite contains 180 runs: 6families×6difficulty levels×5seeds,6\ families× 6\ difficulty levels× 5\ seeds, (7) each generated with ntrain=400n_train=400 context rows and ntest=300n_test=300 query rows. We also run an extreme suite at levels 66–88 to probe saturation behavior under stronger noise and higher nuisance dimension. To establish the core relationship at high resolution, we run a large-scale study on the warped circle, the family with the clearest single-loop topology and natural class balance, across all nine difficulty levels with ntrain=6667n_train=6667 and ntest=5000n_test=5000. As in the main suite, we use five seeds per level, giving 4545 runs. We have also repeated the same design at half scale (ntrain=3333n_train=3333, ntest=2500n_test=2500) to test scale-invariance which the results are briefly mentioned as patterns are similar to the large-scale run. We further run fine-tuning experiments and embedding-scope robustness checks to verify that the main observations are not artifacts of the default setup. 5 Experimental Results We first establish the topology–reliability relationship at high resolution using the large-sample warped circle case study. We then return to the full six-family benchmark and show how the same pattern generalizes, where it breaks, and which parts of the signature remain stable across scale. 5.1 Large-sample case study: the warped circle at scale To probe the core claim with minimal confounds, we study the warped circle (S1S^1) in depth at ntrain=6667n_train=6667 and ntest=5000n_test=5000. This family has a single interpretable H1H_1 feature and near-balanced labels by construction, so a difficulty ladder can change its geometry without changing the underlying homotopy type. H0H_0 fragmentation strongly tracks reliability at scale. At scale, the H0H_0 fragmentation descriptors track true-probability reliability with strong monotone associations, as shown in Table 2 and Figure 2. Pooled over all 4545 runs, the H0H_0 fragmentation count correlates with MAR at ρ=0.92ρ=0.92 and with Bayes error at ρ=0.94ρ=0.94, while H0H_0 total persistence correlates at ρ=−0.94ρ=-0.94 with both metrics. The relationship is not tied to a single error definition: the same fragmentation count also tracks observed error, overconfidence, and wrong-overconfidence. The durable H1H_1 signal points in the same direction from the loop side, with long-lived loop persistence falling as difficulty increases. At high resolution, the two homological dimensions agree on the same failure pattern: reliability degrades as the representation fragments. H1H_1 area saturates at high sampling density. In contrast, raw H1H_1 area is a weaker predictor at scale. Figure 2(c) shows that with ntest=5000n_test=5000 densely sampled points, the circle’s single loop is already resolved by difficulty level 22, after which H1H_1 area plateaus even as error keeps climbing. The H0H_0 descriptors do not show the same ceiling. The fragmentation count rises monotonically from about 90009000 to more than 2300023000, and total persistence falls across the full range. At high resolution, because the magnitude of H1H_1 saturates, the stress signal therefore shifts to the H0H_0 fragmentation and the loss of durable H1H_1 structure. This saturation is expected. For a fixed neighborhood size k, the k-nearest-neighbor graph on N points has a finite cycle capacity, on the order of (k−1)N(k-1)N independent 11-cycles. At high density, even mild difficulty can push the graph close to that capacity, compressing the dynamic range of raw loop counts and areas. Consistent with this, H1H_1 count rises by 3 times across levels at ntest=300n_test=300, but only by less than 2 times at ntest=5000n_test=5000. Persistent homology also stabilizes as dense samples converge to the underlying manifold, so the large finite-sample swings in H1H_1 area at small n should not be expected to persist indefinitely [7, 29]. The H0H_0 fragmentation count and durable-H1H_1 persistence have no comparable raw-area ceiling, and they continue to change monotonically with difficulty. This hints why H0H_0 fragmentation is the more scale-stable reliability signal. The hard-versus-easy contrast is extreme. Splitting the case study into its three easiest and three hardest levels, Bayes error rises from 0.015 to 0.223, and MAR rises from 0.022 to 0.244. The same split shows H0H_0 total persistence collapsing from 69 to 17 and the H0H_0 fragmentation count nearly doubling. These are among the largest standardized effects we observe, helped by the low variance that dense sampling affords. This case study gives the least confounded version of the paper’s central relationship. Once sampling noise is reduced, H0H_0 fragmentation and durable-H1H_1 collapse become strong readouts of where TabPFN’s predictions become unreliable. The descriptor that weakens, raw H1H_1 area, does so for a concrete reason: the loop has already been resolved, so the raw area has little room left to grow. Table 2: warped circle case study at scale (n=45n=45, ntest=5000n_test=5000). Pooled Spearman correlations between embedding topology and true-probability reliability. H0H_0 fragmentation is strongly associated with reliability; raw H1H_1 area saturates once the loop is densely resolved. Descriptor vs. MAR vs. Bayes error H0H_0 total persistence −0.94-0.94 −0.94-0.94 H0H_0 fragmentation count 0.920.92 0.940.94 Durable H1H_1 persistence −0.86-0.86 −0.87-0.87 H1H_1 area (saturates) 0.520.52 0.540.54 (a) (b) (c) Figure 2: warped circle case study at scale (4545 runs, ntest=5000n_test=5000), with points in (a,b) colored by difficulty level. (a) H0H_0 fragmentation count vs MAR (ρ=0.92ρ=0.92) and (b) H0H_0 total persistence vs MAR (ρ=−0.94ρ=-0.94): the color gradient shows that fragmentation moves monotonically with difficulty while simultaneously tracking calibration, so a single scatter conveys both the level progression and the reliability relationship. (c) H1H_1 area rises then saturates by level 2 once the loop is densely resolved, which is why raw H1H_1 area is a weaker large-sample predictor than the ceiling-free H0H_0 descriptors. 5.2 The pattern generalizes across six topology families Across the full six-family benchmark, Figure 3 indicates that TabPFN’s internal topology changes systematically as the input geometry becomes more difficult. Comparing the easiest and hardest difficulty tertiles, split by ground-truth difficulty level, the loop- and fragmentation-related descriptors all increase with topological stress: H1H_1 area grows from 686 to 1086, the number of H1H_1 loops from 643 to 1014, durable H1H_1 persistence from 152 to 216, and the H0H_0 fragmentation count from 484 to 744. TabPFN’s observed error rises across the same difficulty range as seen in Figure 3b. The H0H_0 fragmentation count has the largest hard-versus-easy effect among the reported topology descriptors, exceeding the loop-activity metrics and matching the case-study result that fragmentation is the dominant signal. Model reliability shifts in the same direction: MAR, Bayes error, and the wrong-overconfidence rate all rise. A complementary movement appears in H0H_0 total persistence, which decreases as difficulty grows. The representation graph produces more connected-component events, but each is shorter-lived: the mean H0H_0 bar lifetime falls. This is the first sign of the H0/H1H_0/H_1 scissors developed in Section 5.4: the task representation becomes more eventful, but less stable. Together, these results show that the large-sample warped circle relationship is not a peculiarity of the circle. Across linked, knotted, surface, and control geometries, harder inputs push TabPFN toward a more fragmented and more cyclic internal representation, and that movement coincides with a measurable loss of reliability. (a) (b) Figure 3: (a) Embedding-topology descriptors in the easy vs hard difficulty tertiles; all loop and fragmentation-count metrics increase under topological stress (Cohen’s d and Mann–Whitney significance annotated). (b) TabPFN’s observed error grows with difficulty level in the main topology-difficulty suite across all six families. 5.3 H0H_0 and H1H_1 predict different reliability axes The topological descriptors correlate strongly with true-probability reliability metrics presented in Table 3 and Figure 4. Pooled across all 180180 runs, H1H_1 area correlates positively with MAR, observed error, and Bayes error. Loop activity is therefore not only a geometric artifact: it tracks datasets on which both decisions and probabilities worsen. The H0H_0 descriptors are especially informative for calibration. The H0H_0 fragmentation count is the single strongest predictor of MAR, whereas H0H_0 total persistence and H0H_0 early-layer mass decrease as reliability worsens. Calibration failure is therefore tied to fragmentation and to the loss of stable, early-formed grouping structure, echoing the H0H_0 dominance seen in the high-resolution case study in Section 5.1. Figure 5 ranks the standardized effect sizes of the hard-versus-easy contrast. The two dimensions also separate by failure type in the reported metrics. H1H_1 area is positively associated with MAR, Bayes error, and observed error, while early H0H_0 mass is the strongest predictor of Bayes error in Table 3. Thus H0H_0 and H1H_1 are not redundant; they capture complementary aspects of TabPFN’s internal task geometry. H0H_0 tracks coarse organization and fragmentation, while H1H_1 tracks loop complexity. Table 3: Pooled Spearman correlations between embedding-topology descriptors and true-probability reliability metrics. Larger values of every reliability metric indicate worse reliability. p∗<0.05^*p<0.05, p∗<0.01^**p<0.01, p∗∗<0.001^***p<0.001. Descriptor MAR Bayes error Observed error H1H_1 area 0.45∗∗0.45^*** 0.33∗∗0.33^*** 0.34∗∗0.34^*** H0H_0 count 0.51∗∗0.51^*** 0.37∗∗0.37^*** 0.25∗∗0.25^*** H0H_0 total −0.39∗∗-0.39^*** −0.28∗∗-0.28^*** −0.06-0.06 H0H_0 early frac. −0.41∗∗-0.41^*** −0.42∗∗-0.42^*** −0.35∗∗-0.35^*** (a) (b) (c) (d) Figure 4: Dataset-level relationships between embedding topology and reliability (each point is one of the 180180 runs, colored by family). (a) H1H_1 area vs MAR; (b) H1H_1 area vs Bayes error; (c) H0H_0 fragmentation count vs MAR, the strongest single predictor; (d) H0H_0 total persistence vs MAR, showing that shorter-lived clusters accompany worse calibration. Inset annotations report the pooled Spearman ρ. Figure 5: Standardized effect sizes (Cohen’s d) for the hard-versus-easy tertile contrast. Loop activity and the H0H_0 fragmentation count rise with difficulty while Bayes error increases; H0H_0 total persistence (Section 5.4) moves in the opposite direction. 5.4 The H0/H1H_0/H_1 scissors effect The most interpretable pattern is the H0/H1H_0/H_1 scissors effect seen in Figure 6. As difficulty increases, H1H_1 area grows while H0H_0 total persistence falls; for example, in the warped circle, H1H_1 area grows by roughly 2.4×2.4× from level 0 to level 5, while H0H_0 total persistence falls from 33 to 14. The H0H_0 count rises at the same time. In words, loop structure proliferates while stable connected-component structure erodes. A unified fragmentation principle. The high-resolution case study in Section 5.1 and a matched half-scale run (ntest=2500n_test=2500) show that the scissors is, more fundamentally, a fragmentation principle acting in both homological dimensions: under stress the model produces more features, but they survive for less time. Table 4 reports how each descriptor correlates with difficulty level as sampling density grows from ntest=300n_test=300 to 5000. Feature counts rise with difficulty in both dimensions at every scale. Durable structure is more revealing: H0H_0 total persistence is negatively correlated with difficulty at all scales, and durable H1H_1 persistence reverses sign as resolution increases. At low sampling density, noisy short-lived bars inflate the apparent magnitude of loop activity; at high resolution, harder topology still produces more loop events, but those loops become less stable. The durable-H1H_1 sign flip comes from two competing effects. Difficulty increases the number of cycles while shortening their lifetimes, so the fraction of loop persistence carried by durable bars falls at every scale, from 0.26 to 0.20 at ntest=300n_test=300 and from 0.35 to 0.11 at ntest=5000n_test=5000. Whether the absolute durable mass rises or falls depends on which effect wins. At small n, the graph is far from its cycle ceiling and the count inflates nearly threefold across levels, pulling even the durable component upward. At large n, total growth is capped by saturation while the durable fraction collapses, so durable mass falls. Dense sampling builds a genuinely long-lived loop that noise can then destroy; sparse sampling never builds one as cleanly, so noise mostly inflates cycle counts. This is why the sign of the durable trend, but not the underlying fragmentation, depends on resolution. TabPFN is not simply creating more topology in a uniform way. Under topological stress it fragments its representation in both dimensions: durable coarse organization in H0H_0 collapses, and durable relational structure in H1H_1 collapses as well. Raw H1H_1 area rises only until the manifold is densely resolved, after which it saturates. The scale-invariant signature of stress is therefore the proliferation of short-lived features together with the loss of durable structure. Table 4: The scissors is a scale-invariant fragmentation principle. Spearman ρ between each descriptor and difficulty level for the warped circle (levels 0–8, n=45n=45 runs per sample size). Feature counts rise with difficulty in both dimensions at every scale; durable H1H_1 persistence reverses sign as resolution increases, revealing that loops fragment just as connected components do. ρ(⋅,difficulty level)ρ(·,\ difficulty level) ntest=300n_test=300 25002500 50005000 H0H_0 count 0.980.98 0.960.96 0.940.94 H1H_1 count 0.930.93 0.770.77 0.650.65 H1H_1 area 0.950.95 0.750.75 0.530.53 H0H_0 total persistence (durable) −0.91-0.91 −0.94-0.94 −0.94-0.94 Durable H1H_1 persistence 0.790.79 −0.77-0.77 −0.87-0.87 (a) (b) (c) Figure 6: The H0/H1H_0/H_1 scissors across difficulty levels (mean ± s.d. per family). (a) H1H_1 area rises with difficulty; (b) H0H_0 total persistence falls; (c) H0H_0 count rises. Loop structure proliferates while stable cluster structure fragments. 5.5 Trefoil knot reveals a second failure mode The per-family analysis in Table 5 shows that the topology–error relationship, while positive for five of six families, inverts for the trefoil knot. The trefoil generator uses a per-dataset median split, so class balance does not drift with difficulty. The inversion therefore reflects a different representational behavior: as the knot becomes harder to resolve, TabPFN appears to simplify or collapse its representation instead of generating more loop structure. This pattern points to a second failure mode. For most topology families, reduced reliability is associated with increased topological complexity. For the trefoil, reduced reliability instead appears as representational collapse: as the knot becomes harder to resolve, the model simplifies the entangled structure rather than generating richer loop geometry. The collapse registers in both homological dimensions. The H0H_0 fragmentation count, which is positively coupled to error for the other structured families, also inverts for the trefoil. The inversion is strongest for decision-error metrics such as observed and Bayes error, and weakest for MAR, consistent with a loss of discriminative structure that hurts decisions without creating a systematic over- or under-confidence bias. The trefoil suggests a boundary of the complexity-driven regime: beyond a critical entanglement, added difficulty no longer enriches the representation but dismantles it. Table 5: Per-family Spearman correlation between H1H_1 area and observed error. Five families are positive; the trefoil knot inverts, indicating a collapse rather than a complexity-driven failure mode. p∗<0.05^*p<0.05, p∗∗<0.001^***p<0.001; n.s. not significant. Family ρ(H1 area,obs. error)ρ(H_1 area,\ obs.\ error) Hopf link 0.83∗∗0.83^*** Sphere 0.61∗∗0.61^*** Warped circle 0.59∗∗0.59^*** Swiss roll (control) 0.43∗0.43^* Torus 0.160.16 (n.s.) Trefoil knot −0.47∗-0.47^** 5.6 Behavior under extreme stress We also investigate an extreme suite at difficulty levels 66–88, with noise up to 1.51.5 and up to 200200 nuisance dimensions. Three observations support the moderate-difficulty interpretation. First, the trefoil’s inverted relationship disappears: once the knot and its scaled copy overlap almost completely, the representation has already collapsed, leaving little residual topology–error relationship to measure. Second, the H0/H1H_0/H_1 scissors persists. Across the extreme levels, H1H_1 area and the H0H_0 fragmentation count continue to rise with difficulty while H0H_0 total persistence continues to fall. Third, topology still predicts decision error strongly: H1H_1 area correlates with observed error at ρ=0.84ρ=0.84. Its correlation with MAR, however, saturates toward zero, as expected once both the true probability and the model move toward chance under overwhelming noise. The extreme regime therefore separates two claims that are coupled at moderate difficulty: topology continues to track decision failure, while the calibration-residual signal saturates when the task itself approaches chance. 5.7 Swiss roll as a negative control The Swiss roll is topologically equivalent to a two-dimensional sheet and has no true loop structure, despite being geometrically curved. It therefore serves as a negative H1H_1 control: if our H1H_1 descriptors were responding only to generic curvature, noise, or sampling artifacts, Swiss roll would show effects comparable to genuinely loop-bearing families such as the Hopf link or warped circle. The Swiss roll does show some H1H_1 growth with increasing noise and a moderate topology–error correlation in Table 5, but the effect is substantially weaker than for genuinely loop-bearing topologies such as the Hopf link. This is the expected leakage for a geometric control: noise can create spurious cycles, but the response is weaker than when the data contain true loop-like structure. The H0H_0 signal is similarly muted; its fragmentation count barely tracks MAR compared with the genuinely structured families. Both homological dimensions therefore respond more strongly to intrinsic topology than to curvature or noise alone. 5.8 Timing of H1H_1 resolution Using πhist(α=1) _hist(α=1) and birth-persistence heatmaps in Figure 7 and Figure 8, we analyze where H1H_1 features appear and resolve across TabPFN layers. For the entangled families, increasing difficulty shifts persistent H1H_1 activity toward later layers. The πhist _hist peak layer rises with difficulty for the trefoil knot and the Hopf link, while the simpler families show no significant shift. The depth shift is therefore specific to topologies that are genuinely hard to disentangle. At the main benchmark scale, the birth-persistence heatmaps corroborate this at the bar level: relative to easy datasets, hard datasets carry more persistent H1H_1 mass and shift their H0H_0 mass toward short lifetimes. This qualitatively echoes prior LLM zigzag findings in [12], where long-lived H1H_1 features tend to appear in middle layers and late-layer changes reflect output-oriented rearrangement. The interpretation differs, however. In LLMs, topology describes how prompts are reorganized during language processing. In TabPFN, topology describes how an in-context tabular task geometry is constructed and whether that construction is reliable. Timing adds a depth axis to the same story: hard topology changes not only the amount and stability of topological structure, but also where in the network the surviving loop structure resolves. (a) (b) Figure 7: Layerwise timing of H1H_1 resolution. (a) Persistence-weighted birth histogram πhist(α=1) _hist(α=1) for easy, medium, and hard datasets. (b) The πhist _hist peak layer as a function of difficulty; entangled families resolve their loops deeper in the network. Each individual run’s peak falls on one of the 1212 discrete TabPFN layers; the plotted curve is the mean peak layer over the five seeds, so it can take fractional values (e.g., a value of 6.56.5 indicates that the peak lies between layers 66 and 77 when averaged across seeds). (a) (b) (c) (d) Figure 8: Birth–persistence heatmaps for the Hopf link (log count of cycles by birth layer and persistence). (a,b) H1H_1 for easy vs hard; (c,d) H0H_0 for easy vs hard. At the main benchmark scale, hard datasets show more persistent H1H_1 mass and a shift of H0H_0 mass toward short lifetimes. 5.9 Fine-tuning under topological stress We also evaluate whether small-scale fine-tuning can repair hard topology regimes by comparing matched base and fine-tuned pairs [34]. No reliability metric improves significantly: pooled Wilcoxon signed-rank tests give p=0.79p=0.79 for observed error, p=0.56p=0.56 for Bayes error, and p=0.62p=0.62 for MAR. Under extreme stress, the effect is slightly adverse. At the hardest difficulty level, fine-tuning raises the mean signed residual and the overconfidence rate, yielding a model that is more confident but no more correct. Reduced reliability is therefore unlikely to be only a matter of insufficient task-specific adaptation. The limitation appears tied to the pretrained prior, or to the internal representation topology that difficult geometries induce. 5.10 Robustness across embedding scopes Finally, we repeat the topology analysis in two additional embedding scopes: support feature-token representations and support label-token representations. The input-topology signal is stable across these scopes. The correlation between input noise and embedding H1H_1 count is ρ=0.78ρ=0.78, 0.710.71, and 0.720.72 for the three scopes, respectively; for H0H_0 total persistence, the corresponding correlations are ρ=0.83ρ=0.83, 0.820.82, and 0.800.80. The signal is therefore not an artifact of reading out one particular representation surface. Topological stress appears across the in-context task representation. 6 Discussion 6.1 TabPFN topology as in-context task geometry The main conclusion is that TabPFN topology should be interpreted as the topology of an inferred task. TabPFN predicts by conditioning on the entire support-query context, and its internal representations encode that task-level geometry. When input topology is simple and aligned with the model’s learned prior, TabPFN can form stable coarse groupings. When topology becomes difficult, noisy, or poorly aligned with the prior, the internal topology becomes more fragmented and more cyclic. This produces the observed H0/H1H_0/H_1 scissors effect. This view complements recent efforts to understand what TabPFN internalizes about a task, such as whether it recovers causal structure [33] or can be repurposed as a feature extractor [38]; here the object of study is the reliability of the inferred task geometry rather than its predictive content. This interpretation is consistent with recent evidence that tabular in-context generalization depends strongly on the prediction tasks induced during pretraining. The authors show that even a single real table can support broad transfer when it yields a sufficiently rich collection of self-supervised tasks [24]. From this perspective, the geometry formed by TabPFN at inference time reflects an interaction between the supplied context and task-solving strategies encoded during pretraining. Our results suggest that topological fragmentation may arise when the new context induces a task geometry that is difficult to represent using those learned strategies. 6.2 Comparison with other works Our work is methodologically close to the LLM zigzag persistence framework [12], which tracks evolving point clouds of hidden representations across transformer layers. That work identifies phases of prompt processing and uses topological descriptors for layer pruning. In TabPFN, the same mathematical machinery reveals a different phenomenon. The most actionable signal is not layer redundancy, but dataset-level reliability. This difference is natural: TabPFN predictions are conditioned on an entire tabular context, so the topology of the inferred in-context task geometry becomes the relevant object of study. Furthermore, HalluZig shows that topological signatures of evolving attention graphs can distinguish hallucinated from factual LLM generations [32]. Our work differs in both object and setting. HalluZig analyzes attention topology in autoregressive language generation, while we analyze representation topology in tabular in-context inference. The common theme is that zigzag persistence can expose reliability-relevant structure in transformer internals, but the appropriate unit of analysis depends on the model and task. 6.3 Practical implications The results suggest several applications and directions for future explorations. First, topology can serve as a dataset-level reliability diagnostic. Before trusting TabPFN on a new tabular context, one could compute internal topological descriptors and flag contexts that resemble high-risk regimes. In addition, topology may be useful for monitoring population drift. If a deployed dataset gradually shifts toward more fragmented H0H_0 or more active H1H_1, that may indicate increasing reliability risk. Lastly, the results motivate topology-aware pretraining or data augmentation. If TabPFN is less reliable on certain topological regimes, future pretraining distributions could explicitly include more complex or knotted tabular structures. Taken together with the findings of [24], our results raise the possibility that geometric and topological diversity constitutes one dimension of task richness. Increasing sample size may improve estimation within a given regime, but broader structural coverage may be required to improve generalization across regimes. This perspective suggests a topology-aware approach to pretraining-task construction. Rather than increasing the number of synthetic rows while sampling from a fixed or limited family of generators, one could diversify the intrinsic geometry of the generated tasks, including connected, loop-bearing, linked, knotted, and geometrically curved but loop-free regimes. The objective would not be to reproduce particular manifolds literally, but to expand the range of representation structures that the model must organize during pretraining. Testing whether such coverage improves reliability on unseen complex geometries is an important direction for future work. 6.4 Limitations and future work This study is primarily based on controlled synthetic data. This design is useful because the true conditional probabilities are known by construction, allowing us to measure MAR, Bayes error, and overconfidence relative to the data-generating process. Real-world tabular datasets may contain additional structure not captured by our generators, including heterogeneous feature types, missingness patterns, categorical variables, domain-specific constraints, distribution shifts, and non-geometric sources of label noise [16, 28]. The synthetic benchmark therefore provides a controlled stress test of TabPFN’s behavior under known geometric and topological conditions, but it does not replace evaluation on diverse real-world benchmarks and tasks [11, 25]. A second limitation is that our topological summaries are coarse. The descriptors used in this study, such as H0H_0 and H1H_1 counts, total persistence, and birth-persistence summaries, characterize global properties of the representation topology. They are useful for identifying context-level reliability patterns, but they do not directly explain which support rows, query rows, or features are responsible for reduced reliability. In this sense, our analysis provides a diagnostic of the in-context task geometry as a whole rather than a mechanistic attribution method. Attention-based topology, especially query-to-support or feature-wise attention topology, may provide a more local view of how TabPFN routes information through the context [32, 21]. The graph construction also introduces methodological choices. We follow prior zigzag representation work by building k-nearest-neighbor graphs at each layer, but the resulting topology depends on the choice of k, the distance metric, the embedding scope, and the maximum simplex dimension. Although our robustness checks suggest that the main context-level signal is not specific to one embedding scope, different graph constructions may emphasize different aspects of the representation geometry. In particular, k-nearest-neighbor graphs preserve relative neighborhood structure but do not explicitly encode absolute distances. Future work could compare k-nearest-neighbor complexes with distance-thresholded (e.g., Vietoris–Rips) constructions [9, 26]. Finally, TabPFN v2 [20] has only 12 transformer layers, giving fewer depth positions than typical LLMs. This limits the resolution of layerwise phase analysis compared with recent zigzag studies of large language models, which often analyze 30 or more layers. The smaller depth makes broad early/middle/late trends visible, but it limits our ability to identify fine-grained processing phases. Future work could apply the same framework to deeper and larger tabular foundation models [14, 15, 25, 2], compare multiple TabPFN versions, or analyze intermediate sub-blocks within each TabPFN layer, such as feature-wise attention, row-wise attention, and MLP updates, to obtain a finer view of the evolution of the in-context task geometry. 7 Conclusion We introduced a zigzag persistent homology framework for analyzing TabPFN’s internal representations across layers. Using controlled synthetic topologies with known true probabilities, we found that TabPFN’s representation topology predicts dataset-level reliability. Harder input geometries induce a dual topological stress signature: increased H1H_1 loop activity and increased H0H_0 fragmentation, together with shorter-lived connected-component structure. These descriptors correlate with Bayes error, MAR, and overconfidence. We further find that small-scale fine-tuning does not reliably repair these hard-topology regimes, indicating that the reduced reliability is tied to the pretrained prior rather than to insufficient task-specific adaptation. Our results connect recent topological analyses of transformer representations to tabular foundation models and suggest a new direction for reliability diagnostics in TabPFN: topology-aware monitoring of the in-context task geometry. Acknowledgments We would like to thank Travis Ens for his continued involvement throughout this project, from its early development to the revision of this manuscript, and for his valuable feedback and constructive suggestions, which helped improve the organization and presentation of this paper. References [1] S. Ö. Arik and T. Pfister (2021) Tabnet: attentive interpretable tabular learning. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35, p. 6679–6687. Cited by: §2.1. [2] M. Bouadi, P. Seth, A. Tanna, and V. K. Sankarapu (2025) Orion-msp: multi-scale sparse attention for tabular in-context learning. arXiv preprint arXiv:2511.02818. Cited by: §2.1, §6.4. [3] L. Breiman (2001) Random forests. Machine learning 45 (1), p. 5–32. Cited by: §2.1. [4] G. Carlsson and V. de Silva (2010) Zigzag persistence. Foundations of Computational Mathematics 10 (4), p. 367–405. External Links: Document Cited by: §1, §3.2. [5] G. Carlsson (2009) Topology and data. Bulletin of the American Mathematical Society 46 (2), p. 255–308. External Links: Document Cited by: §2.2. [6] T. Chen and C. Guestrin (2016) Xgboost: a scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, p. 785–794. Cited by: §2.1. [7] D. Cohen-Steiner, H. Edelsbrunner, and J. Harer (2007) Stability of persistence diagrams. Discrete & Computational Geometry 37 (1), p. 103–120. External Links: Document Cited by: §5.1. [8] Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, B. Chang, et al. (2024) A survey on in-context learning. In Proceedings of the 2024 conference on empirical methods in natural language processing, p. 1107–1128. Cited by: §2.1. [9] H. Edelsbrunner and J. Harer (2010) Computational topology: an introduction. American Mathematical Society. External Links: ISBN 9780821849255 Cited by: §2.2, §6.4. [10] H. Edelsbrunner, D. Letscher, and A. Zomorodian (2002) Topological persistence and simplification. Discrete & Computational Geometry 28 (4), p. 511–533. External Links: Document Cited by: §2.2. [11] N. Erickson, L. Purucker, A. Tschalzev, D. Holzmüller, P. M. Desai, D. Salinas, and F. Hutter (2025) Tabarena: a living benchmark for machine learning on tabular data. arXiv preprint arXiv:2506.16791. Cited by: §2.1, §6.4. [12] Y. Gardinazzi, K. Viswanathan, G. Panerai, A. Ansuini, A. Cazzaniga, and M. Biagetti (2025) Persistent topological features in large language models. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 18811–18830. External Links: Link Cited by: §1, §2.3, §3.2, §3.2, §3.3, §5.8, §6.2. [13] Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko (2021) Revisiting deep learning models for tabular data. Advances in neural information processing systems 34, p. 18932–18943. Cited by: §2.1. [14] L. Grinsztajn, K. Flöge, O. Key, F. Birkel, P. Jund, B. Roof, B. Jäger, D. Safaric, S. Alessi, A. Hayler, et al. (2025) Tabpfn-2.5: advancing the state of the art in tabular foundation models. arXiv preprint arXiv:2511.08667. Cited by: §A.2, §1, §2.1, §6.4. [15] L. Grinsztajn, K. Flöge, O. Key, F. Birkel, P. Jund, B. Roof, M. Manium, S. B. Hoo, M. Bühler, A. Garg, et al. (2026) TabPFN-3: technical report. arXiv preprint arXiv:2605.13986. Cited by: §A.2, §1, §2.1, §6.4. [16] L. Grinsztajn, E. Oyallon, and G. Varoquaux (2022) Why do tree-based models still outperform deep learning on typical tabular data?. Advances in neural information processing systems 35, p. 507–520. Cited by: §2.1, §6.4. [17] A. Haeri, J. Vitrano, and M. Ghelichi (2025) Generative ai enhanced financial risk management information retrieval. arXiv:2504.06293. External Links: Link Cited by: §2.1. [18] W. Han, G. Tan, L. Li, Z. Feng, C. Zhou, X. Liu, and L. Hu (2026) Topological feature-driven tabpfn model for prediction of enlarged hemorrhage and edema after tumor resection in meningiomas. Frontiers in Medicine 13, p. 1808831. Cited by: §1. [19] N. Hollmann, S. Müller, K. Eggensperger, and F. Hutter (2023) TabPFN: a transformer that solves small tabular classification problems in a second. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2.1. [20] N. Hollmann, S. Müller, L. Purucker, A. Krishnakumar, M. Körfer, S. B. Hoo, R. T. Schirrmeister, and F. Hutter (2025) Accurate predictions on small data with a tabular foundation model. Nature 637 (8045), p. 319–326. External Links: Document, Link Cited by: §1, §2.1, §3.1, §3.2, §6.4. [21] J. Hu and M. Ghelichi (2026) Noise immunity in in-context tabular learning: an empirical robustness analysis of tabpfn’s attention mechanisms. ArXiv:2604.04868. External Links: Link Cited by: §A.2, §1, §2.1, §6.4. [22] M. B. Kursa (2014) Robustness of random forest-based gene selection methods. BMC bioinformatics 15 (1), p. 8. Cited by: §2.1. [23] M. Q. Le and D. Taylor (2022) Persistent homology with k-nearest-neighbor filtrations reveals topological convergence of pagerank. arXiv preprint arXiv:2206.04725. Cited by: §3.2. [24] J. Ma, N. Shaheen, A. Labach, A. Mhedhbi, F. Hutter, A. L. Caterini, and V. Thomas (2025) Generalization can emerge in tabular foundation models from a single table. arXiv preprint arXiv:2511.09665. Cited by: §6.1, §6.3. [25] J. Ma, V. Thomas, R. Hosseinzadeh, A. Labach, H. Kamkari, J. C. Cresswell, K. Golestan, G. Yu, A. L. Caterini, and M. Volkovs (2024) Tabdpt: scaling tabular foundation models on real data. arXiv preprint arXiv:2410.18164. Cited by: §2.1, §6.4, §6.4. [26] C. Maria, J. Boissonnat, M. Glisse, and M. Yvinec (2014) The GUDHI library: simplicial complexes and persistent homology. In Mathematical Software – ICMS 2014, p. 167–174. External Links: Document Cited by: §6.4. [27] G. Naitzat, A. Zhitnikov, and L. Lim (2020) Topology of deep neural networks. Journal of Machine Learning Research 21 (184), p. 1–40. External Links: Link Cited by: §2.2. [28] A. Nawaz, A. Ahmad, and S. S. Khan (2025) Assessing the robustness of tabular prior-data fitted network classifier. In 1st ICML Workshop on Foundation Models for Structured Data, Cited by: §2.1, §6.4. [29] P. Niyogi, S. Smale, and S. Weinberger (2008) Finding the homology of submanifolds with high confidence from random samples. Discrete & Computational Geometry 39 (1–3), p. 419–441. External Links: Document Cited by: §5.1. [30] L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin (2018) CatBoost: unbiased boosting with categorical features. Advances in neural information processing systems 31. Cited by: §2.1. [31] B. Rieck, M. Togninalli, C. Bock, M. Moor, M. Horn, T. Gumbsch, and K. Borgwardt (2019) Neural persistence: a complexity measure for deep neural networks using algebraic topology. In International Conference on Learning Representations, External Links: Link Cited by: §2.2. [32] S. N. Samaga, G. Gonzalez Arroyo, and T. K. Dey (2026) HalluZig: hallucination detection using zigzag persistence. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, p. 3466–3482. External Links: Document, Link Cited by: §1, §2.4, §6.2, §6.4. [33] O. Swelam, L. Purucker, J. Robertson, H. Raum, J. Boedecker, and F. Hutter (2025) Does tabpfn understand causal structures?. arXiv preprint arXiv:2511.07236. Cited by: §2.1, §6.1. [34] A. Tanna, P. Seth, M. Bouadi, and V. K. Sankarapu (2026) Exploring fine-tuning for tabular foundation models. arXiv preprint arXiv:2601.09654. Cited by: §2.1, §5.9. [35] B. Van Breugel and M. Van Der Schaar (2024) Why tabular foundation models should be a research priority. arXiv preprint arXiv:2405.01147. Cited by: §2.1. [36] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: §2.1. [37] J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov (2023) Transformers learn in-context by gradient descent. In International Conference on Machine Learning, p. 35151–35174. Cited by: §2.1. [38] H. Ye, S. Liu, and W. Chao (2025) A closer look at TabPFN v2: understanding its strengths and extending its capabilities. arXiv:2502.17361. External Links: Link Cited by: §A.2, §1, §2.1, §6.1. [39] A. Zomorodian and G. Carlsson (2005) Computing persistent homology. Discrete & Computational Geometry 33 (2), p. 249–274. External Links: Document Cited by: §2.2. This appendix provides additional details about the experimental setups used in the main paper. Appendix A Experimental Details A.1 Synthetic Data Generation All six topology families are produced by a common pipeline; they differ only in the base manifold, its embedding map, and the label score. For a fixed family, difficulty level, and random seed we (i) sample points on a base manifold, (i) add isotropic off-manifold Gaussian noise, (i) append pure-noise nuisance dimensions, and (iv) assign a smooth ground-truth label probability from the intrinsic manifold coordinates. Because the label is a known function of those coordinates, the true conditional probability P∗(y∣x)P^*(y x) is available for every point, which is what makes the true-probability reliability metrics of the main text well defined. Let z denote the intrinsic coordinate(s) of a sampled point, Φ the family embedding into ℝ2R^2 or ℝ3R^3, and ς(u)=1/(1+e−u) (u)=1/(1+e^-u) the logistic function. Every dataset is constructed as x0 x_0 =WΦ(z)+ε, =W\, (z)+ , ε ∼(0,σ2I), (0,σ^2I), (8) x x =[x0;ξ], = [\,x_0\,;\ ξ\, ], ξ ξ ∼(0,Im), (0,I_m), (9) η(z) η(z) =ς(κg(z)), = \! (κ\,g(z) ), y y ∼Bernoulli(η(z)), (η(z) ), (10) where σ is the off-manifold noise level, m the number of appended nuisance dimensions, W an anisotropic warp that scales the first ambient axis by a factor w, g(z)g(z) a family-specific score, and κ the label sharpness. We fix κ=4.0κ=4.0 in every run so that the intrinsic label (Bayes) noise is held constant and geometric difficulty is decoupled from label difficulty. For each point we store the true class probabilities [ 1−η,η][\,1-η,\ η\,], the Bayes-optimal label [η≥12]1[η≥ 12], and the Bayes confidence max(η,1−η) (η,1-η). Unless otherwise noted, each dataset uses ntrain=400n_train=400 context rows and ntest=300n_test=300 query rows, and every (family, level) pair is generated with five random seeds. The large-sample case study of the main text reuses the warped circle generator with ntrain=6667n_train=6667 and ntest=5000n_test=5000 (and a half-scale 3333/25003333/2500 variant), again with five seeds. Family embeddings and label scores. Writing t,θ,ϕ∼(0,2π)t,θ,φ (0,2π), u∼(−1,1)u (-1,1), and s∼(0,1)s (0,1) for the intrinsic coordinates, the six families are: • Warped circle (S1⊂ℝ2S^1 ^2): radius r=1+τsin(3θ)r=1+τ (3θ) and Φ(θ)=(rcosθ,rsinθ) (θ)= (r θ,\ r θ ); score g=sinθg= θ. The twist τ deforms the radius at frequency 33 while preserving the homotopy type (S1S^1). • Torus (T2⊂ℝ3T^2 ^3): with R=3R=3 and r=1r=1, Φ(θ,ϕ)=((R+rcosθ)cosϕ,(R+rcosθ)sinϕ,rsinθ) (θ,φ)= ((R+r θ) φ,\ (R+r θ) φ,\ r θ ); score g=sinϕg= φ. • Sphere (S2⊂ℝ3S^2 ^3): with ρ=1−u2ρ= 1-u^2, Φ(u,v)=(ρcosv,ρsinv,u) (u,v)= (ρ v,\ ρ v,\ u ) sampled uniformly on the sphere; score g=ug=u, a smooth hemisphere split by latitude. • Hopf link (two linked S1⊂ℝ3S^1 ^3): half of the points lie on ring A=(cost,sint, 0)A=( t,\ t,\ 0) and half on ring B=(δ,cost,sint)B=(δ,\ t,\ t), where δ is the inter-ring separation; the score g=∥x−cB∥−∥x−cA∥g= x-c_B - x-c_A contrasts distances to the ring centers cA=(0,0,0)c_A=(0,0,0) and cB=(δ,0,0)c_B=(δ,0,0), so the label encodes ring membership. • Trefoil knot (knotted S1⊂ℝ3S^1 ^3): the base knot is γ(t)=(sint+2sin2t,cost−2cos2t,−sin3t)γ(t)= ( t+2 2t,\ t-2 2t,\ - 3t ); class 0 is γ(t)γ(t) and class 11 is the scaled, shifted copy sfγ(t)+0.2s_f\,γ(t)+0.2. The score g=∥x1:3∥−mg= x_1:3 -m thresholds the ambient radius at the per-dataset median m of the training radii, which keeps the two classes balanced at every difficulty level; a fixed threshold instead skews toward one class as the copies overlap at high difficulty. • Swiss roll (ℝ2R^2 sheet ⊂ℝ3 ^3, negative control): with angle a=2πnturnss+πa=2π\,n_turns\,s+π and independent height h∼(0,10)h (0,10), Φ(s,h)=(acosa,h,asina) (s,h)= (a a,\ h,\ a a ); score g=s−12g=s- 12. The sheet has no intrinsic loop, so any H1H_1 signal is noise-induced. In every family, the first ambient axis of the embedding is multiplied by the warp factor w before noise is added. Difficulty schedule. Each family is escalated across nine ordered levels: levels 0–55 form the main suite with 6×6×5=1806× 6× 5=180 runs and levels 66–88 the extreme suite with 6×3×5=906× 3× 5=90 runs. All families share the noise, nuisance, and warp ladder of Table 6, with two minor exceptions: the sphere uses σ=0.30σ=0.30 at level 44, and the torus uses σ=0.30σ=0.30 at level 44 and σ=0.50σ=0.50 at level 55. Beyond this shared ladder, each structured family also tightens one geometry-specific knob that drives its components together or increases its curvature; the torus and sphere have no secondary knob and are stressed by noise, nuisance, and warp alone. Table 6: Shared difficulty ladder used by all six families. Levels 0–55 are the main suite; levels 66–88 are the extreme suite. The sphere uses σ=0.30σ=0.30 at level 44, and the torus uses σ=0.30σ=0.30 at level 44 and σ=0.50σ=0.50 at level 55. Level 0 1 2 3 4 5 6 7 8 Noise σ 0.02 0.05 0.10 0.18 0.28 0.45 0.70 1.00 1.50 Nuisance dims m 0 0 4 10 20 30 50 100 200 Warp w 1.0 1.0 1.2 1.5 1.8 2.0 2.5 3.0 3.5 Table 7: Family-specific secondary difficulty knob by level. Smaller ring separation and smaller trefoil scale factor push the two components together, while more Swiss-roll turns increase curvature. The torus and sphere have no secondary knob. Level 0 1 2 3 4 5 6 7 8 Warped circle: twist τ 0.00 0.10 0.20 0.30 0.45 0.60 0.80 1.00 1.20 Hopf link: separation δ 1.5 1.2 1.0 0.8 0.5 0.3 0.2 0.1 0.05 Trefoil: scale factor sfs_f 2.0 1.8 1.5 1.3 1.15 1.05 1.02 1.01 1.005 Swiss roll: turns nturnsn_turns 1.0 1.5 2.0 2.5 3.5 5.0 7.0 10.0 15.0 A.2 Model configuration. Newer TabPFN releases include v2.5 and v3 models [14, 15]. In this study we use a TabPFN v2 checkpoint with feature group size =1=1, similar to the setup in [38, 21] to remove artifacts from estimator ensembling. To reduce randomness from inference-time ensembling, we use one estimator and disable feature shuffling when fitting the model. A.3 Compute resources. All computational tests reported in this paper were conducted using a single NVIDIA Tesla T4 GPU.