Paper deep dive
Stemma: Induced Decision Regions Reveal LLM Provenance
Keyu Zhang, Vadim Safronov, Andrew Martin
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:LLM provenance testing asks whether a suspect LLM belongs to the same lineage as a source. Existing black-box methods largely infer this relationship from response-level characteristics, but these characteristics may shift under adaptation or deployment even when the underlying meaning remains unchanged, weakening the reliability of provenance evidence. To address this limitation, we introduce induced decision regions by mapping open-ended outputs into a finite decision space, thereby abstracting away surface-form variation and reframing provenance testing as measuring the inheritance of decision regions. Empirical analysis shows that the source's induced regions are preserved more strongly in related models than in unrelated models. Building on this signal, we propose Stemma, a practical black-box LLM fingerprinting method that operationalises stability, robustness, and specificity as complementary probe-selection principles for reliably estimating induced decision region inheritance. Across 770 source-suspect pairs drawn from 56 public checkpoints and spanning diverse model-weight transformations, Stemma achieves 0.967 AUC and 87.8% TPR at 1% FPR, substantially outperforming four representative baselines. It further achieves 0.995 AUC and 93.5% TPR at 1% FPR on 1,260 pairs covering 91 deployment instances, demonstrating robustness to diverse inference-time deployment settings.
Tags
Links
- Source: https://arxiv.org/abs/2607.25880v1
- Canonical: https://arxiv.org/abs/2607.25880v1
Trouble viewing inline? Open PDF directly →
Full Text
92,714 characters extracted from source content.
Expand or collapse full text
Stemma: Induced Decision Regions Reveal LLM Provenance Keyu Zhang Vadim Safronov Andrew Martin Department of Computer Science, University of Oxford keyu.zhang,vadim.safronov,andrew.martin@cs.ox.ac.uk Abstract LLM provenance testing asks whether a suspect LLM belongs to the same lineage as a source. Existing black-box methods largely infer this relationship from response-level characteristics, but these characteristics may shift under adaptation or deployment even when the underlying meaning remains unchanged, weakening the reliability of provenance evidence. To address this limitation, we introduce induced decision regions by mapping open-ended outputs into a finite decision space, thereby abstracting away surface-form variation and reframing provenance testing as measuring the inheritance of decision regions. Empirical analysis shows that the source’s induced regions are preserved more strongly in related models than in unrelated models. Building on this signal, we propose Stemma, a practical black-box LLM fingerprinting method that operationalises stability, robustness, and specificity as complementary probe-selection principles for reliably estimating induced decision region inheritance. Across 770 source–suspect pairs drawn from 56 public checkpoints and spanning diverse model-weight transformations, Stemma achieves 0.967 AUC and 87.8% TPR at 1% FPR, substantially outperforming four representative baselines. It further achieves 0.995 AUC and 93.5% TPR at 1% FPR on 1,260 pairs covering 91 deployment instances, demonstrating robustness to diverse inference-time deployment settings. 1 Introduction Large language models (LLMs) have become widely deployed infrastructure for text generation, code assistance, and reasoning. Across their development and deployment lifecycles, models may undergo fine-tuning, merging, compression, or distillation, while prompt templates and decoding settings further shape their observable behaviour (Dettmers et al., 2023; Yu et al., 2024; Shi et al., 2024). This complexity obscures model provenance, raising intellectual-property and licensing concerns when protected models are reused without authorisation (Stalnaker et al., 2025), as well as accountability concerns when unsafe behaviours, biases, or vulnerabilities propagate to downstream models (Hammoud et al., 2024; Zhang et al., 2024). These concerns motivate model provenance testing, which assesses whether a suspect model belongs to the same lineage as a source. Black-box model fingerprinting provides a practical approach to provenance testing. It characterises a source model through observable behavioural signatures that are preserved across provenance-related models while remaining distinguishable from independently developed models (Jiang et al., 2026). Because verification requires only query access to the suspect model, it remains applicable when model weights, internal activations, training data, and development history are unavailable. The central challenge is to identify signatures that are both persistent under adaptation and specific to the source. In classification models, the fixed label space naturally partitions inputs into decision regions. Existing methods thus exploit this structure by selecting boundary-adjacent, adversarial, or error-region inputs and measuring whether source and suspect models assign them to the same regions (Cao et al., 2021; Godinot et al., 2025; Guan et al., 2022). Derived models often inherit these fine-grained structures, whereas independently trained models are less likely to reproduce them. This paradigm does not transfer directly to LLMs. Open-ended generation produces an unstructured output space in which the same meaning may take many surface forms, making decision regions difficult to define and compare from black-box outputs. Existing black-box LLM fingerprinting methods instead rely largely on output similarity, elicited behaviours, or perturbation responses (Nikolic et al., 2025; Gubri et al., 2024; Shao et al., 2026). Because these signals depend on particular response realisations, they may shift under prompting, deployment, or model adaptation, weakening the reliability of provenance testing. We address this limitation by introducing induced decision regions for LLMs. By mapping outputs into a finite decision space, we abstract away surface-form variation, make these regions observable, and reframe provenance testing as measuring the inheritance of decision regions. Our analysis shows that related models preserve the source’s induced regions more strongly than unrelated models. Building on this formulation, we propose Stemma111The name Stemma is inspired by stemmatic analysis, which infers textual lineage from inherited variants. Appendix K develops this analogy. as a black-box LLM fingerprinting method for provenance testing. Stemma detects induced decision region inheritance through probes selected for stability across equivalent input representations, robustness to boundary shifts, and specificity against unrelated models. Across various model families, adaptation types, and deployment settings, Stemma achieves stronger and more consistent provenance separation than existing methods. Contributions. Our contributions are summarised as follows. • We introduce induced decision regions for LLMs, reframing black-box provenance testing from comparing response-level characteristics to measuring the inheritance of decision regions, and show that related models preserve these regions more strongly than unrelated models. • We propose and open-source Stemma, a black-box LLM fingerprinting method that tests provenance through induced decision region inheritance, with fingerprint construction guided by stability, robustness, and specificity. • We conduct comprehensive evaluations across 56 public checkpoints and 91 deployment instances, spanning diverse model families, adaptation types, and deployment settings. Stemma substantially outperforms four representative black-box baselines across multiple metrics, demonstrating consistently stronger provenance separation. 2 Background and Problem Setting 2.1 Related Work Methods for model ownership and provenance testing fall into watermarking and fingerprinting (Ye et al., 2026). Watermarking embeds identifiable signals into model parameters or behaviours, requiring prior control of the protected model and restricting verification to models marked in advance (Uchida et al., 2017; Adi et al., 2018; Li et al., 2023; Gloaguen et al., 2025). Fingerprinting is non-invasive, instead exploiting distinctive characteristics that arise naturally during model development. Depending on access to the suspect model, fingerprinting methods can be classified as white-box or black-box: white-box methods use parameters, activations, or internal representations and therefore require internal access (Jia et al., 2021; Zhang et al., 2025; Yu and Wang, 2024), whereas black-box methods rely only on behaviours elicited through controlled queries, offering broader applicability but making reliable provenance testing more challenging. Existing black-box LLM fingerprinting methods can be broadly grouped by how they obtain and characterise observable behaviour. The first group directly characterises responses to selected queries. Model Provenance Testing measures next-token agreement between the source and suspect relative to independently trained control models, while DuFFin compares response similarity over sampled questions (Nikolic et al., 2025; Yan et al., 2026). LLMmap learns representations from selected queries and responses for model identity recognition, whereas ErrorTrace derives distinctive error signatures for model family attribution (Pasquini et al., 2025; Zang et al., 2025). Recent API-auditing methods further exploit output-rank distributions or response stability near estimated knowledge boundaries (Zhu et al., 2026; Fang et al., 2026). The second group constructs adversarially optimised prompts to elicit target behaviours. TRAP uses such prompts for model identification, while ProFLingo and RoFL test whether responses persist in downstream models (Gubri et al., 2024; Jin et al., 2024; Tsai et al., 2025). LLMPrint induces and tests inheritance of preferences between target-token pairs (Hu et al., 2026). A third line, represented by ZeroPrint, characterises response variation under semantic perturbations through zeroth-order estimates of local Jacobians (Shao et al., 2026). Across these approaches, fingerprints are derived from response-level characteristics, including response patterns, elicited behaviours, and perturbation effects. These characteristics may shift with prompting, deployment, or weight adaptation without commensurate semantic changes, limiting the stability and comparability of provenance evidence. Prior work on deep classifiers shows that decision regions provide a structured and directly comparable basis for black-box provenance testing. IPGuard constructs probes near the source’s decision boundaries and tests whether the suspect model preserves the corresponding region assignments (Cao et al., 2021). Model Lineage Closeness Analysis measures preservation of decision geometry through differences in decision boundary distances and prediction agreement (Tang et al., 2025). ADV-TRA extends boundary-sensitive probes to adversarial trajectories that traverse decision boundaries, improving tolerance to boundary shifts (Xu et al., 2024). Open-ended LLM generation, however, does not naturally expose a finite decision space over which analogous regions can be defined and compared. Stemma therefore maps open-ended outputs into such a space, abstracting away surface-form variation and testing provenance through the inheritance of the decision regions. 2.2 Problem Setting We study pairwise black-box model provenance testing for LLMs. Given a source model S and a suspect model instance T, the objective is to determine whether T is provenance-related to S. A model instance comprises an underlying model and the deployment configuration through which its behaviour is exposed. The pair is related if the underlying models of S and T are identical or linked through model derivation, possibly under different deployment configurations. Otherwise, the pair is unrelated, even if the models share similarities in architecture, training, or performance. Access assumptions. We assume access to the source model’s next-token logits for fingerprint construction, whereas the suspect model is available only through a black-box interface that accepts textual inputs and returns generated responses without exposing its parameters, logits, activations, training data, or development history. This asymmetric setting captures a practical provenance-auditing scenario in which the evaluator controls the source model and seeks to identify related model variants or deployed instances through exposed interfaces. Adversarial scope. We consider a non-adaptive setting in which the provider may know the general fingerprinting method but not the test-specific probes. The suspect model may undergo ordinary model development and deployment transformations, but neither the model nor its interface is adapted to evade the method by detecting, filtering, or altering responses to fingerprint queries. 3 Induced Decision Regions for LLM Provenance 3.1 Formalisation Formally, let Q denote the query space, O the observable model-output space, and =1,…,CD=\1,…,C\ a finite decision space. For an LLM instance M, let oM(q)∈o_M(q) denote its output on query q∈q . Given a score extraction rule g~:×→ℝC g:Q×O ^C and a decision extraction rule g:×→g:Q×O , we define the model-specific soft and discrete decision maps as h~M(q)=g~(q,oM(q))∈ℝC,hM(q)=g(q,oM(q))∈. h_M(q)= g (q,o_M(q) ) ^C, h_M(q)=g (q,o_M(q) ) . (1) The soft map h~M h_M assigns each query a real-valued score vector representing the model’s relative preferences over the available decisions, whereas the discrete map hMh_M assigns it the selected decision. Depending on the available access, the value hM(q)h_M(q) is obtained either by taking argmaxd∈h~M,d(q) _d h_M,d(q), with ties resolved by a fixed deterministic rule, or by extracting it directly from generated text. In both cases, the resulting decision lies in the same finite space D. The discrete decision map hMh_M induces a partition of the query space by grouping queries assigned to the same decision. For each decision d∈d , we define the induced decision region of M as ℛdM=q∈∣hM(q)=d.R^M_d=\q h_M(q)=d\. (2) Conceptually, we refer to the arrangement of these regions and their separating boundaries as the induced decision geometry of M. For provenance testing, we use induced decision region inheritance to denote the extent to which induced decision geometry is preserved between a source model S and a suspect model T. Since this geometry cannot be compared exhaustively over Q, we assess its preservation using a finite probe set ⊂P . When soft decision vectors are available for both models, inheritance can be assessed at a finer resolution through their similarity over P: Sim^(S,T)=1||∑q∈sim(h~S(q),h~T(q)), Sim_P(S,T)= 1|P| _q sim ( h_S(q), h_T(q) ), (3) where sim(⋅,⋅)sim(·,·) denotes a similarity measure between soft decision vectors. When comparison is restricted to discrete decisions, inheritance is assessed through alignment with the source region assignments over P: ^(S,T)=1||∑q∈hT(q)=hS(q). A_P(S,T)= 1|P| _q 1\h_T(q)=h_S(q)\. (4) Higher values of Sim Sim_P and A_P indicate stronger estimated inheritance and hence stronger evidence of a provenance relationship. 3.2 Empirical Evidence We next examine whether induced decision region inheritance is observable in LLM behaviour. In general, any evaluation interface that maps open-ended outputs to a finite decision space can induce such regions, including constrained response formats, semantic output categories, and verifier-defined behavioural tests. In this paper, we instantiate this idea using multiple-choice questions, where the answer options define the decision space D, as formalised in Appendix A. To construct the visualisation in Figure 1, we use MMLU questions (Hendrycks et al., 2020) and cyclically permute each question’s answer options, treating each permutation as a separate query. For each query, we compute the next-token choice-label logits, normalise them over the four choices, and map the probabilities back to the canonical options. Averaging these probabilities across permutations yields a four-dimensional soft decision vector for each model–question pair while mitigating order and label biases (Zheng et al., 2024a; Pezeshkpour and Hruschka, 2024). We then order the vector dimensions by the source’s averaged preference ranking, such that RiR_i corresponds to its i-th ranked option, and apply the same ordering to the suspect vectors. Finally, each vector is projected into two dimensions. Details of the projection and annotation metrics are provided in Appendix B. Figure 1: Induced decision region inheritance on MMLU using 40 randomly sampled questions (left) and 40 Stemma-selected questions (right). The source is Qwen-2.5-7B, while the related and unrelated suspects are Qwen-2.5-7B-Instruct and Qwen3-1.7B, respectively. Each point represents a model’s projected decision for one question: grey points denote source decisions, while green and orange points denote suspect decisions that preserve or change the corresponding source-region assignment, respectively. Lines connect the source and suspect decisions corresponding to the same question. Annotations report average option-preference similarity (SimSim) and the fraction of matching region assignments (A). For the randomly sampled probes, the related suspect exhibits higher soft decision similarity than the unrelated suspect, 0.75 versus 0.56, and preserves more source region assignments, 32/40 versus 21/40. Thus, random probes already reveal inheritance in both soft decision preferences and discrete region assignments. Because induced decision geometry cannot be exhaustively observed, any finite probe set captures only a partial view of region inheritance, with its discriminative strength depending on which parts of the geometry are sampled. Stemma-selected probes yield substantially clearer separation, with soft decision similarity of 0.80 versus 0.33 and source-region alignment of 39/40 versus 10/40. Stemma therefore does not create the underlying provenance signal, but makes induced decision region inheritance more observable and discriminative. 4 Stemma Design 4.1 Overview Stemma is a black-box LLM fingerprinting method that tests provenance through induced decision region inheritance. It instantiates induced decision regions using multiple-choice questions, whose candidate options define a finite decision space. The method comprises three stages, as illustrated in Figure 2. Decision interface calibration selects and fixes a prompt template for each model to enable reliable decision extraction. Fingerprint construction selects informative probes by evaluating the source model’s induced decisions for stability, robustness, and specificity. Fingerprint verification queries the suspect model with these probes and estimates induced decision region inheritance by measuring how strongly the suspect preserves the source decisions recorded in the fingerprint. Figure 2: Overview of the Stemma workflow. 4.2 Decision Interface Calibration Different multiple-choice prompt formats may elicit substantially different output behaviours, including unparseable responses and strong label bias, making induced decisions difficult to extract reliably. Stemma therefore calibrates each model’s decision interface on held-out questions by excluding severely label-biased templates from a predefined set and selecting the template with the highest valid-choice extraction rate among the remainder. Because these templates preserve the same underlying questions and candidate options, calibration varies their interface rendering, while extracted decisions remain comparable in the same canonical decision space. Calibration is performed once before fingerprint construction and verification, after which the selected template is fixed for all subsequent queries to that model. The candidate prompt templates and calibration configuration are provided in Appendix C. 4.3 Fingerprint Construction Stemma constructs a finite source fingerprint by selecting question probes whose induced decisions are stable, robust, and specific. Stability. Stability requires a probe to remain in the same induced decision region across semantically equivalent input representations, reducing sensitivity to option order and label preferences. Accordingly, Stemma evaluates each candidate under cyclic permutations of its answer options, such that every semantic option appears once under each displayed choice label. After mapping the resulting decisions to the canonical options, stability is measured by their consistency rate across permutations, with a higher rate indicating greater stability. Robustness. Robustness requires a probe to lie well within its assigned induced decision region, so that its region assignment remains stable under small boundary shifts. A probe may be consistent across option permutations yet remain close to a competing boundary, making its region assignment vulnerable to downstream adaptation or deployment changes. Stemma measures robustness by computing, for each option permutation, the log-probability margin between the source-assigned option and the highest-scoring alternative, and then averaging these margins across permutations. Larger average margins indicate greater robustness. Specificity. Specificity requires a probe’s source decision to be uncommon among unrelated models, so that its preservation provides discriminative evidence of induced decision region inheritance. Because gold-answer decisions are often shared across related and unrelated models, Stemma first filters out questions whose source decisions match their gold answers. However, non-gold decisions may still be common because of question ambiguity, annotation errors, or attractive distractors. Stemma therefore measures how often an unrelated background model reproduces the source decision, with lower reproduction rates indicating greater specificity. Probe Selection and Fingerprint Formation. For each candidate q, Stemma obtains source-model next-token choice-label logits under cyclic option permutations, maps the resulting decisions to the canonical options, and defines hS(q)h_S(q) as the most frequently selected canonical decision. The three selection principles are operationalised using the permutation consistency rate s(q)s(q) for stability, the average log-probability margin r(q)r(q) for robustness, and the background decision alignment rate b(q)b(q) together with the non-gold requirement hS(q)≠d∗(q)h_S(q)≠ d^*(q) for specificity. Here, d∗(q)d^*(q) denotes the gold answer. Stemma first applies the eligibility filters to the candidate pool, yielding Selig=q∈cand|s(q)>τs,r(q)≥τr,b(q)<τb,hS(q)≠d∗(q),Q_S^elig= \q _cand\; |\;s(q)> _s,\;r(q)≥ _r,\;b(q)< _b,\;h_S(q)≠ d^*(q) \, (5) where τs _s and τr _r are the minimum stability and robustness thresholds, respectively, and τb _b is the maximum background-alignment threshold. It then ranks the eligible candidates and forms the fingerprint from the top K probes: ℱS=(q,hS(q))|q∈TopKq′∈Selig[wrr~(q′)−wbb~(q′)],F_S= \ (q,h_S(q) )\; |\;q _q _S^elig [w_r r(q )-w_b b(q ) ] \, (6) where r~ r and b~ b denote the transformed robustness and background alignment scores, respectively, and wrw_r and wbw_b control their contributions. Exact score transformations, weights, and thresholds are provided in Appendix C. 4.4 Fingerprint Verification Given a source fingerprint ℱSF_S and a suspect model T, Stemma queries each selected probe through T’s calibrated prompt template using the same cyclic option permutations as in fingerprint construction. A rule-based extractor maps each generated response to either a displayed option or an invalid outcome ⊥ . Valid extracted options are mapped back to the canonical decision space and compared with the stored source decision, while invalid outcomes are treated as non-aligned. These permutation-level comparisons are aggregated using the fingerprint alignment score: A(S,T)=1KC∑(q,hS(q))∈ℱS∑π∈Π(q)[d^T(q,π)=hS(q)].A(S,T)= 1KC _ (q,h_S(q) ) _S _π∈ (q)1 [ d_T(q,π)=h_S(q) ]. (7) Here, Π(q) (q) denotes the C cyclic permutations of probe q, and d^T(q,π)∈∪⊥ d_T(q,π) ∪\ \ denotes the suspect outcome after mapping each valid extracted option back to the canonical decision space. Thus, A(S,T)∈[0,1]A(S,T)∈[0,1] is the fraction of the KCKC probe–permutation observations that preserve the source region assignment, with invalid outcomes contributing zero. Higher values indicate stronger induced decision region inheritance and hence stronger evidence of a provenance relationship. 5 Experiments 5.1 Experimental Setup Benchmark and evaluation protocol. We curate 7 provenance groups with 14 source models from publicly available Hugging Face checkpoints (Hugging Face, 2026), considering model provenance, documentation quality, download popularity, and diversity across model families, scales, and downstream transformations. Each group contains one pretrained source, one instruction-tuned source, and 6 additional variants covering task- or domain-specific fine-tuning, parameter-efficient adaptation, model merging, quantisation, and distillation. The benchmark spans Qwen, Llama, Mistral, Falcon, and OLMo, with model sizes ranging from 1.7B to 14B parameters. Three Qwen-based groups provide hard negatives with similar architectures and training recipes but distinct provenance. Each source is evaluated against every non-self checkpoint. A pair is labelled positive when the source and suspect belong to the same curated provenance group and negative otherwise. This yields 770 ordered source–suspect pairs, comprising 98 positive and 672 negative pairs. Each checkpoint is queried using the interface recommended by its model card or tokenizer configuration, with completion-style checkpoints receiving raw prompts and chat-oriented checkpoints using their tokenizer chat templates. By default, responses are generated using stochastic decoding with temperature 0.70.7, top-p 0.90.9, and top-k 5050. The complete benchmark is provided in Appendix D. Experimental setup and baselines. Unless otherwise stated, Stemma constructs a fingerprint of 40 probes for each source model from a candidate pool of 3,000 questions randomly sampled from MMLU. Phi-3.5-mini-instruct is used as the default background model, which has no known provenance relationship with any benchmark model. We compare Stemma with representative black-box LLM fingerprinting baselines, including LLMmap, LLMPrint, Model Provenance Testing (MPT), and ZeroPrint (Pasquini et al., 2025; Hu et al., 2026; Nikolic et al., 2025; Shao et al., 2026). All methods are evaluated under the benchmark and evaluation protocol described above. Experiments are conducted on NVIDIA L40S GPUs with 48 GB of memory. Detailed Stemma and baseline configurations are provided in Appendices C and F, respectively. Evaluation metrics. We evaluate all methods using their continuous provenance scores and report AUC to assess overall ranking performance. Since false-positive provenance claims can have substantial legal and reputational consequences, we additionally report standardised pAUC over the FPR range [0,0.05][0,0.05] and TPR at 1% FPR to assess performance under stringent low-FPR constraints. We also report the discriminability index d′d to quantify the separation between positive and negative score distributions. For baseline comparisons, the best and second-best results are shown in bold and underlined, respectively. Exact implementation details for these metrics are provided in Appendix E. 5.2 Effectiveness in Provenance Testing Table 1: Overall comparison between Stemma and baselines on the main provenance benchmark. Method Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d LLMmap 0.480 0.487 0.000 -0.298 0.851 0.712 0.388 1.471 0.665 0.568 0.102 0.628 LLMPrint 0.428 0.498 0.020 -0.183 0.615 0.580 0.143 0.265 0.516 0.541 0.071 0.054 MPT 0.461 0.498 0.020 -0.276 0.722 0.692 0.327 1.045 0.556 0.613 0.194 0.500 ZeroPrint 0.535 0.496 0.020 0.003 0.775 0.755 0.469 1.199 0.617 0.610 0.194 0.603 Stemma 0.964 0.959 0.918 3.045 0.970 0.938 0.857 2.945 0.967 0.944 0.878 2.951 Table 1 shows that Stemma consistently outperforms all baselines for pretrained and instruction-tuned sources, as well as overall, with gains in overall discrimination and low-FPR performance. The gap is pronounced for pretrained sources, where source fingerprints use raw completion prompts while many deployed suspects are queried through chat templates. Under this interface mismatch, baselines approach random performance, suggesting that their provenance signals are sensitive to prompt-induced changes in response realisation. Baselines improve for instruction-tuned sources, where both fingerprint construction and verification use chat templates, but do not close the gap to Stemma despite this prompt-format alignment. This suggests that induced decision region inheritance offers both reduced sensitivity to surface-form variation and stronger separation between related and unrelated model pairs. To isolate interface mismatch, we further evaluate an all-raw setting in which both fingerprint construction and verification use raw prompts. Stemma remains competitive in overall ranking, achieving 0.965 AUC compared with 0.988 for MPT, while attaining the strongest overall low-FPR performance with 0.952 pAUC and 0.898 TPR at 1% FPR, versus 0.938 and 0.816, respectively, for the strongest baseline. Thus, even under conditions favourable to the baselines, Stemma is more reliable under strict false-positive constraints. Detailed results are provided in Appendix G. 5.3 Robustness under Deployment Variations Table 2: Robustness under inference-time deployment variants. Method Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d LLMmap 0.481 0.487 0.000 -0.190 0.901 0.754 0.452 1.760 0.689 0.599 0.113 0.793 LLMPrint 0.499 0.489 0.000 0.050 0.615 0.522 0.000 0.413 0.551 0.516 0.000 0.196 MPT 0.477 0.487 0.000 -0.252 0.780 0.727 0.452 1.225 0.570 0.621 0.226 0.563 ZeroPrint 0.623 0.496 0.000 0.210 0.868 0.813 0.607 1.697 0.670 0.642 0.208 0.789 Stemma 0.995 0.965 0.929 3.414 0.996 0.980 0.964 4.158 0.995 0.974 0.935 3.603 To evaluate robustness to inference-time deployment changes, we construct 11 alternative variants for each of the 7 instruction-tuned checkpoints, spanning system and role-play prompts, chain-of-thought prompting, retrieval augmentation, and decoding settings. Together with the 14 source checkpoints, this yields 91 model instances and 1,260 source–suspect pairs, comprising 168 positive and 1,092 negative pairs. Full details are provided in Appendix H. Table 2 shows that Stemma maintains near-perfect AUC and substantially stronger low-FPR performance across these variations. Existing methods perform particularly poorly for pretrained sources and improve for instruction-tuned sources, but still fall consistently below Stemma. These results demonstrate Stemma’s robustness under diverse black-box deployment configurations, consistent with reduced sensitivity to deployment-induced surface-form variation. 5.4 Ablation Study Figure 3: Ablation study of probe selection, pool size and probe budget, probe dataset, and background model. The ⋄ symbol marks the default configuration. The probe selection panel of Figure 3 isolates successive selection stages under the same query budget. The Random probes setting retains the same cyclic-permutation evaluation and aggregation but samples uniformly from the full candidate pool, corresponding to the random setting in Section 3.2. Its performance confirms that induced decision region inheritance is observable without targeted selection, although random probes provide limited discrimination. Restricting candidates to the non-gold pool improves specificity, while subsequent filtering removes probes that fail the stability or robustness requirements. Finally, ranking the retained candidates prioritises the most informative probes and yields the strongest performance. Together, these incremental gains support the proposed selection principles, while the larger gains in pAUC indicate that targeted selection is particularly valuable for distinguishing difficult-to-separate model pairs. The pool size and probe budget panel examines how performance changes with the number of candidate questions and selected probes. A pool of 1,000 questions yields noticeably weaker performance. Results generally improve at 3,000–4,000 questions compared with 2,000, while increasing to 5,000–6,000 provides no consistent further benefit, indicating that larger pools are not necessarily better. For the probe budget, performance largely plateaus once k reaches 40, and most larger-budget settings cluster around a pAUC of 0.940.94, indicating limited sensitivity to the exact budget. The corresponding AUC results follow the same overall pattern, but with less pronounced differences. Thus, Stemma performs consistently across a broad range of pool sizes and probe budgets. Differences across probe datasets are more pronounced in pAUC than in AUC, with MMLU-Pro and MMLU achieving stronger low-FPR performance than the two commonsense-oriented datasets. The background-model panel shows that performance is relatively insensitive to background choice. Although combining all three backgrounds yields modest improvements, a single unrelated background model remains competitive. Overall, the results suggest that knowledge-oriented questions with clearer answer boundaries may provide more specific provenance evidence, while a single unrelated background model appears practically adequate. Detailed results are provided in Appendix I. 5.5 Computational Cost Figure 4: Median runtime cost of fingerprint construction (blue) and verification (orange). Figure 4 compares the wall-clock costs under the same GPU setting, with measurement details provided in Appendix J. Stemma requires approximately 4.5 minutes for one-time fingerprint construction, remaining within the range of existing methods and over an order of magnitude faster than LLMPrint. Verification takes 38 seconds, only tens of seconds slower than the faster baselines and modest relative to the substantial detection gains. Collectively, Stemma maintains reasonable construction cost and practical verification time. 6 Discussion and Conclusion Limitations and Future Work. Our current instantiation relies on multiple-choice questions and therefore assumes that the evaluated models can reliably process such questions and produce stable option-level decisions. Although practical provenance testing is likely to focus on capable models, this requirement may limit applicability to weaker models. However, multiple-choice questions are only one mechanism for inducing decision regions, and future work could explore more general interfaces for obtaining structured and observable decisions. We also do not consider adaptive attackers with knowledge of the probe distribution. Because our candidate probes are drawn primarily from a single public question set, an attacker could deliberately alter the suspect model’s behaviour on the corresponding question distribution, thereby weakening detection. This risk could be mitigated by constructing candidate pools from a diverse mixture of datasets or privately constructed question sets. Future work should further characterise the robustness limits of provenance testing under adaptive evasion. Conclusion. We introduced induced decision regions for LLMs by mapping open-ended outputs into a finite decision space, abstracting away surface-form variation and reframing provenance testing as measuring the inheritance of decision regions. Building on this formulation, we proposed Stemma, a black-box LLM fingerprinting method that selects probes according to stability, robustness, and specificity, allowing region inheritance to be assessed with a limited query budget. Across extensive evaluations, Stemma consistently demonstrated strong provenance separation, showing that induced decision region inheritance provides a reliable signal for LLM provenance testing. Ethics Statement Stemma scores provide statistical evidence of a potential provenance relationship rather than definitive proof of model ownership or unauthorised reuse. The method could be misused to support false provenance claims, and false positives may cause legal or reputational harm. Stemma should therefore not be used as the sole basis for attribution or enforcement decisions, but interpreted alongside model documentation, licensing and development records, and other technical evidence. Our experiments use only publicly available model checkpoints and benchmark datasets and involve no human subjects or private user data. Reproducibility Statement The complete Stemma procedure is described in Section 4, with prompt calibration, fingerprint construction, fingerprint verification, and hyperparameter settings specified in Appendix C. The models used in the main experiments, provenance-group construction, and pair-labelling protocol are documented in Appendix D. Appendix E provides the exact evaluation-metric definitions and implementation details, while Appendix F describes the baseline implementations and configurations. The all-raw and deployment-robustness benchmark settings are specified in Appendices G and H, respectively. Full ablation results and runtime measurements are reported in Appendices I and J. The open-source implementation and configurations for Stemma are available at https://github.com/kerryzhangcode/Stemma. Acknowledgements We gratefully acknowledge support from a Department of Computer Science Scholarship at the University of Oxford, funded by a generous sponsor. We also acknowledge the use of the University of Oxford Advanced Research Computing (ARC) facility in carrying out this work (10.5281/zenodo.22558). References Y. Adi, C. Baum, M. Cisse, B. Pinkas, and J. Keshet (2018) Turning your weakness into a strength: watermarking deep neural networks by backdooring. In Proceedings of the 27th USENIX Conference on Security Symposium, SEC’18, USA, p. 1615–1631. External Links: ISBN 978-1-931971-46-1 Cited by: §2.1. T. L. Andrews (2016) Analysis of variation significance in artificial traditions using Stemmaweb. Digital Scholarship in the Humanities 31 (3), p. 523–539. External Links: ISSN 2055-7671, Document Cited by: Appendix K. X. Cao, J. Jia, and N. Z. Gong (2021) IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, ASIA CCS ’21, New York, NY, USA, p. 14–25. External Links: Document, ISBN 978-1-4503-8287-8 Cited by: §1, §2.1. T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer (2023) QLoRA: Efficient Finetuning of Quantized LLMs. In Advances in Neural Information Processing Systems, Vol. 36, p. 10088–10115. Cited by: §1. Y. Fang, Y. Feng, B. Li, and M. Zhou (2026) KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing. arXiv. External Links: 2605.29524, Document Cited by: §2.1. T. Gloaguen, R. Staab, N. Jovanović, and M. Vechev (2025) LLM Fingerprinting via Semantically Conditioned Watermarks. In The Fourteenth International Conference on Learning Representations, Cited by: §2.1. A. Godinot, E. Le Merrer, C. Penzo, F. Taïani, and G. Trédan (2025) Queries, representation & detection: the next 100 model fingerprinting schemes. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI’25/IAAI’25/EAAI’25, Vol. 39, p. 16817–16825. External Links: Document, ISBN 978-1-57735-897-8 Cited by: §1. J. Guan, J. Liang, and R. He (2022) Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks. In Advances in Neural Information Processing Systems, Vol. 35, p. 36571–36584. Cited by: §1. M. Gubri, D. Ulmer, H. Lee, S. Yun, and S. J. Oh (2024) TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 11496–11517. External Links: Document Cited by: §1, §2.1. H. A. A. K. Hammoud, U. Michieli, F. Pizzati, P. Torr, A. Bibi, B. Ghanem, and M. Ozay (2024) Model Merging and Safety Alignment: One Bad Model Spoils the Bunch. In Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 13033–13046. External Links: Document Cited by: §1. T. Heikkilä and T. Roos (2016) Thematic Section on Studia Stemmatologica. Digital Scholarship in the Humanities 31 (3), p. 520–522. External Links: ISSN 2055-7671, Document Cited by: Appendix K. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2020) Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations, Cited by: Appendix C, §3.2. Y. Hu, Z. Jiang, M. Li, O. Ahmed, Z. Huang, C. Hong, and N. Z. Gong (2026) Fingerprinting LLMs via Prompt Injection. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), M. Liakata, V. P. Moreira, J. Zhang, and D. Jurgens (Eds.), San Diego, California, United States, p. 11795–11810. External Links: Document, ISBN 979-8-89176-390-6 Cited by: §2.1, §5.1. Hugging Face (2026) Hugging face hub. Note: https://huggingface.co/models Cited by: §5.1. H. Jia, C. A. Choquette-Choo, V. Chandrasekaran, and N. Papernot (2021) Entangled Watermarks as a Defense against Model Extraction. In 30th USENIX Security Symposium (USENIX Security 21), p. 1937–1954. External Links: ISBN 978-1-939133-24-3 Cited by: §2.1. Y. Jiang, Y. Gao, C. Zhou, H. Hu, S. Chen, A. Fu, and W. Susilo (2026) Intellectual property protection for deep learning model and dataset intelligence. Engineering Applications of Artificial Intelligence 163, p. 113024. External Links: ISSN 0952-1976, Document Cited by: §1. H. Jin, C. Zhang, S. Shi, W. Lou, and Y. T. Hou (2024) ProFLingo: A Fingerprinting-based Intellectual Property Protection Scheme for Large Language Models. In 2024 IEEE Conference on Communications and Network Security (CNS), p. 1–9. External Links: ISSN 2994-5895, Document Cited by: §2.1. T. Kojima, S. (. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa (2022) Large Language Models are Zero-Shot Reasoners. Advances in Neural Information Processing Systems 35, p. 22199–22213. Cited by: Appendix H. L. Li, B. Jiang, P. Wang, K. Ren, H. Yan, and X. Qiu (2023) Watermarking LLMs with Weight Quantization. In Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, p. 3368–3378. External Links: Document Cited by: §2.1. I. Nikolic, T. Baluta, and P. Saxena (2025) Model Provenance Testing for Large Language Models. In Advances in Neural Information Processing Systems, Vol. 38, p. 34126–34153. Cited by: §1, §2.1, §5.1. D. Pasquini, E. M. Kornaropoulos, and G. Ateniese (2025) LLMmap: Fingerprinting for Large Language Models. In 34th USENIX Security Symposium (USENIX Security 25), p. 299–318. External Links: ISBN 978-1-939133-52-6 Cited by: §2.1, §5.1. P. Pezeshkpour and E. Hruschka (2024) Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions. In Findings of the Association for Computational Linguistics: NAACL 2024, K. Duh, H. Gomez, and S. Bethard (Eds.), Mexico City, Mexico, p. 2006–2017. External Links: Document Cited by: §3.2. S. Shao, Y. Li, H. Yao, Y. Chen, Y. Yang, and Z. Qin (2026) Reading between the lines: Towards reliable black-box LLM fingerprinting via zeroth-order gradient estimation. In Proceedings of the ACM Web Conference 2026, Www ’26, New York, NY, USA, p. 2637–2648. External Links: Document, ISBN 979-8-4007-2307-0 Cited by: §1, §2.1, §5.1. C. Shi, H. Yang, D. Cai, Z. Zhang, Y. Wang, Y. Yang, and W. Lam (2024) A Thorough Examination of Decoding Methods in the Era of LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 8601–8629. External Links: Document Cited by: §1. T. Stalnaker, N. Wintersgill, O. Chaparro, L. A. Heymann, M. Di Penta, D. M. German, and D. Poshyvanyk (2025) An Empirical Analysis of Machine Learning Model and Dataset Documentation, Supply Chain, and Licensing Challenges on Hugging Face. ACM Transactions on Software Engineering and Methodology. External Links: ISSN 1049-331X, Document Cited by: §1. C. Tang, L. Zhang, Q. Zhao, X. Zhuang, and X. Li (2025) Model Lineage Closeness Analysis. Proceedings of the AAAI Conference on Artificial Intelligence 39 (19), p. 20796–20804. External Links: Document Cited by: §2.1. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom (2023) Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv. External Links: 2307.09288, Document Cited by: Appendix H. Y. Tsai, C. Guo, J. Yang, and L. van der Maaten (2025) RoFL: Robust Fingerprinting of Language Models. arXiv. External Links: 2505.12682, Document Cited by: §2.1. Y. Uchida, Y. Nagai, S. Sakazawa, and S. Satoh (2017) Embedding Watermarks into Deep Neural Networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, ICMR ’17, New York, NY, USA, p. 269–277. External Links: Document, ISBN 978-1-4503-4701-3 Cited by: §2.1. L. Wang, W. Xu, Y. Lan, Z. Hu, Y. Lan, R. K. Lee, and E. Lim (2023) Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki (Eds.), Toronto, Canada, p. 2609–2634. External Links: Document Cited by: Appendix H. N. Wang, Z.y. Peng, H. Que, J. Liu, W. Zhou, Y. Wu, H. Guo, R. Gan, Z. Ni, J. Yang, M. Zhang, Z. Zhang, W. Ouyang, K. Xu, W. Huang, J. Fu, and J. Peng (2024) RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 14743–14777. External Links: Document Cited by: Appendix H. T. Xu, C. Wang, G. Liu, Y. Yang, K. Peng, and W. Liu (2024) United We Stand, Divided We Fall: Fingerprinting Deep Neural Networks via Adversarial Trajectories. In Advances in Neural Information Processing Systems, Vol. 37, p. 69299–69328. External Links: Document Cited by: §2.1. Y. Yan, H. Tang, S. Yan, and E. Dai (2026) DuFFin: A Dual-Level Fingerprinting Framework for LLMs IP Protection. In Findings of the Association for Computational Linguistics: EACL 2026, V. Demberg, K. Inui, and L. Marquez (Eds.), Rabat, Morocco, p. 5168–5184. External Links: Document, ISBN 979-8-89176-386-9 Cited by: §2.1. P. Ye, H. Ren, Z. Li, A. Yan, H. Yan, S. Wang, and J. Li (2026) Securing Large Language Models: A Survey of Watermarking and Fingerprinting Techniques. ACM Computing Surveys 58 (7), p. 1–35. External Links: ISSN 0360-0300, 1557-7341, Document Cited by: §2.1. L. Yu, B. Yu, H. Yu, F. Huang, and Y. Li (2024) Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. In Proceedings of the 41st International Conference on Machine Learning, p. 57755–57775. External Links: ISSN 2640-3498 Cited by: §1. R. Yu and X. Wang (2024) Neural Lineage. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4797–4807. External Links: ISSN 2575-7075, Document Cited by: §2.1. C. Zang, X. Meng, W. Chen, T. Cong, Z. Yaxing, D. Qi, Z. Li, and S. Guo (2025) ErrorTrace: A Black-Box Traceability Mechanism Based on Model Family Error Space. In Advances in Neural Information Processing Systems, Vol. 38, p. 68550–68578. Cited by: §2.1. J. Zhang, D. Liu, C. Qian, L. Zhang, Y. Liu, Y. Qiao, and J. Shao (2025) REEF: Representation Encoding Fingerprints for Large Language Models. International Conference on Learning Representations 2025, p. 48092–48117. Cited by: §2.1. J. Zhang, J. Chi, Z. Li, K. Cai, Y. Zhang, and Y. Tian (2024) BadMerging: Backdoor attacks against model merging. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, Ccs ’24, New York, NY, USA, p. 4450–4464. External Links: Document, ISBN 979-8-4007-0636-3 Cited by: §1. C. Zheng, H. Zhou, F. Meng, J. Zhou, and M. Huang (2024a) Large Language Models Are Not Robust Multiple Choice Selectors. International Conference on Learning Representations 2024, p. 19426–19454. Cited by: §3.2. M. Zheng, J. Pei, L. Logeswaran, M. Lee, and D. Jurgens (2024b) When “A Helpful Assistant” Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 15126–15154. External Links: Document Cited by: Appendix H. X. Zhu, Y. Ye, T. A. Qiu, H. Zhu, S. Tan, A. Mannan, J. Michala, R. A. Popa, and W. Neiswanger (2026) Auditing black-box LLM APIs with a rank-based uniformity test. In The Fourteenth International Conference on Learning Representations, Cited by: §2.1. Appendix A Multiple-Choice Instantiation of Induced Decision Regions Let Q be a set of multiple-choice questions, where each question q∈q contains C candidate options defining the finite decision space =1,…,CD=\1,…,C\. When decision scores are available, let h~M,d(q) h_M,d(q) denote the score assigned by model M to option d, such as a next-token logit or normalised choice probability. The soft decision vector is h~M(q)=(h~M,1(q),…,h~M,C(q))∈ℝC, h_M(q)= ( h_M,1(q),…, h_M,C(q) ) ^C, (8) with the corresponding discrete decision given by hM(q)=minargmaxd∈h~M,d(q),h_M(q)= *arg\,max_d h_M,d(q), (9) where ties are resolved in favour of the lowest-indexed decision. Under text-only access, we extend the observable outcome space with an invalid outcome: ⊥=∪⊥,D_ =D∪\ \, (10) where ⊥ denotes a response from which no valid option can be extracted. The extracted discrete outcome is then hM(q)=g(q,oM(q))∈⊥.h_M(q)=g (q,o_M(q) ) _ . (11) Thus, valid outcomes under both observation modes lie in the same canonical decision space D. For each decision d∈d , the corresponding induced decision region is ℛdM=q∈∣hM(q)=d.R^M_d=\q h_M(q)=d\. (12) The collection ℛdMd∈\R^M_d\_d partitions the valid-decision subset q∈∣g(q,oM(q))∈\q g(q,o_M(q)) \ of the multiple-choice question space, defining the induced decision geometry of M. Responses mapped to ⊥ lie outside this partition and are excluded from fingerprint construction. For a probe set ⊆P , induced decision region inheritance can be assessed at different resolutions. When soft decision vectors are available for both models, as in the visualisation presented in Section 3.2, preservation is assessed by averaging the similarity between corresponding soft decision vectors over P, Sim^(S,T)=1||∑q∈sim(h~S(q),h~T(q)). Sim_P(S,T)= 1|P| _q sim ( h_S(q), h_T(q) ). (13) When such fine-grained information is unavailable, as in the black-box verification setting described in Section 4.4, preservation is instead assessed through alignment of discrete decisions over P, ^(S,T)=1||∑q∈hT(q)=hS(q)≠⊥. A_P(S,T)= 1|P| _q 1 \h_T(q)=h_S(q)≠ \. (14) Thus, invalid outcomes are treated as non-aligned observations. When answer-option permutations are used, extracted scores and decisions are mapped back to the underlying answer options before aggregation or comparison, as specified in the corresponding sections. Appendix B Induced Decision Region Visualisation This appendix describes how the visualisation in Figure 1 is constructed and how its panel annotations are computed. The visualisation is used solely for diagnostic analysis and does not affect Stemma probe selection or provenance scoring. Source-relative projection. For each question, we rank the answer options by the source model’s permutation-averaged option probabilities. This defines a source-relative labelling of the finite decision space D. In the four-choice MMLU setting, the corresponding induced decision regions are visualised as R1,…,R4R_1,…,R_4, where R1R_1 corresponds to the source model’s most preferred option, R2R_2 to its second most preferred option, and so on. The same source-defined ordering is applied to every model M. For each question q, the resulting soft decision vector is h~M(q)=(h~M,1(q),h~M,2(q),h~M,3(q),h~M,4(q)), h_M(q)= ( h_M,1(q), h_M,2(q), h_M,3(q), h_M,4(q) ), where the entries are the permutation-averaged probabilities assigned by M to the source-ranked options R1,…,R4R_1,…,R_4. To project this four-dimensional vector into two dimensions, we arrange the four regions as the quadrants of a square, with R1,…,R4R_1,…,R_4 occupying the upper-left, upper-right, lower-right, and lower-left quadrants, respectively. The projected coordinates are defined as xM(q)=tanh(τ[maxh~M,2(q),h~M,3(q)−maxh~M,1(q),h~M,4(q)]),x_M(q)= \! (τ [ \ h_M,2(q), h_M,3(q)\- \ h_M,1(q), h_M,4(q)\ ] ), yM(q)=tanh(τ[maxh~M,1(q),h~M,2(q)−maxh~M,3(q),h~M,4(q)]).y_M(q)= \! (τ [ \ h_M,1(q), h_M,2(q)\- \ h_M,3(q), h_M,4(q)\ ] ). The x-coordinate compares the strongest preference among the right-side regions with that among the left-side regions, while the y-coordinate compares the strongest preference among the upper-side regions with that among the lower-side regions. The tanh transformation bounds the displayed coordinates to [−1,1][-1,1], and τ controls the projection scale. We use τ=2τ=2 in the figure. Displayed points and region preservation. For each model–question pair, let zM(q)=(xM(q),yM(q))z_M(q)=(x_M(q),y_M(q)) denote its displayed two-dimensional coordinate. The figure plots zS(q)z_S(q) for the source model S as a grey point and zT(q)z_T(q) for the suspect model T as a coloured point. The connecting line represents the projected displacement zT(q)−zS(q)z_T(q)-z_S(q) and is used only as a visual aid, with shorter lines indicating smaller shifts in the projected space. Under this source-relative labelling, the discrete decision map is hM(q)=minargmaxi∈1,2,3,4h~M,i(q),h_M(q)= *arg\,max_i∈\1,2,3,4\ h_M,i(q), where the outer minimum resolves ties in favour of the first decision under the fixed ordering. The corresponding induced decision region is ℛiM=q∈∣hM(q)=i,R_i^M=\q h_M(q)=i\, which is represented by quadrant RiR_i in the visualisation. Suspect points are coloured green when hT(q)=hS(q)h_T(q)=h_S(q), indicating source region alignment, and orange otherwise, indicating a source region change. Annotation metrics. Let P denote the displayed question set. Each panel reports soft decision similarity. For this visualisation, we instantiate the similarity function for normalised soft decision vectors as sim(a,b)=1−12∥a−b∥1.sim(a,b)=1- 12 a-b _1. The resulting similarity score is Sim=Sim^(S,T)=1||∑q∈(1−12‖h~T(q)−h~S(q)‖1).Sim= Sim_P(S,T)= 1|P| _q (1- 12 \| h_T(q)- h_S(q) \|_1 ). Each panel also reports source region alignment, A=^(S,T)=1||∑q∈hT(q)=hS(q).A= A_P(S,T)= 1|P| _q 1\h_T(q)=h_S(q)\. Higher values of SimSim and A indicate stronger preservation of the source model’s soft decision preferences and discrete region assignments, respectively. Appendix C Stemma Implementation and Configuration Details This appendix specifies the implementation and main experimental configuration of Stemma’s three stages: prompt calibration, fingerprint construction, and fingerprint verification. Unless otherwise stated, all random sampling and stochastic generation use a fixed random seed of 42 for reproducibility. Dataset. Probe candidates are sampled from the MMLU test split across all subjects (Hendrycks et al., 2020). Unless otherwise stated, we randomly sample 3,000 candidate questions for each source model. Each candidate is evaluated under cyclic permutations of its answer options to assess the stability of the source decision across different option orders and label assignments. Prompt calibration. Before fingerprint construction and verification, Stemma independently calibrates a multiple-choice prompt template for each evaluated model using 100 examples from the MMLU validation split, which is disjoint from the candidate probe pool. Calibration selects a prompt format that reliably elicits parseable option labels, facilitating both white-box choice label scoring during fingerprint construction and black-box answer extraction during fingerprint verification. We evaluate 6 simple prompt templates. To avoid severe label bias, we discard any template for which the most frequently predicted label accounts for more than 0.70.7 of the calibration predictions. Among the remaining templates, we select the one with the highest valid-choice extraction rate, breaking ties according to the predefined template order, and fix it for all subsequent queries to that model. The set of prompt templates used in calibration is shown below. In these templates, question denotes the question text, options denotes the formatted answer options, and choices denotes the available option labels. Simple answer cue MMLU-style prompt Strict label-only prompt Single-character response High-constraint prompt Answer cue with trailing newline Fingerprint construction. For each source model, Stemma computes next-token choice-label log-probabilities for every candidate question under all cyclic option permutations. Choice labels are scored using tokenizer-aware matching. When multiple token IDs correspond to the same visible option label, their log-probabilities are combined using log-sum-exp. Under each permutation, the highest-probability option is selected and mapped back to its canonical option identity. The canonical source decision is then determined by voting across the resulting permutation-level decisions. Candidate questions are filtered and ranked according to stability, robustness, and specificity. During hard filtering, a candidate is retained only if the same canonical wrong option receives more than half of the source model’s permutation votes (>0.5>0.5), has a non-negative average wrong-label top margin (≥0≥ 0), and is selected by fewer than half of the background model’s permutation-level predictions (<0.5<0.5). Unless otherwise stated, we use microsoft/Phi-3.5-mini-instruct as the background model for specificity evaluation. During soft ranking, the surviving candidates are ranked using the following standardised score: Score(q)=1.0⋅clip(z(r(q)),−2,2)−0.5⋅z(b(q)),Score(q)=1.0·clip (z(r(q)),-2,2 )-0.5· z(b(q)), (15) where r(q)r(q) is the average wrong-label top logit margin across permutations, b(q)b(q) is the fraction of background-model predictions that select the same canonical wrong option, and z(⋅)z(·) denotes standardisation over the surviving candidate pool for the corresponding source model. The standardised robustness term is clipped to [−2,2][-2,2] to prevent extreme margins from dominating the ranking. The resulting source-specific fingerprint comprises the 40 highest-ranked probes and their canonical wrong labels, as determined by permutation voting. Fingerprint verification. For each source–suspect pair, the suspect model is queried on the corresponding source probes under the same cyclic option permutations used during fingerprint construction. Generation is limited to 16 new tokens. A rule-based regular-expression extractor maps each response to either a predicted option label or an invalid outcome ⊥ , with valid labels mapped back to their canonical option identities. The fingerprint alignment score is computed as defined in Equation 7, with invalid outcomes treated as non-aligned. With 40 probes and four cyclic permutations, the main configuration yields 160 probe–permutation observations for each model pair. Appendix D Model Benchmark Table 3 presents the complete model benchmark used in our experiments. It comprises 56 publicly available Hugging Face checkpoints organised into 7 provenance groups across 5 model families, covering both fine-grained relationships among closely related models and diverse model-development transformations. Model selection considers provenance clarity, documentation quality, download popularity, and diversity across model families, scales, and transformation types. Table 3: Complete model benchmark. Provenance group Repository ID Model type Query interface Qwen-2.5-7B Qwen/Qwen2.5-7B Pretrained raw Qwen/Qwen2.5-7B-Instruct Instruct chat Qwen/Qwen2.5-Coder-7B-Instruct Fine-tune chat zjudai/flowertune-medical-lora-qwen2.5-7b-instruct Adapter chat SeeFlock/task-12-Qwen-Qwen2.5-7B-Instruct Adapter chat Locutusque/StockQwen-2.5-7B Merge chat Qwen/Qwen2.5-7B-Instruct-GPTQ-Int8 Quantisation chat Lansechen/Qwen2.5-7B-Open-R1-Distill Distillation chat Qwen-2.5-14B Qwen/Qwen2.5-14B Pretrained raw Qwen/Qwen2.5-14B-Instruct Instruct chat oxyapi/oxy-1-small Fine-tune chat Qwen/Qwen2.5-14B-Instruct-1M Fine-tune chat ToastyPigeon/qwen-story-test-qlora Adapter chat v000000/Qwen2.5-14B-Gutenberg-Instruct-Slerpeno Merge chat Qwen/Qwen2.5-14B-Instruct-GPTQ-Int8 Quantisation chat alibaba-pai/DistilQwen2.5-DS3-0324-14B Distillation chat Qwen3-1.7B Qwen/Qwen3-1.7B-Base Pretrained raw Qwen/Qwen3-1.7B Instruct chat mlabonne/Qwen3-1.7B-abliterated Fine-tune chat HuggingFaceTB/qwen3-1.7b-gsm8k-sft Fine-tune chat txmedai/ClinicalEase-Qwen3-1.7B Adapter chat kurakurai/Luth-1.7B-Instruct Merge chat Qwen/Qwen3-1.7B-GPTQ-Int8 Quantisation chat prithivMLmods/Regulus-Qwen3-R1-Llama-Distill-1.7B Distillation chat Llama-3.1-8B meta-llama/Llama-3.1-8B Pretrained raw meta-llama/Llama-3.1-8B-Instruct Instruct chat RedHatAI/Llama-3.1-8B-tldr Fine-tune raw chchen/Llama-3.1-8B-Instruct-PsyCourse-fold7 Adapter chat zjudai/flowertune-medical-lora-llama-3.1-8b-instruct Adapter chat Xiaojian9992024/Llama3.1-8B-ExtraMix Merge chat hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 Quantisation chat arcee-ai/Llama-3.1-SuperNova-Lite Distillation chat Mistral-7B-v0.3 mistralai/Mistral-7B-v0.3 Pretrained raw mistralai/Mistral-7B-Instruct-v0.3 Instruct chat KurmaAI/AQUA-7B Fine-tune chat chaymaemerhrioui/mistral-Brain_Model_ACC_Trainer Adapter chat zjudai/flowertune-medical-lora-mistral-7b-instruct-v0.3 Adapter chat grimjim/Mistral-7B-Instruct-demi-merge-v0.3-7B Merge chat RedHatAI/Mistral-7B-Instruct-v0.3-GPTQ-4bit Quantisation chat eganwo/mistral7b-distilled-from-deepseek-r1-qwen32b Distillation chat Falcon3-7B tiiuae/Falcon3-7B-Base Pretrained raw tiiuae/Falcon3-7B-Instruct Instruct chat ehristoforu/falcon3-ultraset Fine-tune chat jahyungu/Falcon3-7B-Instruct-v1-Easy Adapter chat jahyungu/Falcon3-7B-Instruct-v1-Hard Adapter chat suayptalha/Falcon3-Jessi-v0.4-7B-Slerp Merge chat tiiuae/Falcon3-7B-Instruct-GPTQ-Int8 Quantisation chat RedaAlami/Falcon3-7B-Instruct-Distill-DS-v1 Distillation chat OLMo-2-7B allenai/OLMo-2-1124-7B Pretrained raw allenai/OLMo-2-1124-7B-Instruct Instruct chat allenai/OLMo-2-1124-7B-SFT Fine-tune chat allenai/OLMo-2-1124-7B-Instruct-preview Fine-tune chat jahyungu/OLMo-2-1124-7B-Instruct-Humanities Adapter chat jahyungu/OLMo-2-1124-7B-Instruct-Social-Sciences Adapter chat Alelcv27/Olmo2-7B-Breadcrumbs-v1 Merge chat kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit Quantisation chat Benchmark composition. Each provenance group follows a consistent structure, containing 8 checkpoints: 1 pretrained checkpoint, 1 instruction-tuned checkpoint, and 6 additional variants. For these additional variants, we aim to cover a diverse set of common model-development operations. Specifically, where suitable public checkpoints are available, each group includes 3 task- or domain-specialised variants covering both full fine-tuning and parameter-efficient adaptation, together with 1 model-merge variant, 1 quantised variant, and 1 distillation variant. The only exception is the OLMo-2-1124-7B group, for which we did not find a suitable public distillation variant, so we include an additional fine-tuned or adapter-based variant instead. This design gives each provenance group comparable coverage while capturing the diversity and complexity of real-world model development, thereby providing a more rigorous testbed for evaluating fingerprinting methods under realistic transformation scenarios. For evaluation, within-group pairs are labelled positive, whereas cross-group pairs are labelled negative. Model types. The model type column reports the development operation associated with each checkpoint. Pretrained denotes the base model trained before instruction tuning. Instruct denotes an instruction-tuned checkpoint derived from the pretrained model. Fine-tune denotes a checkpoint further trained on task-specific, domain-specific, or instruction-following data. Adapter denotes parameter-efficient adaptation, where additional or low-rank trainable parameters are used to specialise the model. Merge denotes a checkpoint produced by combining weights from multiple models. Quantisation denotes a numerically compressed checkpoint with reduced weight precision. Distillation denotes a checkpoint derived from a base model and further trained or fine-tuned using teacher-generated data, such as model-generated responses or reasoning traces from a stronger model. Query interfaces. We also distinguish two query interfaces, following the recommended usage in each checkpoint’s model card or tokenizer configuration. The raw interface denotes completion-style prompting, where the prompt is passed directly to the causal language model without a chat wrapper. It is used for pretrained checkpoints and for downstream checkpoints that are released or documented as completion-style models. The chat interface denotes chat-template prompting, where the calibrated prompt is first treated as a user message and then rendered using the tokenizer’s chat template before generation. It is used for instruction-tuned and chat-oriented checkpoints. Chat templates may add model-specific system, user, and assistant formatting, thereby changing the effective input received by the model. For example, when no explicit system message is provided, the Qwen2.5-Instruct tokenizer renders a single user message as follows: Rendered input under the Qwen2.5-Instruct chat template Here, user_prompt denotes the input prompt used for evaluation. We do not explicitly provide a manually constructed system message. Instead, when a checkpoint’s tokenizer chat template defines a default system message, it is inserted automatically during template rendering. Using the interface specified by each checkpoint’s model card or tokenizer configuration helps ensure that the model is evaluated under its intended usage pattern, better matching common usage and realistic deployment practice. Appendix E Evaluation Metrics Each evaluated source–suspect pair is assigned a continuous provenance score, where higher values indicate stronger evidence of a provenance relationship. Related pairs are treated as positive and unrelated pairs as negative. AUC and partial AUC. The area under the receiver operating characteristic curve is defined as AUC=∫01TPR(u)du,AUC= _0^1TPR(u)\,du, where u denotes the false-positive rate. To emphasise performance in the low-FPR regime, we additionally report standardised partial AUC over FPR∈[0,α]FPR∈[0,α], with α=0.05α=0.05. Let Aα=∫0αTPR(u)duA_α= _0^αTPR(u)\,du denote the unstandardised partial area. We apply McClish standardisation: pAUCα=12(1+Aα−α2/2α−α2/2).pAUC_α= 12 (1+ A_α-α^2/2α-α^2/2 ). Under this standardisation, random ranking corresponds to 0.50.5 and perfect ranking to 11. Low-FPR true-positive rate. TPR at 1% FPR is the highest empirical true-positive rate attained without exceeding a false-positive rate of 0.01. We do not interpolate between operating points. Discriminability. We quantify the standardised separation between the positive and negative score distributions using d′=μ+−μ−(σ+2+σ−2)/2,d = _+- _- ( _+^2+ _-^2)/2, where μ+ _+ and μ− _- are the mean positive and negative scores, and σ+2 _+^2 and σ−2 _-^2 are their sample variances. Positive values indicate that related pairs receive higher scores on average, while larger values indicate clearer separation in the intended direction. Negative values indicate reversed mean ordering. Appendix F Baseline Implementations We implement four black-box LLM fingerprinting baselines to compare against Stemma. Unless otherwise stated below, we use the settings and hyperparameters specified in the original papers or official repositories for each baseline, in order to reproduce its intended performance. Since a fine-grained comparison requires a continuous provenance score rather than only a binary decision for each source and suspect model pair, we use the underlying fingerprint similarity or matching score produced by each baseline as its pairwise provenance score. This is natural for these methods, as their final decisions are based on fingerprint similarity even when their decision rules differ. LLMmap. LLMmap fingerprints a model by encoding its behaviour on a fixed set of diagnostic prompts into an open-set feature space. We use the 8 input queries from the original paper and the pretrained open-set feature extractor released with the official implementation. To construct each source gallery representation, we evaluate 100 prompt configurations over these queries and average the resulting feature vectors. During verification, the suspect model is evaluated once on the same 8 queries and its feature vector is compared with the source gallery representation. For both gallery construction and verification, generated responses contain at most 100 new tokens and are truncated to 650 characters before feature extraction. Following the original comparison procedure, we compute the Euclidean distance d between the source and suspect fingerprints and convert it into a similarity score using the inverse-one-plus mapping, s=1/(1+d)s=1/(1+d). This score is used as the pairwise provenance score for each evaluated source and suspect model pair. LLMPrint. LLMPrint constructs black-box fingerprints by learning adversarial suffix probes that induce model-specific preferences over target word pairs. Its original configuration optimises suffixes for 300 word pairs using 1,000 GCG steps per pair. Based on pilot runtime measurements, running the full configuration independently across all three evaluation settings is estimated to require over 240 GPU-days in our environment. We therefore use a reduced configuration consistently across all source models. For each source model, we sample 100 word pairs from the fixed category vocabulary provided by the official repository, such that both words are represented by a single token under the source tokenizer. We then optimise one adversarial suffix per pair for 200 GCG steps under the fixed LLMPrint prompt template. The learned suffixes are used to query both source and suspect models. Following the original scoring procedure, we perform 100 repeated one-token generations for each word pair and record how often each target token is generated. The more frequently generated token determines the model’s binary preference for that pair, and the agreement rate between the source and suspect preferences is used as the pairwise provenance score. Model Provenance Testing. Model Provenance Testing (MPT) detects provenance by comparing next-token continuation behaviour between a source model and a suspect model. Following the official implementation, we sample 5,000 sentence-level prompts from the MPT prompt pool provided in the official repository. For each source model, we construct a continuation cache by querying the model on all prompts with single-token generation, using max_new_tokens=1. Each suspect model is then evaluated on the same prompt set, and its next-token continuations are matched against the cached source continuations. We use the resulting hit rate, namely the fraction of prompts on which the suspect continuation matches the source continuation, as the pairwise provenance score. ZeroPrint. ZeroPrint constructs black-box LLM fingerprints by measuring the response sensitivity of a model to query perturbations. We follow the original setup by using openai_humaneval as the query source, converting each task prompt into a completion-style query, and truncating the query to 20 words. Following the configuration reported in the original ZeroPrint paper, we sample 2 base queries and generate 4 word-substitution variants for each query, where each perturbation replaces 3 words with independently selected top-10 nearest neighbours under GloVe embeddings. Responses are generated with at most 128 new tokens and are not truncated after generation. To reduce generation noise, each model is queried 20 times per prompt, and the resulting responses are embedded with sentence-transformers/all-mpnet-base-v2, matching the official implementation. ZeroPrint then estimates a response-sensitivity fingerprint for each base query from the input and output embedding changes using the Jacobian-based estimator with ridge regularisation α=0.001α=0.001. The resulting fingerprints are aggregated by mean pooling, and the Pearson correlation coefficient between source and suspect fingerprints is used as the pairwise provenance score. Implementation consistency. For all baselines, we use the same benchmark split, model query interfaces, source-model set, and suspect-model set. Each baseline produces a pairwise provenance score for every evaluated source–suspect pair. We compute and report the final results from these pairwise scores using the same evaluation metrics as Stemma. Appendix G All-Raw Query Setting Table 4 provides the full results for the all-raw query-interface setting. In this control setting, all checkpoints are queried with raw completion-style prompts during both fingerprint construction and verification, without applying tokenizer chat templates. By removing model-specific chat formatting, this setting isolates the effect of prompt-format mismatch in the main experiments and evaluates baselines under more favourable input-matched conditions, although it is less representative of realistic deployment. Table 4: Overall comparison under an all-raw interface setting. Method Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d LLMmap 0.577 0.501 0.000 0.223 0.724 0.555 0.102 0.801 0.652 0.519 0.041 0.517 LLMPrint 0.541 0.496 0.000 0.213 0.738 0.602 0.163 0.908 0.625 0.550 0.082 0.507 MPT 0.993 0.946 0.878 3.258 0.991 0.972 0.939 3.333 0.988 0.938 0.816 2.713 ZeroPrint 0.738 0.528 0.020 0.847 0.868 0.710 0.367 1.615 0.807 0.604 0.133 1.214 Stemma 0.968 0.966 0.918 3.419 0.963 0.944 0.878 2.988 0.965 0.952 0.898 3.049 Appendix H Robustness Benchmark In addition to the main checkpoint-level benchmark, we construct a robustness benchmark to evaluate provenance signals under common deployment-time variations. It follows the same 7 provenance groups and uses their pretrained and instruction-tuned checkpoints as source models. Deployment variants are created from the corresponding instruction-tuned checkpoints by changing inference-time settings without modifying model weights. Pretrained checkpoints use the raw query interface, whereas instruction-tuned and chat-oriented checkpoints use the chat interface specified by their tokenizer configuration. We consider five categories of deployment variation. For all prompt-based variants, the additional instruction is supplied through the system-message field of the model’s chat template, while the original multiple-choice prompt remains unchanged in the user message. System prompts. We evaluate two system-prompt variants that reflect commonly used assistant-style deployment settings. The first is a minimal generic helpful-assistant prompt, while the second uses the longer safety-oriented system prompt reported for Llama 2-Chat (Touvron et al., 2023). Minimal assistant system prompt Safety-oriented assistant system prompt Role-play prompts. We evaluate two role-play prompt variants, motivated by prior studies of persona and role prompting (Zheng et al., 2024b; Wang et al., 2024). The first represents a simple occupational role, where the model is assigned a profession-based persona. The second represents a more detailed persona-style role, where the model is given a specific character description and instructed to imitate the assigned personality. Occupational role prompt Persona-style role prompt Chain-of-thought prompts. We evaluate two reasoning-prompt variants that are commonly used to encourage intermediate reasoning during inference. The first is the classic zero-shot chain-of-thought prompt (Kojima et al., 2022), which uses a short instruction to elicit step-by-step reasoning. The second follows the plan-and-solve prompting style (Wang et al., 2023), where the model is first asked to understand the problem and devise a plan before carrying out the solution. Zero-shot chain-of-thought prompt Plan-and-solve prompt Retrieval-augmented generation. We evaluate two retrieval-augmented generation variants using SQuAD-v2 as the retrieval corpus and all-mpnet-base-v2 as the retriever. Each SQuAD-v2 context passage is treated as one retrieval document, and the original question text is used as the retrieval query. The top-1 or top-3 retrieved passages are prepended to the original multiple-choice prompt in descending similarity order. Retrieved passages are inserted in a fixed context block before the original prompt. Decoding settings. We evaluate four decoding variants that cover deterministic and stochastic generation regimes. The first uses greedy decoding, while the remaining three use stochastic decoding with moderate, high, and low sampling settings. Moderate sampling is the default decoding configuration used in the main benchmark and in the default robustness setting. The exact decoding parameters are summarised in Table 5. Table 5: Decoding settings used in the robustness benchmark. Setting Decoding mode Temperature top-p top-k Greedy decoding Deterministic – – – Moderate sampling (default) Stochastic 0.7 0.9 50 High sampling Stochastic 1.2 1.0 0 Low sampling Stochastic 0.3 0.9 10 Together, the robustness benchmark uses 7 pretrained checkpoints as source models and includes the corresponding 7 instruction-tuned checkpoints as both instruct-source models and default suspect models under the moderate-sampling setting. Each instruction-tuned checkpoint is further redeployed under 11 alternative inference-time settings, yielding 77 additional deployment variants. In total, the benchmark contains 91 model or deployment instances: 7 pretrained checkpoints, 7 default instruction-tuned deployments, and 77 additional deployment variants. Under the same pair-construction rule, this gives 1,260 evaluated pairs, including 168 positive pairs and 1,092 negative pairs. Appendix I Full Ablation Results Tables 6–10 report the detailed results underlying the ablation analyses in Section 5.4, including source-type breakdowns where applicable. Probe selection. All variants use the same candidate pool and final budget of 40 probes. The Random probes variant uniformly samples probes from all candidate questions. Non-gold pool samples after restricting candidates to questions whose source decision differs from the gold answer. Filtered pool additionally applies the stability, robustness, and background-specificity filters, but samples uniformly without soft ranking. Stemma further ranks the filtered candidates and selects the top-scoring probes. All variants use the same cyclic-permutation verification and scoring procedure. Table 6: Detailed ablation results for probe selection stages. Selection stage Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d Random probes 0.761 0.675 0.347 0.953 0.741 0.616 0.224 0.821 0.751 0.614 0.143 0.875 Non-gold pool 0.812 0.710 0.367 1.264 0.878 0.795 0.551 1.746 0.844 0.756 0.439 1.494 Filtered pool 0.936 0.818 0.571 2.242 0.957 0.904 0.796 2.551 0.948 0.871 0.674 2.340 Stemma 0.964 0.959 0.918 3.045 0.970 0.938 0.857 2.945 0.967 0.944 0.878 2.951 Pool size and probe budget. We jointly vary the candidate pool size from 1,000 to 6,000 questions and the final probe budget from 10 to 100, while keeping all other settings fixed. Tables 7 and 8 report the corresponding pAUC and AUC results, respectively. The default configuration is shown in bold. As the sweep was conducted independently from the main experiment, the default-setting result differs slightly from the main-run result due to stochastic decoding and hardware-level nondeterminism. Table 7: Detailed pAUC results across candidate pool sizes and probe budgets. Pool size Number of probes k 10 20 30 40 50 60 70 80 90 100 1,000 0.857 0.888 0.904 0.915 0.927 0.920 0.929 0.921 0.916 0.915 2,000 0.881 0.934 0.930 0.934 0.932 0.928 0.932 0.937 0.937 0.936 3,000 0.878 0.945 0.949 0.946 0.956 0.950 0.950 0.953 0.950 0.948 4,000 0.903 0.935 0.940 0.951 0.946 0.951 0.950 0.944 0.945 0.943 5,000 0.908 0.929 0.941 0.939 0.946 0.946 0.942 0.946 0.946 0.940 6,000 0.887 0.900 0.920 0.946 0.938 0.942 0.945 0.944 0.950 0.949 Table 8: Detailed AUC results across candidate pool sizes and probe budgets. Pool size Number of probes k 10 20 30 40 50 60 70 80 90 100 1,000 0.920 0.949 0.954 0.956 0.955 0.957 0.959 0.959 0.958 0.958 2,000 0.946 0.958 0.962 0.960 0.960 0.961 0.962 0.962 0.965 0.965 3,000 0.955 0.972 0.969 0.968 0.970 0.971 0.971 0.970 0.969 0.970 4,000 0.955 0.970 0.968 0.970 0.970 0.970 0.969 0.967 0.969 0.968 5,000 0.958 0.958 0.963 0.965 0.965 0.966 0.967 0.969 0.971 0.970 6,000 0.945 0.956 0.959 0.963 0.963 0.964 0.964 0.964 0.966 0.965 Probe dataset. For the dataset ablation, candidate questions are drawn from the test splits of MMLU and MMLU-Pro and the validation splits of CommonsenseQA and CosmosQA, using up to 3,000 questions per dataset. Decision-interface calibration uses 100 questions from the validation splits of MMLU and MMLU-Pro and the training splits of CommonsenseQA and CosmosQA, ensuring that calibration and fingerprint construction use disjoint splits. The selection procedure and probe budget remain fixed across datasets. Table 9: Detailed ablation results for probe dataset choice. Dataset Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d CommonsenseQA 0.915 0.770 0.490 1.856 0.977 0.961 0.898 2.936 0.948 0.868 0.714 2.232 CosmosQA 0.889 0.783 0.551 1.748 0.951 0.874 0.694 2.601 0.925 0.815 0.582 2.031 MMLU-Pro 0.947 0.938 0.878 2.736 0.948 0.916 0.837 2.673 0.948 0.929 0.857 2.635 MMLU (default) 0.964 0.959 0.918 3.045 0.970 0.938 0.857 2.945 0.967 0.944 0.878 2.951 Background model. For the background-model ablation, we compare Phi-3.5-mini-instruct, Yi-1.5-9B-Chat, and Gemma-2-9B-IT, while keeping all other settings fixed. In the combined setting, we pool the permutation-level decisions from all three models and compute background alignment as the fraction matching the source decision. Table 10: Detailed ablation results for background model choice. Background model Pretrained sources Instruct sources All sources AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d AUC ↑ pAUC ↑ TPR ↑ d′↑d Phi-3.5 (default) 0.964 0.959 0.918 3.045 0.970 0.938 0.857 2.945 0.967 0.944 0.878 2.951 Yi-1.5 0.958 0.940 0.857 2.531 0.964 0.955 0.898 2.853 0.959 0.939 0.857 2.628 Gemma-2 0.955 0.959 0.918 2.654 0.968 0.942 0.878 2.901 0.960 0.946 0.878 2.700 Combined 0.966 0.957 0.918 2.983 0.979 0.958 0.918 3.077 0.973 0.952 0.898 2.950 Appendix J Runtime Measurement Measurement protocol. We measure the wall-clock runtime of all methods on the same NVIDIA L40S GPU with 48 GB of memory. Fingerprint construction is measured per source model and reported as the median over the 14 source checkpoints. Fingerprint verification measures the suspect-side computation required to evaluate one source–suspect pair and is reported as the median over the corresponding verification executions, assuming that the source fingerprint has already been constructed. The measurements cover the principal computational procedures, such as model inference and, where applicable, prompt optimisation involving backward passes, but exclude model loading. Reported runtimes are rounded to the nearest second. LLMmap. The reported construction time includes evaluating each source under 100 prompt configurations on the 8 diagnostic queries, encoding the resulting responses, and averaging the feature vectors to construct the gallery representation. The reported verification time includes evaluating each suspect once on the same 8 queries and encoding its responses using the pretrained feature extractor. LLMPrint. The reported construction time includes optimising one adversarial suffix for each of 100 word pairs, each consisting of two single-token words, over 200 GCG steps, and estimating the source model’s preference direction for each pair using 100 repeated one-token generations. The reported verification time includes querying each suspect model with the fixed optimised suffixes, estimating its preference direction for each pair using 100 repeated one-token generations, and comparing these directions with the stored source preferences. MPT. The reported construction time includes querying each source on 5,000 prompts with single-token generation. The reported verification time includes querying each suspect once on the same prompt set and matching the resulting continuations against the cached source continuations to compute the hit rate. ZeroPrint. The reported construction time includes querying each source on 2 HumanEval base prompts and 4 word-substitution variants per prompt, with 20 generations for each query, embedding the responses, and constructing the Jacobian-based fingerprint. The reported verification time includes applying the same procedure to each suspect. Stemma. The reported construction time includes calibrating the source model and the configured background model on 100 held-out validation questions, followed by evaluating them on 3,000 candidate questions under cyclic option permutations to obtain the source-decision and background-alignment statistics used for probe selection. The reported verification time includes calibrating each suspect on the same held-out set and querying it on the 40 source-specific probes under four cyclic permutations. Appendix K The Stemmatic Analogy Behind Stemma Stemmatic analysis. Manuscript studies draw on several complementary approaches, including palaeography, codicology, and stemmatics. Palaeography examines handwriting and scribal practices, while codicology examines the material construction and production of manuscripts. Stemmatics instead focuses on reconstructing the transmission history of works preserved in multiple manuscript copies, particularly when the original text and many intermediate copies no longer survive and only a set of extant manuscripts, commonly termed witnesses, remains (Heikkilä and Roos, 2016). To infer the genealogy of the transmitted text, stemmatic analysis aligns witnesses at corresponding textual locations and compares their variant readings, including substitutions, omissions, additions, and transpositions. It therefore relies on structured patterns of textual variation rather than treating overall visual or stylistic resemblance as the primary genealogical signal. The genealogical value of a shared variant depends on how readily it could have arisen independently. Variants that are widespread across the textual tradition or easily reproduced provide limited evidence, whereas distinctive variants are more informative because their preservation by multiple witnesses may indicate descent from a common exemplar. Stemmatic inference therefore considers not merely the presence of shared variation, but its genealogical informativeness (Andrews, 2016). The inferred relationships are conventionally represented by a stemma codicum, or stemma, a tree-like representation of textual descent among surviving witnesses and reconstructed intermediate exemplars. Analogy to model provenance. The name Stemma reflects this methodological analogy. Candidate questions correspond to aligned textual locations, model instances to manuscript witnesses, and induced decisions to variant readings. Existing response-based fingerprints derive provenance signals from observable characteristics of generated responses, much as palaeography and codicology draw on handwriting and material features, whereas Stemma maps model behaviour into a common finite decision space and compares structured decision variants across aligned probes. Decisions readily reproduced by unrelated models provide limited provenance evidence, just as common or independently arising readings provide limited evidence of textual affiliation. By contrast, stable, robust, and specific decisions preserved by a suspect resemble distinctive inherited variants. Stemma therefore tests model provenance through structured patterns of induced decision region inheritance rather than through response-level characteristics tied to particular surface realisations.