Paper deep dive
STEB: Style Text Embedding Benchmark
Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 7/5/2026, 7:48:42 AM
Summary
The paper introduces the Style Text Embedding Benchmark (STEB), an open-source benchmark designed to standardize the evaluation of style embeddings. While semantic embeddings (like those in MTEB) are rigorously evaluated, style embedding evaluation has been fragmented. STEB comprises 96 datasets across 77 languages and 55 evaluation tasks, including clustering, pair classification, authorship retrieval, and probing. The benchmark provides two scoring methods: an 'operational score' based on existing field datasets and a 'definitional score' based on a theoretical definition of style. The study finds that specialized style embeddings generally outperform general-purpose semantic embeddings on stylistic tasks, and that semantic models often fail to capture nuanced stylistic features.
Entities (7)
Relation Signals (4)
STEB → includestask → Authorship Verification
confidence 100% · STEB encompasses 96 datasets across 77 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection...
LUAR-CRUD → outperforms → GPT-5.2
confidence 100% · We demonstrate that LUAR-CRUD (Rivera Soto et al., 2021) clearly beats GPT-5.2 on a small AR task
Qwen3-Embedding-8B → performspoorlyon → STEB
confidence 100% · Qwen3-Embedding-8B, a top-5 MTEB model, ranks poorly on STEB.
STEB → addressesgapin → MTEB
confidence 90% · While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented. To bridge this gap, we introduce the Style Text Embedding Benchmark (STEB).
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open-source the STEB code base at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2606.31741v1
- Canonical: https://arxiv.org/abs/2606.31741v1
Trouble viewing inline? Open PDF directly →
Full Text
126,372 characters extracted from source content.
Expand or collapse full text
STEB: Style Text Embedding Benchmark Rafael Rivera Soto Johns Hopkins University Anna Wegmann Utrecht University Cristina Aggazzotti Johns Hopkins University Abstract While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 9696 datasets across 77 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open-source the STEB code base at: https://github.com/rrivera1849/STEB. STEB: Style Text Embedding Benchmark 1 Introduction Figure 1: STEB Score by model category Style embeddings (blue) score above general-purpose semantic models (orange) on stylistic tasks; dashed lines mark the best model in each category. Qwen3-Embedding-8B, a top-5 MTEB model, ranks poorly on STEB. Representation learning has driven progress in NLP in the last decade, including on information retrieval, clustering, classification, and semantic search. While such models like SBERT-models (Reimers and Gurevych, 2019), E5 (Wang et al., 2024a), LLM2Vec (BehnamGhader et al., 2024), and EmbeddingGemma (Vera et al., 2025) focus on capturing semantic meaning, a more disregarded parallel field focuses on the style of text.111Semantic meaning (or content) and style are hard to disentangle and might not be fully disjoint Wegmann et al. (2026). Style representations have proven valuable in applications such as authorship attribution (A), style transfer, and AI-generated text detection (Horvitz et al., 2024a, b; Khan et al., 2024; Kim et al., 2025a; Rivera Soto et al., 2021, 2023; Wegmann et al., 2022), and might support the development of more style-aware LLMs Wegmann et al. (2026). Currently, evaluation approaches for style embeddings are inconsistent. Works vary considerably in the tasks and datasets they evaluate on and in evaluation protocol decisions like preprocessing, encoded text length, and documents per embedding. For example, LUAR (Rivera Soto et al., 2021) evaluates only on authorship retrieval, STAR adds author clustering (Huertas-Tato et al., 2024), and LISA (Patel et al., 2023) uses the STEL evaluation framework (Wegmann and Nguyen, 2021; Wegmann et al., 2022). The Massive Text Embedding Benchmark (MTEB) (Muennighoff et al., 2023), positioned as a general-purpose text embedding benchmark, offers no alternative as it includes no style-specific tasks, and many of its leading models (Bai et al., 2023; Lee et al., 2025; Li et al., 2023; Wang et al., 2024a; Xiao et al., 2023) are trained on objectives such as semantic similarity, information retrieval, and natural language inference (NLI) that benefit less from stylistic features (see Fig. 1). As a result, cross-work comparisons are unreliable, and progress on style embeddings is hard to quantify. We introduce the open-source Style Text Embedding Benchmark (STEB). STEB consists of 9696 datasets across 77 languages, organized into 55 evaluation tasks (clustering, pair classification, order alignment, retrieval, and probing) under fixed evaluation protocols (e.g., metrics, long document handling) covering applications like AI-text detection (ATD), authorship verification (AV) and authorship retrieval (AR). STEB reports two complementary scores. The operational score reflects the datasets and tasks that the field has built to study style. It inherits the emphasis of the field—most prominently A work—without committing to an a priori definition. The definitional score instead reweighs results according to the style definition proposed in Wegmann et al. (2026). We evaluate 4040 models spanning style embeddings, general-purpose semantic embeddings, masked language models (MLMs), causal language models (CLMs) and non-neural baselines. We find that no single model dominates across STEB. Instead, the best representation depends on whether the goal is strong performance on a specific downstream application or broad coverage of stylistic attributes, as well as on the degree to which the task is entangled with semantics. Notably, recent semantic embeddings that lead MTEB (e.g., Qwen-Embedding-8B) perform substantially worse than specialized style representations, highlighting a gap in MTEB’s evaluation of stylistic tasks. Furthermore, re-evaluating a prior multilingual setup under a different protocol for fairly handling long texts produces substantially different rankings, reinforcing the importance of standardized evaluation procedures. We also find that off-the-shelf MLMs perform surprisingly well at capturing linguistic features and remain competitive on AV and AR, suggesting promising directions for future research. Overall, we hope that STEB serves as a “yardstick” for measuring progress toward better style representations. Why not just use LLM prompting? While generative models can solve an increasing number of problems, text representations remain highly relevant and widely used in practice (Enevoldsen et al., 2025; Warner et al., 2025) for several reasons: (i) Text representations are more efficient: state-of-the-art encoder models are smaller (millions vs. billions of parameters) and only use a single forward pass, whereas generative models incur cost proportional to the output length; (i) they are competitive on discriminative tasks, matching or outperforming much larger generative models (Warner et al., 2025); and (i) they scale much better to larger document pools for applications like RAG (Ram et al., 2023) or clustering, and might reduce hallucinations (Gao et al., 2024). We demonstrate that LUAR-CRUD (Rivera Soto et al., 2021) clearly beats GPT-5.2 on a small AR task, while using ≫ 750× fewer FLOPs (Tab. 1; full setup in App. E). Model R@1 R@8 MRR TFLOPs USD LUAR-CRUD 83.0 95.0 87.8 ≈32≈ 32 0 GPT-5.2 59.0 69.0 63.4 ≫24 24k 22 Table 1: Embeddings outperform LLMs in AR at a fraction of the compute. The GPT-5.2 FLOP estimate uses 1 billion active parameters as a conservative lower bound. 2 Related works Benchmarks for style representations Evaluation of style representations has mostly focused on authorship tasks Wegmann et al. (2026). The longest-running effort in this space is the PAN shared task series at CLEF, which has run continuously since the late 2000s and describes itself as covering “digital text forensics and stylometry”.222https://pan.webis.de/ Early editions established cross-domain A and AV as standard tasks (Juola and Stamatatos, 2013; Kestemont et al., ; Stamatatos et al., 2014, 2015, 2018a), while later editions broadened to author profiling, style change detection, multi-author writing style analysis, hate-speech spreader profiling, and ATD (Ayele et al., 2024; Bevendorff et al., 2020, 2021, 2025a, 2026; Stamatatos et al., 2023). Most of the style embeddings we evaluate, evaluate on at least one PAN split. For AR, LUAR (Rivera Soto et al., 2021) introduces its own evaluation splits, which have since continued to be used (e.g., Man et al. (2026a)). Few evaluation approaches have used more theoretical definition-based understandings. For example, STEL (Wegmann and Nguyen, 2021) introduces a parallel-text ranking evaluation over a small set of style dimensions, and STEL-or-Content (Wegmann et al., 2022) extends it with a content-controlled variant that penalizes models reliant upon semantics. Patel et al. (2025) further extends STEL by prompting LLMs to generate synthetic instances across 4040 stylistic features. 3 The Style Text Embedding Benchmark STEB consists of 9696 datasets across 77 languages, organized into 55 evaluation tasks. In Section 3.1, we outline these tasks, define the metrics used for scoring each, and discuss how each serves to evaluate the quality of style embeddings. In § 3.2, we provide information about the included datasets and their properties. In § 3.3, we explain the operational and the definitional STEB score. In § 3.4, we mention principles that we considered, and should generally be considered, when extending STEB. 3.1 Tasks and evaluation See Tab. 12 for examples of all task types in STEB. Pair classification The goal of pair classification is to determine whether two input texts share the same label (e.g., same author in AV). To perform this, we embed both inputs and compute their cosine similarity, expecting high similarity between same-label pairs and low similarity otherwise. We implement two variants: (1) All-to-All, where every sample in the set is compared against every other sample providing an exhaustive evaluation, and (2) Predefined, where the pairs are restricted to a predefined list of pairs. This latter setting is critical for controlling for confounding variables (e.g., enforcing cross-topic pairs). We report the area under the receiver operating curve (AUROC). Clustering While semantic embeddings are optimized to cluster documents by topic, style embeddings should induce clusters corresponding to authorship (Andrews and Bishop, 2019), register (e.g., formality vs. informality) (Patel et al., 2025), and LM provenance (Rivera Soto et al., 2023). We evaluate whether style embeddings are capable of clustering text samples across style-relevant labels. Following the MTEB (Muennighoff et al., 2023) protocol, we use a mini-batch k-means algorithm with a batch size of 3232 and k equal to the number of ground-truth labels. We report the V-measure (Rosenberg and Hirschberg, 2007). Authorship retrieval AR has emerged as a primary application of style embeddings (Agarwal et al., 2025; Fincke and Boschee, 2024; Kim et al., 2025a; Man et al., 2026b; Rivera Soto et al., 2021). Unlike traditional information retrieval, which optimizes for semantic relevance between a query and a document, AR aims to identify other texts written by a specific author from within a large candidate pool. We use standard retrieval benchmarks where topic and authorship may correlate but are not the sole signal. We report the mean reciprocal rank (MRR). Order alignment Order alignment evaluates whether a style embedding ranks a set of texts in the same stylistic order as a reference set. In its simplest form, this corresponds to the STEL task introduced by Wegmann and Nguyen (2021), which uses parallel texts. A pair of texts has to be aligned to the order of another base pair of texts using stylistic information (see App. Fig. 2). Every order alignment task also has a distractor variant. Here, the unordered set includes one additional text that shares the same topic as the ordered texts but stylistically unrelated to any of them, a model that relies on semantic features more than style will mis-align it. In a setup with a one element ordered set, and with a two element unordered set including one distractor, this is equivalent to the STEL-or-Content task Wegmann et al. (2022). We report two accuracies, one for the original and one for the distractor variant. To achieve a high score in this task, an embedding must capture features other than semantics, and in the distractor variant it must ignore semantic similarity altogether. Probing The probing task assesses which linguistic features are encoded in an embedding by training a linear classifier on top of frozen representations (Conneau et al., 2018). We extract foundation-level linguistic features using the LFTK toolkit (Lee and Lee, 2023a), excluding features that may correlate with semantics (e.g., entity counts). Each continuous feature is discretized into five quantile bins, and the train/validation/test splits are balanced across the discretized labels. We then train a logistic regression probe on the frozen embeddings, selecting the L2 regularization strength from 10−5,10−4,10−3,10−2\10^-5,10^-4,10^-3,10^-2\ via a validation set, and report test accuracy averaged across all features. 3.2 STEB datasets An overview of all datasets can be found in § C. The added datasets can generally be grouped into 6 categories: those that target capabilities with respect to single linguistic features (7 datasets), authorial styles (58), dialectal styles (3), registers (12), genres (4), historical styles (3) and demographics (9). 3.3 STEB Scores With STEB, we evaluate models on the most common datasets and tasks that the field has built to study style. We aggregate the results differently for the operational score and the definitional score. Operational STEB Score While we do not align the operational score with a definition, we take care to not overweigh groups of correlated datasets. We automatically discover redundancies following Olmo et al. (2026) using the Spearman correlation between the model rankings on each dataset and apply hierarchical clustering (Ward linkage with a 0.50.5 distance threshold, cf. Ward Jr., 1963). Scores are macro-averaged within each and then across clusters. Definitional STEB Score To contrast to the operational view of style (cf. § 1), we also evaluate embeddings based on an a priori definition of style, namely the definition introduced in Wegmann et al. (2026). We evaluate whether embeddings (i) encode linguistic features, (i) capture the stylistic patterns of different objects of study (e.g., dialect, genre, idiolect), and (i) are more sensitive to stylistic than content information. See App. Tab. 13 for an overview of the dataset-to-cluster assignment. 3.4 Design principles We provide code and datasets with STEB that are meant to be extended by the community. We emphasize the principles on which we designed STEB and that we recommend the community to follow: Tasks where style matters Fundamentally, STEB targets scenarios where the semantics of a text is neither the sole nor strongest signal. Where possible, we enforce topic-controlled settings that penalize models that rely on any semantic shortcuts (e.g., Tab. 4). As such, each STEB dataset should benefit from representations that capture stylistic information and should be more difficult for models that heavily rely on semantic information. Length control To preserve ecological validity (de Vries et al., 2020), we evaluate models on the full text length inherent to each dataset. This approach preserves the natural length distributions between domains, for example, retaining the verbosity of blog posts versus the brevity of Reddit comments. For models with fixed context windows (e.g., 512512 tokens), we employ a chunk-and-pool strategy where inputs exceeding the limit are segmented in sentence-boundary-aware chunks. We embed each chunk independently and apply mean pooling to derive a single document representation (see Fig. 4 for a visual of the approach). Reproducibility and extensibility For reproducibility, all stochastic steps in evaluation (e.g., k-means initialization, probe training) use fixed random seeds. To facilitate community adoption, STEB is open-source and easy to extend. We welcome contributions from the community in adding new tasks and datasets. 4 Models We test non-neural (§ 4.1) as well as automatically-learned representations (§ 4.2). 4.1 Non-neural representations We test various representations using predefined, linguistically motivated features. Note that the strongest non-neural representation generally combines several of these features and tailors the set to the specific dataset or task Wegmann et al. (2026). Function word frequencies Function words (i.e., stop words) like determiners (the) and pronouns (you) are useful across text analyses, such as A Kestemont (2014); Mosteller and Wallace (1963), genre/text type classification Venglařová and Matlach (2024), and deception detection Liu et al. (2012). We count the frequencies of 390 function words and 69 function phrases (instead of), saving these counts as a vector per text Aggazzotti and Smith (2025). TF-IDF-weighted n-grams N-grams—sequences of n consecutive units (i.e., characters, tokens, part-of-speech (POS) tags)—have long been used for text analysis, especially for A Houvardas and Stamatatos (2006); Peng et al. (2003). Following others (e.g., Weerasinghe and Greenstadt, 2020), we weight the n-grams by their Term Frequency-Inverse Document Frequency (TF-IDF) and fit the vectorizer to a 10 billion token sample of FineWeb Penedo et al. (2024); see § D for more details.333Note that n-grams can capture topic as well as style signals, depending on the n-gram unit and their frequency. Stylometric features There is no agreed upon fixed set of best stylometric features Juola (2006); Nini (2023). There are perhaps equally as many feature extraction tools for various use cases. Due to its popularity in the NLP community, we use LFTK Lee and Lee (2023b) and combine its surface and POS features, which we found to perform best across datasets while remaining fast. NeuroBiber NeuroBiber Alkiek et al. (2025) is a transformer-based system that extracts 96 syntactic and lexical “Biber features”, such as function words, subordination types, and tenses Biber (1988); Biber and Conrad (2019). Model Clustering All-to-All Predefined Order Align. Retrieval Probing STEB score Num. Datasets 53 24 26 11 4 4 122 Style- specific LUAR-MUD 20.03 62.33 72.33 20.81 75.67 53.04 50.70 LUAR-CRUD 19.65 61.00 73.12 19.80 77.64 53.75 50.82 StyleDistance 18.46 59.82 66.21 44.02 49.61 57.87 49.33 mStyleDistance 7.35 53.83 53.71 31.38 20.04 45.96 35.38 MSR 16.67 64.34 71.01 21.85 63.49 51.01 48.06 CISR 15.28 59.71 66.43 33.81 45.87 52.54 45.61 STAR 25.92 64.56 73.28 23.93 66.96 49.86 50.75 LISA 12.74 60.91 62.40 16.48 42.96 46.38 40.31 Sem- antic all-mpnet-base-v2 8.48 60.41 63.87 14.62 48.99 45.64 40.34 E5-large-v2 11.06 60.20 64.70 15.82 49.91 49.03 41.79 Qwen3-Embedding-8B 2.67 51.61 52.79 23.11 35.17 42.96 34.72 MLM BERT-large-cased 12.02 60.99 67.11 21.57 58.88 55.16 45.96 RoBERTa-large 13.21 60.97 68.79 20.71 61.07 57.92 47.11 DeBERTa-v3-large 21.44 59.37 70.45 24.23 63.24 53.41 48.69 ModernBERT-large 16.58 60.19 65.38 20.56 52.17 54.06 44.82 CLM OPT-1.3B 3.83 52.40 55.59 24.08 39.07 45.82 36.80 Qwen3.5-4B-Base 4.23 52.80 57.41 24.88 37.15 43.02 36.58 Non- neural Function words 7.17 55.17 60.88 14.51 43.14 37.98 36.47 TFIDF n-grams 6.78 57.18 64.45 13.72 50.95 55.74 41.47 Stylometric 9.67 56.88 60.48 25.87 39.52 53.30 40.95 NeuroBiber 8.24 55.10 57.27 14.13 9.06 51.64 32.57 Table 2: Overall STEB (operational) results (×100× 100). Bold = best, underline = 2nd best per column. Metrics: V-measure (Clustering), AUC (All-to-All and Predefined Pair Classification), distractor accuracy (Order Alignment), MRR (Retrieval), and average accuracy (Probing). Scores are macro-averaged across automatically discovered dataset clusters within each task, then averaged across tasks to produce the overall STEB score. Num. Datasets = datasets contributing to each task; a dataset may contribute to more than one task. The full table is in App. Tab. 6. Object of Study Ling. Feat. Content-Ind. Avg. Model Genre Register Time Demo. Dialect Idiolect Avg. Order Al. Num. Datasets 4 7 3 3 3 29 49 7 11 67 Top 5 StyleDistance 62.84 40.01 46.21 33.67 56.91 60.03 49.94 74.02 44.23 56.06 StyleDist. Synth. 55.45 38.81 43.27 33.13 55.64 45.26 45.26 70.87 46.10 54.08 CISR 62.43 36.09 41.52 33.03 54.76 58.72 47.76 65.85 32.31 48.64 DeBERTa large 63.26 38.25 46.94 35.37 43.37 68.26 49.24 69.56 18.72 45.84 RoBERTa-base 63.79 42.52 45.94 32.84 40.16 59.83 47.51 77.95 10.27 45.24 Style emb. STAR 68.68 39.79 47.25 40.99 49.39 72.24 53.06 67.66 14.08 44.93 LUAR-MUD 66.67 38.72 42.14 35.53 44.30 76.11 50.58 70.50 11.29 44.12 mStyleDistance 49.41 32.83 35.04 32.25 42.83 37.23 38.27 56.17 38.32 44.25 LUAR-CRUD 65.80 38.14 41.30 35.32 41.52 76.99 49.85 70.37 10.57 43.59 MSR 66.39 39.08 41.98 37.05 41.52 68.69 49.12 66.52 12.35 42.66 LISA 56.89 37.95 37.38 32.95 27.19 54.00 41.06 64.02 7.10 37.39 Table 3: Top 5 models for STEB Score (definitional). Attributes are grouped into three clusters after Wegmann et al. (2026) and averaged: Object of Study (Genre, Style, Time, Demographics, Dialect, Idiolect), Linguistic Features, and Content-Independence. See App. Tab. 13 for selected datasets and App. Tab. 9 for full results. 4.2 Neural models We benchmark models with state-of-the-art results on various style and semantic embedding tasks, grouped into four families: style embeddings, semantic embeddings, MLMs, and CLMs. Style embeddings Most style embedding models we evaluate are trained contrastively, differing primarily in how they construct positive and negative pairs. LUAR (Rivera Soto et al., 2021) and STAR (Huertas-Tato et al., 2024) treat texts by the same author as positives and different authors as negatives, learning authorship representations from Reddit and various social media sources, respectively. CISR (Wegmann et al., 2022) further constrains positives to be same-author but different-topic pairs, encouraging content-independent style representations. StyleDistance (Patel et al., 2025) constructs synthetic hard-positives and hard-negatives designed to isolate stylistic correlations from topical ones. mStyleDistance (Qiu et al., 2025) extends this approach to the multilingual setting. MSR (Kim et al., 2025b) learns multilingual authorship embeddings using probabilistic content masking to suppress content features and language-aware batching to reduce cross-lingual easy-negatives. LISA (Patel et al., 2023) departs from purely contrastive training, learning interpretable style embeddings via a linear projection over an EncT5 encoder. Semantic embeddings These models are trained for general-purpose text similarity, primarily on semantic retrieval and NLI data. We include models like all-mpnet-base-v2 (Reimers and Gurevych, 2019) that perform well according to the semantic textual similarity evaluations and various models that perform(ed) well according to MTEB Muennighoff et al. (2023): Qwen3-Embedding-8B (Zhang et al., 2025), GTE (Li et al., 2023), E5 (Wang et al., 2024a), BGE (Xiao et al., 2023), and Jina Embeddings v3 (Sturua et al., 2025). Masked language models (MLMs) We include encoder-only transformers pre-trained with masked language modeling and without embedding-specific fine-tuning. Specifically, we include RoBERTa (Liu et al., 2019), DeBERTa-v3 (He et al., 2023), and ModernBERT (Warner et al., 2025). Since these models do not produce a single vector, we obtain embeddings by mean-pooling the token-level hidden states from the last layer, weighted by the attention mask. Causal language models (CLMs) Auto-regressive models are pretrained with next-token prediction and have no embedding-specific training. We include them to assess whether large-scale causal language modeling implicitly captures stylistic features. We include GPT-2 XL (Radford et al., 2019), OPT-1.3B (Zhang et al., 2022) and the Qwen family at multiple scales. We use the hidden state of the last non-padding token as the embedding, as it is the only position with full sequence context. 5 Results We discuss operational and definitional STEB results (§ 5.1) and provide a per-application (§ 5.2) and a multilingual (§ 5.3) analysis; all but the last are restricted to English. 5.1 Overall Results STEB (operational) results are in Tab. 2 and selected STEB (definitional) results in Tab. 3; full results for all 4040 models are in App. Tab. 6 and Tab. 9. Model ATD ATD (Adv.) Authorship Verification AR Avg. Overall Easy Medium Hard Top 5 deberta-v3-large 36.03 100.00 73.29 77.68 62.92 55.87 63.24 68.14 STAR 22.79 100.00 77.52 88.37 76.02 58.26 66.96 66.82 LUAR-MUD 28.20 75.01 76.54 86.84 73.62 61.10 75.67 63.85 deberta-v3-base 29.55 99.24 69.40 73.46 59.80 54.75 55.17 63.34 LUAR-CRUD 25.20 72.61 76.35 86.60 75.92 61.23 77.64 62.95 Style Embeddings StyleDistance 30.10 70.08 70.44 75.52 69.17 62.18 49.61 55.06 mStyleDistance 15.85 5.48 54.41 60.49 49.63 50.53 20.04 23.95 StyleDistance (Synthetic) 22.76 44.03 59.00 64.82 55.28 52.62 31.53 39.33 MSR 17.38 0.12 73.88 88.92 73.43 55.99 63.49 38.72 CISR 22.78 64.33 71.57 79.39 70.74 65.56 45.87 51.14 LISA 11.61 6.06 65.03 84.41 66.71 47.86 42.96 31.41 Table 4: Top 5 models across application clusters, plus the remaining style embeddings. ATD = AI-Text Detection, AR = Authorship Retrieval. Authorship Verification: Overall aggregates a superset of datasets (PAN13–15, PAN20–21, Enron, and the PAN22–26 style-change benchmarks), while Easy/Medium/Hard are computed over the PAN22–26 style-change subset only—they do not average to the Overall column. To avoid double-counting, only Overall contributes to Avg. Bold = best, underline = 2nd best per column (across all models). Each column reports the macro-averaged score (×100× 100) within the corresponding manual cluster. No single winner across task categories and style definitions There is no consistent winner across categories, for neither STEB (definitional) nor STEB (operational). While StyleDistance wins for the definitional, STAR, LUAR-CRUD, and LUAR-MUD score similarly for the operational score. STAR’s advantage is in clustering and predefined pair classification, while it falls behind on retrieval, where LUAR-CRUD leads by over 1010 points, and order alignment, where StyleDistance leads by nearly 1919 points. Models like STAR and LUAR might profit from content-entangled features for different objects of study that are punished more heavily for the definitional STEB score. These patterns confirm previous observations that style is conceptualized differently across the literature, and that different training and evaluation objectives—motivated by different style definitions—result in different strengths and weaknesses across downstream tasks (Wegmann et al., 2026). Style embeddings consistently outperform semantic embeddings. Every style-specific model except LISA and mStyleDistance outperform the best general-purpose semantic embedding on both STEB scores. This disparity is especially pronounced in order alignment, AR, and predefined pair classification. Semantic embeddings are optimized on objectives that favor matching texts primarily by semantic content, and the gap on STEB suggests style features are not well captured by semantic objectives alone. These findings underscore the need for a dedicated style benchmark, as semantic-oriented evaluation fails to capture the capabilities of style-specific models. MLM pre-training captures stylistic information. All MLMs we evaluate outperform every semantic embedding and every causal LM on both STEB scores, despite having no embedding-specific fine-tuning. DeBERTa-v3-large outperforms some style embeddings on retrieval and matches style-specific models on authorship verification, without any author-specific training (Tab. 2). RoBERTa leads over style embeddings in sensitivity to linguistic features (probing column of Tab. 2; linguistic-feature column of Tab. 3), suggesting that style-specific fine-tuning can come at the cost of linguistic feature sensitivity. Joint training on MLM and style objectives may help recover this sensitivity in style-specific models. The newer ModernBERT variants (only shown in App. Tab. 6 & Tab. 9) underperform, suggesting that more recent MLM recipes have shifted away from preserving linguistic feature information. Together, these results suggest that new MLM objectives are a promising direction for future research. Non-neural models are competitive on certain tasks. TFIDF n-grams is the strongest non-neural model, followed closely by Stylometric. Both generally outperform causal LMs and perform similarly to general-purpose sentence embedders, but lag behind style-specific and masked models. They perform best on tasks like probing and predefined classification, but degrade on tasks like clustering and retrieval that often require richer representations. Despite TFIDF n-grams being effectively a bag-of-words model, it nearly matches sophisticated sentence embedders. Also, although the literature has shown no general-purpose best set of stylometric features, using surface and POS tag features does fairly well across datasets and tasks.444Tailoring stylometric features to each dataset/task would likely produce even higher results, and we encourage testing specialized feature sets for more representative performance. CISR and StyleDistance models perform best at content independence. StyleDistance, CISR, and mStyleDistance perform the best at content-independence (i.e., the distractor variant of the order alignment tasks). This is expected as they were all trained similarly with hard negatives to improve content independence. Across models, though, there is still a lot of room for improvement on content-independence. STAR is most sensitive to style across stylistics attributes. STAR performs the best at recovering distinctive patterns across “objects of study” (e.g., genre, time, registers) beyond authorial styles. This might be influenced by its being one of few models that was trained on several different domains (i.e., social media, blogs, and books). Including more diverse training datasets (and potentially training tasks, cf. Wegmann et al., 2026) might thus be a promising direction for more generalizability. 5.2 Per-application analysis To answer questions such as “Which model is best for AI-text detection?”, we manually group datasets into four application-oriented clusters (Tab. 4): ATD, ATD where text has been modified to evade detection (Adv.), AV, and AR. For AV we additionally report three sub-columns over the PAN style-change benchmarks ordered by how content-entangled the pairs are: Easy, Medium, and Hard, where Hard pairs do not share any semantic content. The top five comprise three style-specific embeddings (STAR, LUAR-MUD, LUAR-CRUD) and two DeBERTa-v3 variants. In ATD, DeBERTa-v3-large555While striking, this result is explained by the fact that DeBERTav3 adds a replaced token detection objective, where a discriminator is trained to detect which tokens in a sequence were replaced by the MLM (He et al., 2023). This objective is none other than ATD in a pre-ChatGPT era (first DeBERTa version was published in 2021)! leads in the standard setting, while in the adversarial setting, STAR and DeBERTa-v3-large perform best. In AV and AR, the style-specific embedding models take the lead, suggesting that they are better suited for capturing the fine-grained nuances of everyday authors. The Easy/Medium/Hard AV columns reveal a finer pattern. On the Easy and Medium sets (where topic can be exploited), both STAR and LUAR perform well. However, in the Hard setting (where topic correlations cannot be exploited), CISR and StyleDistance perform best, suggesting that disentangling style from content matters for such applications. Model Pair Class. Retrieval Avg. Top 5 deberta-v3-large 74.03 47.77 60.90 StyleDistance 72.40 44.45 58.42 roberta-large 71.91 44.02 57.96 CISR 70.08 37.93 54.01 deberta-v3-base 71.85 35.42 53.63 Style embeddings LUAR-MUD 69.50 32.38 50.94 LUAR-CRUD 72.37 30.07 51.22 mStyleDistance 59.94 14.69 37.32 StyleDist. Synth. 64.03 19.46 41.74 MSR 69.43 28.38 48.90 STAR 71.32 35.80 53.56 LISA 62.67 26.70 44.68 Table 5: Top 5 multilingual STEB results (×100× 100). Pair Class. averages AUC across 13 PAN13/14/15 AV datasets (Dutch, Greek, Spanish); Retrieval averages MRR across 4 PAN18 cross-domain A datasets (French, Italian, Polish, Spanish). Bold = best, underline = 2nd best per column (across all models). 5.3 Multilingual analysis STEB includes non-English datasets across six languages (Dutch, French, Greek, Italian, Polish, Spanish) drawn from the PAN13/14/15 AV shared tasks and the PAN18 cross-domain A shared task. These datasets are excluded from the main STEB scores (Tab. 2 and Tab. 3) as most evaluated models are monolingual English encoders. Here, we investigate whether multilingual style embeddings perform better than monolingual English style embeddings on these multilingual applications. We derive a single score by averaging across all dataset metrics and across the independent tasks (pair classification, retrieval). Multilingual style-representations underperform their English counterparts Tab. 5 shows that the top-5 by multilingual average comprises two DeBERTa-v3 variants, RoBERTa-large, and two style embeddings (StyleDistance, CISR). The explicitly multilingual style embeddings (MSR, mStyleDistance) do not appear in the top five. These results suggest that current multilingual training strategies are not yet effective, and as such, they remain a challenge for future work.666To stress-test this conclusion, we reproduce Kim et al. (2025b)’s multilingual evaluation setup on the same PAN13/14/15 multilingual AV datasets and style embeddings. We find that STEB’s default chunk-and-pool strategy for handling large texts is the main driver. Under per-document fixed truncation, MSR achieves the top score, while with the chunk-and-pool strategy, StyleDistance and CISR perform best. These results underscore the necessity of shared evaluation protocols, since without them our conclusions would differ materially. See § F for the full comparison. The two style embeddings that do appear are notable for having been trained so as to separate semantic content from style, suggesting that this separation might be a useful strategy for the generalizability of style embeddings across languages. 6 Conclusion We introduced STEB, a benchmark for evaluating embeddings across style-related NLP tasks, spanning 55 tasks, 9696 datasets, and 77 languages. Across 4040 models, we find that no single embedding is best. Style embeddings that retain semantic signal lead on authorship-related applications where author and topic correlate, while content-disentangled style embeddings lead on content-controlled applications where topic shortcuts are removed. On ATD, MLMs outperform style-specific embeddings, and on predicting second-order style features (e.g., genre, dialect, register, time), the authorship-focused STAR model leads. Multilingual style embeddings underperform their English counterparts on the multilingual subset of STEB, highlighting cross-lingual style as an open problem. We release STEB to enable consistent, reproducible comparisons and to lower the cost of measuring progress on style. We welcome contributions from the community in improving the benchmark. Limitations Leakage of datasets / contamination Style embeddings are commonly trained and tested on similar datasets, often using datasets for training and testing with differing splits between publications. Representations might be tested on some data that they have seen during training. For example, the StyleDistance models were trained on one of the seven “linguistic features” datasets and on two of the eleven “Content Independence” datasets in Tab. 3. Excluding these shrinks the difference to CISR to ≈ 2 percentage points on content independence and make the synthetic version and CISR have almost the same overall performance. However, the overall ranking of the top models is preserved. Bounded by dataset coverage Both the empirical clustering in §5.1 and the attribute clustering in §5.1 are bounded by what existing datasets expose. In particular, most of the field’s emphasis has been on authorship-related tasks, and as such other axes of analysis remain underexplored. We call on the community to create datasets that explore aspects other than authorship and to expand STEB with these additions. Languages other than English remains underrepresented It is a fact of the field that not as many non-English datasets are available. As such, we inherit this limitation. Moreover, only recently has there been an effort to extend style embedding models to multilingual settings (Kim et al., 2025a; Qiu et al., 2025). References Agarwal et al. (2025) Shantanu Agarwal, Joel Barry, Steven Fincke, and Scott Miller. 2025. Cross-Genre Authorship Attribution via LLM-Based Retrieve-and-Rerank. arXiv preprint. ArXiv:2510.16819 [cs]. Aggazzotti and Smith (2025) Cristina Aggazzotti and Elizabeth Allyn Smith. 2025. A stylometric analysis of speaker attribution from speech transcripts. Preprint, arXiv:2512.13667. Alkiek et al. (2025) Kenan Alkiek, Anna Wegmann, Jian Zhu, and David Jurgens. 2025. Neurobiber: Fast and interpretable stylistic feature extraction. arXiv preprint. ArXiv:2502.18590 [cs]. Almeida et al. (2011) Tiago A. Almeida, José María G. Hidalgo, and Akebo Yamakami. 2011. Contributions to the study of SMS spam filtering: new collection and results. In Proceedings of the 11th ACM symposium on Document engineering, DocEng ’11, pages 259–262, New York, NY, USA. Association for Computing Machinery. Alva-Manchego et al. (2020) Fernando Alva-Manchego, Louis Martin, Antoine Bordes, Carolina Scarton, Benoît Sagot, and Lucia Specia. 2020. ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4668–4679, Online. Association for Computational Linguistics. Andrews and Bishop (2019) Nicholas Andrews and Marcus Bishop. 2019. Learning invariant representations of social media users. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1684–1695, Hong Kong, China. Association for Computational Linguistics. Ayele et al. (2024) Abinew Ali Ayele, Nikolay Babakov, Janek Bevendorff, Xavier Bonet Casals, Berta Chulvi, Daryna Dementieva, Ashaf Elnagar, Dayne Freitag, Maik Fröbe, Damir Korenčić, Maximilian Mayerl, Daniil Moskovskiy, Animesh Mukherjee, Alexander Panchenko, Martin Potthast, Francisco Rangel, Naquee Rizwan, Paolo Rosso, Florian Schneider, Alisa Smirnova, Efstathios Stamatatos, Elisei Stakovskii, Benno Stein, Mariona Taulé, Dmitry Ustalov, Xintong Wang, Matti Wiegmann, Seid Muhie Yimam, and Eva Zangerle. 2024. Overview of PAN 2024: Multi-author writing style analysis, multilingual text detoxification, oppositional thinking analysis, and generative ai authorship verification condensed lab overview. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 15th International Conference of the CLEF Association, CLEF 2024, Grenoble, France, September 9–12, 2024, Proceedings, Part I, page 231–259, Berlin, Heidelberg. Springer-Verlag. Bai et al. (2023) Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. 2023. Qwen technical report. Preprint, arXiv:2309.16609. Barbieri et al. (2020) Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1644–1650, Online. Association for Computational Linguistics. BehnamGhader et al. (2024) Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2vec: Large language models are secretly powerful text encoders. In First Conference on Language Modeling. Bevendorff et al. (2023) Janek Bevendorff, Ian Borrego-Obrador, Mara Chinea-Ríos, Marc Franco-Salvador, Maik Fröbe, Annina Heini, Krzysztof Kredens, Maximilian Mayerl, Piotr Pęzik, Martin Potthast, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, Benno Stein, Matti Wiegmann, Magdalena Wolska, and Eva Zangerle. 2023. Overview of PAN 2023: Authorship Verification, Multi-Author Writing Style Analysis, Profiling Cryptocurrency Influencers, and Trigger Detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 459–481. Springer, Cham. Bevendorff et al. (2021) Janek Bevendorff, Berta Chulvi, Gretel Liz De La Peña Sarracén, Mike Kestemont, Enrique Manjavacas, Ilia Markov, Maximilian Mayerl, Martin Potthast, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, Benno Stein, Matti Wiegmann, Magdalena Wolska, and Eva Zangerle. 2021. Overview of PAN 2021: Authorship verification, profiling hate speech spreaders on twitter, and style change detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 12th International Conference of the CLEF Association, CLEF 2021, Virtual Event, September 21–24, 2021, Proceedings, page 419–431, Berlin, Heidelberg. Springer-Verlag. Bevendorff et al. (2022) Janek Bevendorff, Berta Chulvi, Elisabetta Fersini, Annina Heini, Mike Kestemont, Krzysztof Kredens, Maximilian Mayerl, Reynier Ortega-Bueno, Piotr Pęzik, Martin Potthast, Francisco Rangel, Paolo Rosso, Efstathios Stamatatos, Benno Stein, Matti Wiegmann, Magdalena Wolska, and Eva Zangerle. 2022. Overview of PAN 2022: Authorship Verification, Profiling Irony and Stereotype Spreaders, and Style Change Detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 382–394. Springer, Cham. Bevendorff et al. (2025a) Janek Bevendorff, Daryna Dementieva, Maik Fröbe, Bela Gipp, André Greiner-Petter, Jussi Karlgren, Maximilian Mayerl, Preslav Nakov, Alexander Panchenko, Martin Potthast, Artem Shelmanov, Efstathios Stamatatos, Benno Stein, Yuxia Wang, Matti Wiegmann, and Eva Zangerle. 2025a. Overview of PAN 2025: Voight-kampff generative ai detection, multilingual text detoxification, multi-author writing style analysis, and generative plagiarism detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 16th International Conference of the CLEF Association, CLEF 2025, Madrid, Spain, September 9–12, 2025, Proceedings, page 388–411, Berlin, Heidelberg. Springer-Verlag. Bevendorff et al. (2025b) Janek Bevendorff, Daryna Dementieva, Maik Fröbe, Bela Gipp, André Greiner-Petter, Jussi Karlgren, Maximilian Mayerl, Preslav Nakov, Alexander Panchenko, Martin Potthast, Artem Shelmanov, Efstathios Stamatatos, Benno Stein, Yuxia Wang, Matti Wiegmann, and Eva Zangerle. 2025b. Overview of PAN 2025: Generative AI detection, multilingual text detoxification, multi-author writing style analysis, and generative plagiarism detection. In Advances in Information Retrieval, pages 434–441. Springer, Cham. Bevendorff et al. (2026) Janek Bevendorff, Maik Fröbe, André Greiner-Petter, Andreas Jakoby, Maximilian Mayerl, Preslav Nakov, Henry Plutz, Martin Potthast, Benno Stein, Minh Ngoc Ta, Yuxia Wang, and Eva Zangerle. 2026. Overview of PAN 2026: Voight-kampff generative ai detection, text watermarking, multi-author writing style analysis, generative plagiarism detection, and reasoning trajectory detection. Preprint, arXiv:2602.09147. Bevendorff et al. (2020) Janek Bevendorff, Bilal Ghanem, Anastasia Giachanou, Mike Kestemont, Enrique Manjavacas, Ilia Markov, Maximilian Mayerl, Martin Potthast, Francisco Rangel, Paolo Rosso, Günther Specht, Efstathios Stamatatos, Benno Stein, Matti Wiegmann, and Eva Zangerle. 2020. Overview of PAN 2020: Authorship verification, celebrity profiling, profiling fake news spreaders on twitter, and style change detection. In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22–25, 2020, Proceedings, page 372–383, Berlin, Heidelberg. Springer-Verlag. Biber (1988) Douglas Biber. 1988. Variation across Speech and writing. New York, NY: Cambridge University Press. Biber and Conrad (2019) Douglas Biber and Susan Conrad. 2019. Register, Genre, and Style, 2nd edition. Cambridge University Press, Cambridge, UK. Carlson et al. (2018) Keith Carlson, Allen Riddell, and Daniel Rockmore. 2018. Evaluating prose style transfer with the Bible. Royal Society Open Science, 5(10):171920. Conneau et al. (2018) Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, and Marco Baroni. 2018. What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2126–2136, Melbourne, Australia. Association for Computational Linguistics. Danescu-Niculescu-Mizil et al. (2013) Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250–259, Sofia, Bulgaria. Association for Computational Linguistics. de Vries et al. (2020) Harm de Vries, Dzmitry Bahdanau, and Christopher D. Manning. 2020. Towards ecologically valid research on language user interfaces. ArXiv, abs/2007.14435. Enevoldsen et al. (2025) Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, Ömer Veysel Çağatan, Akash Kundu, Martin Bernstorff, Shitao Xiao, Akshita Sukhlecha, Bhavish Pahwa, Rafał Poświata, Kranthi Kiran Gv, Shawon Ashraf, Daniel Auras, Björn Plüster, Jan Philipp Harries, Loïc Magne, Isabelle Mohr, Dawei Zhu, Hippolyte Gisserot-Boukhlef, Tom Aarsen, Jan Kostkan, Konrad Wojtasik, Taemin Lee, Marek Suppa, Crystina Zhang, Roberta Rocca, Mohammed Hamdy, Andrianos Michail, John Yang, Manuel Faysse, Aleksei Vatolin, Nandan Thakur, Manan Dey, Dipam Vasani, Pranjal A. Chitale, Simone Tedeschi, Nguyen Tai, Artem Snegirev, Mariya Hendriksen, Michael Günther, Mengzhou Xia, Weijia Shi, Xing Han Lù, Jordan Clive, Gayatri K, Maksimova Anna, Silvan Wehrli, Maria Tikhonova, Henil Shalin Panchal, Aleksandr Abramov, Malte Ostendorff, Zheng Liu, Simon Clematide, Lester James Validad Miranda, Alena Fenogenova, Guangyu Song, Ruqiya Bin Safi, Wen-Ding Li, Alessia Borghini, Federico Cassano, Lasse Hansen, Sara Hooker, Chenghao Xiao, Vaibhav Adlakha, Orion Weller, Siva Reddy, and Niklas Muennighoff. 2025. MMTEB: Massive Multilingual Text Embedding Benchmark. In The Thirteenth International Conference on Learning Representations (ICLR). Fincke and Boschee (2024) Steven Fincke and Elizabeth Boschee. 2024. Separating Style from Substance: Enhancing Cross-Genre Authorship Attribution through Data Selection and Presentation. arXiv preprint. ArXiv:2408.05192 [cs]. Gao et al. (2024) Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint. ArXiv:2312.10997 [cs]. Gehring and Paaßen (2025) Lukas Gehring and Benjamin Paaßen. 2025. Assessing LLM text detection in educational contexts: Does human contribution affect detection? Preprint, arXiv:2508.08096. Groenwold et al. (2020) Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating African-American Vernacular English in Transformer-Based Text Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5877–5883, Online. Association for Computational Linguistics. Gupta et al. (2025) Abhay Gupta, Jacob Cheung, Philip Meng, Shayan Sayyed, Kevin Zhu, Austen Liao, and Sean O’Brien. 2025. EnDive: A Cross-Dialect Benchmark for Fairness and Performance in Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 16830–16855, Suzhou, China. Association for Computational Linguistics. Halvani (2017) Oren Halvani. 2017. Enron authorship verification corpus. https://data.mendeley.com/datasets/n77w7mygwg/1. He et al. (2023) Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. DeBERTav3: Improving deBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations. Horvitz et al. (2024a) Zachary Horvitz, Ajay Patel, Chris Callison-Burch, Zhou Yu, and Kathleen McKeown. 2024a. ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):18216–18224. Horvitz et al. (2024b) Zachary Horvitz, Ajay Patel, Kanishk Singh, Chris Callison-Burch, Kathleen McKeown, and Zhou Yu. 2024b. TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 13376–13390, Miami, Florida, USA. Association for Computational Linguistics. Houvardas and Stamatatos (2006) John Houvardas and Efstathios Stamatatos. 2006. N-gram feature selection for authorship identification. In Proceedings of the 12th International Conference on Artificial Intelligence: Methodology, Systems, and Applications, AIMSA’06, page 77–86, Berlin, Germany. Springer. Huang et al. (2024) Baixiang Huang, Canyu Chen, and Kai Shu. 2024. Can Large Language Models Identify Authorship? In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 445–460, Miami, Florida, USA. Association for Computational Linguistics. Huertas-Tato et al. (2024) Javier Huertas-Tato, Alejandro Martín, and David Camacho. 2024. Understanding writing style in social media with a supervised contrastively pre-trained transformer. Know.-Based Syst., 296(C). Ide et al. (2008) Nancy Ide, Collin Baker, Christiane Fellbaum, Charles Fillmore, and Rebecca Passonneau. 2008. MASC: the Manually Annotated Sub-Corpus of American English. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), Marrakech, Morocco. European Language Resources Association (ELRA). Ide et al. (2013) Nancy Ide, Keith Suderman, Collin Baker, Rebecca Passonneau, and Christiane Fellbaum. 2013. Manually annotated sub-corpus third release. Web Download. Juola (2006) Patrick Juola. 2006. Authorship attribution. Foundations and Trends in Information Retrieval, 1(3):233–334. Juola and Stamatatos (2013) Patrick Juola and Efstathios Stamatatos. 2013. Overview of the author identification task at PAN 2013. In Conference and Labs of the Evaluation Forum. Kang et al. (2019) Dongyeop Kang, Varun Gangal, and Eduard Hovy. 2019. (Male, Bachelor) and (Female, Ph.D) have different connotations: Parallelly Annotated Stylistic Language Dataset with Multiple Personas. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1696–1706, Hong Kong, China. Association for Computational Linguistics. Kestemont (2014) Mike Kestemont. 2014. Function words in authorship attribution. From black magic to theory? In Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLFL), pages 59–66, Gothenburg, Sweden. Association for Computational Linguistics. (43) Mike Kestemont, Michael Tschuggnall, Efstathios Stamatatos, Walter Daelemans, Günther Specht, Benno Stein, and Martin Potthast. Overview of the Author Identification Task at PAN-2018. Khan et al. (2021) Aleem Khan, Elizabeth Fleming, Noah Schofield, Marcus Bishop, and Nicholas Andrews. 2021. A Deep Metric Learning Approach to Account Linking. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5275–5287, Online. Association for Computational Linguistics. Khan et al. (2024) Aleem Khan, Andrew Wang, Sophia Hager, and Nicholas Andrews. 2024. Learning to generate text in arbitrary writing styles. Preprint, arXiv:2312.17242. Kim et al. (2025a) Junghwan Kim, Haotian Zhang, and David Jurgens. 2025a. Leveraging multilingual training for authorship representation: Enhancing generalization across languages and domains. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 34855–34880, Suzhou, China. Association for Computational Linguistics. Kim et al. (2025b) Junghwan Kim, Haotian Zhang, and David Jurgens. 2025b. Leveraging multilingual training for authorship representation: Enhancing generalization across languages and domains. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 34867–34892, Suzhou, China. Association for Computational Linguistics. Kortmann et al. (2020) Bernd Kortmann, Kerstin Lunkenheimer, and Katharina Ehret. 2020. The electronic world atlas of varieties of English. Available online at https://ewave-atlas.org/. Krishna et al. (2020) Kalpesh Krishna, John Wieting, and Mohit Iyyer. 2020. Reformulating Unsupervised Style Transfer as Paraphrase Generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 737–762, Online. Association for Computational Linguistics. Kuzman et al. (2023) Taja Kuzman, Igor Mozetič, and Nikola Ljubešić. 2023. Automatic Genre Identification for Robust Enrichment of Massive Text Collections: Investigation of Classification Methods in the Era of Large Language Models. Machine Learning and Knowledge Extraction, 5(3):1149–1175. Laippala et al. (2023) Veronika Laippala, Samuel Rönnqvist, Miika Oinonen, Aki-Juhani Kyröläinen, Anna Salmela, Douglas Biber, Jesse Egbert, and Sampo Pyysalo. 2023. Register identification from the unrestricted open Web using the Corpus of Online Registers of English. Language Resources and Evaluation, 57(3):1045–1079. Lee and Lee (2023a) Bruce W. Lee and Jason Lee. 2023a. LFTK: Handcrafted features in computational linguistics. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 1–19, Toronto, Canada. Association for Computational Linguistics. Lee and Lee (2023b) Bruce W. Lee and Jason Lee. 2023b. LFTK: Handcrafted features in computational linguistics. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 1–19, Toronto, Canada. Association for Computational Linguistics. Lee et al. (2025) Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. NV-embed: Improved techniques for training LLMs as generalist embedding models. In The Thirteenth International Conference on Learning Representations. Li et al. (2024) Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. MAGE: Machine-Generated text detection in the wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 36–53. Li et al. (2023) Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. Preprint, arXiv:2308.03281. Liu et al. (2012) Xiong Liu, Jeffrey Hancock, Guangfan Zhang, Roger Xu, David Markowitz, and Natalya Bazarova. 2012. Exploring linguistic features for deception detection in unstructured text. In Hawaii International Conference on System Sciences. Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692. Lyu et al. (2021) Yiwei Lyu, Paul Pu Liang, Hai Pham, Eduard Hovy, Barnabás Póczos, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2021. StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2116–2138, Online. Association for Computational Linguistics. Man et al. (2026a) Hieu Man, Van-Cuong Pham, Nghia Trung Ngo, Franck Dernoncourt, and Thien Huu Nguyen. 2026a. Explainable disentangled representation learning for generalizable authorship attribution in the era of generative ai. Preprint, arXiv:2604.21300. Man et al. (2026b) Hieu Man, Van-Cuong Pham, Nghia Trung Ngo, Franck Dernoncourt, and Thien Huu Nguyen. 2026b. Explainable Disentangled Representation Learning for Generalizable Authorship Attribution in the Era of Generative AI. arXiv preprint. ArXiv:2604.21300 [cs]. Mosteller and Wallace (1963) Frederick Mosteller and David L. Wallace. 1963. Inference in an authorship problem: A comparative study of discrimination methods applied to the authorship of the disputed Federalist Papers. Journal of the American Statistical Association, 58(302):275–309. Muennighoff et al. (2023) Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistics. Nevalainen et al. (2021) Terttu Nevalainen, Helena Raumolin-Brunberg, Samuli Kaislaniemi, Mikko Laitinen, Minna Nevala, Arja Nurmi, Minna Palander-Collin, Tanja Säily, and Anni Sairio. 2021. CEECES 1 = Corpus of Early English Correspondence Extension Sampler part 1. Version 2. XML conversion and encoding by Lassi Saario. Nevalainen et al. (2022) Terttu Nevalainen, Helena Raumolin-Brunberg, Samuli Kaislaniemi, Mikko Laitinen, Minna Nevala, Arja Nurmi, Minna Palander-Collin, Tanja Säily, and Anni Sairio. 2022. CEECES 2 = Corpus of Early English Correspondence Extension Sampler part 2. XML conversion and encoding by Lassi Saario. Ni et al. (2019) Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 188–197, Hong Kong, China. Association for Computational Linguistics. Nini (2023) Andrea Nini. 2023. A Theory of Linguistic Individuality for Authorship Analysis. Elements in Forensic Linguistics. Cambridge University Press. Olmo et al. (2026) Team Olmo et al. 2026. Olmo 3. Preprint, arXiv:2512.13961. Patel et al. (2023) Ajay Patel, Delip Rao, Ansh Kothary, Kathleen McKeown, and Chris Callison-Burch. 2023. Learning interpretable style embeddings via prompting LLMs. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15270–15290, Singapore. Association for Computational Linguistics. Patel et al. (2025) Ajay Patel, Jiacheng Zhu, Justin Qiu, Zachary Horvitz, Marianna Apidianaki, Kathleen McKeown, and Chris Callison-Burch. 2025. StyleDistance: Stronger content-independent style embeddings with synthetic parallel examples. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8662–8685, Albuquerque, New Mexico. Association for Computational Linguistics. Penedo et al. (2024) Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. 2024. The FineWeb Datasets: Decanting the web for the finest text data at scale. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Peng et al. (2003) Fuchun Peng, Dale Schuurmans, Shaojun Wang, and Vlado Keselj. 2003. Language independent authorship attribution using character level language models. In Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics - Volume 1, EACL ’03, page 267–274, USA. Association for Computational Linguistics. Qiu et al. (2025) Justin Qiu, Jiacheng Zhu, Ajay Patel, Marianna Apidianaki, and Chris Callison-Burch. 2025. mStyleDistance: Multilingual Style Embeddings and their Evaluation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16917–16931, Vienna, Austria. Association for Computational Linguistics. Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Ram et al. (2023) Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-Context Retrieval-Augmented Language Models. Transactions of the Association for Computational Linguistics, 11:1316–1331. Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. Rivera Soto et al. (2023) Rafael Rivera Soto, Kailin Koch, Aleem Khan, Barry Y. Chen, Marcus Bishop, and Nicholas Andrews. 2023. Few-shot detection of machine-generated text using style representations. In The Twelfth International Conference on Learning Representations (ICLR). Rivera Soto et al. (2021) Rafael Rivera Soto, Olivia Elizabeth Miano, Juanita Ordonez, Barry Y. Chen, Aleem Khan, Marcus Bishop, and Nicholas Andrews. 2021. Learning universal authorship representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 913–919, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Rosenberg and Hirschberg (2007) Andrew Rosenberg and Julia Hirschberg. 2007. V-measure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 410–420, Prague, Czech Republic. Association for Computational Linguistics. Schler et al. (2006) Jonathan Schler, Moshe Koppel, Shlomo Argamon, and James Pennebaker. 2006. Effects of Age and Gender on Blogging. pages 199–205, Stanford, USA. Sharoff (2018) Serge Sharoff. 2018. Functional Text Dimensions for the annotation of web corpora. Corpora, 13(1):65–95. Stamatatos et al. (2014) Efstathios Stamatatos, Walter Daelemans, Ben Verhoeven, Martin Potthast, Benno Stein, Patrick Juola, Miguel A Sanchez-Perez, Alberto Barrón-Cedeño, et al. 2014. Overview of the author identification task at PAN 2014. In CEUR workshop proceedings, volume 1180, pages 877–897. CEUR-WS. Stamatatos et al. (2023) Efstathios Stamatatos, Krzysztof Kredens, Piotr Pezik, Annina Heini, Janek Bevendorff, Benno Stein, and Martin Potthast. 2023. Overview of the authorship verification task at PAN 2023. In CLEF 2023: Conference and Labs of the Evaluation Forum. Stamatatos et al. (2015) Efstathios Stamatatos, Martin Potthast, Francisco Rangel, Paolo Rosso, and Benno Stein. 2015. Overview of the PAN/CLEF 2015 Evaluation Lab. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 518–538. Springer, Cham. Stamatatos et al. (2018a) Efstathios Stamatatos, Francisco Rangel, Michael Tschuggnall, Benno Stein, Mike Kestemont, Paolo Rosso, and Martin Potthast. 2018a. Overview of PAN 2018. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 267–285. Springer, Cham. Stamatatos et al. (2018b) Efstathios Stamatatos, Francisco Rangel, Michael Tschuggnall, Benno Stein, Mike Kestemont, Paolo Rosso, and Martin Potthast. 2018b. Overview of pan 2018: Author identification, author profiling, and author obfuscation. In International conference of the cross-language evaluation forum for european languages, pages 267–285. Springer. Sturua et al. (2025) Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao. 2025. Jina embeddings v3: Multilingual text encoder with low-rank adaptations. In Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6–10, 2025, Proceedings, Part V, page 123–129, Berlin, Heidelberg. Springer-Verlag. Vajjala and Lučić (2018) Sowmya Vajjala and Ivana Lučić. 2018. OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 297–304, New Orleans, Louisiana. Association for Computational Linguistics. Venglařová and Matlach (2024) Klára Venglařová and Vladimír Matlach. 2024. Beyond content: discriminatory power of function words in text type classification. Digital Scholarship in the Humanities, 39(2):765–789. Vera et al. (2025) Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, Daniel Cer, Alice Lisak, Min Choi, Lucas Gonzalez, Omar Sanseviero, Glenn Cameron, Ian Ballantyne, Kat Black, Kaifeng Chen, Weiyi Wang, Zhe Li, Gus Martins, Jinhyuk Lee, Mark Sherwood, Juyeong Ji, Renjie Wu, Jingxiao Zheng, Jyotinder Singh, Abheesht Sharma, Divyashree Sreepathihalli, Aashi Jain, Adham Elarabawy, AJ Co, Andreas Doumanoglou, Babak Samari, Ben Hora, Brian Potetz, Dahun Kim, Enrique Alfonseca, Fedor Moiseev, Feng Han, Frank Palma Gomez, Gustavo Hernández Ábrego, Hesen Zhang, Hui Hui, Jay Han, Karan Gill, Ke Chen, Koert Chen, Madhuri Shanbhogue, Michael Boratko, Paul Suganthan, Sai Meher Karthik Duddu, Sandeep Mariserla, Setareh Ariafar, Shanfeng Zhang, Shijie Zhang, Simon Baumgartner, Sonam Goenka, Steve Qiu, Tanmaya Dabral, Trevor Walker, Vikram Rao, Waleed Khawaja, Wenlei Zhou, Xiaoqi Ren, Ye Xia, Yichang Chen, Yi-Ting Chen, Zhe Dong, Zhongli Ding, Francesco Visin, Gaël Liu, Jiageng Zhang, Kathleen Kenealy, Michelle Casbon, Ravin Kumar, Thomas Mesnard, Zach Gleicher, Cormac Brick, Olivier Lacombe, Adam Roberts, Qin Yin, Yunhsuan Sung, Raphael Hoffmann, Tris Warkentin, Armand Joulin, Tom Duerig, and Mojtaba Seyedhosseini. 2025. Embeddinggemma: Powerful and lightweight text representations. Preprint, arXiv:2509.20354. Wang et al. (2024a) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024a. Text embeddings by weakly-supervised contrastive pre-training. Preprint, arXiv:2212.03533. Wang et al. (2024b) Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024b. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1369–1407, St. Julian’s, Malta. Association for Computational Linguistics. Ward Jr. (1963) Joe H. Ward Jr. 1963. Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301):236–244. Warner et al. (2025) Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and Iacopo Poli. 2025. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2526–2547, Vienna, Austria. Association for Computational Linguistics. Weerasinghe and Greenstadt (2020) Janith Weerasinghe and Rachel Greenstadt. 2020. Feature vector difference based neural network and logistic regression models for authorship verification. Notebook for PAN at CLEF. Wegmann et al. (2026) Anna Wegmann, Cristina Aggazzotti, Rafael Rivera Soto, and Dong Nguyen. 2026. A Survey on Representing Linguistic Style: Challenges and Opportunities. Wegmann and Nguyen (2021) Anna Wegmann and Dong Nguyen. 2021. Does it capture STEL? a modular, similarity-based linguistic style evaluation framework. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7109–7130, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Wegmann et al. (2022) Anna Wegmann, Marijn Schraagen, and Dong Nguyen. 2022. Same author or just same topic? towards content-independent style representations. In Proceedings of the 7th Workshop on Representation Learning for NLP, pages 249–268, Dublin, Ireland. Association for Computational Linguistics. Wu et al. (2024) Junchao Wu, Runzhe Zhan, Derek F Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S Chao. 2024. Detectrl: Benchmarking llm-generated text detection in real-world scenarios. Advances in Neural Information Processing Systems, 37:100369–100401. Xiao et al. (2023) Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. Preprint, arXiv:2309.07597. Xu et al. (2012) Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for Style. In Proceedings of COLING 2012, pages 2899–2914, Mumbai, India. The COLING 2012 Organizing Committee. Yannakoudakis et al. (2011) Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A New Dataset and Method for Automatically Grading ESOL Texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180–189, Portland, Oregon, USA. Association for Computational Linguistics. Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: Open Pre-trained Transformer language models. Preprint, arXiv:2205.01068. Zhang et al. (2025) Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Appendix Appendix A Full aggregate results The full tables of results for the experiments in § 5.1, § 5.2, and § 5.3 are shown in Tab. 6, Tab. 7, and Tab. 8, respectively. Model Clustering All-to-All Predefined Order Align. Retrieval Probing STEB score Num. Datasets 53 24 26 11 4 4 122 Style specific LUAR-MUD 20.03 62.33 72.33 20.81 75.67 53.04 50.70 LUAR-CRUD 19.65 61.00 73.12 19.80 77.64 53.75 50.82 StyleDistance 18.46 59.82 66.21 44.02 49.61 57.87 49.33 StyleDistance (Synthetic) 13.46 57.02 57.67 42.76 31.53 56.69 43.19 mStyleDistance 7.35 53.83 53.71 31.38 20.04 45.96 35.38 MSR 16.67 64.34 71.01 21.85 63.49 51.01 48.06 CISR 15.28 59.71 66.43 33.81 45.87 52.54 45.61 STAR 25.92 64.56 73.28 23.93 66.96 49.86 50.75 LISA 12.74 60.91 62.40 16.48 42.96 46.38 40.31 Semantic embedders all-mpnet-base-v2 8.48 60.41 63.87 14.62 48.99 45.64 40.34 GTE-base-en-v1.5 10.76 60.04 60.33 15.62 49.36 50.64 41.13 GTE-large-en-v1.5 11.48 60.99 62.39 16.20 49.60 48.87 41.59 E5-base-v2 9.28 60.40 63.42 15.26 49.62 52.13 41.69 E5-large-v2 11.06 60.20 64.70 15.82 49.91 49.03 41.79 BGE-base-en-v1.5 7.93 59.40 62.97 15.82 51.44 49.45 41.17 BGE-large-en-v1.5 7.15 59.84 63.81 16.23 48.79 47.23 40.51 Jina Embeddings v3 7.90 60.58 64.60 17.11 47.35 45.62 40.53 Qwen3-Embedding-8B 2.67 51.61 52.79 23.11 35.17 42.96 34.72 Masked language models BERT-large-cased 12.02 60.99 67.11 21.57 58.88 55.16 45.96 BERT-large-uncased 11.18 60.44 66.92 21.18 60.23 52.48 45.41 RoBERTa-base 12.59 60.97 66.49 22.47 50.63 61.32 45.74 RoBERTa-large 13.21 60.97 68.79 20.71 61.07 57.92 47.11 DeBERTa-v3-base 19.42 58.35 66.82 23.41 55.17 51.73 45.81 DeBERTa-v3-large 21.44 59.37 70.45 24.23 63.24 53.41 48.69 ModernBERT-base 14.05 59.28 64.13 19.20 51.25 53.91 43.63 ModernBERT-large 16.58 60.19 65.38 20.56 52.17 54.06 44.82 Causal language models GPT-2 XL 4.42 51.68 53.49 26.32 28.82 48.66 35.57 OPT-1.3B 3.83 52.40 55.59 24.08 39.07 45.82 36.80 Qwen2-0.5B 3.77 51.59 52.78 25.51 25.70 42.34 33.62 Qwen3-0.6B-Base 3.73 51.43 53.14 26.01 23.67 41.78 33.29 Qwen3.5-0.8B-Base 4.21 52.73 57.23 25.07 32.83 43.30 35.89 Qwen3.5-2B-Base 3.97 52.46 57.09 24.61 35.13 42.61 35.98 Qwen3.5-4B-Base 4.23 52.80 57.41 24.88 37.15 43.02 36.58 Pre-defined features Function words 7.17 55.17 60.88 14.51 43.14 37.98 36.47 TFIDF FineWeb 1-2-grams 6.78 57.18 64.45 13.72 50.95 55.74 41.47 TFIDF FineWeb 1-3-grams 6.37 56.97 64.45 13.67 49.00 55.51 40.99 TFIDF Reddit 1-2-grams 6.79 56.93 64.45 13.84 49.77 55.54 41.22 TFIDF Reddit 1-3-grams 6.52 56.68 64.25 13.84 50.26 55.34 41.15 Stylometric 9.67 56.88 60.48 25.87 39.52 53.30 40.95 NeuroBiber 8.24 55.10 57.27 14.13 9.06 51.64 32.57 Table 6: Full STEB (operational) results (×100× 100). Bold = best, underline = 2nd best per column. Metrics: V-measure (Clustering), AUC (All-to-All and Predefined Pair Classification), distractor accuracy (Order Alignment), MRR (Retrieval), and average accuracy (Probing). Scores are macro-averaged across automatically discovered dataset clusters within each task, then averaged across tasks to produce the overall STEB score. Num. Datasets counts the datasets contributing to each task; a dataset may contribute to more than one task. Model ATD ATD (Adv.) Authorship Verification AR Avg. Overall Easy Medium Hard Top 5 deberta-v3-large 36.03 100.00 73.29 77.68 62.92 55.87 63.24 68.14 STAR 22.79 100.00 77.52 88.37 76.02 58.26 66.96 66.82 LUAR-MUD 28.20 75.01 76.54 86.84 73.62 61.10 75.67 63.85 deberta-v3-base 29.55 99.24 69.40 73.46 59.80 54.75 55.17 63.34 LUAR-CRUD 25.20 72.61 76.35 86.60 75.92 61.23 77.64 62.95 StyleDistance 30.10 70.08 70.44 75.52 69.17 62.18 49.61 55.06 ModernBERT-large 26.39 60.69 68.10 85.02 64.09 50.75 52.17 51.84 CISR 22.78 64.33 71.57 79.39 70.74 65.56 45.87 51.14 roberta-large 23.13 35.31 71.34 85.49 66.43 52.61 61.07 47.71 bert-large-uncased 17.53 30.22 69.39 82.49 65.10 50.94 60.23 44.34 bert-large-cased 16.61 27.57 69.36 85.59 64.93 49.53 58.88 43.11 ModernBERT-base 24.85 27.94 66.71 83.86 61.03 49.89 51.25 42.69 StyleDistance (Synthetic) 22.76 44.03 59.00 64.82 55.28 52.62 31.53 39.33 MSR 17.38 0.12 73.88 88.92 73.43 55.99 63.49 38.72 roberta-base 24.83 8.29 69.02 84.16 64.37 50.31 50.63 38.19 TFIDF Reddit 1-3-grams 12.37 7.17 65.83 73.90 60.50 50.98 50.26 33.91 TFIDF Reddit 1-2-grams 11.93 4.64 66.20 75.72 60.34 50.79 49.77 33.13 TFIDF FineWeb 1-2-grams 12.77 0.81 66.44 77.03 60.69 51.03 50.95 32.74 TFIDF FineWeb 1-3-grams 13.36 1.25 66.28 75.46 61.09 51.12 49.00 32.47 gte-base-en-v1.5 14.30 0.16 63.76 87.36 67.91 43.27 49.36 31.89 Stylometric 24.10 0.87 62.87 77.32 50.71 46.58 39.52 31.84 e5-large-v2 9.48 0.20 67.16 88.54 68.08 48.21 49.91 31.69 Function words 14.25 5.20 63.10 68.40 58.25 52.30 43.14 31.42 LISA 11.61 6.06 65.03 84.41 66.71 47.86 42.96 31.41 e5-base-v2 9.33 0.11 65.97 87.83 67.96 46.38 49.62 31.26 gte-large-en-v1.5 8.36 0.20 64.93 86.67 67.65 44.15 49.60 30.77 bge-base-en-v1.5 6.13 0.01 65.05 86.64 65.66 43.76 51.44 30.66 all-mpnet-base-v2 6.25 0.04 66.00 87.36 66.33 44.30 48.99 30.32 jina-embeddings-v3 6.56 0.01 67.15 88.83 68.93 44.80 47.35 30.27 bge-large-en-v1.5 5.97 0.01 65.90 87.12 66.79 44.15 48.79 30.17 Qwen3.5-4B-Base 9.01 1.86 57.55 62.97 51.11 47.61 37.15 26.39 opt-1.3b 8.13 0.03 54.78 55.99 51.32 50.35 39.07 25.50 Qwen3.5-2B-Base 8.27 1.27 57.02 61.82 50.61 47.77 35.13 25.42 Qwen3.5-0.8B-Base 9.25 1.80 57.33 62.35 51.71 47.86 32.83 25.30 mStyleDistance 15.85 5.48 54.41 60.49 49.63 50.53 20.04 23.95 Qwen3-Embedding-8B 6.95 0.11 52.75 62.78 52.41 47.54 35.17 23.74 gpt2-xl 12.52 0.09 52.84 53.91 51.43 49.49 28.82 23.57 NeuroBiber 20.34 3.18 59.02 71.14 55.94 52.30 9.06 22.90 Qwen2-0.5B 8.29 0.38 52.94 61.94 50.58 48.08 25.70 21.83 Qwen3-0.6B-Base 8.29 0.34 53.29 61.53 50.27 47.91 23.67 21.39 Table 7: Scores across application clusters, sorted in descending order. ATD = AI-Text Detection, AR = Authorship Retrieval. Authorship Verification: Overall aggregates a superset of datasets (PAN13–15, PAN20–21, Enron, and the PAN22–26 style-change benchmarks), while Easy/Medium/Hard are computed over the PAN22–26 style-change subset only—they do not average to the Overall column. To avoid double-counting, only Overall contributes to Avg. Bold = best, underline = 2nd best per column (across all models). Each column reports the macro-averaged score (×100× 100) within the corresponding manual cluster. Model Pair Class. Retrieval Avg. deberta-v3-large 74.03 47.77 60.90 StyleDistance 72.40 44.45 58.42 roberta-large 71.91 44.02 57.96 bert-large-cased 71.33 39.11 55.22 CISR 70.08 37.93 54.01 deberta-v3-base 71.85 35.42 53.63 STAR 71.32 35.80 53.56 roberta-base 66.01 39.70 52.86 bert-large-uncased 69.36 34.92 52.14 LUAR-CRUD 72.37 30.07 51.22 LUAR-MUD 69.50 32.38 50.94 TFIDF Reddit 1-2-grams 63.02 38.52 50.77 TFIDF Reddit 1-3-grams 62.61 37.34 49.98 MSR 69.43 28.38 48.90 TFIDF FineWeb 1-3-grams 61.94 34.53 48.24 TFIDF FineWeb 1-2-grams 62.42 31.98 47.20 e5-large-v2 67.05 24.60 45.83 e5-base-v2 66.20 24.99 45.59 ModernBERT-base 69.09 20.55 44.82 LISA 62.67 26.70 44.68 Stylometric 67.38 19.37 43.37 StyleDistance (Synthetic) 64.03 19.46 41.74 bge-base-en-v1.5 62.86 18.90 40.88 ModernBERT-large 66.95 13.80 40.37 jina-embeddings-v3 64.02 16.23 40.12 all-mpnet-base-v2 60.04 19.17 39.61 bge-large-en-v1.5 62.93 14.61 38.77 mStyleDistance 59.94 14.69 37.32 gte-large-en-v1.5 62.78 10.16 36.47 Qwen3.5-0.8B-Base 56.83 15.68 36.25 Qwen3.5-2B-Base 55.72 16.41 36.07 Qwen3.5-4B-Base 55.93 14.52 35.23 Function words 54.25 15.16 34.71 gte-base-en-v1.5 58.53 10.41 34.47 opt-1.3b 55.30 11.55 33.43 NeuroBiber 54.98 10.59 32.79 Qwen3-Embedding-8B 54.56 10.51 32.54 Qwen3-0.6B-Base 54.64 10.41 32.53 Qwen2-0.5B 53.43 9.41 31.42 gpt2-xl 48.75 12.55 30.65 Table 8: Full Multilingual STEB results (×100× 100), sorted in descending order. Pair Class. averages AUC across 13 PAN13/14/15 authorship verification datasets (Dutch, Greek, Spanish); Retrieval averages MRR across 4 PAN18 cross-domain A datasets (French, Italian, Polish, Spanish). Bold = best, underline = 2nd best per column (across all models). Object of Study Ling. Feat. Content-Ind. Avg. Model Genre Register Time Demo. Dialect Idiolect Avg. Order Al. † Num. Datasets 4 7 3 3 3 29 49 7 11 67 Style specific StyleDistance 62.84 40.01 46.21 33.67 56.91 60.03 49.94 74.02 44.23 56.06 StyleDistance (Synth.) 55.45 38.81 43.27 33.13 55.64 45.26 45.26 70.87 46.10 54.08 CISR 62.43 36.09 41.52 33.03 54.76 58.72 47.76 65.85 32.31 48.64 STAR 68.68 39.79 47.25 40.99 49.39 72.24 53.06 67.66 14.08 44.93 mStyleDistance 49.41 32.83 35.04 32.25 42.83 37.23 38.27 56.17 38.32 44.25 LUAR-MUD 66.67 38.72 42.14 35.53 44.30 76.11 50.58 70.50 11.29 44.12 LUAR-CRUD 65.80 38.14 41.30 35.32 41.52 76.99 49.85 70.37 10.57 43.59 MSR 66.39 39.08 41.98 37.05 41.52 68.69 49.12 66.52 12.35 42.66 LISA 56.89 37.95 37.38 32.95 27.19 54.00 41.06 64.02 7.10 37.39 Semantic embedders Qwen3-Embedding-8B 39.57 33.88 37.26 26.92 27.05 43.96 34.77 57.68 24.10 38.85 E5-base-v2 55.91 37.26 39.72 30.50 22.85 57.80 40.67 69.27 5.11 38.35 E5-large-v2 57.58 37.70 40.81 31.38 22.16 58.53 41.36 67.54 5.22 38.04 GTE-base-en-v1.5 53.62 36.51 39.44 31.76 23.10 56.56 40.17 65.77 5.31 37.08 BGE-base-en-v1.5 54.69 36.67 36.96 30.75 23.10 58.25 40.07 65.79 5.22 37.03 BGE-large-en-v1.5 56.15 37.31 38.23 29.71 22.47 57.35 40.20 65.54 5.28 37.01 Jina Embeddings v3 56.24 37.51 39.77 31.10 23.74 57.25 40.94 63.80 5.53 36.75 GTE-large-en-v1.5 57.09 36.67 41.20 31.38 24.01 57.26 41.27 63.40 5.45 36.71 all-mpnet-base-v2 56.07 37.55 39.03 31.47 23.31 57.50 40.82 62.18 5.06 36.02 Masked language models DeBERTa-v3-large 63.26 38.25 46.94 35.37 43.37 68.26 49.24 69.56 18.72 45.84 RoBERTa-base 63.79 42.52 45.94 32.84 40.16 59.83 47.51 77.95 10.27 45.24 RoBERTa-large 65.84 40.01 46.43 33.07 42.07 66.21 48.94 75.48 9.86 44.76 DeBERTa-v3-base 58.34 36.63 44.97 33.72 46.08 62.29 47.01 68.28 18.49 44.59 BERT-large-cased 66.24 41.99 45.11 34.79 41.23 64.12 48.91 73.58 9.22 43.90 ModernBERT-large 63.99 39.77 44.29 32.75 42.28 60.13 47.20 73.38 10.35 43.64 ModernBERT-base 62.23 38.76 43.14 32.67 37.71 58.98 45.58 72.90 9.64 42.71 BERT-large-uncased 62.24 39.90 45.27 32.80 44.03 64.81 48.17 68.55 10.89 42.54 Causal language models Qwen3.5-0.8B-Base 46.66 35.59 40.51 28.29 32.34 45.08 38.08 60.61 26.03 41.57 Qwen2-0.5B 44.76 36.01 40.57 28.24 34.66 39.32 37.26 60.46 26.76 41.49 GPT-2 XL 41.36 32.22 36.15 27.29 34.00 40.83 35.31 58.54 29.14 41.00 Qwen3.5-2B-Base 45.71 34.81 39.04 28.33 33.39 46.08 37.89 59.49 25.49 40.96 Qwen3.5-4B-Base 44.42 34.62 38.93 28.43 34.70 47.35 38.07 58.63 25.72 40.81 Qwen3-0.6B-Base 42.81 34.26 38.22 28.26 35.77 38.48 36.30 57.99 28.12 40.80 OPT-1.3B 43.20 33.19 38.79 28.76 30.68 46.92 36.92 57.85 24.29 39.69 Pre-defined features Stylometric 54.60 38.59 34.43 32.44 35.49 51.19 41.12 67.36 24.89 44.46 Function words 51.00 35.58 43.56 31.56 27.06 53.12 40.31 63.21 9.52 37.68 TFIDF Reddit 1-2-grams 49.91 33.96 36.19 31.67 27.71 57.98 39.57 66.90 5.66 37.38 TFIDF FineWeb 1-2-grams 49.19 33.79 36.20 31.70 26.31 58.70 39.32 67.13 5.55 37.33 TFIDF Reddit 1-3-grams 48.90 33.75 36.37 31.84 25.33 58.05 39.04 66.55 5.62 37.07 TFIDF FineWeb 1-3-grams 48.49 33.55 36.30 31.75 24.81 57.64 38.76 66.68 5.47 36.97 NeuroBiber 47.31 32.38 34.49 31.46 28.07 34.04 34.62 64.11 7.99 35.58 Table 9: Full STEB (definitional) results. Results are grouped into three clusters after Wegmann et al. (2026) and averaged: Object of Study (Genre, Register, Time, Demographics, Dialect, Idiolect), Linguistic Features, and Content-Independence. The Avg. column under Object of Study reports the mean of the six objects of study. Models are sorted by overall Avg. (descending). The datasets used for each column are given in App. Tab. 13. Order Alignment† uses the distractor variant of the order-alignment task for the respective datasts and no other task for that dataset. The other columns use all available tasks for the included dataset and the acc variant of order alignment. Bold = best, underline = 2nd best per column (across all models). Appendix B Task examples We display an example of the order alignment task in Figure 2. Task: Align the order of set B to that of set A w.r.t. style: 1. 2. Set A r u a fan of them or something? Are you one of their fans? Set B Oh, and also that young physician got an unflattering haircut Oh yea and that young dr got a bad haircut Solution: B2, B1 Figure 2: Order Alignment Example. Set A is written in an informal and a formal style, respectively. Set B is written in the reverse stylistic order. The task is to reorder B to match A’s style sequence. We aim for sets to only include sentences showing the same content. The “distractor” variant is signified by the modifications in grey and red. Figure was taken and slightly modified from Wegmann et al. (2022). Appendix C Dataset details Table LABEL:tab:dataset-attribute-overview summarizes which stylistic attribute each STEB dataset is intended to probe, using the seven categories around which this section is organized. A check mark indicates the category the dataset is grouped under in our analyses; some datasets could plausibly fit additional categories (e.g. the Corpus of Diverse Styles spans dialect, time, and genre) but we keep a single assignment for clarity. Table 10: STEB datasets by intended stylistic attribute. Feat. = linguistic feature, Dial. = dialect, Reg. = register, Gen. = genre, Demo. = demographics, Auth. = authorship. Each row receives a single check mark for the bucket the dataset is grouped under in this appendix; a few datasets span multiple attributes but are assigned to a single bucket for clarity. Dataset Feat. Dial. Reg. Gen. Time Demo. Auth. STEL_feature ✓ SynthSTEL_feature ✓ StylePTB ✓ probing_amazon ✓ probing_blog ✓ probing_reddit ✓ probing_stackexchange ✓ twitter_aave_sae ✓ endive ✓ eWAVE ✓ STEL_register ✓ SynthSTEL_register ✓ OneStopEnglishCorpus ✓ graded_formality ✓ ASSET ✓ wikipedia_politeness ✓ stackexchange_politeness ✓ enron_spam ✓ sms_spam ✓ telegram-spam-ham ✓ hate_speech ✓ hate_speech_and_offensive_language ✓ corpus-of-diverse-styles ✓ core ✓ x_genre ✓ masc_text_genre ✓ parallel_shakespeare ✓ ceeces ✓ bible_versions ✓ fce_l1 ✓ blog_age ✓ blog_gender ✓ pastel_age ✓ pastel_education ✓ pastel_gender ✓ pastel_ethnic ✓ pastel_politics ✓ pastel_tod ✓ pan13_authorship_verification_english_test ✓ pan14_authorship_verification_corpus1_english_essays_test ✓ pan14_authorship_verification_corpus1_english_novels_test ✓ pan14_authorship_verification_corpus2_english_essays_test ✓ pan14_authorship_verification_corpus2_english_novels_test ✓ pan15_authorship_verification_english_test ✓ pan20_authorship_verification_test ✓ pan21_authorship_verification_test ✓ enron_authorship_corpus ✓ pan18_style_change ✓ pan22_style_change_basic ✓ pan22_style_change_advanced ✓ pan22_style_change_sentence ✓ pan23_style_change_easy ✓ pan23_style_change_medium ✓ pan23_style_change_hard ✓ pan24_style_change_easy ✓ pan24_style_change_medium ✓ pan24_style_change_hard ✓ pan25_style_change_easy ✓ pan25_style_change_medium ✓ pan25_style_change_hard ✓ pan26_style_change_easy ✓ pan26_style_change_medium ✓ pan26_style_change_hard ✓ pan18_cross_domain_authorship_attribution_english ✓ amazon ✓ fanfiction ✓ stackexchange_retrieval ✓ gede_essay_detection ✓ machine_text_detection_DetectRL_arxiv ✓ machine_text_detection_DetectRL_direct_prompt ✓ machine_text_detection_DetectRL_writing_prompt ✓ machine_text_detection_DetectRL_xsum ✓ machine_text_detection_DetectRL_yelp_review ✓ machine_text_detection_DetectRL_paraphrase_attacks ✓ machine_text_detection_DetectRL_perturbation_attacks ✓ machine_text_detection_DetectRL_prompt_attacks ✓ machine_text_detection_M4_arxiv ✓ machine_text_detection_M4_peerread ✓ machine_text_detection_M4_reddit ✓ machine_text_detection_M4_wikihow ✓ machine_text_detection_M4_wikipedia ✓ machine_text_detection_MAGE_cmv ✓ machine_text_detection_MAGE_eli5 ✓ machine_text_detection_MAGE_hswag ✓ machine_text_detection_MAGE_roct ✓ machine_text_detection_MAGE_sci_gen ✓ machine_text_detection_MAGE_squad ✓ machine_text_detection_MAGE_tldr ✓ machine_text_detection_MAGE_wp ✓ machine_text_detection_MAGE_xsum ✓ machine_text_detection_MAGE_yelp ✓ machine_text_detection_PAN24_news ✓ machine_text_detection_PAN25_26_essays ✓ machine_text_detection_PAN25_26_fiction ✓ machine_text_detection_PAN25_26_news ✓ machine_text_detection_PAN25_collaborative ✓ C.1 Feature datasets STEL_feature added as order alignment task. We split the publicly available STEL version Wegmann and Nguyen (2021); Wegmann et al. (2022) into registers (cf. Section C.3) and linguistic features (i.e., number substitution, contraction or emoji use characteristic). STEL_feature consist of ≈400≈ 400 instances total. The STEL characteristics are publicly released. Note that in order alignment tasks we pair each style transfer pair with every other pair from the same style category—thus, we do not use the predefined pairings from STEL or SynthSTEL. SynthSTEL_feature added as order alignment task. We split the SynthSTEL dataset Patel et al. (2025) into 8 register (cf. Section C.3) and 32 linguistic features (i.e., everything else, including features like All Lower Case / Proper Capitalization). We combine the train and test split, leading to 100 pairs per feature. SynthSTEL is released under the MIT license per its HuggingFace dataset card at https://huggingface.co/datasets/StyleDistance/synthstel. StylePTB added as order alignment task. StylePTB (Lyu et al., 2021) is a style-transfer benchmark with ≈60k≈ 60k parallel pairs over 21 individual styles. We keep 15 and remove six out of the 21 dimensions as some change meaning (e.g., antonym replacement) or have no consistent style change within the same dimension (e.g., synonym replacement uses WordNet without clear stylistic commonalities in the selected synonyms like market replaced with marketplace and years with twelvemonths). Note that the distribution is heavily skewed: to_future has ≈7k≈ 7k pairs while p_back_to_front has only 7. To keep as many categories as possible, we combine the train/dev/test partitions. The text is provided in tokenized form: we do not detokenise. The StylePTB transformations are released under C BY 4.0 (https://github.com/lvyiwei1/StylePTB/blob/master/LICENSE). Probing added as probing task across four domains (probing_amazon, probing_blog, probing_reddit, probing_stackexchange). We extract 4646 foundation-level linguistic features per document using the LFTK toolkit (Lee and Lee, 2023b): 1212 token-level counts (words, stop-words, punctuation, syllables, characters, plus token frequency norms) and 3434 POS-tag counts (total and unique counts for each of 1717 Universal Dependencies POS tags). Each feature is independently quantile-binned into discrete labels and the resulting per-feature classification task is solved with a small MLP probe on top of frozen embeddings. The reported scores are macro-averaged across the 4646 features. The underlying source texts are sampled from publicly available Amazon product reviews (Ni et al., 2019), blog posts (via Kaggle https://w.kaggle.com/datasets/rtatman/blog-authorship-corpus), Reddit comments (via HuggingFace dataset AnnaWegmann/StyleEmbeddingData), and Stack Exchange posts, yielding 27,58427,584, 39,11139,111, 16,44716,447, and 9,2539,253 records respectively after binning and balancing. The scripts used to create the probing datasets will be released publicly on our GitHub repository. C.2 Dialectal datasets Twitter AAE added as order alignment task. We use a set of parallel constructed tweets written originally in AAE variety and rewritten in SAE by annotatorsGroenwold et al. (2020). The dataset consists of ≈2k≈ 2k instances. It can be accessed via the EMNLP 2020 supplementary archive at https://aclanthology.org/attachments/2020.emnlp-main.473.OptionalSupplementaryMaterial.zip. There is no license file; we treat it as research-use only consistent with its release as paper supplementary material. EnDive added as clustering and all-to-all pair classification task. We add parallel sentences from the EnDive benchmark Gupta et al. (2025) for Jamaican English, African American English, Colloquial Singaporean English, Indian English and Chicano English across 12 NLU benchmarks777specifically logic_bench_yn, logic_bench_mcq svamp mbpp humaneval gsm8k, folio, boolq, copa, multirc, sst-2, wsc at https://huggingface.co/datasets/abhaygupta1266/. The dataset was created using few-shot prompting with examples from native speakers. License information is not provided, but we assume it to be permissive based on the fact that it is meant to be a public benchmarks freely shared on HuggingFace. eWAVE added as clustering and all-to-all pair classification task. We add dialect example sentences from the Electronic World Atlas of Varieties of English (Kortmann et al., 2020) Cross-Linguistic Data Formats (CLDF) release at https://zenodo.org/records/17433568 and https://github.com/cldf-datasets/ewave. We include 53 varieties ranging from ≈20≈ 20 to ≈200≈ 200 instances. Varieties include Hong Kong English, Minx English, Bahamian Creole and Indian English. The CLDF release is licensed C BY 4.0. C.3 Register datasets STEL_register added as order alignment tasks. We split the publicly available888The full formality dataset of 815 items is not publicly available, so we use the public version of STEL with 100 instances of two pair seach, see https://github.com/nlpsoc/STEL/ STEL version Wegmann and Nguyen (2021); Wegmann et al. (2022) into registers (i.e., formality and complexity dimension) and linguistic features (cf. C.1). The registers consist of ≈200≈ 200 instances each. SynthSTEL_register added as order alignment task. We split the SynthSTEL dataset Patel et al. (2025) into 8 register (e.g., formal tone, offensive language, sarcasm) and 32 linguistic features (cf. Section C.1). Note that depending on ones definition of “register”, some categories like positive sentiment expression might not be considered style Wegmann et al. (2026). We combine the train and test split, leading to 100 pairs per feature. SynthSTEL is released under the MIT license per its HuggingFace dataset card at https://huggingface.co/datasets/StyleDistance/synthstel. OneStopEnglish added as order alignment task. We use the OneStopEnglish corpus Vajjala and Lučić (2018) at https://github.com/nishkalavallabhi/OneStopEnglishCorpus/. It provides 189 parallel texts at three reading levels (beginner, intermediate, advanced) compiled from news articles rewritten by English teachers to suit the levels of the learners. It is shared with Creative Commons Attribution Share Alike 4.0 International, see https://github.com/nishkalavallabhi/OneStopEnglishCorpus/blob/master/LICENSE.markdown. Level Example 1 Lol your always so convincing. 2 Lol, you’re always so convincing. 3 You’re always so convincing. 4 You are always very convincing. 5 You are consistently persuasive. Table 11: Graded formality example. Graded Formality Dataset added as order alignment task. From TweetEval Barbieri et al. (2020) we take the ≈4k≈ 4k Tweets classified as non-offensive at https://huggingface.co/datasets/cardiffnlp/tweet_eval/. From SMS Spam Almeida et al. (2011), we take the ≈4k≈ 4k SMS classified not spam at https://huggingface.co/datasets/ucirvine/sms_spam. We use gpt-5-mini-2025-08-07 rewrite the utterances more formally 4 times, see Figure 3. Both datasets are publicly shared on HuggingFace. An example of the resulting dataset can found in Table 11. An example of the resulting sentences are: “Lol your always so convincing.”, “Lol, you’re always so convincing.”, “You’re always so convincing.”, “You are always very convincing.”, “You are consistently persuasive.”. ASSET added as order alignment task. We use the ASSET corpus (Alva-Manchego et al., 2020), a multi-reference sentence-simplification benchmark covering ≈2k≈ 2k English Wikipedia source sentences with ten crowdsourced simplifications per sentence. To turn ASSET into an order-alignment task on the simple axis, we form one length-two ordered pair per source sentence by sampling one of its ten reference simplifications uniformly at random as the simpler element and pairing it with the original Wikipedia sentence as the more-complex element. We combine the validation and test splits on HuggingFace https://huggingface.co/datasets/facebook/asset. The HuggingFace card shows the license as C BY SA 4.0. wikipedia_politeness added as clustering and all-to-all pair classification task. We add the Wikipedia Politeness Corpus Danescu-Niculescu-Mizil et al. (2013) shared at https://zissou.infosci.cornell.edu/convokit/datasets/wikipedia-politeness-corpus/. The corpus consists of ≈4k≈ 4k polite, neutral and impolite texts of Wikipedia editor requests. The corpus is shared with a C BY license v4.0, see https://convokit.cornell.edu/documentation/wiki_politeness.html. stackexchange_politeness added as clustering and all-to-all pair classification task. We add the Stack Exchange Politeness Corpus of Danescu-Niculescu-Mizil et al. (2013) shared at https://zissou.infosci.cornell.edu/convokit/datasets/stack-exchange-politeness-corpus/. We use the same labels as wikipedia_politeness. It consists of ≈6k≈ 6k Stack Exchange requests. The corpus is shared with a C BY license v4.0, see https://convokit.cornell.edu/documentation/stack_politeness.html C.4 Genre datasets Corpus of Diverse Styles added as clustering, all-to-all pair classification and order alignment task. We use the Corpus of Diverse Styles (Krishna et al., 2020) taken from HuggingFace at https://huggingface.co/datasets/billray110/corpus-of-diverse-styles. The original dataset totals ≈400k≈ 400k records across 11 “styles” (i.e., lyrics, three decade slices of the Corpus of Historical American English, Tweets, African American English Tweets, the writing of James Joyce, the Bible, the Switchboard telephone-speech transcripts, English poetry, and Shakespeare). Note that while we could distribute the labels between dialect, time and genre, we assign everything to genre for simplicity. The code is MIT-licensed (https://github.com/martiansideofthemoon/style-transfer-paraphrase/blob/master/LICENSE). CORE added as clustering and all-to-all pair classification task. We add the Corpus of Online Registers of English (CORE) Laippala et al. (2023) as found at https://github.com/TurkuNLP/CORE-corpus. We use the sub-labels as the relevant classes for pair classification, i.e., everything but the main labels IN, NA, HI, LY, SP, IP, ID, OP. If a document falls in several categories, we have it show up in several. We use documents from any file across the train, dev and test sets. This dataset was licensed under C BY-SA 4.0. x_genre added as clustering and all-to-all pair classification task. We add the English slice of the X-GENRE genre-classification dataset (Kuzman et al., 2023) shared at https://huggingface.co/datasets/TajaKuzmanPungersek/X-GENRE-text-genre-dataset. X-GENRE merges FTD (Sharoff, 2018) and CORE Laippala et al. (2023) into a joint 9-label schema using classification models. Compared to CORE, 15 sub-labels and their texts are discarded. The added dataset consists of ≈2k≈ 2k English documents. We merge the train/dev/test partitions. X-GENRE is released under C BY-SA 4.0 on HuggingFace. MASC added as clustering and all-to-all pair classification task. We add the Manually Annotated Sub-Corpus of American English (MASC) 3.0.0 (Ide et al., 2008, 2013) at http://w.anc.org/data/masc/downloads. MASC is a corpus including texts from 19 genres in American English (e.g., debate transcripts, e-mail, jokes). We create 150 samples per genre. MASC is distributed without license or other restrictions, see https://anc.org/data/masc/about/. C.5 Time datasets Shakespeare added as order alignment task. We add the parallel Shakespeare corpus of Xu et al. (2012) at https://github.com/cocoxu/Shakespeare. It includes labels for historical (i.e., Shakespearean) and contemporary English, providing ≈ 20k line-aligned pairs of originals and their modern paraphrases for 17 plays. The license file only includes a request for citation at https://github.com/cocoxu/Shakespeare/blob/master/README_LICENSE, thus it is likely to be shared at least for research purposes. CEECES added as clustering and all-to-all pair classification. We add the Corpus of Early English Correspondence Extension Sampler (CEECES), parts 1 and 2 (Nevalainen et al., 2021, 2022). It consists of ≈15k≈ 15k paragraphs from ≈2.5k≈ 2.5k historical letters labeled by six 20-year periods from 1680-1800. Both Zenodo records are licensed under Creative Commons Attribution Non Commercial 4.0 International and Creative Commons Attribution Non Commercial No Derivatives 4.0 International at https://zenodo.org/records/6411789 and https://zenodo.org/records/5887101. Bible versions added as clustering and all-to-all pair classification task. We add the parallel Bible corpus of Carlson et al. (2018) shared at https://github.com/keithecarlson/StyleTransferBibleData. The dataset comprises eight verse-aligned English Bible translations including, for example, the American Standard Version (from 1901) and the Kings James Version (from 1611). We remove verses that are identical (ignoring whitespacing and casing). The dataset includes no specific license, but all included Bibles are considered public domain and the associated paper was released with the Creative Commons Attribution License. C.6 Demographic datasets fce_l1 added as clustering and all-to-all pair classification task for distinguishing different native language speakers. We add the the publicly released subset of the Cambridge Learner Corpus’s First Certificate in English exam corpus (CLC FCE) (Yannakoudakis et al., 2011) at https://ilexir.co.uk/datasets/index.html. It consists of ≈2.5k≈ 2.5k responses for ≈1.2k≈ 1.2k English Exams by students of 16 different native languages. The dataset is released with a non-commercial research and educational license. blog_age, blog_gender added as clustering and all-to-all pair classification tasks. We add the Blog Authorship Corpus Schler et al. (2006) via HuggingFace at https://huggingface.co/datasets/barilan/blog_authorship_corpus. We use the validation split to construct age (i.e., age buckets of 10s, 20s and 30s) and gender labeled texts. The corpus is meant for non-commercial research use, see https://u.cs.biu.ac.il/˜koppel/BlogCorpus.htm. PASTEL_age, PASTEL_education, PASTEL_gender, PASTEL_ethnic, PASTEL_politics added as clustering and all-to-all pair classification tasks. We add PASTEL (Kang et al., 2019) corpus at https://github.com/dykang/PASTEL. It consists of ≈8k≈ 8k five-sentence stories written by crowd annotators. Texts are labeled with demographic information of annotators, including six age, eight education, three gender and, nine ethnic and three politics labels. There is no explicit license information provided, but since the dataset is released as a benchmark, we assume at least a intended research-purpose license. Task Type Example Solution Order Alignment Task: Align the order of the set B to that of set A w.r.t. style: Set A Set B santa was to fat, and the woman was driving. Santa was excessively overweight while the woman was driving. They cannot see anything in the beginning. Well, they probably can’t see anything at first. B2 B1 Pair Classification Task: Are the two texts written in the same style? Text A Text B Santa was excessively overweight while the woman was driving. santa was to fat, and the woman was driving. different style Clustering Task: Cluster the texts by style: Text T1. Round him peered Lenehan. T2. O lead me onward to the loneliest shade, T3. All Star Classic Game 1 Orlando 09 Game 1 - West Coast vs East T4. ( Goes up and comes down to center, shrieking and laughing. T1 → joyce T2 → poetry T3 → tweets T4 → coha_1890 Authorship Retrieval Task: Retrieve the target written by the same author as the query: Query Q. Bought as a Christmas gift but the case is nice Targets T1. Very well made and very loud! Feelnmich safer having it in my hunting bag! T2. Perfect for any animal lover - it says it all. The decal came shipped really nicely and shipped fast - T1 (author 100066) Probing Task: Predict each stylistic feature label from the text: Text Want to get one of these: But don’t have enough money. Alas. $249 for the smallest one seems a little steep.… Features (which quintile does the feature fall into?) n_adj=1, n_verb=0, t_word=0, t_syll=0, … (42 more features) n_adj → 1 n_verb → 0 t_word → 0 t_syll → 0 Table 12: Examples of each STEB Tasks. C.7 Authorship datasets PAN Authorship Verification added as predefined pair classification task. We use the official test sets of the PAN authorship-verification shared tasks: PAN13 (Juola and Stamatatos, 2013), PAN14 (Stamatatos et al., 2014), PAN15 (Stamatatos et al., 2015), PAN20 (Bevendorff et al., 2020), and PAN21 (Bevendorff et al., 2021). PAN13, PAN14, and PAN15 contain multiple languages, including Greek, and Spanish in PAN13, Dutch, Greek, and Spanish in PAN14 and PAN15. PAN20 and PAN21 are large-scale cross-domain fanfiction verification benchmarks, we subsample each to a balanced 500-pair subset (250 same-author, 250 different-author) with a fixed seed for reproducibility. All corpora are available at Zenodo, and the download links can be found in the PAN website: https://pan.webis.de/data.html. All PAN datasets are available for research use. PAN Style Change added as predefined pair classification task. We use the publicly datasets of the PAN Style Change Detection shared tasks: PAN18 (Stamatatos et al., 2018a), PAN22 (Bevendorff et al., 2022), PAN23 (Bevendorff et al., 2023), PAN24 (Ayele et al., 2024), PAN2025 (Bevendorff et al., 2025a), and PAN26 (Bevendorff et al., 2026). For each of the aforementioned style cahnge datasets, consecutive paragraphs or sentence pairs (depending on the datasets granularity) are taken to be same- or different-author trials. PAN23–26 ship three difficulty levels (easy, medium, hard) where each subsequent difficulty level has more topic overlap. The PAN26 raw pairs are heavily imbalanced (up to ≈ 96% same-author in the medium split). We apply per-document stratified down-sampling that keeps all different-author pairs and matches the same-author count. All corpora are available at Zenodo, and the download links can be found in the PAN website: https://pan.webis.de/data.html. All PAN datasets are available for research use under the C BY 4.0 license. Enron Authorship Corpus (Halvani, 2017) added as predefined pair classification task. We add the 80-author Enron e-mail authorship-verification corpus at https://prod-dcd-datasets-public-files-eu-west-1.s3.eu-west-1.amazonaws.com/f0527105-2774-423e-80ca-c692b70b6cb. Each verification case is comprised of 55 documents, where 44 documents come from one author and 11 comes from another. We concatenate the 44 documents together separated by a double newline so as to create two pairs of text for the verification trail. The corpora is available for research use under the C BY 4.0 licence. C.8 AI-text detection datasets Standard ATD variants use per-generator labels (human plus one label per LLM). Adversarial variants use binary human-vs-LLM labels because the question is whether embeddings still separate human from machine text after evasion techniques have been applied. GEDE added as a clustering task for distinguishing human and the various LLMs. The Generative Essay Detection in Education (GEDE) dataset Gehring and Paaßen (2025) is a collection of three academic essay datasets (Argument Annotated Essays (AAE), PERSUADE 2.0, and British Academic Written English (BAWE)) plus machine-generated versions of those essays at various levels of contribution using GPT-4o-mini and Llama-3.3-70b-Instruct. There are a total of 916 human essays and 12,703 LLM essays. The dataset is provided under a C BY-NC-SA 4.0 DEED Attribution-NonCommercial-ShareAlike 4.0 International license at https://github.com/lukasgehring/Assessing-LLM-Text-Detection-in-Educational-Contexts. DetectRL (Wu et al., 2024) added as five standard machine-text-detection datasets (DetectRL_arxiv, direct_prompt, writing_prompt, xsum, yelp_review and three adversarial datasets (DetectRL_paraphrase, perturbation, prompt_attacks, binary human-vs-LLM). Source: https://github.com/NLP2CT/DetectRL. M4 (Wang et al., 2024b) added as five clustering datasets over English-only domains: arxiv, peerread, reddit, wikihow, and wikipedia. Each domain contains data from 55 different LLMs (Cohere, davinci, ChatGPT, Dolly-v2, and BloomZ), but we skip BloomZ due to the problematic nature of the data where parts of the prompt are in the “generated" text. Source: https://github.com/mbzuai-nlp/M4. The data is available for research use and has previously been used in SemEval 2024 (https://github.com/mbzuai-nlp/SemEval2024-task8/). MAGE Li et al. (2024) added as ten clustering datasets over the HuggingFace dataset yaful/MAGE: cmv, eli5, hswag, roct, sci_gen, squad, tldr, wp, xsum, and yelp. Each domain’s records are labelled by generator model only, collapsing across generation methods (continuation, specified, topical), giving 27 unique machine labels plus human per domain. The data is available for public use in HuggingFace (yaful/MAGE). PAN24 Generative Authorship added as clustering and all-to-all pair classification task. We use the test partition of the PAN24 Generative Authorship Detection (Ayele et al., 2024) task. All PAN datasets are available for research use under the C BY 4.0 license. PAN25/26 Generative AI Detection (Task 1) added as three clustering datasets (PAN25_26_essays, fiction, news). The PAN25 and PAN26 editions of Task 1 (Bevendorff et al., 2025b, 2026) share the same underlying validation data. We split it by the genre field. Each record is labelled by its model field, yielding a multi-class clustering task with human plus all participating LLMs per genre. All PAN datasets are available for research use under the C BY 4.0 license. PAN25 Human-AI Collaborative Text Classification added as clustering task. The PAN25 Task 2 development set (Bevendorff et al., 2025b) labels each text with one of six human-AI collaboration categories (e.g. fully human-written, human-initiated, then machine-continued, machine-initiated, then human-continued). We perform the clustering across these different labels. All corpora are available at Zenodo, and the download links can be found in the PAN website: https://pan.webis.de/data.html. All PAN datasets are available for research use under the C BY 4.0 license. C.9 Authorship retrieval datasets PAN18 Cross-Domain Authorship Attribution (Stamatatos et al., 2018b) added as retrieval task. Each problem provides a set of candidate authors with one or more known texts and a set of unknown query texts to be attributed. We concatenate each candidate’s known texts into a single target document and add each unknown text as a query with the corresponding ground-truth label. The PAN18 dataset also contains data in French, Italian, Polish, and Spanish. All PAN datasets are available for research use under the C BY 4.0 license. Amazon and Reddit added as retrieval task. We use the same retrieval test splits as Rivera Soto et al. (2021), subsampling each dataset to 10001000 query and target pairs to make the evaluation of all models tractable within our compute and timing constraints (the original datasets contain over 100,000100,000 queries and targets). We re-distribute the subsampled test splits for research purposes only. StackExchange added as retrieval task. StackExchange data is readily available for use from: https://archive.org/download/stackexchange. It is licensed by C BY-SA 3.0 which allows research use. We collect a small sample of StackExchange data spanning 500500 authors who contribute to any community at random. The retrieval setup is similar to Rivera Soto et al. (2021), where multiple documents serve as a query and multiple documents serve as a target. Appendix D TF-IDFngrams model details While many approaches compare the frequencies of particular n-grams, simply counting their presence in documents does not reveal broader contextual information, such as how common the n-gram is in general. Therefore, n-grams can be weighted using their Term Frequency-Inverse Document Frequency (TF-IDF), which considers how often an n-gram appears in a particular document compared to how rare it is in a corpus overall. This measurement increases the weight of rare words, which might be distinguishing of an author, and reduces the weight of common words. The authorship verification baseline for the PAN competition Stamatatos et al. (2023), for instance, uses TF-IDF-weighted character 4-grams. We experimented with fitting the vectorizer to a few diverse datasets—Corpus of Diverse Styles (Krishna et al., 2020), Reddit Million User Dataset Khan et al. (2021), and FineWeb Penedo et al. (2024)—but found performance to be similar for all. After experimenting with various n values for each of these models, we found that character 3-5-grams, token 1-2-grams, and POS tag 1-2-grams worked best overall. Based on these findings, for the main paper, we chose a TFIDFngrams model fit to a 10 billion token sample of FineWeb Penedo et al. (2024), a large collection of cleaned English web data from CommonCrawl, with these n-gram settings. Tab. 6 through Tab. 9 show results for a few variations of TF-IDF n-gram models based on the dataset it was trained on (Reddit, FineWeb) and the chosen n (1-2, 1-3) for the token n-grams and POS tag n-grams. The character n-gram values were not decreased because very short character n-grams occur extremely often but do not carry much discriminative information; this makes the feature matrix larger and sparser, increases computational cost, and can introduce noise that hurts generalization. Object of Study Ling. Feat. Content Independence Dataset Gen. Register Time Demo. Dialect Idiolect Order Alignment† CORE ✓ x_genre ✓ MASC ✓ Corpus of Diverse Styles ✓ ✓ STEL_register ✓ ✓ SynthSTEL_register ✓ ✓ Graded Formality ✓ ✓ OneStopEnglish ✓ ✓ ASSET ✓ ✓ wikipedia_politeness ✓ stackexchange_politeness ✓ Bible versions ✓ CEECES ✓ Shakespeare ✓ ✓ Blog ✓ FCE L1 ✓ PASTEL ✓ EnDive ✓ Twitter AAE/SAE ✓ ✓ eWAVE ✓ Enron authorship corpus ✓ PAN13 AV ✓ PAN14 AV ✓ PAN15 AV ✓ PAN20 AV ✓ PAN21 AV ✓ PAN18 SC ✓ PAN22 SC ✓ PAN23 SC ✓ PAN24 SC ✓ PAN25 SC ✓ PAN26 SC ✓ PAN18 cross-domain A ✓ amazon ✓ fanfiction ✓ stackexchange_retrieval ✓ STEL_feature ✓ ✓ SynthSTEL_feature ✓ ✓ StylePTB ✓ ✓ probing_amazon ✓ probing_blog ✓ probing_reddit ✓ probing_stackexchange ✓ Table 13: Datasets used for each cluster of the STEB Score (definitional). This mirrors the three clusters of Tab. 3 and Tab. 9 after Wegmann et al. (2026): Object of Study (Genre, Register, Time, Demographics, Dialect, Idiolect), Linguistic Features (Ling. Feat.), and Content-Independence (Order Alignment task). Idiolect includes authorship verification (PAN style-change and PAN authorship-verification editions, plus Enron) and authorship retrieval (PAN18 cross-domain A English, Amazon, fanfiction, stackexchange_retrieval). PAN rows are collapsed across tasks and difficulty levels (AV = authorship verification, SC = style change, A = authorship attribution); Order Alignment† uses the distractor variant of the order-alignment task for the respective datasts and no other task for that dataset. The other columns use all available tasks for the included dataset and the acc variant of order alignment. Appendix E Embeddings vs. prompting In practice: Why not just use prompting? To demonstrate this concretely, we compare prompting GPT-5.2 against LUAR-CRUD on authorship retrieval, using a relatively small corpus of 100 query and 100 target collections of 16 sentences each, created with Amazon Reviews data (§ E). The dataset is a subsample of the STEB Amazon retrieval dataset described in § C.9. As no established prompts exist for retrieval (likely due to scaling issues, cf. § E), we adapt the linguistically informed prompts created for authorship verification by Huang et al. (2024), with minor grammatical corrections. We provide the full prompt in App. Fig. 3. Table 1 reports the same performance measures on retrieval as STEB (Section 3). We additionally report tera floating point operations (TFLOPs) and API costs in US dollars (GPT-5.2 only). LUAR-CRUD, a model from 2021, outperforms GPT-5.2 released in 2025 while being at least 500× more efficent than causal models with 1B+ parameters. System Message Respond with a JSON object with two elements: "analysis": Reasoning behind your answer. "answer": A list of all candidate IDs (integers 0 to N-1) sorted from most to least likely to be written by the same author as the query text. The list must contain every candidate ID exactly once. User Prompt You are given a set of texts written by one unknown person (the query author) and several sets of candidate texts written by several known authors (the candidate authors). Rank all candidate sets of texts by how likely each was written by the query author. Analyze the writing style only, disregarding the differences in topic and content. Base your reasoning on linguistic features such as phrasal verbs, modal verbs, punctuation, rare words, affixes, quantities, humor, sarcasm, typographical errors, and misspellings. Query texts: [16 QUERY DOCUMENTS UNKNOWN AUTHOR] Candidates (rank all N by likelihood of matching the query author): Candidate 0: [16 TARGET DOCUMENTS AUTHOR 0] Candidate 1: [16 TARGET DOCUMENTS AUTHOR 1] ⋮ Candidate N−1N-1: [16 TARGET DOCUMENTS AUTHOR N−1N-1] Figure 3: Prompt template for A via LLM-based stylistic analysis. We display the prompt used with GPT-5.2 in Figure 3. The lower efficiency of LLM models is unsurprising: authorship retrieval requires comparing a query against every target collection in the pool, which scales poorly with prompting approaches. As the target pool grows, it may even exceed the model’s context window or lead to large costs in US dollars or FLOPs.999This parallels problems faced in information retrieval and RAG Gao et al. (2024). To illustrate, a common Reddit authorship retrieval dataset Khan et al. (2021) contains 120k authors with 16 texts each, requiring approximately 5.1M tokens per retrieval query, which is considerably larger than GPT-5.2’s context window of 400k.101010See https://developers.openai.com/api/docs/models/gpt-5.2 Even if the context window was not restricted, retrieval on the 120k dataset would require about 20 billion TFLOPs for a 1 billion parameter causal model and 40k TFLOPS for models like LUAR. Appendix F Multilingual truncation analysis The main-text multilingual ranking (Tab. 5) is computed under STEB’s default protocol, which segments long inputs into sentence-boundary-aware chunks and mean-pools the chunk embeddings (§ G). To isolate the effect of this evaluation choice, we reproduce the multilingual evaluation setup of Kim et al. (2025b): the same PAN13/14/15 authorship verification datasets (Dutch, Greek, Spanish), the same pre-defined pair classification protocol, the same style embeddings, and the same AUC metric. The only change is that each document is truncated to the model’s maximum length rather than chunked and pooled as in STEB. Tab. 14 compares the two protocols. Under truncation, MSR moves from rank 44 to rank 11, while without truncation it ranks 44th, underperforming the style embeddings trained on English data. These results underscore the necessity of shared evaluation protocols, as without them our conclusions would differ materially. Chunk & Pool Truncate Model AUC Rank AUC Rank StyleDistance 72.40 1 58.40 4 CISR 70.08 2 60.31 2 LUAR-MUD 69.50 3 58.52 3 MSR 69.43 4 67.51 1 mStyleDistance 59.94 5 56.88 5 Table 14: Mean AUC (×100× 100) on the PAN13/14/15 multilingual AV datasets used by Kim et al. (2025b), under STEB’s default chunk-and-pool protocol versus per-document truncation. Bold = best per protocol. Appendix G Chunking large documents We demonstrate STEBs strategy for embedding large inputs into a single representation in Fig. 4. Note that this approach is fair to all encoders, as each one is able to observe the same amount of tokens regardless of what their token limits are. Long Input Document (e.g. 2048 tokens) Chunk 1 [0-512)Chunk 2 [512-1024)Chunk N(Remainder)SegmentationEncodingsEncoderEncoderEncoderEmbeddings1N∑ 1N PoolingFinal Vector Figure 4: Example of STEBs chunk-and-pool strategy. In the example, it is assumed that the model’s maximum context-length is 512512 tokens. The long input document is chunked up into segments of 512512 tokens that respect sentence boundaries. Each chunk is then embedded individually be the encoder, and finally we mean-pool across the chunks derive our final embedding. Appendix H Use of AI Assistants We used AI-assistants (Claude Code, Cursor) to help us develop the STEB code-base. That said, all the code underwent human-review, and we rigorously tested the code-base for correctness. We also used LLMs to give us ideas for rephrasing and shortening excerpts of the paper. Appendix I Compute Requirements We ran each model on a single H100 or A100 40Gb GPU. No single run took longer than 24hrs, and the largest model evaluated has 8B parameters. We upper-bound our GPU expenditure at 960960 GPU hours (2424 hours times 4040 models). Please note that this is a very large over-estimate, most models took less than 88 hours to run. Appendix J Sample Counts per Dataset We specify the number of classes, number of samples per class, and total number of texts embedding in LABEL:tab:dataset-sample-counts. Table 15: Per-dataset sample counts in STEB. “# Classes” refers to classes for clustering and pair-classification tasks, trials for pre-defined pair classification, style groups for order alignment, unique author/item ids for retrieval, and probing features for probing. “Samples/Class” is the per-class evaluation budget. “Total” is the total number of texts the embedder is asked to encode. Dataset # Classes Samples/Class Total Clustering (53 datasets) bible_versions 8 200 1,600 blog_age 3 200 600 blog_gender 2 200 400 ceeces 6 200 1,200 core 38 25 950 corpus-of-diverse-styles 11 200 2,200 eWAVE 53 29 1,537 endive 6 200 1,200 enron_spam 2 200 400 fce_l1 15 25 375 gede_essay_detection 4 200 800 hate_speech 2 200 400 hate_speech_and_offensive_language 3 200 600 machine_text_detection_DetectRL_arxiv 5 200 1,000 machine_text_detection_DetectRL_direct_prompt 5 200 1,000 machine_text_detection_DetectRL_paraphrase_attacks 2 200 400 machine_text_detection_DetectRL_perturbation_attacks 2 200 400 machine_text_detection_DetectRL_prompt_attacks 2 200 400 machine_text_detection_DetectRL_writing_prompt 5 200 1,000 machine_text_detection_DetectRL_xsum 5 200 1,000 machine_text_detection_DetectRL_yelp_review 5 200 1,000 machine_text_detection_M4_arxiv 6 200 1,200 machine_text_detection_M4_peerread 6 200 1,200 machine_text_detection_M4_reddit 6 200 1,200 machine_text_detection_M4_wikihow 5 200 1,000 machine_text_detection_M4_wikipedia 5 200 1,000 machine_text_detection_MAGE_cmv 28 60 1,680 machine_text_detection_MAGE_eli5 28 73 2,044 machine_text_detection_MAGE_hswag 28 66 1,848 machine_text_detection_MAGE_roct 28 76 2,128 machine_text_detection_MAGE_sci_gen 28 36 1,008 machine_text_detection_MAGE_squad 28 55 1,540 machine_text_detection_MAGE_tldr 28 57 1,596 machine_text_detection_MAGE_wp 28 70 1,960 machine_text_detection_MAGE_xsum 28 86 2,408 machine_text_detection_MAGE_yelp 28 42 1,176 machine_text_detection_PAN24_news 14 200 2,800 machine_text_detection_PAN25_26_essays 13 25 325 machine_text_detection_PAN25_26_fiction 12 25 300 machine_text_detection_PAN25_26_news 14 59 826 machine_text_detection_PAN25_collaborative 6 200 1,200 masc_text_genre 19 150 2,850 pastel_age 6 25 150 pastel_education 8 25 200 pastel_ethnic 9 25 225 pastel_gender 3 55 165 pastel_politics 3 200 600 pastel_tod 5 200 1,000 sms_spam 2 200 400 stackexchange_politeness 3 200 600 telegram-spam-ham 2 200 400 wikipedia_politeness 3 200 600 x_genre 9 46 414 All-to-All Pair Classification (24 datasets) bible_versions 8 200 1,600 blog_age 3 200 600 blog_gender 2 200 400 ceeces 6 200 1,200 core 38 25 950 corpus-of-diverse-styles 11 200 2,200 eWAVE 53 29 1,537 endive 6 200 1,200 enron_spam 2 200 400 fce_l1 15 25 375 hate_speech 2 200 400 hate_speech_and_offensive_language 3 200 600 masc_text_genre 19 150 2,850 pastel_age 6 25 150 pastel_education 8 25 200 pastel_ethnic 9 25 225 pastel_gender 3 55 165 pastel_politics 3 200 600 pastel_tod 5 200 1,000 sms_spam 2 200 400 stackexchange_politeness 3 200 600 telegram-spam-ham 2 200 400 wikipedia_politeness 3 200 600 x_genre 9 46 414 Pre-Defined Pair Classification (25 datasets) enron_authorship_corpus 80 2 160 pan13_authorship_verification_english_test 30 2 60 pan14_authorship_verification_corpus1_english_essays_test 100 2 200 pan14_authorship_verification_corpus1_english_novels_test 100 2 200 pan14_authorship_verification_corpus2_english_essays_test 200 2 400 pan14_authorship_verification_corpus2_english_novels_test 200 2 400 pan15_authorship_verification_english_test 500 2 1,000 pan18_style_change 1,492 2 2,984 pan20_authorship_verification_test 500 2 1,000 pan21_authorship_verification_test 500 2 1,000 pan22_style_change_advanced 9,537 2 19,074 pan22_style_change_basic 2,141 2 4,282 pan22_style_change_sentence 22,105 2 44,210 pan23_style_change_easy 2,826 2 5,652 pan23_style_change_hard 4,112 2 8,224 pan23_style_change_medium 7,013 2 14,026 pan24_style_change_easy 2,471 2 4,942 pan24_style_change_hard 4,131 2 8,262 pan24_style_change_medium 4,592 2 9,184 pan25_style_change_easy 10,247 2 20,494 pan25_style_change_hard 10,648 2 21,296 pan25_style_change_medium 12,759 2 25,518 pan26_style_change_easy 71,108 2 142,216 pan26_style_change_hard 54,141 2 108,282 pan26_style_change_medium 24,406 2 48,812 Order Alignment (11 datasets) ASSET 1 400 400 OneStopEnglishCorpus 1 600 600 STEL_feature 3 200 600 STEL_register 2 392 784 StylePTB 14 50 700 SynthSTEL_feature 32 200 6,400 SynthSTEL_register 8 200 1,600 corpus-of-diverse-styles 11 400 4,400 graded_formality 2 1,000 2,000 parallel_shakespeare 1 400 400 twitter_aave_sae 1 400 400 Retrieval (4 datasets) amazon 500 195 97,787 fanfiction 500 86 43,411 pan18_cross_domain_authorship_attribution_english 50 5 259 stackexchange_retrieval 500 125 62,980 Probing (4 datasets) probing_amazon 46 27,584 27,584 probing_blog 46 39,111 39,111 probing_reddit 46 16,447 16,447 probing_stackexchange 46 9,253 9,253