Paper deep dive
BehaviorFormer: Discovering Behavioral Patterns via Clusterable Next-Token Prediction
Tobias Schneider, Danial Fahrner, Soren Antebi, Daniel Uedelhoven, Lennard Bodden, Sandra Halscheidt, Rafet Sifa
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Discovering behavioral patterns in open-ended games is challenging due to the lack of clear objectives or ground-truth annotations. In this work, we extend a next-token prediction Transformer with the deep clustering algorithm DCSS to learn meaningful representations with well-separated clusters. In a controlled Minecraft session with 26 players, the method discovers 23 distinct behavioral patterns that generalize across the studied players, further refining structures present in self-supervised representations into clearly separable clusters. We group the clusters into seven interpretable categories covering stationary, roaming, travel, mining, base work, logging, and farming behavior.
Tags
Links
- Source: https://arxiv.org/abs/2610.16322v1
- Canonical: https://arxiv.org/abs/2610.16322v1
Trouble viewing inline? Open PDF directly â
Full Text
23,788 characters extracted from source content.
Expand or collapse full text
BehaviorFormer: Discovering Behavioral Patterns via Clusterable Next-Token Prediction Thanks: This research has been funded by the Federal Ministry of Education and Research of Germany and the state of North-Rhine Westphalia as part of the Lamarr-Institute for Machine Learning and Artificial Intelligence. Tobias Schneider1,2, Danial Fahrner1, Soren Antebi1, Daniel Uedelhoven1, Lennard Bodden1, Sandra Halscheidt1, Rafet Sifa1,2 Affiliation: 1Fraunhofer IAIS, Sankt Augustin, Germany 2University of Bonn, Bonn, Germany Abstract Discovering behavioral patterns in open-ended games is challenging due to the lack of clear objectives or ground-truth annotations. In this work, we extend a next-token prediction Transformer with the deep clustering algorithm DCSS to learn meaningful representations with well-separated clusters. In a controlled Minecraft session with 26 players, the method discovers 23 distinct behavioral patterns that generalize across the studied players, further refining structures present in self-supervised representations into clearly separable clusters. We group the clusters into seven interpretable categories covering stationary, roaming, travel, mining, base work, logging, and farming behavior. Index Terms: Behavioral Modeling, Deep Clustering, Self-Supervised Learning, Representation Learning ©2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. I Introduction Player behavioral modeling is challenging, especially in open-ended games such as Minecraft, where players follow no clear goal. Unlike objective-driven games, where behavior is primarily shaped by the game objective, open-ended games encourage players to play however they want, leaving no ground truth for what counts as a behavioral pattern. This makes manual analysis based on hand-crafted features and self-reported annotations challenging. It requires defining behavioral patterns upfront, embeds strong prior assumptions, and struggles to capture unexpected behavior that does not fit into the predefined taxonomy. Self-reported annotations are subjective and expensive to collect, while hand-crafted features fail to represent the complex temporal structure of player behavior. This motivates automatic behavioral pattern discovery directly from raw behavior data using deep learning. Mapping player behavior onto a small set of behavioral patterns compresses complex behavioral data into interpretable high-level behavior that both humans and algorithms can reason about. This can be useful for various applications such as behavioral analytics, player modeling for personalization, content design and balancing, or churn and engagement analysis. Self-supervised sequence models such as Transformers [1] can capture temporal dynamics but do not explicitly model behavioral patterns. We address this by combining a Transformer with the Deep Clustering using Soft Silhouette (DCSS) [2] algorithm to jointly learn representations and compact, well-separated clusters corresponding to behavioral patterns. We apply this approach to a controlled Minecraft session with 26 players, finding 23 global behavioral patterns that can be categorized into seven interpretable groups. The discovered patterns generalize across players, capturing globally shared behavior rather than individual playstyles. I Related Work Prior work on Minecraft player modeling uses supervised classification on manually extracted features and self-reported labels. MĂŒller et al. [3] distinguish between building, mining, fighting, and exploring behavior with 65% accuracy. While this shows that high-level behavioral patterns can be derived from raw game logs to some extent, the approach is limited for behavioral discovery as categories and features must be defined in advance. The moderate performance suggests that hand-crafted representations only partially capture the ambiguity and temporal structure of open-ended player behavior. Similarly, prior work on player profiling with traditional machine learning algorithms relies on hand-crafted features [4, 5, 6]. This work shows that clustering can produce actionable player profiles, but the resulting structure still depends on feature engineering and game-specific design knowledge. Player2Vec [7] reduces this dependency by treating game event logs as token sequences and training a Transformer with a next-token prediction (NTP) objective to learn player representations without explicit behavior labels. Pu et al. [8] take a similar approach using masked language modeling. However, since the objectives are primarily designed for sequence modeling, cluster structure emerges only implicitly. Deep clustering methods instead learn representations and clusters jointly. DEC [9] and IDEC [10] encourage compact clusters through a KL-divergence objective, but do not explicitly optimize inter-cluster separation. DCSS [2] addresses this with a differentiable soft silhouette objective that rewards both compactness and separation of clusters, making it a promising method for behavioral pattern discovery. I Method I-A Clustering Objective The DCSS objective combines a reconstruction loss with a soft silhouette clustering loss and an entropy regularizer to prevent trivial solutions: â=ârâeâc+λ1â(1âSâfâ(hrâ(X)))âλ2âHÂŻâ(hrâ(X)),L=L_rec+ _1 (1-Sf(h_r(X)) )- _2 H(h_r(X)), (1) where ârâeâcL_rec is the chosen reconstruction loss, Sâfâ(â )Sf(·) is the soft silhouette score over the cluster assignment probabilities hrâ(X)h_r(X), HÂŻâ(â ) H(·) is the mean entropy of the assignment probabilities, and λ1,λ2 _1, _2 balance the three terms. In this work, we replace DCSSâs autoencoder with a Transformer for its sequence modeling capabilities, using the final tokenâs hidden state as input to the clustering network hrh_r. As our inputs are discrete tokens, we use NTP cross-entropy as ârâeâcL_rec instead of MSE. I-B Dataset Construction We collected data from one session with 26 individuals playing the game Minecraft over 6 hours. To motivate diverse behavior, the players were not given any concrete goal and had complete freedom in what to do during the session, except to form groups of 2-6 players. The session was recorded in a controlled environment, allowing verification of qualitative analysis, while maintaining rich behavioral patterns. The following game events were logged: Breaking or placing a block, picking up or crafting an item, interacting with a chest or furnace, and death of an entity such as an enemy monster. We also collected the position of each player at an interval of 0.5 seconds, further contextualizing the logged events. To construct a dataset for NTP using Transformers, we tokenized the collected logs. For the event logs, each unique event is assigned an action token. For the movement logs, both direction and magnitude are discretized: direction into 8 directional bins, and magnitude into thresholds suited to Minecraft, differentiating between standing still, sneaking, walking, sprinting, and very fast motion when using a boat. Each possible combination is assigned one unique motion token. The resulting vocabulary contains 532 tokens. We split the data into sequences of 64 tokens, corresponding to approximately 30 seconds of gameplay. This length was chosen in preliminary experiments, as shorter sequences fail to capture meaningful behavioral patterns while longer sequences mix multiple patterns per sequence. In total, we collected 11,234 sequences across 26 players. For evaluation of NTP loss on unseen data, we hold out 60 randomly sampled sequences per player as a validation set in each run, leaving 9,674 sequences for training. We provide the Transformer with two learned additive positional embeddings: a token position embedding TpâoâsT_pos based on the tokenâs index in the sequence, and a tick position embedding TtâiâcâkT_tick shared across all tokens within the same 0.5 s window, capturing events that occur simultaneously. I-C Hyperparameter Selection In preliminary experiments, we identified parameters with significant impact on performance and swept them over an extensive hyperparameter grid as shown in Table I. For the Transformer, we vary embedding dimension d, depth L, and DCSS-stage learning rate η. For DCSS, we vary the number of clusters K, clustering loss weight λ1 _1, pretraining epochs NpreN_pre, and clustering temperature Ïc _c. We train with a batch size of 256 and a pretraining learning rate of 10â410^-4, use the default λ2=0.01 _2=0.01 from the original DCSS paper, 2 attention heads, no dropout, and early stopping with a patience of 10 epochs on validation loss. To identify a robust selection of hyperparameters that consistently leads to good clustering, we focus on four key metrics aggregated across ten experiments: 1. Silhouette score: measures how compact and separated clusters are, which we interpret as a measure of cluster quality. 2. Adjusted Rand Index (ARI): quantifies agreement between two clusterings, adjusted for chance. We report the mean pairwise ARI across 10 independent runs as a measure of cluster assignment stability. 3. Cluster utilization: quantifies the ratio of clusters with assigned samples, which we interpret as coverage. 4. Loss: measures next-token prediction performance using cross-entropy on the validation set, which we interpret as representation fidelity. IV Results & Discussion Fig. 1: T-SNE visualization of embedding spaces learned by NTP alone (left) and DCSS (right), colored by DCSS cluster assignment. In the DCSS space, sub-clusters detected via HDBSCAN are indicated by connecting lines within their parent cluster. IV-A Hyperparameter Analysis No configuration is optimal across all four objectives, as they exhibit clear trade-offs (e.g. cluster quality vs. representation fidelity). However, with multi-objective optimization we selected a robust configuration that achieves near-optimal values on all objectives. Table I depicts the impact of modulating each hyperparameter one at a time on each metric, using the selected configuration as baseline. TABLE I: Local ablation study for selected hyperparameters. The selected parameter value and the best value for each metric are shown in bold. Parameter Silhouetteâ ARIâ Lossâ Utilizationâ 10 99.7±0.399.7±0.3 68.4±33.468.4±33.4 1.259±.0061.259±.006 92.0±22.092.0±22.0 K 25 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 50 99.3±0.099.3±0.0 95.8±4.695.8±4.6 1.255±.0021.255±.002 85.4±3.585.4±3.5 64 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 d 128 99.4±0.199.4±0.1 55.9±40.755.9±40.7 1.271±.0131.271±.013 71.6±32.271.6±32.2 256 98.8±0.798.8±0.7 64.3±45.064.3±45.0 1.292±.0131.292±.013 75.2±25.375.2±25.3 1 99.2±0.199.2±0.1 88.5±7.588.5±7.5 1.268±.0021.268±.002 96.0±1.996.0±1.9 L 2 99.5±0.099.5±0.0 98.3±0.598.3±0.5 1.261±.0031.261±.003 92.0±3.392.0±3.3 3 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 1e-4 99.5±0.299.5±0.2 83.1±16.883.1±16.8 1.261±.0021.261±.002 94.4±2.894.4±2.8 η 5e-4 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 1e-3 99.7±0.299.7±0.2 34.1±44.334.1±44.3 1.258±.0041.258±.004 57.6±41.157.6±41.1 10 99.5±0.199.5±0.1 68.8±19.368.8±19.3 1.261±.0041.261±.004 83.2±13.283.2±13.2 NpreN_pre 25 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 50 99.3±0.199.3±0.1 98.4±1.198.4±1.1 1.256±.0021.256±.002 94.8±5.794.8±5.7 λ1 _1 0.01 54.3±4.754.3±4.7 77.5±10.577.5±10.5 1.252±.0031.252±.003 97.6±2.897.6±2.8 0.1 95.3±0.795.3±0.7 87.1±17.087.1±17.0 1.256±.0021.256±.002 94.0±4.794.0±4.7 0.5 99.2±0.099.2±0.0 98.5±0.498.5±0.4 1.255±.0021.255±.002 93.6±2.893.6±2.8 1.0 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 5 99.5±0.299.5±0.2 75.0±37.875.0±37.8 1.257±.0041.257±.004 82.4±29.182.4±29.1 Ïc _c 10 99.5±0.099.5±0.0 98.2±0.798.2±0.7 1.255±.0031.255±.003 95.6±3.095.6±3.0 20 99.6±0.099.6±0.0 98.7±0.398.7±0.3 1.256±.0021.256±.002 86.8±4.686.8±4.6 Both stability and coverage of clustering seem to heavily depend on the number of clusters K. The selected configuration does not achieve optimal coverage, meaning that only 24 of the 25 clusters are used. A small embedding dimension d of 64 is crucial for good performance across all metrics. The method seems to benefit from deeper models, as higher L improves fidelity while maintaining stability and coverage. The model is sensitive to the learning rate η, as both lower and higher learning rates show instability and inconsistent coverage. Longer pretraining improves clustering stability and coverage. While the DCSS paper uses a λ1 _1 of 0.01, we found that much higher weighting for the soft silhouette loss was required to achieve good separability and stability, which might originate from significant differences in the dataset and model architecture. Also, we observed that the clustering softmax temperature Ïc _c can benefit from slightly lower values to prevent low cluster utilization, while the DCSS paper uses a value of 20. Our results show that the method is sensitive to the selected hyperparameters, making their selection crucial. The selected configuration achieves highly separable clusters with high stability and good coverage. The loss only increases by 1%1\% over using raw NTP, indicating that the clustering does not hurt NTP capabilities significantly, which suggests meaningful clustering, but requires further qualitative analysis. Figure 1 shows t-SNE [11] visualizations of the embedding spaces learned by NTP alone and DCSS. The samples are colored by DCSS cluster assignment to allow comparison of global structure across the methods. The NTP space already exhibits cluster-like structures, but applying clustering algorithms directly to it does not yield satisfactory results, as many structures lack clear boundaries. In contrast, the DCSS space shows 24 well-separated clusters of various sizes, with some exhibiting sub-cluster structures detected via HDBSCAN [12]. The shared coloring reveals that most clusters extracted by DCSS correspond to structures already present in the NTP space, but with noisy boundaries and overlap between neighboring groups. This confirms that DCSS refines latent structure already present in the NTP representations into clearly separable clusters, with negligible impact on representation fidelity. IV-B Qualitative Analysis TABLE I: Discovered behavioral clusters, grouped into seven interpretable categories. Letters in parentheses reference consistent IDs. Group Clusters Description N Stationary Stationary (A) Idle or low-activity without movement 6318 Roaming Relocating (B), Wandering (C), Crop tending (D), Mixed (E), Resource gathering (F), Aquatic foraging (G), Wood gathering (H), Provisioning (I) Local movement with some opportunistic activity 3392 Travel Short (P), Moderate (L), Long (M), Boat (T) Directed long-range movement 486 Base work Crafting (U), Building (O), Remodeling (K), Earthwork (S) Construction and item production at bases 424 Mining Tunneling (N), Caving (R), Excavating (J) Underground resource extraction 423 Logging Oak/Birch (V), Spruce (W) Intensive wood logging 105 Farming Farming (Q) Planting and harvesting crops 80 To interpret the clusters, we compute statistics over action and motion token distributions, player height, position displacement, distance to the nearest player base, player proximity, session timing, and inter-cluster transition probabilities. We then manually assign post-hoc cluster labels in Table I that reflect the dominant behavior, and merge them into seven interpretable categories with shared high-level behavior. One of the 24 utilized clusters (X), with only 6 assigned samples, was excluded from the analysis due to insufficient size, resulting in a total of 23 clusters used for analysis. The Stationary (A) cluster accounts for over half of samples and exhibits an exceptionally high self-transition rate of 0.71, meaning players remain in this state across consecutive sequences far more often than in any other cluster. Especially for this cluster, the analysis of sub-clusters was essential for validating cluster interpretations. We found that the largest sub-cluster consists of pure idle behavior with neither actions nor movement, while smaller sub-clusters include some actions. However, the diverse activities are unified into a single cluster because they are all dominated by low movement. Roaming clusters account for the second largest portion, capturing players between focused and stationary work, performing opportunistic actions such as collecting resources while on the move. These clusters have low self-transition rates below 0.1, meaning players typically stay briefly before going back to stationary patterns or into focused work. Together, Stationary and Roaming account for over 85% of samples, showing that intensive activity is comparatively rare. Focused work clusters (Mining, Base work, Farming, and Logging) make up less than 10% of the samples, but exhibit the highest activity with an average of 11 to 17 action tokens per sequence. Mining clusters differ in extraction strategy: Tunneling (N) captures systematic tunnel construction, while Caving (R) reflects natural cave exploration, both with the goal of finding ores. Excavating (J) captures large-scale stone breaking without a clear structure. These interpretations are confirmed by low player height well below surface level. Base work clusters capture near-base construction and maintenance. Earthwork (S) in particular appears predominantly in the first half of the session, which aligns with player base setup. Farming (Q) captures farm construction and crop harvesting, and is the most action-intensive cluster overall. As each action is assigned a unique token, cutting down trees of types that grow in different regions ends up in different clusters (V, W). Travel clusters form a progression from short overland sprints (P) to steady boat travel (T), and consist almost entirely of motion tokens. They appear especially in the sessionâs second half, capturing targeted travel and raiding that occurred later in the session. Boat travel is the second most persistent state with a self-transition rate of 0.38, reflecting sustained ocean crossings across many consecutive sequences. Nearly half of all clusters cover at least 24 of 26 players, and every cluster covers at least 10, confirming that the discovered clusters capture shared behavior rather than individual playstyles. Common patterns like Stationary and Roaming span nearly all players, while specialized ones such as Farming, Logging, or Boat travel cover fewer players, as expected since not every player engages in every activity. Although combat clusters are missing due to the relevant game events not being logged, the extracted patterns align with behavior observed during the controlled session, confirming that the method recovers meaningful behavioral structure without requiring self-reported annotations. V Conclusion In this work, we combined a Transformer with the DCSS clustering objective to enable automatic behavioral pattern discovery from raw gameplay logs in open-ended games. We found that careful hyperparameter selection is crucial for achieving high-quality and stable results. Applied to a controlled Minecraft session, the method yields 23 well-separated clusters that group into seven interpretable behavioral categories. The discovered patterns are global, capturing shared behavior across players rather than individual playstyles, and correspond to refinements of structures already present in pure NTP representations. These results motivate further work to validate generalization across games and longer sessions, investigate the impact of sequence construction, tokenization, and architecture choices, and explore methods for dynamically determining the number of clusters. References [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ć. Kaiser, and I. Polosukhin, âAttention is all you need,â Advances in neural information processing systems, vol. 30, 2017. [2] G. Vardakas, I. Papakostas, and A. Likas, âDeep clustering using the soft silhouette score: Towards compact and well-separated clusters,â Machine Learning, vol. 115, no. 4, p. 81, Mar 2026. [Online]. Available: https://doi.org/10.1007/s10994-026-07026-w [3] S. MĂŒller, M. Kapadia, S. Frey, S. Klinger, R. P. Mann, B. Solenthaler, R. W. Sumner, and M. Gross, âStatistical analysis of player behavior in minecraft,â in Proceedings of the 10th International Conference on the Foundations of Digital Games. Society for the Advancement of the Science of Digital Games, 2015. [4] A. Drachen, R. Sifa, C. Bauckhage, and C. Thurau, âGuns, swords and data: Clustering of player behavior in computer games in the wild,â in 2012 IEEE Conference on Computational Intelligence and Games (CIG), 2012, p. 163â170. [5] R. Sifa, A. Drachen, C. Bauckhage, C. Thurau, and A. Canossa, âBehavior evolution in tomb raider underworld,â in 2013 IEEE Conference on Computational Inteligence in Games (CIG). IEEE, 2013, p. 1â8. [6] R. Sifa, A. Drachen, and C. Bauckhage, âProfiling in games: Understanding behavior from telemetry,â Social interactions in virtual worlds: An interdisciplinary perspective, p. 337â375, 2018. [7] T. Wang, M. Honari-Jahromi, S. Katsarou, O. Mikheeva, T. Panagiotakopoulos, S. Asadi, and O. Smirnov, âplayer2vec: A language modeling approach to understand player behavior in games,â arXiv preprint arXiv:2404.04234, 2024. [8] J. Pu, J. Lin, X. Mao, J. Tao, X. Shen, Y. Shang, and R. Wu, âUnsupervised representation learning of player behavioral data with confidence guided masking,â in Proceedings of the ACM Web Conference 2022, ser. W â22. New York, NY, USA: Association for Computing Machinery, 2022, p. 3396â3406. [Online]. Available: https://doi.org/10.1145/3485447.3512275 [9] J. Xie, R. Girshick, and A. Farhadi, âUnsupervised deep embedding for clustering analysis,â in International conference on machine learning. PMLR, 2016, p. 478â487. [10] X. Guo, L. Gao, X. Liu, and J. Yin, âImproved deep embedded clustering with local structure preservation.â in Ijcai, vol. 17, 2017, p. 1753â1759. [11] L. van der Maaten and G. Hinton, âVisualizing data using t-sne,â Journal of Machine Learning Research, vol. 9, no. 86, p. 2579â2605, 2008. [Online]. Available: http://jmlr.org/papers/v9/vandermaaten08a.html [12] R. J. Campello, D. Moulavi, and J. Sander, âDensity-based clustering based on hierarchical density estimates,â in Pacific-Asia conference on knowledge discovery and data mining. Springer, 2013, p. 160â172.