Paper deep dive

Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition

Kaisen Yang, Qingle Liu, Kejin Wang, Yicheng Zhao, Jieming Li, Shenghan Zheng, Ruize Yang, Bojun Yang, Heng Gong, Xiang Gao, Lanyue Zhang, Kaiyu Zhong, Zhuo Liu, Shaoxuan Li, Chengxi Li, Yong Yan, Weixuan Zhang, Tianwei Luo, Situ Wang, Youjie Zheng, Sihan Zhao, Shengyuan Wang, Huan-ang Gao, Jiazheng Xu, Xiaohui Xie, Wentao Han, Hongning Wang

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 117

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 10/11/2026, 2:07:09 AM

Summary

This paper introduces Adversarial Heuristic Learning (AHL), a paradigm where AI agents refine game policies and software without changing model weights. The authors present AAArena, a benchmark with 12 adversarial games and 1,920 human programs. Experiments show that Opus5.5 with Claude Code achieved top rankings in 6 games, highlighting the potential of learning from replays and opponent selection, while noting challenges in complex rule understanding.

Entities (8)

AAArena · benchmark · 99%Tsinghua University · organization · 99%Adversarial Heuristic Learning · method · 98%Claude-Code · tool · 95%Opus5.5 · model · 95%AntWar · game · 90%MiracleAquaWar · game · 90%SnakeGo · game · 90%

Relation Signals (6)

AAArena → contains → 12 adversarial games

confidence 99% · We introduce AAArena, a benchmark comprising 12 authentic adversarial games

AAArena → contains → 1,920 archived human programs

confidence 99% · comprising 12 authentic adversarial games and 1,920 archived human programs

Adversarial Heuristic Learning → uses → AI Agents

confidence 98% · AHL, a paradigm that uses AI agents as learning engines

Opus5.5 → achievedrank → rank-1

confidence 95% · Opus5.5 with Claude Code earns 6 gold medals (rank-1 finishes)

Opus5.5 → usedwith → Claude-Code

confidence 95% · Opus5.5 with Claude Code earns 6 gold medals

AI Agents → learnsfrom → replays

confidence 92% · agents learn from both on-policy replays of their own matches and off-policy replays of other players’ matches

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Adversarial games have driven advances from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remain challenging. AI agents offer an alternative by turning game experience into revisions of executable policies. Building on heuristic learning (HL), we formalize Adversarial Heuristic Learning (AHL), a paradigm that uses AI agents as learning engines to refine game policies and supporting software while keeping model weights fixed. We introduce AAArena, a benchmark comprising 12 authentic adversarial games and 1,920 archived human programs, with an evaluation protocol modeled on real-world game competitions. Agents interpret rules, choose opponents, analyze replays, and revise game agents to achieve their highest ranking within fixed match and evaluation budgets. We evaluate \val{completedmodels} model and harness configurations: Opus5.5 with Claude Code earns 6 gold medals, while no evaluated configuration tops the remaining 6 human ladders. Performance is generally weaker in games with more complex rule specifications. Further experiments show that opponent selection and dense feedback support policy improvement, and that agents learn from both on-policy replays of their own matches and off-policy replays of other players' matches. These results highlight HL's potential in adversarial games and identify persistent challenges in game understanding, strategy implementation, and long-horizon policy development.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

113,486 characters extracted from source content.

Expand or collapse full text

Can AI Agents Learn Their Way to the Top? Evaluating Heuristic Learning in a Long-Running Game Agent Competition Kaisen Yang 1,∗,† , Qingle Liu 1,∗,† , Kejin Wang 1,∗ , Yicheng Zhao 1,∗ , Jieming Li 1,∗ , Shenghan Zheng 1,∗ , Ruize Yang 1,∗ , Bojun Yang 1,∗ , Heng Gong 1 , Xiang Gao 1 , Lanyue Zhang 1 , Kaiyu Zhong 1 , Zhuo Liu 1 , Shaoxuan Li 1 , Chengxi Li 1 , Yong Yan 1 , Weixuan Zhang 1 , Tianwei Luo 1 , Situ Wang 1 , Youjie Zheng 1 , Sihan Zhao 1 , Shengyuan Wang 1,2 , Huan-ang Gao 1 , Jiazheng Xu 1 , Xiaohui Xie 1 , Wentao Han 1 , Hongning Wang 1,‡ 1 Department of Computer Science and Technology, Tsinghua University 2 College of AI, Tsinghua University ∗ Core Contributor; † Project Lead; ‡ Corresponding Author. Adversarial games have driven advances from heuristic search to reinforcement learning, yet learning and adapting strategies from limited samples remain challenging. AI agents offer an alternative by turning game experience into revisions of executable policies. Building on heuristic learning (HL), we formalize Adversarial Heuristic Learning (AHL), a paradigm that uses AI agents as learning engines to refine game policies and supporting software while keeping model weights fixed. We introduce AAArena, a benchmark comprising 12 authentic adversarial games and 1,920 archived human programs, with an evaluation protocol modeled on real-world game competitions. Agents interpret rules, choose opponents, analyze replays, and revise game agents to achieve their highest ranking within fixed match and evaluation budgets. We evaluate 7 model and harness configurations: Opus5.5 with Claude Code earns 6 gold medals (rank-1 finishes), while no evaluated configuration tops the remaining 6 human ladders. Performance is generally weaker in games with more complex rule specifications. Further experiments show that opponent selection and dense feedback support policy improvement, and that agents learn from both on-policy replays of their own matches and off-policy replays of other players’ matches. These results highlight HL’s potential in adversarial games and identify persistent challenges in game understanding, strategy implementation, and long-horizon policy development. Contact: yks23@mails.tsinghua.edu.cn, lql24@mails.tsinghua.edu.cn Project page: https://aaarena.net Code: https://github.com/THU-CST-SAST/AAArena 1 Introduction Adversarial games have long served as a testing ground for artificial intelligence. Early systems relied on hand-crafted heuristics, evaluation functions, and tree search; reinforcement learning and self-play subsequently advanced experience-driven policy optimization (Campbell et al., 2002; Coulom, 2007; Silver et al., 2018). Opponents form part of each player’s environment, and changes in their strategies shift the learning target. Adapting and revising strategies in a changing environment with limited interaction remains a challenge (Hernandez-Leal et al., 2019). AI agents now work in code repositories (Yang et al., 2024). They also operate graphical interfaces (Xie et al., 2024), while agent-design and recursive program-improvement methods let models revise the software supporting their behaviour (Hu et al., 2025c; Zelikman et al., 2024). These capabilities suggest that AI agents can serve as learning engines. In Learning Beyond Gradients, Weng (2026) describes heuristic learning 1 arXiv:2610.12341v1 [cs.AI] 8 Oct 2026 MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden Pacman SnakeGo Rollman MoneCraft AntWar LostSpace AquaWar Generals Dorado Miracle LOTA AntWar2 2198 2723 1253 2003 527 1052 23422567 1947 2697 2005 2755 1444 1744 2134 3034 2305 2905 20472572 2558 3308 2336 3086 GLM-5.3Kimi K3 Qwen 3.8 DeepSeek V4 Pro LongCat 2.0 GPT6-sol Opus5.5 Human SOTA Figure 1 AAArena overview. Left: the 12 games in ascending rule-description size (abstract syntax tree node count), with player counts, role symmetry, and information structure (Table 1). Right: retained-champion Elo for seven models and human SOTA. Each spoke reports native Elo, with game-specific tick intervals; higher Elo lies farther outward. Human SOTA forms the dashed regular dodecagon, with its Elo labeled in red. Model colours match Figure 3. (HL), in which an AI agent accumulates experience in editable software structures. The agent reads failures, logs, and replays to revise programs and their supporting software. Recent work also studies program-policy revision and tournament-based code evolution (Garnier et al., 2026; Wang et al., 2026; Fu et al., 2026; Yang et al., 2026a). The Tsinghua University Agent Competition is an annual programming competition in which contestants build game agents to compete in adversarial games (Saiblo). Each year, organizers introduce a new game and publish its rules. Contestants interpret the rules, design executable strategies, and improve their programs through freely chosen matches to compete for leaderboard positions. Can AI agents undertake the same process, learning from rules and game feedback to continually improve the game agents they build? This requires translating game understanding into executable policies and planning tests and revisions within a limited budget. We formalize this feedback-driven program improvement as Adversarial Heuristic Learning (AHL), an instantiation of HL in adversarial tasks. The AI agent revises code policies and supporting software from game experience; the game agent executes these policies to compete. The base model’s weights remain fixed. We build Agents for Agents Arena (AAArena) from the competition’s past games, runnable backends, and archived human programs, turning this development process into a unified evaluation task. Across 12 original games,AAArenabudgets selected matches and full-pool evaluations separately to assess whether AI agents can improve their programs and climb human-program ladders. Our strongest configuration, Opus5.5 with Claude Code, earns 6 gold medals (rank 1 in the frozen human ladders) across 12 games; no evaluated model tops the remaining 6 ladders. Our contributions are threefold. • We formalize adversarial heuristic learning (AHL), a paradigm that uses AI agents as learning engines to learn from adversarial matches and iteratively refine executable game policies. •We introduceAAArena, a benchmark comprising 12 authentic adversarial games and 1,920 archived human programs, with an evaluation protocol modeled on real-world adversarial game competitions. •We evaluate seven AI model–harness configurations onAAArena, with the strongest topping six 2 human ladders, while the remaining six games generally have more complex rules and remain untopped by any evaluated configuration. Further experiments show the importance of opponent selection and dense feedback, and the potential of off-policy experience from other players’ replays to overcome learning plateaus. Case studies reveal persistent challenges in game understanding, strategy implementation, and long-horizon policy development. 2 Adversarial Heuristic Learning MiracleAquaWar LostSpace SnakeGo AntWar Generals Rollman 2P / Asym / ● AntWar2 Agent for Agent Iteration Loop Read Write Replay Rule Replay Dictionary SDK & Demo Leaderboard Game Pack Replays History Agent Code Policy Evolves Environment Validator Fast legality and runtime check Frozen Pool Agent’s Gaming Agent Vs Human’s Gaming Agent Run on AI9 / Saiblo Platform Pacman MoneCraft LOTADorado Notes Codes Static Iterating Run with persistent session Led by Goal · Learn from replays · Code as policy / gaming agent · Keep codeexplainable & editable x No internet searching x No reading rivals’ code Principles 흅 풕−ퟏ →흅 풕 Select Action 1. Small Match 2. Full Eval > > > > Passed Code Selected Rival(s) Dense if Small Match Binary if Full Eval Update Read AAArena: 12 Adversarial Agent Games 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ● 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ◑ 2P / Sym / ○ 4P / Sym / ◑ # Players / Symmetry / Information Sym | Asym Perfect / Partial / Hidden Figure 2 Agent for agent iteration loop. An AI agent uses game resources, replays, and retained notes to revise an executable game-agent policy. Validated snapshots enter small matches against selected opponents or full-pool evaluations against the frozen pool. Small matches return dense replays; full-pool evaluations return outcomes, Elo, and rank. The agent retains development history across revisions. 2.1 Preliminaries Game agent. An adversarial gameGhas a shared environment andn ≥2 game agents, with conflicting objectives for at least two agents. Games can assign asymmetric roles and restrict each agent’s observations. Each game agent i executes a code policy P i that selects actions as a i t ∼ π P i (·| h i t ), whereh i t is its permitted observation history anda i t is its action. The environment processes these actions to determine the outcome, while the submitted program acts autonomously throughout the match. AI agent. An AI agentA θ develops the game agent’s policy between matches;θdenotes the base model’s parameters. It reads game resources, source code, and feedback to revise the policy and its supporting software. In Agents for Agents, the resulting game agent’s competitive performance measures the AI agent’s policy-development ability. Heuristic learning. Heuristic learning (HL) incorporates experience through an AI agent’s direct revisions to a software system (Weng, 2026). Feedback may include rewards, tests, logs, or replays. The agent can revise policy logic, state representations, tests, configuration, and memory. We call the executable policy and its supporting software and records the heuristic system. 2.2 Heuristic learning in adversarial games Definition 1 (Adversarial heuristic learning). Adversarial heuristic learning (AHL) is an instantiation of HL in adversarial tasks. An AI agent uses experience and feedback to revise a heuristic system and improve competitive performance, while the base model’s parameters θ remain fixed. 3 The heuristic system consists of an executable policyPand development stateC, which stores supporting tools, tests, experiment records, and memory. Figure 2 summarizes the interaction loop; Algorithm 1 gives its formal form. The AI agent acquires game experience, interprets feedback, and revises the heuristic system. It chooses how to collect experience and organize revisions; the algorithm leaves feedback sources, evaluation schedules, and stopping criteria to the task setting. Algorithm 1 Adversarial Heuristic Learning (AHL) Input: AI agent A θ with fixed parameters θ; game resources R; initial policy P 0 and development state C 0 1: (P,C)← (P 0 ,C 0 ) 2: while the stopping criterion remains unmet do 3: Acquire game experience through interactions or available records 4: Interpret feedback using A θ and (R,P,C) 5: Revise P and its supporting state C from the feedback 6: end while 7: Output: executable policy P 3 AAArena 3.1 Games and opponent pools The 12 competition games cover tactical combat, territory control, resource management, and survival. Each has a verified, frozen opponent pool (Table 1). Agents receive rules, SDKs, replay documentation, a leaderboard, and a permitted reference program. Appendices A and B describe the games and resources. Rule-description size. Kolmogorov complexityK(R) =min p:U(p)=R |p|is the length of the shortest program describingRon a fixed universal machineU(Vitányi, 2020). We reportC AST (G) =|AST(R G )|, the node count of the rules’ abstract syntax tree (AST), and rule atoms (RA), the count of independently changeable rule propositions. These representation-dependent counts describe rule specifications; they do not estimate exact Kolmogorov complexity or optimal-play difficulty. The same rule-analysis procedure gives Go 501 AST nodes and 125 rules, and chess 1,551 AST nodes and 238 rules. The suite spans 1,127–6,368 AST nodes and 219–1,008 rules. GameStrategic setting Pool programs AST nodes Rules (RA) PacmanCompetitive maze collection44 1,127219 SnakeGoSnake movement and territory141 1,538308 RollmanAsymmetric maze pursuit64 1,539312 MoneCraft Mining and resource control112 1,823347 AntWarTower defence and economy114 2,160368 LostSpace Multiplayer survival and escape111 2,297426 AquaWar Hidden-identity tactical combat170 2,531439 GeneralsTerritorial and army control197 2,683461 DoradoResource competition and army development323 4,241616 MiracleHex-grid unit tactics253 4,391581 LOTAHero control and lane combat200 4,513684 AntWar2Ordered tower-defence operations191 6,368 1,008 Table 1 The 12 games, ordered by increasing abstract syntax tree (AST) node count for their rule representations. RA counts rule atoms, or independently changeable rule propositions. Pool sizes count executable programs, not unique human participants. 4 3.2 Interaction interfaces Small matches support targeted diagnosis by letting the AI agent select opponents and inspect detailed replays. Full-pool evaluations measure competitive progress against the complete opponent pool and return Elo, rank, and per-opponent outcomes. The first channel concentrates feedback on specific strategic hypotheses; the second supplies a common reference for comparing snapshots. Both evaluate immutable programs through the same game referee and return feedback for subsequent revisions. 3.3 Evaluation protocol Each run receives two budgets:B s small-match units (one per requested opponent) andB ℓ full-pool evaluations. Default limits areB s = 128 andB ℓ = 16. Within these limits, the agent chooses when to revise, validate locally, or request either kind of feedback. The final result is the highest officially evaluated Elo attained during the run, together with its corresponding rank. IfVis the set of officially evaluated snapshots, including the common initialization baseline, then Score(B s ,B ℓ ) = max P∈V br(P ). We retain the policy attaining this score, allowing later revisions to explore without discarding a stronger snapshot. Opponent Elo ratings stay fixed throughout the run. Appendix B specifies ranking, budget accounting, and the baseline’s separate allowance. 4 Experiments 4.1 Performance of AI Agents Developing Game Agents Table 2 Main results across 12 adversarial games, ordered by rule complexity. Rule complexity uses AST node counts (Section 3.1). A green check marks at least one configuration reaching rank 1; a red cross marks none. Each model entry gives Elo (↑) and rank (↓) under a 128-small/16-full budget; a gold medal marks rank 1. Bold marks the best value per row. All models use max reasoning effort. Opus5.5 uses Claude Code; others use Codex. Rule complexity: low (top) to high (bottom)↓ Game Rule complexity Topped Opus5.5GPT6-solGLM-5.3Kimi K3 DeepSeek V4 ProQwen 3.8 LongCat 2.0 Elo RankElo RankElo RankElo RankElo RankElo RankElo Rank Pacman1,127✓ 2703.912367.612281.612196.32 2096.84 2236.111835.0 10 SnakeGo1,538✓ 1860.911129.64 1141.24 1165.93 1097.05 1107.55 1153.34 Rollman1,539✓ 1010.21 581.21581.21636.91807.71636.91534.21 MoneCraft1,823✓ 2387.112418.212359.212288.23 2288.23 2288.23 2069.99 AntWar2,160 × 1405.15 1311.76 1464.25 1130.29 1167.29 1355.05 1028.3 11 LostSpace2,297✓ 2220.211625.27 1980.12 1916.12 1956.82 1797.73 1052.9 77 AquaWar2,531✓ 1552.511552.511494.111494.111464.511539.711116.1 33 Generals2,683 × 1689.6 10 1528.4 13 1441.2 17 1433.3 17 1340.8 23 1624.9 10 1010.3 58 Dorado4,241 × 2252.42 2271.52 2210.32 2023.4 16 2187.83 2035.8 14 1512.4 156 Miracle4,391 × 1704.86 1469.5 20 1436.8 26 1432.8 26 1448.8 24 1500.2 18 1500.2 18 LOTA4,513 × 2224.2 15 2140.3 22 2147.4 22 2133.2 22 2140.3 22 2099.0 25 1585.1 74 AntWar26,368 × 2234.45 1440.3 34 1977.19 1644.2 19 1569.6 25 2183.56 1460.0 31 Gold medals↑6442231 Setup. We evaluate seven model–harness configurations with the default 128-small/16-full budgets and retained-champion scoring from Section 3.3. All models usemaxreasoning effort. Opus5.5 uses Claude Code; the other models use Codex, so the comparison includes both model and harness differences. Runs stop upon reaching rank 1. We repeat each setting three times and report the median result; each run contributes its best officially evaluated policy. Main runs use on-policy replays from matches involving the learner’s own 5 policy. 1 Appendix B.1 describes the implementation. We use GLM-5.3 for subsequent analyses because its complete interaction logs and fixed harness support comparisons across runs. Results. AI-developed game agents top six frozen human ladders; every evaluated configuration falls short of rank 1 on the other six (Table 2 and Figures 1 and 3). Opus5.5 with Claude Code earns 6 gold medals and the highest or tied-highest Elo on nine games. Several configurations reach rank 1 on Rollman and AquaWar, whereas only Opus5.5 with Claude Code does so on LostSpace and SnakeGo. The remaining gaps also differ across games. The best policy ranks second on Dorado, while the best policies on LOTA, Generals, and Miracle remain outside the top five. Gold medals summarize rank-1 coverage; per-game Elo and rank preserve these differences without averaging across game-specific rating scales. Five of the six games with smaller rule descriptions have a rank-1 finish, compared with one of the six games with larger descriptions (Table 2). This pattern links larger rule specifications with gaps in competitive policy development. 0 250 500 750 1000 1250 Rollman 0 500 1000 1500 2000 2500 3000 Pacman 0 500 1000 1500 2000 AntWar 0 500 1000 1500 2000 AquaWar 0 500 1000 1500 2000 Generals 0 500 1000 1500 2000 2500 LostSpace 0 500 1000 1500 2000 Miracle 0 500 1000 1500 2000 2500 Dorado 0 500 1000 1500 2000 2500 3000 MoneCraft 0 500 1000 1500 2000 2500 LOTA 0 500 1000 1500 2000 2500 SnakeGo 0 500 1000 1500 2000 2500 AntWar2 Figure 3 Per-game Elo of the retained champion. Each game orders model configurations by Elo. Takeaway. The strongest configuration surpasses human champions in six games, while challenging games remain beyond the evaluated agents. Performance is generally weaker in games with more complex rule specifications, highlighting the challenge of translating rule understanding into competitive policies. 4.2 How Do Policies Evolve During Learning? 4.2.1 Evolution Patterns Across Games We examine the 12 GLM-5.3 runs under the default 128-small/16-full budget to distinguish how policies improve across games. Each trajectory records the Elo and rank of individual full-pool submissions, the order of small-match and full-pool requests, and token use. Submission scores can fall after a revision even though the retained-champion score keeps the best result. The trajectories follow three patterns: Success: reaching rank 1. Rollman, MoneCraft, Pacman, and AquaWar reach rank 1 and stop before exhausting the full-pool evaluation budget. These runs show that feedback-driven program revision can produce policies that top the human-program ladders within the given interaction limits. Progress: gains with regressions. AntWar, Dorado, LostSpace, SnakeGo, and AntWar2 improve overall, but individual revisions can reduce performance. Dorado makes several consecutive gains in the middle of its run, whereas AntWar2 suffers a sharp intermediate drop and LostSpace ends with prolonged fluctuations. Retaining the best evaluated policy preserves earlier gains when later edits regress. 1 Here, on-policy and off-policy label replay sources, not reinforcement-learning algorithm classes. 6 Stagnation: persistent plateaus. Generals, Miracle, and LOTA show extended intervals with little improve- ment after initial adaptation. Further revisions and evaluations repeatedly return to similar performance levels, showing that continued interaction does not consistently yield stronger policies within these runs. Appendix B.6 presents the complete trajectories in Figures 11–13. We next trace the code changes behind Dorado’s progress to show how match feedback leads to specific policy revisions. 4.2.2 Case Study: Dorado GLM-5.3 improves its Dorado policy from rank 41 to rank 2. Figure 4 traces its performance, and Table 3 shows the corresponding code changes. Dorado has two players, each controlling a base and recruiting heroes from four classes. Mining supplies gold for recruitment and for upgrading heroes near the base. Destroying the opposing base wins; simultaneous destruction draws. If both bases remain at the turn limit, gold plus the level-based value of each player’s heroes determines the result. Correct mine allocation. The central mine stays open, while other mines have limited opening windows. Each mine uses the highest individual mining rate; sending several heroes to one mine does not add their yields. The agent excludes dead miners from occupancy counts, penalizes assigning several miners to the same mine, and tracks mine opening windows. After these revisions, the policy reaches rank 14. Concentrate upgrades. The agent then upgrades two core heroes while preserving mining income. The next full-pool evaluation raises the policy from rank 14 to rank 3 (+147.7 Elo), the largest adjacent gain. Refine counterattacks. The agent later allows a one-hero counterattack only after sustained nearby aggression and reaches rank 2. Other attempts, including a four-hero roster and unconditional use of theSacrificeability, regress to ranks 46 and 43. The agent restores its best policy before trying another revision. (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1800 2000 2200 Full-pool evaluation Elo Rank 41363935463543291814368942 Leader; final upper (#1) Initial upper (#40) Initial lower (#41) Final lower (#2) 13672108144 1 25 50 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 Increment (M) 1481216 0 50 100 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Figure 4 GLM-5.3 on Dorado. Full-pool Elo and rank, small–full interaction order, and token use under the 128/16 budget. Appendix B.5 specifies budget-slot placement, batch markers, rank annotations, and token accounting. 7 Table 3 Dorado champion milestones. Mechanism descriptions with added (+) and deleted (-) C++ diff excerpts. Ellipses omit context. Full-pool eval.Elo Rank Policy adjustmentCode excerpt 2 1782.036 Keep mining; attack only a vulnerable enemy base. +232 enemyBase != NULL && enemyBase->hp < 1800 && (int)myHeroes.size() >= 4; 4 1800.735Use allliving heroesto defend a serious threat. +245 int count = seriousThreat ? (int)defenders.size() : 2; 6 1802.835Triggerfull defencefor active threats or recent damage. +228 const bool seriousThreat = ... || baseThreat || recentBaseHit; 8 1862.829 Exclude dead miners from occupancy; spread assignments. +343 if (alive && mineUsable(it->second, round)) occ[it->second]++; -477 ... + 600L * occ[m]; +472 ... + 2000L * occ[m]; 9 1977.918 Penalize duplicate mine assignments more strongly. -472 ... + 2000L * occ[m]; +472 ... + 5000L * occ[m]; 10 2040.114 Track mine opening windows and prioritize open mines. +217 s_mineExistUntil[i] = round + exist->val[0] - 1; + 36 #define MINE_OPEN_OCCUPANCY_PENALTY 50000L 11 2187.83 Concentrate upgrades on a two-hero core; preserve mining income. +296 if (s_dutyId < 0 && (int)s_leveledCore.size() < 2 ...) +316 if (!coreHammerguard && type == "hammerguard") priority = 0; 16 2210.32 Allow one-hero counterattacks only after sustained aggression. +410 s_aggressionRounds = tightAggression ? s_aggressionRounds + 1 : 0; +412 const bool corePush = activeCoreHeroes >= 2 && aggressionThreat; 4.3 Does More Interaction Overcome Performance Plateaus? We test whether more interaction resolves GLM-5.3’s weak results on Generals, Miracle, LOTA, and AntWar2. Each continuation keeps the existing workspace and evaluation ledger and adds 256 small-match units and 32 full-pool evaluations. The total budget is 384/48, three times the original 128/16. Table 4 compares the best policies at both budget levels. Table 4 Threefold-budget continuation for GLM-5.3. Elo and rank of the retained champion at each budget. The extension adds 256 small units and 32 full-pool evaluations to the same run. A positive∆Rank indicates an improved position. 128 / 16384 / 48Change GameEloRankEloRank∆Elo∆Rank Generals1441.2171441.217+0.0+0 Miracle1436.8261486.819+50.0+7 LOTA2147.4222147.422+0.0+0 AntWar21977.192117.06+139.8+3 Tripling the budget leaves two retained champions unchanged and improves two; none reaches rank 1 (Table 4). Generals and LOTA keep their earlier best scores, while Miracle moves from rank 26 to 19 and AntWar2 from rank 9 to 6. Figure 5 shows flat intervals and fluctuations through the extended runs. These outcomes describe a finite continuation of four GLM-5.3 runs under the same development protocol. Takeaway. Tripling the interaction budget yields limited further improvement, suggesting that the default budget brings these runs close to a performance plateau. Within the tested range, more interaction alone does not overcome the remaining performance gaps. 4.4 Which Feedback Choices Affect Policy Improvement? We select one representative game from each trajectory category: Pacman (Success), AntWar (Progress), and Miracle (Stagnation). All ablations use GLM-5.3 with max reasoning effort and the same 128-small/16-full budget. Each axis varies one component while keeping its remaining settings fixed. We report retained- champion Elo, with exact Elo/rank tables in Appendix C. 8 Threefold budget Generals best Elo 1441.2, rank 17 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1163248 1200 1400 1600 1800 2000 2200 Full-pool evaluation Elo Rank Peaks 22 17 20 18 18 17 17 17 20 17 20 18 171717 17 18171717 17 18 17 18 18 18 Leader (#1) Initial upper (#26) Initial lower (#27) Final upper (#16) Final lower (#17) 1108216324432 1 50 100 150 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 20 40 Increment (M) 1163248 0 100 200 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Miracle best Elo 1486.8, rank 19 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1163248 1400 1600 1800 2000 Full-pool evaluation Elo Rank Peaks 2727 26 27 26 2628 19 55 19 20 20 20 20 20201920 20 23 20 2020 19 23 Leader (#1) Initial upper (#26) Initial lower (#27) Final upper (#18) Final lower (#19) 1108216324432 1 100 200 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 30 Increment (M) 1163248 0 50 100 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation LOTA best Elo 2147.4, rank 22 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1163248 2000 2200 2400 2600 Full-pool evaluation Elo Rank Peaks 2229 23 2225 22 25 22 28 27 2525 25 2725282425 24 252928 Leader (#1) Initial/final upper (#21) Initial/final lower (#22) 1108216324432 1 25 50 75 100 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 5 10 15 Increment (M) 1163248 0 20 40 60 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation AntWar2 best Elo 2117.0, rank 6 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1163248 1500 2000 Full-pool evaluation Elo Rank Peaks 14 50 13 16 13 1313 9 11 9 14 8 9 7 9 6 7 7 7 7 777 7 7 7 77 Leader (#1) Initial upper (#15) Initial lower (#16) Final upper (#5) Final lower (#6) 1108216324432 1 50 100 150 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 30 40 Increment (M) 1163248 0 100 200 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Figure 5 Policy performance under a threefold budget. Full-pool Elo, interaction order, and token use for four continued runs. Ranks appear at every third full-pool evaluation and local Elo peaks. Plot conventions appear in Appendix B.5. 9 4.4.1 Does Opponent Selection Affect Performance? We compare four opponent-selection strategies. Model selection lets the agent choose opponents, as in the main experiment. The ladder starts at rank 30 and moves toward stronger opponents without returning to weaker ones. Random sampling draws uniformly from the pool; top-four uses only the four highest-ranked opponents. All conditions use dense feedback and a small-match batch size of four. The ladder leads on Pacman and AntWar, while model selection leads on Miracle (Figure 6). Model selection outperforms random and top-four selection in all three games; the ladder trails random selection on Miracle. Top-four selection yields the lowest Elo throughout. One hypothesis is that gradually stronger or agent-selected opponents provide more actionable feedback than repeated mismatches. ModelLadderRandomTop-4 0 500 1000 1500 2000 2500 3000 2097 #4 2128 #3 2014 #6 1897 #10 Elo Pacman ModelLadderRandomTop-4 0 500 1000 1500 2000 2500 3000 1180 #8 1312 #6 1154 #9 1056 #11 Elo AntWar ModelLadderRandomTop-4 0 500 1000 1500 2000 2500 3000 1482 #20 1449 #24 1457 #24 1406 #31 Elo Miracle Figure 6 Opponent selection. GLM-5.3 retained-champion Elo and rank with model, ladder, random, and top-four selection; all use dense feedback and batch size 4. Plot conventions appear in Appendix B.5. Takeaway. Opponent selection is important for effective policy development: agents need opponents whose matches provide useful information for improvement. Agent-directed selection and progressively stronger opponents both perform well, showing the value of choosing opponents for learning. 4.4.2 Does Dense Replay Feedback Affect Performance? We compare detailed small-match replays and their translations with binary win/non-win feedback. Both conditions keep conversation history, agent-selected opponents, and full-pool feedback; the agent chooses the batch size. Dense feedback yields higher retained-champion Elo in all three games, with gains that vary by game (Figure 7). Detailed replays supply state, action, and consequence information for diagnosing decision errors; binary feedback supplies the outcome alone. The gains support retaining this diagnostic context in AHL under the tested budgets. DenseBinary 0 500 1000 1500 2000 2500 3000 2236 #1 1710 #11 Elo Pacman DenseBinary 0 500 1000 1500 2000 2500 3000 1119 #9 881 #22 Elo AntWar DenseBinary 0 500 1000 1500 2000 2500 3000 1528 #15 1461 #23 Elo Miracle Figure 7 Replay feedback. Dense replays versus binary win/non-win feedback under the same 128/16 budget. Labels give Elo and rank. Dense feedback improves performance in all three representative games. 10 Takeaway. A wider feedback channel enables more efficient policy development. Dense replays provide decision-level information that agents can use to diagnose failures and guide code revisions, improving performance under a fixed budget. 4.4.3 Does the Number of Opponents per Small Match Matter? We compare 1, 2, 4, and 8 opponents per small-match submission, keeping model selection and dense feedback. The total allowance remains 128 opponent tests and 16 full-pool evaluations. Intermediate batches perform best: four opponents on Pacman and Miracle, and two on AntWar (Figure 8). Eight opponents per request spend the small-match allowance in 16 full-sized requests, compared with 32–64 requests for batches of 2–4. The ranking is consistent with a trade-off between matchup breadth per request and opportunities to test revised policies. The observed preference for 2–4 opponents concerns these three games and settings. 1248 0 500 1000 1500 2000 2500 3000 3500 2041 #5 2196 #2 2704 #1 2160 #2 Batch size Elo Pacman 1248 0 500 1000 1500 2000 2500 3000 3500 1130 #9 1433 #5 1209 #8 1119 #9 Batch size Elo AntWar 1248 0 500 1000 1500 2000 2500 3000 3500 1339 #45 1491 #19 1519 #16 1465 #23 Batch size Elo Miracle Figure 8 Small-match batch size. Number of opponents per submission, with a fixed total allowance of 128 opponent tests and 16 full-pool evaluations. Labels give Elo and rank. The best size is 4, 2, and 4 for Pacman, AntWar, and Miracle, respectively. Takeaway. The ablations show that opponent selection, replay detail, and batch size all affect retained- champion performance under a fixed budget. We next vary which players generate the replay evidence. 4.5 Can Agents Learn from Off-Policy Replays? We test whether off-policy replays from matches between other policies support policy improvement. On Pacman, AntWar, and Miracle, GLM-5.3 chooses replays from a catalog of matches between frozen-pool opponents. It can request 128 replay views and 16 full-pool evaluations of its own policy. Reasoning effort, game resources, opponent ladders, and budget ceilings match the on-policy setting. Only the replay source changes; the agent still receives full-pool feedback on its own policy. All three off-policy runs improve from their first evaluated policy to the retained champion: Pacman advances from rank 10 to 1, AntWar from 41 to 19, and Miracle from 27 to 15 (Figure 9). Miracle provides the clearest contrast with learning from the agent’s own matches. Its on-policy trajectory plateaus with a retained champion at rank 26, while the independent off-policy run reaches rank 15 (Figure 10). This contrast suggests that other players’ experience can support progress beyond the plateau observed with self-generated replays. On Pacman, both settings reach rank 1; on AntWar, on-policy learning performs better, reaching rank 5 compared with 19 off-policy. 11 On-policy and off-policy learning Pacman best rank: on-policy 1, off-policy 1 (a) Full-pool Elo(b) Token usage 1481216 1800 2000 2200 2400 Full-pool evaluation Elo On rank 101052441 Off rank 101010105443235821 Leader / Final lower (#1) Initial upper (#9) Initial lower (#10) 0 2 4 6 8 Increment (M) 1481216 0 10 20 30 Full-pool evaluation Cumulative (M) On-policyOff-policy On-policy incrementOff-policy increment Markers: full evaluations; bars: intervening tokens AntWar best rank: on-policy 5, off-policy 19 (a) Full-pool Elo(b) Token usage 1481216 0 1000 2000 Full-pool evaluation Elo On rank 11985855555555555 Off rank 411113320201920202020201919202020 Leader (#1) Initial upper (#40) Initial lower (#41) Final upper (#18) Final lower (#19) 0 20 40 Increment (M) 1481216 0 50 100 150 Full-pool evaluation Cumulative (M) On-policyOff-policy On-policy incrementOff-policy increment Markers: full evaluations; bars: intervening tokens Miracle best rank: on-policy 26, off-policy 15 (a) Full-pool Elo(b) Token usage 1481216 1000 1500 2000 Full-pool evaluation Elo On rank 27392727292726272726272627272827 Off rank 27589445792626234938351815324549 Leader (#1) Initial upper (#26) Initial lower (#27) Final upper (#14) Final lower (#15) 0 10 20 30 Increment (M) 1481216 0 20 40 60 Full-pool evaluation Cumulative (M) On-policyOff-policy On-policy incrementOff-policy increment Markers: full evaluations; bars: intervening tokens Figure 9 On-policy and off-policy learning trajectories. Full-pool Elo and ranks, cumulative tokens, and token increments for GLM-5.3 with self-generated and other players’ replays. Both conditions receive full-pool feedback on their own policies. Appendix B.5 specifies rank rows, reference levels, and token accounting. 12 On-policyOff-policy 0 500 1000 1500 2000 2500 3000 2281.6 #1 2404.4 #1 Elo Pacman On-policyOff-policy 0 500 1000 1500 2000 2500 3000 1464.2 #5 930.7 #19 Elo AntWar On-policyOff-policy 0 500 1000 1500 2000 2500 3000 1436.8 #26 1528.2 #15 Elo Miracle Figure 10 On-policy versus off-policy replay learning. Retained-champion Elo and rank for GLM-5.3 under the same 128/16 budget ceilings. On-policy scores are the main-table results. Labels give Elo and rank; each panel uses the same zero-based Elo scale. Both Pacman runs terminate early upon reaching rank 1. Takeaway. Off-policy replays broaden the experience available to AHL and can support progress beyond the plateaus observed with on-policy learning. Other players’ experience can suggest alternative strategies, with the potential to outperform learning from the agent’s own matches. 5 Related Work Table 5 Representative game-agent evaluation protocols. Feedback denotes information available to the evaluated AI agent:✓indicates availability and a dash indicates absence in the named setting. Traces include in-game observations or post-match logs. CodeClash refers to its standard, closed-code tournament; GACL to code generation. WorkAI output OpponentsFeedbackProtocol Scores Traces Direct play GameBench (Costarelli et al., 2024) ActionsHuman + AI players✓ Direct play TextArena (competitive) (Guertler et al., 2025) ActionsHuman + AI players✓ Direct play Program development GACL (Game Agent Coding League, n.d.) ProgramsAI-developed programs – One-shot Vibe-coding (IR) (Danassis and Goel, 2025) ProgramsHuman-written + AI-developed programs ✓– Preset CATArena (games) (Fu et al., 2026) ProgramsAI-developed programs✓ Preset CodeClash (tournament) (Yang et al., 2026a) ProgramsAI-developed programs✓ Preset C:Ladder (Yang et al., 2026a)ProgramsHuman-written programs ✓ Preset EvoPolicyGym (Wang et al., 2026) ProgramsNone✓ Agent-scheduled AAArena (ours)ProgramsHuman-written programs ✓ Agent-scheduled Programs are executable code policies. Preset means fixed refinement rounds, tournament rounds, or opponent order; Agent- scheduled means that the agent chooses evaluation requests within a budget. These labels describe the outer schedule. Scores include outcomes, returns, and rankings; traces include observations, match logs, and replays. C:Ladder starts from a human solution. IR denotes iterative refinement. Rows describe specific evaluation conditions: GACL covers code generation, and CodeClash covers its default code-opaque tournaments. EvoPolicyGym budgets single-agent episodes and hides validation feedback;AAArenauses frozen human programs and returns full-pool scores during development under a separate evaluation budget. 13 Competitive program development. Competitive program-development benchmarks differ in their outputs, opponents, feedback, and revision protocols (Table 5). GACL evaluates generated game agents and ProxyWar includes code repair (Game Agent Coding League, n.d.; Peng et al., 2026); the vibe-coding tournament compares student and model programs with a self-play refinement condition (Danassis and Goel, 2025). CATArena supplies tournament logs and peer code across revision rounds (Fu et al., 2026), while CodeClash studies competitive code evolution and C:Ladder advances through ranked human programs after each win (Yang et al., 2026a). EvoPolicyGym budgets episodes in single-agent environments and uses hidden validation (Wang et al., 2026).AAArenacombines 12 custom competition games, archived human-written programs, and agent-selected testing, with separate budgets for small matches and full-pool evaluations. HL and program evolution. Weng’s HL incorporates experience through revisions to program policies and supporting software (Weng, 2026). Programmatic reinforcement learning and LLM-guided program search also use programs as policy representations (Verma et al., 2018; Bastani et al., 2018; Eberhardinger et al., 2025; Kuang et al., 2025). Related systems improve controllers, algorithms, and agent software (Garnier et al., 2026; Romera-Paredes et al., 2023; Novikov et al., 2025; Hu et al., 2025c; Zelikman et al., 2024), or revise rewards and training procedures (Ma et al., 2024; Chi et al., 2026). Trajectory distillation extracts reusable skills from replay (Yang et al., 2026b). AHL applies HL to competitive game-agent development, where the AI agent revises policies and supporting software with a fixed base model. Direct game-playing evaluation. Direct game-playing benchmarks measure the model’s decisions during matches. BALROG and LMGame-Bench evaluate action selection in interactive games (Paglieri et al., 2025; Hu et al., 2025a). GameBench, GameArena, TextArena, GAMEBoT, ZeroSumEval, and Kaggle Game Arena extend this evaluation to competitive players (Costarelli et al., 2024; Hu et al., 2025b; Guertler et al., 2025; Lin et al., 2025; Khan et al., 2025; Doerschuk-Tiberi et al., 2026). These settings measure the player, whereas AAArena measures the AI agent developing the autonomous program that plays the game. 6 Future Work ExtendingAAArena. We plan to add games, controlled rule variants, and changing opponent populations to study adaptation to changes in the task and competition. These extensions would broaden evaluation beyond per-game rankings to adaptation speed, cross-game transfer, and continual learning. Heuristic and reinforcement learning. We aim to compare how code revisions and parameter updates learn from experience and assign credit for improvements. Such comparisons should account for feedback access and interaction budgets. Hybrid learners could use HL to revise policy structure and reinforcement learning to train its components. From replay use to active learning. External replays could help agents form strategic hypotheses before choosing opponents to test them. We plan to study how agents allocate replay views and matches to resolve uncertainty. Agents could turn validated strategies into reusable policy modules. 7 Conclusion We presentAAArenato evaluate AI agents that develop executable game policies through adversarial heuristic learning (AHL). The benchmark combines 12 real-world competition games, archived human programs, and a unified two-budget protocol. Across 7 AI models, Opus5.5 with Claude Code earns 6 gold medals; no evaluated configuration tops the remaining 6 human ladders. Dense feedback supports policy improvement in the ablations, and agents also improve by learning from both on-policy and off-policy replays. Qualitative analysis of code revisions and trajectories points to challenges in game understanding, strategy implementation, and long-horizon planning. By making this development process measurable,AAArenaprovides a foundation for research on heuristic learning and self-evolving AI agents. 14 Acknowledgement We thank the many members of the Student Association of the Department of Computer Science and Technology at Tsinghua University for developing and maintaining the Tsinghua University Agent Competition over the years and for supporting this work. We thank Z.ai for sponsoring this work. We also thank all participants in past editions of the Tsinghua University Agent Competition. References Osbert Bastani, Yewen Pu, and Armando Solar-Lezama. Verifiable reinforcement learning via policy extraction. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URLhttps://proceedings.neurips.c/ paper_files/paper/2018/file/e6d8545daa42d5ced125a4bf747b3688-Paper.pdf. Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs. Biometrika, 39(3-4):324–345, 1952. doi: 10.1093/biomet/39.3-4.324. URL https://doi.org/10.1093/biomet/39.3-4.324. Murray Campbell, A.Joseph Hoane, and Feng-hsiung Hsu. Deep blue. Artificial Intelligence, 134(1-2):57–83, January 2002. ISSN 0004-3702. doi: 10.1016/s0004-3702(01)00129-1. URLhttp://dx.doi.org/10.1016/S0004-3702(01) 00129-1. Yizhe Chi, Wenyi Li, Deyao Hong, Xiaoqiu Wang, Mingju Gao, Kaisen Yang, Bingxiang He, Youjie Zheng, Calvin Xiao, and Qinhuai Na. Ai4ai-bench: Benchmarking llm agents in algorithmic design for recursive self-improvement, 2026. URL https://arxiv.org/abs/2608.20318. Anthony Costarelli, Mat Allen, Roman Hauksson, Grace Sodunke, Suhas Hariharan, Carlson Cheng, Wenjie Li, Joshua Clymer, and Arjun Yadav. Gamebench: Evaluating strategic reasoning abilities of llm agents, 2024. URL https://arxiv.org/abs/2406.06613. Rémi Coulom. Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search, pages 72–83. Springer Berlin Heidelberg, 2007. ISBN 9783540755388. doi: 10.1007/978-3-540-75538-8_7. URLhttp://dx.doi.org/10.1007/ 978-3-540-75538-8_7. Panayiotis Danassis and Naman Goel. Can vibe coding beat graduate cs students? an llm vs. human coding tournament on market-driven strategic planning, 2025. URL https://arxiv.org/abs/2511.20613. Bovard Doerschuk-Tiberi, Yao Yan, Justin Chiu, Hann Wang, Timothy Chung, Martyna Plomecka, John Schultz, Jon Lipovetz, Clayton Drazner, Yuchen Zhuang, Jaimie Hwang, Nate Keating, Riley Jones, Andrew Lee, Oran Kelly, Ian Gemp, Michael Aaron, Laurel Prince, Kate Larson, Jeff Moser, Harrison Jobe, Chad Woodford, Siqi Liu, Andrew Wang, Bo Chang, Christopher D’Mello, Diane Chaleff, Addison Howard, Johnny Yip, Chuck Sugnet, Antonio Gulli, Meghan O’Connell, Will Cukierski, Nenad Tomasev, Dima Yeroshenko, Kinjal Parekh, Roxanne Daniel, Marc Lanctot, Domino Weir, Elsa Dong, Daniel Hennes, Melissa Nalubwama, Robert Fraser, Ryan Trostle, Jun Peng, Tom Mason, Lloyd Hightower, Chiamaka Chukwuka, Yuexiang Zhai, Phoebe Kirk, Yi Su, Yuting Han, Jie Ren, Chris Prichard, Sahand Sharifzadeh, Karim Hakimzadeh, DJ Sterling, Meg Risdal, Kate Olszewska, Ya Xu, Orhan Firat, and Minmin Chen. Game arena: Strategic llm evaluation in competitive environments, 2026. URL https://arxiv.org/abs/2609.31473v1. Manuel Eberhardinger, James Goodman, Alexander Dockhorn, Diego Perez-Liebana, Raluca D. Gaina, Duygu Çakmak, Setareh Maghsudi, and Simon Lucas. From code to play: Benchmarking program search for games using large language models, 2025. URL https://arxiv.org/abs/2412.04057. Lingyue Fu, Xin Ding, Linyue Pan, Yaoming Zhu, Shao Zhang, Lin Qiu, Weiwen Liu, Weinan Zhang, Xuezhi Cao, Xunliang Cai, Jiaxin Ding, and Yong Yu. Catarena: Evaluating evolutionary capabilities of code agents via iterative tournaments, 2026. URL https://arxiv.org/abs/2510.26852. Game Agent Coding League. Game Agent Coding League for LLMs. Official benchmark documentation, n.d. URL https://gameagentcodingleague.com/. Accessed September 28, 2026; publication date not stated. Paul Garnier, Jonathan Viquerat, and Elie Hachem. Heuristic learning for active flow control using coding agents, 2026. URL https://arxiv.org/abs/2607.11565. Leon Guertler, Bobby Cheng, Simon Yu, Bo Liu, Leshem Choshen, and Cheston Tan. Textarena, 2025. URL https://arxiv.org/abs/2504.11442. 15 Pablo Hernandez-Leal, Bilal Kartal, and Matthew E. Taylor. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems, 33(6):750–797, October 2019. ISSN 1573-7454. doi: 10.1007/s10458-019-09421-1. URL http://dx.doi.org/10.1007/s10458-019-09421-1. Lanxiang Hu, Mingjia Huo, Yuxuan Zhang, Haoyang Yu, Eric P. Xing, Ion Stoica, Tajana Rosing, Haojian Jin, and Hao Zhang. lmgame-bench: How good are llms at playing games?, 2025a. URLhttps://arxiv.org/abs/2505.15146. Lanxiang Hu, Qiyu Li, Anze Xie, Nan Jiang, Ion Stoica, Haojian Jin, and Hao Zhang. Gamearena: Evaluating llm reasoning through live computer games, 2025b. URL https://arxiv.org/abs/2412.06394. Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems, 2025c. URLhttps://arxiv.org/abs/ 2408.08435. Haidar Khan, Hisham A. Alyahya, Yazeed Alnumay, M Saiful Bari, and Bülent Yener. Zerosumeval: Scaling llm evaluation with inter-model competition, 2025. URL https://arxiv.org/abs/2504.12562. Zhiyi Kuang, Ryan Rong, YuCheng Yuan, and Allen Nie. Learning game-playing agents with generative code optimization, 2025. URL https://arxiv.org/abs/2508.19506. Wenye Lin, Jonathan Roberts, Yunhan Yang, Samuel Albanie, Zongqing Lu, and Kai Han. GAMEBoT: Transparent assessment of LLM reasoning in games. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7656–7682, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long.378. URL https://aclanthology.org/2025.acl-long.378/. Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models, 2024. URL https://arxiv.org/abs/2310.12931. Alexander Novikov, Ngân V ̃u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. Alphaevolve: A coding agent for scientific and algorithmic discovery, 2025. URL https://arxiv.org/abs/2506.13131. Davide Paglieri, Bartłomiej Cupiał, Samuel Coward, Ulyana Piterbarg, Maciej Wolczyk, Akbir Khan, Eduardo Pignatelli, Łukasz Kuciński, Lerrel Pinto, Rob Fergus, Jakob Nicolaus Foerster, Jack Parker-Holder, and Tim Rocktäschel. Balrog: Benchmarking agentic llm and vlm reasoning on games, 2025. URL https://arxiv.org/abs/2411.13543. Wenjun Peng, Xinyu Wang, and Qi Wu. Proxywar: Dynamic assessment of llm code generation in game arenas, 2026. URL https://arxiv.org/abs/2602.04296. Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with large language models. Nature, 625(7995): 468–475, December 2023. ISSN 1476-4687. doi: 10.1038/s41586-023-06924-6. URLhttp://dx.doi.org/10.1038/ s41586-023-06924-6. Saiblo. Saiblo. https://w.saiblo.net/, n.d. Accessed October 9, 2026. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419): 1140–1144, December 2018. ISSN 1095-9203. doi: 10.1126/science.aar6404. URLhttp://dx.doi.org/10.1126/science. aar6404. Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. Programmatically interpretable reinforcement learning. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5045–5054. PMLR, 10–15 Jul 2018. URL https://proceedings.mlr.press/v80/verma18a.html. Paul M.B. Vitányi. How incomputable is kolmogorov complexity? Entropy, 22(4):408, April 2020. ISSN 1099-4300. doi: 10.3390/e22040408. URL http://dx.doi.org/10.3390/e22040408. Zhilin Wang, Han Song, Runzhe Zhan, Jusen Du, Jiacheng Chen, Tianle Li, Qingyu Yin, Yulun Wu, Zhennan Shen, Tong Zhu, Yanshu Li, Guanjie Chen, Derek F. Wong, Yafu Li, Yu Cheng, and Yang Yang. Evopolicygym: Evaluating autonomous policy evolution in interactive environments, 2026. URL https://arxiv.org/abs/2607.02440. 16 Jiayi Weng. Learning beyond gradients.https://trinkle23897.github.io/learning-beyond-gradients/, May 2026. Blog post. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024. URL https://arxiv.org/abs/2404.07972. John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering, 2024. URLhttps://arxiv.org/abs/ 2405.15793. John Yang, Kilian Lieret, Joyce Yang, Carlos E. Jimenez, Muhtasham Oblokulov, Aryan Siddiqui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Codeclash: Benchmarking goal-oriented software engineering, 2026a. URL https://arxiv.org/abs/2511.00839v2. Kaisen Yang, Zheng Jiang, Yuzhao Peng, Houde Qian, Boshi Zhang, Youjie Zheng, Shijin Hong, Qingle Liu, Ruoyu Han, Bohan Lyu, Bingxiang He, Eren Cai, Calvin Xiao, and Qinhuai Na. Scalable behaviour cloning on browser using via skill distillation, 2026b. URL https://arxiv.org/abs/2606.32014. Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (stop): Recursively self-improving code generation, 2024. URL https://arxiv.org/abs/2310.02304. A The evaluated games The suite shares an evaluation interface, but its policies solve different decision problems. The descriptions below refer to the public rule and SDK bundles used in the experiments; Table 1 lists pool sizes. Pacman. Pacman is a separate, symmetric competitive maze game. Players collect ordinary andhigh-value beans , avoidmonsters, and placeminesthat affect movement choices. Collection, collision avoidance, and disruption of the opponent jointly determine the score. Its rules and opponent pool are distinct from Rollman’s asymmetric contest. SnakeGo. SnakeGo is a territory game in whichsnakesmove, collect items, claimcells, and can split or use special actions. Local collision avoidance interacts with longer-horizon territory growth, making a safe immediate move potentially costly at settlement. Rollman. Rollman is an asymmetric maze game. One side collectsbeansand uses temporary skills; the other coordinatesghoststo intercept it. The roles differ in both action space and objective, making movement planning and coordinated pursuit distinct policy problems. MoneCraft. MoneCraft is a mining and resource-control game on a symmetric map. Players capturemines that generatecurrencyand can teleport, placemines, or purchase protection. The policy trades movement and disruption against sustained resource production; end-gamecurrencyandmine ownershipdetermine the result under the referee’s rules. AntWar. AntWar combines tower defence with economic management. Players build and upgradetowers, improve theirbases, and timesuper-weapons, whileantsfollow environment-driven paths. The policy must balance present defence with spending that changes future pressure. LostSpace. LostSpace is a multiplayer survival-and-escape game on a layered map that contracts over time. Players collectsuppliesandkeys, usetools, fight or evade rivals, and seek escape. Ranked outcomes create trade-offs between immediate combat, survival, and the timing of an escape attempt. AquaWar. AquaWar is a hidden-identity tactical game. Players selectfish, make assertions about hidden opponent identities, and choose attacks or type-specific skills. Roster design, inference, and information- sensitive timing all affect the outcome. 17 Generals. Generals combines territorial expansion, army movement, neutral assets, production, technology, and tactical abilities. An effective policy coordinates economic development with the timing and direction of attacks rather than optimizing either in isolation. Dorado. Dorado combines resource competition withheroand army development. Players chooseheroes, produce and commandunits, mine neutral resources, and invest inupgrades. The policy coordinates movement, combat, harvesting, and abilities to threaten opposingbaseswhile maintaining the resources needed to continue production. Miracle. Miracle is a hex-grid strategy game with heterogeneousunitsand an initial drafting stage. A player selectscreaturesand anartifact, then summons, moves, attacks, and uses abilities to destroy the opponent’s miracle. Resource limits and unit interactions make tactical coordination central to the policy. LOTA. LOTA is a lane-based combat game. Each player chooses aheroand controls movement, attacks, abilities, skill development, andrevivalwhileunitsadvance along fixedlanes. Victory depends on destroying the opposingbaseor the referee’s settlement criteria at the turn limit. Effective control couples immediate fights with sustained pressure on defensive structures. AntWar2. AntWar2 extends tower-defence strategy withordered operation listswhose accepted prefix changes subsequent decisions. The agent must coordinatetower placement, economic development, and weapon timing while respecting sequence-dependent legality. B Arena resources and evaluation details Public resources and opponents. We freeze the versioned opponent programs, their ratings, and the game resources across runs. Some entrants submitted several related versions, so pool sizes count programs, not independent participants. The workspace includes rules, SDKs, replay documentation, a leaderboard, and one reference program (the rank-40 entry). The agent cannot access the other opponents’ source code or evaluator internals. Budget accounting. A small-match request tests up to 8 opponents and costs one unit per opponent. A full-pool evaluation costs one large unit. These budgets constrain feedback access; source edits, model calls, tokens, and wall-clock time have no separate limits. The starter policy’s initial evaluation is free and falls outside the charged full-pool sequence. B.1 AI agent workflow Each model–game run uses one persistent session with a writable workspace. Opus5.5 runs in Claude Code; the other models run in Codex. The agent’s goal is to improve competitive performance. It chooses when to read rules, edit code, test selected opponents, and submit a full-pool evaluation. Small matches return replays; full-pool evaluations return Elo, rank, and per-opponent outcomes. Context compaction preserves the session but shortens its conversation history, so the agent keeps notes to retain experience. We report the run’s highest officially evaluated Elo, including the common initialization baseline, and its corresponding rank. The controller retains the policy snapshot attaining that score. B.2 Elo and rank Each full-pool evaluation estimates the Elobr(P) of policy snapshotPfrom its match outcomes using a fixed-pool Bradley–Terry model (Bradley and Terry, 1952). LetHdenote the frozen opponent pool andr j the fixed rating of opponentj. The candidate’s score in matchmagainst opponentj(m) iss m ∈0, 1 2 , 1 (loss, draw, win). The expected score at rating r is p(r,r j ) = 1 1 + 10 (r j −r)/400 , 18 and the estimate ˆr solves X m p(ˆr,r j(m) ) + p(ˆr, ̄r H ) = X m s m + 1 2 , ̄r H = 1 |H| X j∈H r j . The extra term adds one virtual draw against the complete pool’s mean rating, keepingˆrfinite for perfect win or loss records. Opponent ratings remain fixed across snapshots and runs. The candidate’s rank is 1 +|j ∈ H:r j > ˆr|. We report each game’s native Elo and rank and summarize rank-1 coverage with gold medals. B.3 Gold medals Gold medals count rank-1 finishes against the frozen human ladders. For configurationmand gameg, let q m,g denote its reported rank. Across G = 12 games, we compute Gold(m) = G X g=1 1q m,g = 1. A gold medal denotes first place under this rating protocol; it does not establish a direct head-to-head win against the human leader or optimal play. The pool ratings and rank-1 thresholds remain fixed when we evaluate additional models. Gold medals summarize performance against the archived pools. Full-pool evaluations provide feedback during optimization, so the reported results do not measure generalization to unseen opponents. The committed summary records pin the rating inputs and calculations; they do not independently validate the original human-rating calibration. B.4 Rule complexity: AST and RA We compare rule-description sizes by rewriting each game’s rules in a shared formal pseudocode language. Its domain-neutral vocabulary includes top-level declarations (constant,enum,entity,action,observation, setup,rule,terminal), fifteen statement types, and expressions for arithmetic, comparisons, and function calls. Statement types includelet,require,when,for,update,create,delete,reveal, andreturn. Set membership, distance, and probability are primitives. We explicitly define every game-specific operation, such as splitting a snake or casting a fish skill. Each description covers players, information, state, initialization, randomness, legal actions, transitions, and termination. We exclude communication protocols, rendering, logging, and platform-only timeouts. AST. The primary measure is the node count of the description’s canonical abstract syntax tree, C AST (G) = N structure (d G ) + N expression (d G ), whered G describes gameG.N structure counts declaration, block, and statement nodes;N expression counts calls, names, attributes, literals, collections, subscripts, comparisons, and operators. We exclude parser-only wrapper and context nodes and count each named rule once at its definition and once at every call site. RA. The secondary measure, rule atoms (RA), counts independently changeable rule propositions. Each declaration member or rule statement expresses exactly one such proposition; two effects written in one sentence still count as two. RA is the size of the disjoint union of seven proposition classes: state, action, observation, setup, condition, transition, and outcome. Both measures are invariant to identifier renaming, comments, and formatting. They depend on the chosen language and measure the size of the rule description, not the state-space size or the difficulty of optimal play. B.5 Reading the trajectory plots The single-run plots present three complementary views of policy development. Panel (a) plots each submitted snapshot’s full-pool Elo, with aligned rank annotations and frozen-ladder reference levels. These curves can regress; the row heading separately reports the retained champion. 19 Panel (b) shows interaction order on 144 equally spaced budget slots (128 small-match units and 16 full-pool evaluations). An opponent test or full-pool evaluation occupies one slot. The continued runs use 432 slots (384 small and 48 full). Tests in the same batch share the midpoint of its occupied slots. Small dots give selected opponents’ ladder ranks: green denotes wins, magenta losses, and grey draws. Large diamonds give the learner’s full-pool rank, with rank 1 at the top. In Dorado, 128 opponent tests comprise 16 batches and 256 role outcomes. Panel (c) reports tokens in millions at each full-pool evaluation. Lines show cumulative use; bars count tokens since the preceding evaluation, with the first bar starting at the run’s beginning. Continuation plots annotate ranks at every third full-pool evaluation and at local Elo peaks. The paired replay-source plots use blue circles for own-policy replays and purple diamonds for other-policy replays. Their panel (a) shows both Elo curves and rank rows; dashed levels refer only to the other-policy run’s first evaluated policy and retained champion. Panel (b) shows cumulative tokens as lines and between- evaluation increments as paired bars on the right axis. The ablation bars label retained-champion Elo and rank; colours distinguish conditions, and panels within each figure share a zero-based Elo axis. B.6 Complete Policy Evolution Trajectories These plots show the complete trajectories of the twelve GLM-5.3 runs from Section 4.2. 20 Success Rollman best Elo 581.2, rank 1 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 200 400 600 Full-pool evaluation Elo Rank 42041 Leader; final lower (#1) Initial upper (#3) Initial lower (#4) 13672108144 1 20 40 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 2 4 6 8 10 Increment (M) 1481216 0 10 20 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation MoneCraft best Elo 2359.2, rank 1 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 2320 2340 2360 2380 Full-pool evaluation Elo Rank 1 Leader / Initial/final lower (#1) 13672108144 1 5 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 2 4 6 Increment (M) 1481216 0 2 4 6 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Pacman best Elo 2281.6, rank 1 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1800 2000 2200 Full-pool evaluation Elo Rank 101052441 Leader; final lower (#1) Initial upper (#9) Initial lower (#10) 13672108144 1 10 20 30 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rankDraw 0 2 4 6 8 Increment (M) 1481216 0 10 20 30 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation AquaWar best Elo 1494.1, rank 1 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1350 1400 1450 1500 Full-pool evaluation Elo Rank 427229721 Leader; final lower (#1) Initial upper (#3) Initial lower (#4) 13672 108144 1 25 50 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 5 10 15 Increment (M) 1481216 0 20 40 60 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Figure 11 Success. Full-pool Elo and ranks, interaction order, and token use for runs that reach rank 1 and stop early. Plot conventions appear in Appendix B.5. 21 Progress AntWar best Elo 1464.2, rank 5 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1000 1500 2000 Full-pool evaluation Elo Rank 11985855555555555 Leader (#1) Initial upper (#10) Initial lower (#11) Final upper (#4) Final lower (#5) 13672108144 1 20 40 60 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 20 40 Increment (M) 1481216 0 50 100 150 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Dorado best Elo 2210.3, rank 2 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1800 2000 2200 Full-pool evaluation Elo Rank 41363935463543291814368942 Leader; final upper (#1) Initial upper (#40) Initial lower (#41) Final lower (#2) 13672108144 1 25 50 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 Increment (M) 1481216 0 50 100 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation LostSpace best Elo 1980.1, rank 2 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1700 1800 1900 2000 Full-pool evaluation Elo Rank 3322222222222223 Leader; final upper (#1) Initial upper; final lower (#2) Initial lower (#3) 13672108144 1 20 40 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 30 Increment (M) 1481216 0 50 100 150 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation SnakeGo best Elo 1141.2, rank 4 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1100 1200 Full-pool evaluation Elo Rank 8777587754578557 Leader (#1) Initial upper (#7) Initial lower (#8) Final upper (#3) Final lower (#4) 13672108144 1 20 40 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 5 10 Increment (M) 1481216 0 10 20 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation AntWar2 best Elo 1977.1, rank 9 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1500 2000 Full-pool evaluation Elo Rank 1614501813161313131414131391110 Leader (#1) Initial upper (#15) Initial lower (#16) Final upper (#8) Final lower (#9) 13672108144 1 20 40 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 Increment (M) 1481216 0 50 100 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Figure 12 Progress. Full-pool Elo and ranks, interaction order, and token use for games with fluctuating improvement. Plot conventions appear in Appendix B.5. 22 Stagnation Generals best Elo 1441.2, rank 17 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1200 1400 1600 1800 2000 2200 Full-pool evaluation Elo Rank 27232220172018181817181718172017 Leader (#1) Initial upper (#26) Initial lower (#27) Final upper (#16) Final lower (#17) 13672108144 1 50 100 150 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 30 Increment (M) 1481216 0 50 100 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Miracle best Elo 1436.8, rank 26 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 1400 1600 1800 2000 Full-pool evaluation Elo Rank 27392727292726272726272627272827 Leader (#1) Initial upper; final lower (#26) Initial lower (#27) Final upper (#25) 13672108144 1 25 50 75 100 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 10 20 30 Increment (M) 1481216 0 20 40 60 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation LOTA best Elo 2147.4, rank 22 (a) Full-pool Elo(b) Opponent selection(c) Token usage 1481216 2200 2400 Full-pool evaluation Elo Rank 22252222242923252224282522222523 Leader (#1) Initial/final upper (#21) Initial/final lower (#22) 13672108144 1 25 50 75 100 Interaction slot Ladder rank Small: winSmall: loss Full: achieved rank 0 5 10 15 Increment (M) 1481216 0 10 20 30 Full-pool evaluation Cumulative (M) Cumulative; full evaluation Since preceding full evaluation Figure 13 Stagnation. Full-pool Elo and ranks, interaction order, and token use for games that plateau after initial adaptation. Plot conventions appear in Appendix B.5. 23 C Detailed ablation results Tables 6–8 list the Elo and rank behind Figures 6–8. GameModelLadderRandomTop-4 Pacman 2096.8 / 4 2127.5 / 3 2014.4 / 6 1897.0 / 10 AntWar 1180.5 / 8 1311.7 / 6 1154.4 / 9 1056.2 / 11 Miracle 1482.4 / 20 1448.8 / 24 1457.0 / 24 1406.0 / 31 Table 6 Opponent selection ablation with GLM-5.3. Each cell is Elo / rank under the 128/16 budget. GameDenseBinary Pacman 2236.1 / 1 1710.1 / 11 AntWar 1118.7 / 9 880.6 / 22 Miracle 1528.2 / 15 1461.1 / 23 Table 7 Replay feedback ablation with GLM-5.3. Each cell is Elo / rank under the 128/16 budget. Game1248 Pacman 2040.6 / 5 2196.3 / 2 2703.9 / 1 2160.4 / 2 AntWar 1130.2 / 9 1433.4 / 5 1208.7 / 8 1118.7 / 9 Miracle 1338.7 / 45 1491.2 / 19 1518.7 / 16 1465.3 / 23 Table 8 Small-match batch size ablation with GLM-5.3. Each cell is Elo / rank under the 128/16 budget. D Token consumption The main text summarizes retained-champion performance. Under the same small-match and full-pool limits, the models consume different numbers of tokens. Table 9 reports the token use of every main-table run. Equal interaction budgets permit different token expenditure. Total use varies by more than fivefold: GPT6-sol consumes 260M tokens, between 16M and 30M per game, whereas Qwen 3.8 consumes 1,454M. Qwen 3.8 uses 1.6 times GLM-5.3’s tokens but earns three gold medals versus four. GPT6-sol also earns four with less than a third of GLM-5.3’s tokens. The leading configuration, Opus5.5 with Claude Code, consumes 737M. Token use also varies within a configuration. Kimi K3 consumes 1.5M on AquaWar and 89M on Generals. The feedback ceilings therefore do not impose equal computational expenditure; we report token use alongside competitive performance. GameOpus5.5 GPT6-sol GLM-5.3 Kimi K3 DeepSeek V4 Pro Qwen 3.8 LongCat 2.0 Rollman48.7525.2823.825.506.1114.6818.35 Pacman22.1616.1433.1440.1791.95163.1217.52 AntWar74.1916.08142.9454.59102.80284.2937.56 AquaWar76.9526.3768.681.4960.1113.9220.65 Generals75.4419.66137.2388.88172.7060.9522.30 LostSpace65.3721.58149.8672.03141.51190.16100.95 Miracle89.2919.1561.9934.7847.89107.5549.05 Dorado44.5823.02105.6360.7267.01253.0210.14 MoneCraft56.1622.106.7529.9767.5448.7231.18 LOTA76.4117.3632.5530.7434.04112.3621.92 SnakeGo48.5323.5322.4959.7529.7978.7727.35 AntWar258.7830.07115.7071.0844.08126.5367.20 Total736.61260.34900.78549.70865.53 1454.07424.17 Table 9 Token use of the main-table runs (millions, input plus output; we count cached input once within input). All runs share the 128/16 interaction budget. 24 E Algorithm families in the champion policies Because the agent acts entirely in code space, the final policies also show which algorithms each model chooses to write. We classify the code of all 84 retained champions into six families: • Rule-based control: hand-written if–else priorities, state machines, and fixed openings or scripts. •Heuristic scoring: hand-weighted features that score candidate actions or targets, followed by a greedy choice. • Graph search: BFS, DFS, Dijkstra, A*, distance maps, flood fill, and territory partitions. •Lookahead search: search over a forward model, including simulated rollouts, beam search, minimax, expectimax, and MCTS. •Combinatorial optimization: assignment and matching, dynamic programming, and knapsack-style allocation. •Inference and randomization: belief tracking over hidden information, opponent modelling, and mixed or randomized choices. We analyse source files in each champion’s strategy directory, excluding byte-identical SDK files and paths matching version, backup, debugging, test, or vendor filters. We assign each function a family based on its structure and identifiers. For functions longer than 40 lines, we classify 20-line windows separately. We exclude I/O, parsing, and protocol code as infrastructure. A family’s share is its fraction of the remaining non-comment lines. Table 10 reports each model–game pair and equally weighted means across models and games. GameOpus5.5 GPT6-sol GLM-5.3 Kimi K3 DeepSeek V4 Pro Qwen 3.8 LongCat 2.0Mean Rollman Pacman AntWar AquaWar Generals LostSpace Miracle Dorado MoneCraft LOTA SnakeGo AntWar2 Mean Rule-based controlHeuristic scoringGraph searchLookahead searchCombinatorial optimizationInference & randomization Table 10 Algorithm families in the champion policies. Each bar splits the decision code of one retained champion by algorithm family, weighted by non-comment lines of code. We exclude infrastructure (I/O, parsing, protocol). Row and column means weight each policy equally. Retained champions rely mainly on rules and heuristic scoring. Averaging the 84 champion-level shares equally, these families account for 79%, graph search for 13%, and the other three families for 8%. Lookahead search exceeds 10% in only 7 of the 84 champions, all on AntWar, AntWar2, and Generals. These endpoint classifications describe policy composition; they do not identify which code changes caused improvements during development. Algorithm shares differ across games. Graph search accounts for 33% on Pacman and 28% on Rollman; inference accounts for 30% on AquaWar. MoneCraft and LOTA consist almost entirely of rules. Across models, Opus5.5 has the lowest rule share (47%) and the highest scoring (27%) and lookahead shares (5%); GPT6-sol has the highest graph-search share (18%). The other models range from 58% to 65% rules. Within each game, we correlate non-rule share (1−rule-based fraction) with Elo across the seven configurations, using average ranks for ties. The mean Spearman correlation across 12 games is 0.35, with variation across 25 games. This association does not establish that replacing rules improves a policy. The classification is automatic and approximate. Identifiers and comments influence it; we assign a mixed function to its dominant mechanism. Models retaining the same starter code have similar shares, as on AquaWar and LOTA. 26

Instant research discovery