Paper deep dive

Sensi: Learn One Thing at a Time -- Curriculum-Based Test-Time Learning for LLM Game Agents

Mohsen Arjmandi

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 50

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/22/2026, 5:56:41 AM

Summary

Sensi is an LLM agent architecture designed for the ARC-AGI-3 challenge, utilizing a two-player Observer-Actor framework, a curriculum-based learning system managed by an external state machine, and a database-as-control-plane to achieve high sample efficiency in test-time learning.

Entities (6)

ARC-AGI-3 · benchmark · 100%Sensi · agent-architecture · 100%SQLite · database · 100%Actor · agent-role · 95%Observer · agent-role · 95%DSPy · framework · 90%

Relation Signals (3)

Sensi → targets → ARC-AGI-3

confidence 100% · Sensi, an LLM agent architecture for the ARC-AGI-3 game-playing challenge

Sensi → uses → SQLite

confidence 100% · a database-as-control-plane that makes the agents context window programmatically steerable

Observer → communicateswith → Actor

confidence 95% · Sensi v1 splits the agent into two communicating roles—an Observer... and an Actor

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language model (LLM) agents deployed in unknown environments must learn task structure at test time, but current approaches require thousands of interactions to form useful hypotheses. We present Sensi, an LLM agent architecture for the ARC-AGI-3 game-playing challenge that introduces structured test-time learning through three mechanisms: (1) a two-player architecture separating perception from action, (2) a curriculum-based learning system managed by an external state machine, and (3) a database-as-control-plane that makes the agents context window programmatically steerable. We further introduce an LLM-as-judge component with dynamically generated evaluation rubrics to determine when the agent has learned enough about one topic to advance to the next. We report results across two iterations: Sensi v1 solves 2 game levels using the two-player architecture alone, while Sensi v2 adds curriculum learning and solves 0 levels - but completes its entire learning curriculum in approximately 32 action attempts, achieving 50-94x greater sample efficiency than comparable systems that require 1600-3000 attempts. We precisely diagnose the failure mode as a self-consistent hallucination cascade originating in the perception layer, demonstrating that the architectural bottleneck has shifted from learning efficiency to perceptual grounding - a more tractable problem.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

50,193 characters extracted from source content.

Expand or collapse full text

Sensi: Learn One Thing at a Time—Curriculum-Based Test-Time Learning for LLM Game Agents Mohsen Arjmandi Independent Researcher (CTO, evolutionID) Correspondence: mohsen.arjmandi@gmail.com. Blog: https://freddiespirit.substack.com (Freddie Spirit). Abstract Large language model (LLM) agents deployed in unknown environments must learn task structure at test time, but current approaches require thousands of interactions to form useful hypotheses. We present Sensi, an LLM agent architecture for the ARC-AGI-3 game-playing challenge that introduces structured test-time learning through three mechanisms: (1) a two-player architecture separating perception from action, (2) a curriculum-based learning system managed by an external state machine, and (3) a database-as-control-plane that makes the agent’s context window programmatically steerable. We further introduce an LLM-as-judge component with dynamically generated evaluation rubrics to determine when the agent has learned enough about one topic to advance to the next. We report results across two iterations: Sensi v1 solves 2 game levels using the two-player architecture alone, while Sensi v2 adds curriculum learning and solves 0 levels—but completes its entire learning curriculum in approximately 32 action attempts, achieving 50–94× greater sample efficiency than comparable systems that require 1,600–3,000 attempts. We precisely diagnose the failure mode as a self-consistent hallucination cascade originating in the perception layer, demonstrating that the architectural bottleneck has shifted from learning efficiency to perceptual grounding—a more tractable problem. 1 Introduction The ARC-AGI-3 challenge [1] presents a distinctive test of machine intelligence: agents must play pixel-art puzzle games with no prior knowledge of the rules, mechanics, or win conditions. Unlike standard game-playing benchmarks where the rules are specified or can be inferred from a large training corpus, ARC-AGI-3 requires agents to discover the game’s dynamics entirely through interaction—a form of test-time learning that mirrors how humans approach unfamiliar situations. Current approaches to this challenge are remarkably sample-inefficient. Systems such as Agentica from Symbolica reportedly require 1,600 to 3,000 game interactions to build a working model of a single game’s mechanics. This inefficiency fundamentally limits the practical applicability of LLM agents in settings where test-time compute budgets are constrained, a concern that has become increasingly relevant as the field explores test-time scaling [16, 15, 20]. We introduce Sensi, an LLM agent that evolves across two design iterations to address this sample efficiency problem. The core architectural insight is separation of concerns: perception should be decoupled from action, and learning should be structured as a curriculum rather than an undirected search. Sensi v1 splits the agent into two communicating roles—an Observer that maintains hypotheses about the game world, and an Actor that selects actions to test those hypotheses. This two-player architecture, where cognition is factored into perception and decision-making, enables structured hypothesis accumulation. Using ChatGPT 5.1 as the backbone, v1 solves 2 levels of the ARC-AGI-3 challenge with the notable property that pass@10 equals pass@1: results are consistently reproducible given the same accumulated knowledge. Sensi v2 adds three mechanisms on top of the two-player foundation: (i) a curriculum that orders learning goals into a sequential queue with a state machine managing progression, (i) a database-as-control-plane where the agent’s entire cognitive state resides in SQLite tables that are programmatically injected into prompts each turn, and (i) a sense scorer implementing LLM-as-judge [21] with dynamically generated evaluation rubrics. V2 solves 0 levels—an honest negative result—but completes its full learning curriculum in approximately 32 interactions, a 50–94× improvement in sample efficiency. The failure mode is itself a contribution: we identify a self-consistent hallucination cascade where errors in the perception layer (frame differencing) propagate through the hypothesis pipeline, producing internally coherent but factually incorrect game models that receive high scores from the sense evaluator. This precisely localizes the bottleneck: the architecture works; the perception does not. Our contributions are: 1. A two-player architecture that separates perception from action via distinct Observer and Actor LLM roles communicating through structured hypothesis lists (§3). 2. A curriculum learning system with an external state machine that sequences learning goals and accumulates verified knowledge as facts for subsequent learning items (§4). 3. A database-as-control-plane pattern that makes the LLM’s context window externally programmable through database state, enabling modular and steerable agent behavior (§4.4). 4. An LLM-as-judge with dynamic rubrics for evaluating learning progress, where both the evaluation metric and the scoring are performed by separate LLM calls (§4.3). 5. A sample efficiency analysis demonstrating 50–94× improvement over baselines, together with a precise diagnosis of the hallucination cascade failure mode (§6). 2 Preliminaries 2.1 ARC-AGI-3 Environment ARC-AGI-3 [1] is a game-based evaluation where agents play pixel-art puzzle games without prior knowledge of the rules. At each timestep t, the agent observes a game frame represented as a 3D tensor: ℱt∈ℤ16L×H×WF_t _16^L× H× W (1) where L is the number of grid layers, H and W are the spatial dimensions (up to 64×6464× 64), and each cell takes an integer value in [0,15][0,15] representing a color. The agent selects from a discrete action space: =a0,a1,…,a7A=\a_0,a_1,…,a_7\ (2) where a0a_0 is reset (restarts the game), a1a_1 through a5a_5 are simple actions (directional movement, interact), a6a_6 is a coordinate-parameterized action taking (x,y)(x,y) arguments, and a7a_7 is an additional simple action. After executing action at∈a_t , the agent receives the next frame ℱt+1F_t+1 and a scalar score st∈[0,254]s_t∈[0,254], along with a status code from NOT_PLAYED,NOT_FINISHED,GAME_OVER,WIN\ NOT\_PLAYED, NOT\_FINISHED, GAME\_OVER, WIN\. Crucially, the semantics of each action are not provided to the agent. The agent must discover what each action does, how the game mechanics work, and what constitutes winning—all from raw interaction. 2.2 POMDP Formulation The ARC-AGI-3 problem can be formulated as a partially observable Markov decision process (POMDP) [5] with an important distinction: the agent must discover not only the state dynamics but also the reward function itself. Let ⟨,,T,R,Ω,O⟩ ,A,T,R, ,O denote the POMDP, where S is the (hidden) game state space, A is the action space, T:×→Δ()T:S×A→ (S) is the unknown transition function, R:×→ℝR:S×A is the unknown reward function, Ω is the observation space (rendered frames), and O:→Δ(Ω)O:S→ ( ) is the observation function. The agent’s task is two-fold: (1) learn an approximate model of T and R from interactions, and (2) use this model to select actions that maximize the game score. This dual-discovery requirement distinguishes ARC-AGI-3 from standard POMDP settings where the dynamics are assumed known or learnable from a fixed distribution. 2.3 Test-Time Learning It is important to situate Sensi within the landscape of test-time adaptation methods. We distinguish three paradigms: Test-time compute (TTC). Methods such as chain-of-thought prompting [16] and self-consistency [15] allocate additional inference-time computation to improve output quality. The model’s parameters remain frozen; only the reasoning trajectory is extended. Test-time training (T). Methods such as T [12] and T-Discover [20] perform gradient-based updates to the model’s parameters on each test instance. The model literally learns—in the weight-update sense—at test time. Test-time in-context learning. Sensi occupies a third position: the model’s parameters remain frozen, but the agent accumulates structured knowledge in an external database that is injected into the context window each turn. This is neither pure inference (the agent’s effective knowledge base changes) nor gradient-based training (no parameters are updated), extending in-context test-time learning ideas from frameworks like EvoTest [4]. We call this in-context knowledge accumulation with persistent structured state. The key distinction from standard in-context learning is that the accumulated knowledge persists across context windows through the database, enabling learning trajectories that span hundreds of turns without being limited by context length. 3 Sensi v1: Two-Player Architecture The foundational insight of Sensi is that a single LLM call asked to simultaneously perceive a game state, reason about what changed, maintain hypotheses, and select an action performs poorly at all four tasks. V1 addresses this by splitting cognition into two cooperating roles. 3.1 Observer and Actor Each turn in Sensi v1 involves two LLM calls: Player1(ℱt,ℱt−1,at−1,Δt) _1(F_t,F_t-1,a_t-1, _t) →(Gt,Kt) \;→\;(G_t,K_t) (3) Player2(Gt,Kt) _2(G_t,K_t) →(dt,at) \;→\;(d_t,a_t) (4) where GtG_t is the guesses list (hypothesized game mechanics), KtK_t is the figured-out list (confirmed observations), dt∈guess,informedd_t∈\ guess, informed\ is the decision type indicating whether the action is exploratory or knowledge-based, and Δt _t is a visual description of changes between consecutive frames. The prompt frames this as cooperative play: “you and your friend are playing a game together where you see the board and your friend chooses an action.” This framing leverages the LLM’s capacity for perspective-taking and collaborative reasoning. 3.2 Stochastic Exploration via Confidence-Modulated Agency A distinctive aspect of v1’s design is its treatment of confidence as a dynamic variable that modulates the agent’s exploration behavior. Rather than using fixed exploration schedules (as in ϵε-greedy strategies) or curiosity-driven bonuses [8, 10], Sensi instructs the LLM to act based on what has been figured out—allowing the model to dynamically modulate its behavior based on its evolving epistemic state. This design exploits an underappreciated capacity of LLMs: the ability to model stochastic aspects of human agency. When humans face genuinely unknown situations, their behavior is not purely logical—it includes intuitive leaps, confidence-weighted guesses, and exploratory randomness that collectively help navigate unfamiliar territory. By representing confidence dynamically in the agent’s state (through the distinction between guesses and figured-out items), Sensi allows the LLM to approximate this stochastic exploration naturally, without requiring explicit exploration mechanisms. Formally, the agent’s update rule can be expressed as: Sense(t+1) (t+1) =Sense(t)+LLM(Sense(t),S(t+1)) =Sense(t)+LLM(Sense(t),S(t+1)) (5) S(t+1) S(t+1) ∼P(⋅∣S(t),A(t)) P(· S(t),A(t)) (6) A(t+1) A(t+1) =LLM(Sense(t+1)) =LLM(Sense(t+1)) (7) where Sense(t)=(Gt,Kt)Sense(t)=(G_t,K_t) is the evolving epistemic state. Unlike thinking models that generate stochastic exploration through random token sampling, Sensi’s exploration emerges from the structure of the epistemic state: when the guesses list is large relative to the figured-out list, the Actor naturally behaves more exploratorily; as items migrate from guesses to figured-out, behavior becomes more directed. 3.3 Results and Limitations Sensi v1 was evaluated on ARC-AGI-3 using ChatGPT 5.1 as the backbone model. It successfully solved the first 2 levels of game LS20, discovering 15 correct facts about the game mechanics including player movement rules, energy consumption, key-door interactions, and the win condition. A notable property of v1’s results is that pass@10 = pass@1: given the same figured-out list, the Actor consistently selects the correct action sequence. This contrasts with thinking-model approaches where results vary across runs due to sampling stochasticity, and suggests that the two-player architecture produces more deterministic and reproducible behavior. However, v1 has fundamental limitations: (i) there is no mechanism to control what the agent learns or in what order, (i) there is no way to verify whether the agent has actually learned something correctly, and (i) at level 3, the agent entered a degenerate state where it stopped updating its understanding and repeated single actions across 100+ turns. These limitations motivate the structured learning approach of v2. 4 Sensi v2: Curriculum Learning with Programmable Context The core idea behind v2 is to impose structure on the learning process itself: learn one thing at a time. Instead of allowing the agent to explore an unstructured hypothesis space, v2 introduces a queue of learning items that the agent must master sequentially, with an external state machine managing progression and an LLM-based judge verifying learning outcomes. ℱt,ℱt−1F_t,F_t-1FrameDiffLLM1MetricGenLLM2SenseScoreLLM3Player1LLM4Player2LLM5ata_tΔt _tμI∗ _I^*ϕt,rt _t,r_tGt,KtG_t,K_tSQLite DBitems_to_learngame / inputsguessesfigured_outsfacts, historyfactswrite Gt,KtG_t,K_tonce per itemnot_reachedlearningcompletedpickϕt≥τ _t≥τϕt<τ _t< →K_t→ facts → DBLLM Pipeline (per turn)State Machine Figure 1: Sensi v2 architecture. Each turn involves up to five LLM calls orchestrated through a pipeline. The SQLite database serves as the control plane: all agent state resides in database tables that are queried to construct prompts and updated with each turn’s outputs. The curriculum state machine (left) manages learning progression, promoting figured-out items to facts when a learning item is completed. MetricGen (LLM2) runs only once when a new learning item is activated. 4.1 Turn Architecture Each turn in Sensi v2 involves up to five LLM calls organized as a pipeline: FrameDiff⏟LLM1→MetricGen⏟LLM2→SenseScore⏟LLM3→Player1⏟LLM4→Player2⏟LLM5 FrameDiff_LLM_1\;→\; MetricGen_LLM_2\;→\; SenseScore_LLM_3\;→\; Player_1_LLM_4\;→\; Player_2_LLM_5 (8) Frame Differencing (LLM1LLM_1). The raw grid ℱtF_t is rendered to a scaled pixel image via a color map (ℤ16→RGBZ_16 ). Two consecutive frames are passed to a multimodal LLM that returns a structured JSON diff: ⬇ 1def frame_diff_finder(self, current_frame, prev_frame) -> str: 2 prediction = self.frame_diff_module( 3 prev_frame=DSPyImage(url=encode_image(prev_frame)), 4 current_frame=DSPyImage(url=encode_image(current_frame)), 5 ) 6 return prediction.diff_json # structured JSON Listing 1: Frame differencing module. The diff schema captures added, removed, and moved objects, UI changes, and a high-level summary. This gives the Observer a structured description of what changed rather than forcing it to compare raw pixels. Metric Generation (LLM2LLM_2). Called once per learning item when it enters the learning state. Generates an evaluation rubric for the sense scorer (detailed in §4.3). Sense Scoring (LLM3LLM_3). Called every turn for the active learning item. Produces a score ϕt∈[1,10] _t∈[1,10] and reasoning text (detailed in §4.3). Player 1—Observer (LLM4LLM_4). Receives the game frame, frame diff, previous guesses and figured-out lists, accumulated facts, the current learning target, sense score feedback, and action history. Outputs updated guesses and figured-out lists. The prompt includes guidelines on list curation: keep valid items, edit with new information, remove contradicted items, and promote confirmed guesses to figured-out. Player 2—Actor (LLM5LLM_5). Receives guesses, figured-out lists, facts, and the current learning target. Outputs a decision type (dt∈guess,informedd_t∈\ guess, informed\) and exactly one action. The Actor’s objective is to resolve the Observer’s uncertainties by choosing actions that help discriminate between competing hypotheses. 4.2 Learning Queue and State Machine At the start of each game, Sensi v2 initializes a learning curriculum stored in an SQLite table: ⬇ 1def initialize_items_to_learn(self, game_id, card_id): 2 # Facts: already-known truths 3 fact_items = [ 4 "RESET starts the game", 5 "all available actions: ACTION1, ..., ACTION7, RESET", 6 ] 7 # Learning items: to be mastered sequentially 8 default_items = [ 9 "learn what each action does in the game", 10 "learn how actions affects your energy while playing", 11 "learn how to win the game", 12 ] 13 for item_name in default_items: 14 cur.execute(""" 15 INSERT OR IGNORE INTO items_to_learn 16 (game_id, card_id, item_name, state, threshold) 17 VALUES (?, ?, ?, ’not_reached’, 8) 18 """, (game_id, card_id, item_name)) Listing 2: Curriculum initialization. Each learning item follows a three-state lifecycle managed by a state machine: not_reachedlearningcompletedpicked as next itemϕt≥τ _t≥τϕt<τ _t< _out itemsbecome new factswaiting in queuesense scorer evaluates each turnknowledge accumulation Figure 2: Learning item state machine. Each curriculum item progresses through three states. The self-loop on learning represents repeated evaluation until the sense score meets the threshold τ. Upon completion, accumulated figured-out items are promoted to facts, creating a knowledge accumulation chain. Formally, let ℐ=I1,I2,…,InI=\I_1,I_2,…,I_n\ be the ordered sequence of learning items, each with state σi∈not_reached,learning,completed,fact _i∈\ not\_reached, learning, completed, fact\. At any turn t, the active item is: I∗=mini⁡Ii∣σi∈learning,not_reachedI^*= _i\I_i _i∈\ learning, not\_reached\\ (9) When I∗I^* transitions to completed, its accumulated figured-out entries are promoted to facts: σI∗=completed⟹∀k∈Kt:σk←fact _I^*= completed\; \;∀\,k∈ K_t: _k← fact (10) These new facts become part of the context for subsequent learning items, creating a knowledge accumulation chain: each learning item builds upon the verified (or putatively verified) knowledge from all previous items. 4.3 Sense Scoring: LLM-as-Judge for Learning Progress How does the agent know when it has learned enough about a topic? V2 introduces a sense scorer—a separate LLM call acting as an external judge. Critically, the judge does not use a fixed rubric. Instead, the rubric itself is dynamically generated by another LLM call, creating a two-phase evaluation system. Phase 1: Metric Generation. When a new learning item becomes active, a MetricGeneratorSignature is invoked to produce a verification criterion: ⬇ 1class MetricGeneratorSignature(dspy.Signature): 2 """Given an item the agent needs to learn about a game, 3 generate a metric to verify that the item has been learned.""" 4 5 item_to_learn: str = dspy.InputField( 6 desc="The item/concept the agent needs to learn" 7 ) 8 learning_metric: str = dspy.OutputField( 9 desc="A criteria and description that a judge will use 10 to give a score on how good a grasp the learner has 11 on the item. Score between 1 to 10." 12 ) Listing 3: Dynamic metric generation via DSPy. Phase 2: Sense Scoring. On each subsequent turn, the SenseScorerSignature evaluates the agent’s current understanding against the generated metric: ⬇ 1class SenseScorerSignature(dspy.Signature): 2 """Score the agent’s understanding of a learning item.""" 3 4 item_to_learn: str = dspy.InputField() 5 learning_metric: str = dspy.InputField() 6 facts: List[str] = dspy.InputField() 7 figured_out: List[str] = dspy.InputField() 8 9 sense_score: int = dspy.OutputField( 10 desc="Score from 1-10 indicating learning progress." 11 ) 12 reasoning: str = dspy.OutputField( 13 desc="Brief explanation for the score" 14 ) Listing 4: Sense scoring via DSPy. Let μi _i denote the metric generated for item IiI_i, and ϕt _t the sense score at turn t. The transition condition is: ϕt(I∗,μI∗,t,Kt)≥τI∗⟹σI∗←completed _t(I^*, _I^*,K_t,K_t)\;≥\; _I^*\;\; \;\; _I^*← completed (11) where tK_t is the accumulated fact set and τI∗ _I^* is the threshold (default τ=8τ=8 out of 10). The sense reasoning rtr_t is fed back to Player 1 on the next turn, providing explicit feedback about why the score was what it was and what knowledge is still missing. 4.4 Database as Control Plane One of the most architecturally significant choices in v2 is using an SQLite database as the agent’s programmable context. Every turn, the agent reads its state from and writes its state to six database tables (Table 1). Table 1: Database schema. Six SQLite tables constitute the agent’s externalized cognitive state. Table Purpose items_to_learn Learning curriculum with state machine inputs Key-value store for per-turn game state game Turn-by-turn history of frames, actions, diffs guesses Player 1’s hypothesis lists per turn figured_outs Player 1’s confirmed observations per turn losing_action_seqs Action sequences that led to game over This is not mere persistence—it is a control plane. By modifying database contents, one changes the agent’s behavior without altering code or prompts. Want the agent to skip action learning? Delete those rows. Want to seed it with human-provided facts? Insert them. Want to change the learning order? Reorder the items. The LLM’s context window becomes programmable through database state: Promptt=f(ℱt,DB.facts,DB.item_to_learn,DB.history,DB.sense_score)Prompt_t=f\! (F_t,\;DB. facts,\;DB. item\_to\_learn,\;DB. history,\;DB. sense\_score ) (12) This architecture differs fundamentally from standard prompt engineering or template-based agent frameworks. In those approaches, the prompt structure is static and the content varies only through direct substitution. In Sensi v2, the database mediates between the symbolic control structure (curriculum, state machine) and the neural inference engine (LLM), creating a form of neuro-symbolic programming where the symbolic layer guides the neural layer through structured learning. The practical implication is external steerability: the entire learning process can be monitored, modified, and debugged through standard database operations. This is analogous to how control planes in distributed systems separate the data path (what the system does) from the control path (how the system is configured). 055101014141515202024242525303032320224466881010τ=8τ=8ActionsEnergyWin Cond.Kt→K_t→ factsKt→K_t→ factsTurnSense ScoreItem 1: ActionsItem 2: EnergyItem 3: Win condition Figure 3: Sense score progression over turns (illustrative). The agent’s sense score for each curriculum item rises toward the threshold τ=8τ=8 as it accumulates figured-out items. Vertical dashed lines mark curriculum transitions where one item is completed and the next begins. The entire curriculum is completed in approximately 32 turns. Scores are representative of observed behavior; exact per-turn values vary across runs. 4.5 POMDP with Internal Reward We can frame Sensi v2’s learning process as a POMDP augmented with an internal reward signal. Define the agent’s epistemic state at time t as: ℰt=(t,Gt,Kt,It∗,ϕt,μIt∗)E_t= (K_t,\;G_t,\;K_t,\;I^*_t,\; _t,\; _I^*_t ) (13) where tK_t is the fact set, GtG_t is guesses, KtK_t is figured-out, It∗I^*_t is the active learning item, ϕt _t is the sense score, and μIt∗ _I^*_t is the evaluation metric for the active item. The transition dynamics are: ℰt+1=T(ℰt,at,ℱt+1,LLM1(ℱt,ℱt+1),LLM3(⋅),LLM4(⋅))E_t+1=T\! (E_t,\;a_t,\;F_t+1,\;LLM_1(F_t,F_t+1),\;LLM_3(·),\;LLM_4(·) ) (14) The policy π is the composition of all five LLM modules: at=π(ℰt,ℱt)=LLM5(LLM4(ℱt,Δt,ℰt))a_t=π(E_t,F_t)=LLM_5\! (LLM_4\! (F_t, _t,E_t ) ) (15) What distinguishes this formulation is that the reward signal is internal. The agent does not directly optimize for the game score. Instead, it optimizes for the sense score ϕt _t on the current learning item. The game score is an emergent property of learning the correct things in the correct order. This is analogous to intrinsic motivation in reinforcement learning [10, 8], but implemented through LLM-based self-evaluation rather than curiosity-driven exploration bonuses [13]. 5 Experiments 5.1 Setup Sensi v1 uses ChatGPT 5.1 (Thinking High) as the backbone model for both Observer and Actor roles. Sensi v2 uses Gemini 3.1 Pro for all five pipeline stages. Both versions are implemented using DSPy [6] for structured LLM interactions, with SQLite for state persistence. The sense score threshold is τ=8/10τ=8/10 for all learning items. We note that the use of different backbone models across versions is a confounding factor. This was driven by practical considerations (model availability and multimodal capabilities at time of development) rather than experimental design. We discuss this limitation in §9. 5.2 Baselines We compare against the following: • Sensi v1: The two-player architecture without curriculum learning. • Agentica (Symbolica): A reported system requiring approximately 1,600–3,000 game interactions per game to build understanding. • Random agent: Uniform random action selection as a lower bound. Our primary comparison axis is sample efficiency—the number of game interactions required to complete an exploration or learning phase—rather than win rate alone. This is because sample efficiency determines the practical viability of test-time learning under compute constraints. Sensi v1ℱtF_tPlayer1ObserverPlayer2Actorata_tGt,KtG_t,K_tnext turnSensi v2ℱtF_tFrameDiffLLM1MetricGenLLM2SenseScoreLLM3Player1ObserverPlayer2Actorata_tSQLiteDBGt,KtG_t,K_tshared= v1 corenew= v2 additions Figure 4: Architectural comparison of Sensi v1 and v2. V1 (left) uses a simple two-player loop with shared hypothesis lists. V2 (right) preserves the Player1/Player2 core (blue) but adds frame differencing, dynamic metric generation, sense scoring, and the database-as-control-plane (red/orange). The two-player core is embedded within a richer learning infrastructure. 6 Results and Analysis 6.1 Task Performance We report task performance honestly and upfront: Sensi v1 solves 2 levels; Sensi v2 solves 0 levels. Table 2 summarizes the comparison. Table 2: Comparison of Sensi versions and baselines. Sample efficiency is measured as the number of game interactions to complete a learning or exploration phase. V2 achieves dramatically higher sample efficiency despite solving fewer levels. System Backbone Lvls Won Tries Curric. Judge Random agent — 0 — × × Agentica (Symbolica) Unknown Unknown 1,600–3,000 × × Sensi v1 ChatGPT 5.1 2 Variable × × Sensi v2 Gemini 3.1 Pro 0 ∼ 32 ✓ ✓ The v2 result—0 levels won—deserves contextualization. The agent did complete its entire learning curriculum. It systematically explored what each action does, learned about the energy system, and formed a hypothesis about the win condition. The learning pipeline executed correctly: the state machine transitioned as designed, sense scores rose to meet thresholds, and figured-out items were promoted to facts. The agent learned the wrong things, but it learned them efficiently and systematically. 6.2 Curriculum Execution The curriculum state machine operated correctly across all runs. The agent: 1. Started with the first learning item (“learn what each action does”), 2. Accumulated guesses and figured-out items about action effects, 3. Received rising sense scores as the figured-out list grew, 4. Transitioned to completed when ϕt≥τ=8 _t≥τ=8, 5. Promoted figured-out items to facts, 6. Advanced to the next learning item (“learn about energy”), 7. Repeated the cycle through all curriculum items. This validates the programmable context architecture: the database-mediated curriculum successfully guided the LLM agent through a structured learning process spanning multiple game resets. The agent’s behavior changed appropriately at each curriculum transition, demonstrating that external symbolic control can effectively steer neural inference. 6.3 Sample Efficiency The agent completed its entire learning curriculum in approximately 32 action attempts. For comparison, Agentica from Symbolica reportedly requires 1,600 to 3,000 interactions per game: Sample efficiency ratio=NbaselineNSensi=1600–300032≈50–94×Sample efficiency ratio= N_baselineN_Sensi= 1600--300032≈ 50--94× (16) This 50–94× improvement in sample efficiency is the paper’s central quantitative result. Even though v2 learned incorrect facts, the speed of learning is what matters for the broader research agenda. If an architecture can learn from 32 interactions, the bottleneck is not sample efficiency but grounding—ensuring that what the LLM perceives is actually correct. This is a fundamentally different (and more tractable) problem. In the context of test-time compute research [20], this is especially relevant. Test-time compute is expensive. An architecture that needs 32 interactions to form hypotheses about a new task is fundamentally more practical than one requiring 3,000, particularly when combined with budget-constrained inference pipelines. 6.4 Failure Analysis: Self-Consistent Hallucination Cascade The failure mode is instructive and constitutes a diagnostic contribution. The frame differencing step (LLM1LLM_1) produces natural language descriptions of visual changes between frames. Gemini 3.1 Pro, when examining small pixel-art frames scaled to 10×10×, frequently misidentifies objects, positions, and movements. A player moving left might be described as “object shifted right.” A UI element showing remaining energy might be described as “decorative border pattern.” These errors propagate through the entire pipeline in what we term a self-consistent hallucination cascade: Δtwrong→Ktwrong→ϕthigh→σI∗=completed→t+1wrong _t^wrong\;→\;K_t^wrong\;→\; _t^high\;→\; _I^*= completed\;→\;K_t+1^wrong (17) The cascade operates as follows: 1. Perception error. The frame differencing LLM (LLM1LLM_1) produces an incorrect diff Δtwrong _t^wrong. 2. Hypothesis contamination. Player 1 (LLM4LLM_4) builds figured-out items based on the wrong diff, producing KtwrongK_t^wrong. 3. Spurious validation. The sense scorer (LLM3LLM_3) evaluates the agent’s understanding based on the internal consistency of the figured-out items—not their correspondence to ground truth. Since the items are coherent (just wrong), the scorer assigns a high score ϕthigh _t^high. 4. Premature completion. The high score triggers the state machine transition: σI∗←completed _I^*← completed. 5. Error compounding. The (wrong) figured-out items are promoted to facts t+1wrongK_t+1^wrong, which become part of the context for the next learning item, compounding the error. 1. Perception ErrorFrameDiff (LLM1)Player moves left→ “object shifted right”2. Wrong HypothesisPlayer1 (LLM4)figured_out:“ACTION3 moves right”3. Spurious ValidationSenseScore (LLM3)“Clear directionalmapping” → ϕt=9/10 _t=9/104. Completionϕt≥τ _t≥τσI∗← _I^*← completedKt→K_t→ facts5. Error CompoundsNext learning itembuilds on wrong factsabout actionsGround truth:“ACTION3 moves left”correct (never reaches pipeline)Self-consistent hallucination cascade:Δtwrong→Ktwrong→ϕthigh→σI∗=completed→t+1wrong _t^wrong→ K_t^wrong→ _t^high→ _I^*= completed _t+1^wrong Figure 5: Self-consistent hallucination cascade. A frame differencing error (step 1) propagates through the pipeline. Player1 builds a wrong hypothesis (step 2), which the sense scorer validates because it is internally consistent (step 3). This triggers premature completion (step 4), and the wrong knowledge contaminates subsequent learning items (step 5). The ground truth (green, bottom-left) never enters the pipeline. This diagnosis is precise and actionable. The architecture—the curriculum, the state machine, the sense scoring pipeline—all function correctly. The information flow is exactly right. The perception layer is the single point of failure, and it is the most straightforward component to improve through better vision models, hybrid programmatic-neural pixel analysis, or ground-truth anchoring mechanisms. 7 Discussion Architecture validated independently of task performance. The 0-level result for v2 might appear to invalidate the approach, but we argue the opposite. The curriculum executed correctly, the state machine transitioned properly, the sense scores rose and fell as expected, and the agent systematically worked through its learning queue. The failure is localized to a single, identifiable component (frame differencing), not to the architectural design. This is analogous to validating a sorting algorithm’s correctness even when the comparator function is buggy: the algorithm works; the comparator needs fixing. The bottleneck has shifted. Before Sensi v2, the question was “can LLM agents learn game mechanics efficiently?” After v2, the question becomes “given that they can learn efficiently, how do we ensure they learn correctly?” This is a more tractable problem because perceptual grounding admits direct engineering solutions: programmatic pixel comparison, fine-tuned vision models, structured visual analysis, or hybrid approaches that combine LLM perception with deterministic pixel differencing. The curriculum and learning architecture, by contrast, required novel design. Implications for test-time compute budgets. The 32-try result has practical implications beyond games. Any test-time inference scenario—novel tool use, unfamiliar APIs, new document formats—could benefit from architectures that form structured hypotheses from minimal interactions. A 50–94× reduction in required interactions translates directly to reduced cost and latency in deployment settings where test-time compute is budgeted. Database-as-control-plane as a general pattern. We believe the database-as-control-plane pattern is underexplored in the agent architecture literature. Most agent frameworks manage context through prompt templating—static templates with variable substitution. Sensi v2 demonstrates that externalizing cognitive state to a database and programmatically injecting it into prompts creates more modular, steerable, and debuggable agents. This pattern naturally supports multi-agent coordination, human-in-the-loop oversight, and post-hoc analysis of agent behavior through standard database queries. 8 Related Work Test-time computation and training. The spectrum of test-time adaptation ranges from inference-time reasoning [16, 15] through in-context learning to full gradient-based test-time training [12, 20]. T-Discover [20] is most relevant to our work: it performs reinforcement learning on a single test problem, training the model’s weights to discover solutions. Sensi differs fundamentally in that no gradient updates occur; instead, learning is realized through structured knowledge accumulation in an external database. The framework applies sequential skill acquisition at inference time (no retraining) to improve stability, out-of-distribution generalization, and continual adaptation in agentic workflows. This directly extends ideas in EvoTest [4], an evolutionary test-time learning framework for self-improving systems. This makes Sensi applicable to black-box LLM APIs where weight access is unavailable. LLM agents for games and environments. Voyager [14] builds an open-ended embodied agent with a skill library, curriculum, and code-as-action paradigm in Minecraft. Unlike Sensi, Voyager operates in a well-understood environment with known mechanics. ReAct [19] interleaves reasoning and acting but does not maintain persistent state across episodes. Reflexion [11] adds verbal reinforcement learning through self-reflection, which is conceptually similar to Sensi’s sense scoring but without the curriculum structure or external state machine. SPRING [17] uses game manuals as grounding documents—the opposite of Sensi’s approach, which assumes no prior game knowledge. Curriculum learning. The idea of ordering training examples by difficulty dates to Bengio et al. [2] and has been extensively studied in deep RL [9]. Sensi’s contribution is applying curriculum learning at test time to structure an LLM agent’s exploration, rather than ordering training data during model development. The curriculum is hand-designed in the current work; automatic curriculum generation [9] is a natural extension. LLM self-evaluation. The use of LLMs as judges has been studied extensively [21], primarily for evaluating model outputs in benchmarking contexts. Self-Refine [7] uses self-feedback for iterative improvement. Sensi’s sense scorer extends this paradigm in two ways: (1) the evaluation rubric is itself generated by an LLM, making the system adaptive to arbitrary learning domains, and (2) the score is used as a control signal for a state machine rather than just feedback for refinement. ARC and abstraction reasoning. The ARC benchmark [3] tests abstract reasoning through visual pattern completion. Program synthesis approaches [18] attempt to solve ARC tasks by generating programs. ARC-AGI-3 [1] extends this to interactive game-playing, requiring online learning rather than one-shot inference. Sensi is, to our knowledge, the first architecture to apply curriculum-based test-time learning specifically to the ARC-AGI-3 game-playing challenge. 9 Limitations and Future Work Limitations. Several limitations should be noted: (1) Sensi v2 won 0 game levels, meaning the architecture has not yet demonstrated end-to-end task success with curriculum learning. (2) The curriculum is hand-designed; the choice and ordering of learning items requires human judgment about what is learnable and in what order. (3) V1 and v2 use different backbone models (ChatGPT 5.1 and Gemini 3.1 Pro, respectively), confounding direct comparison of architectural changes with model capability differences. (4) The sample efficiency comparison to Agentica/Symbolica is based on reported numbers rather than controlled experiments with identical conditions. (5) The sense scorer evaluates internal consistency rather than ground-truth correctness, creating a fundamental vulnerability to coherent hallucinations. Future work. The most immediate priority is fixing the perception layer. Candidate approaches include: (i) hybrid programmatic and LLM-based frame differencing, where deterministic pixel comparison provides ground truth that the LLM interprets; (i) ground-truth anchoring for the sense scorer, where a subset of evaluation criteria are verified against actual game state rather than relying solely on LLM judgment; and (i) fine-tuned vision models specifically trained on pixel-art game frames. Beyond perception, several directions are promising: automatic curriculum generation, where the agent determines its own learning order based on initial exploration; cross-game transfer, where facts learned in one game inform hypotheses in structurally similar games; and multi-agent curriculum learning, where multiple agents with different curricula share and validate knowledge through the database-as-control-plane. 10 Conclusion We presented Sensi, an LLM agent architecture that evolves across two iterations to address the sample efficiency problem in test-time game learning. Sensi v1 demonstrates that separating perception from action via a two-player architecture enables systematic hypothesis accumulation, solving 2 ARC-AGI-3 levels. Sensi v2 adds curriculum learning, a database-as-control-plane, and LLM-as-judge evaluation, achieving 50–94× greater sample efficiency than comparable systems—completing its learning curriculum in approximately 32 interactions versus 1,600–3,000 for baselines. V2’s 0-level result is an honest negative finding, but the failure mode is precisely diagnosed: a self-consistent hallucination cascade originating in the perception layer, where frame differencing errors produce internally coherent but factually incorrect knowledge that passes the sense scorer’s consistency checks. This diagnosis shifts the research bottleneck from “how to learn efficiently” to “how to perceive correctly”—a more tractable engineering problem. The broader contribution is methodological: structured test-time learning—with curriculum, state machine, and programmable context—works as an architectural pattern. The 32-try budget suggests that LLM agents can form structured hypotheses about unknown environments with human-like sample efficiency, provided the perception layer is reliable. Fixing perception is the next step; the learning architecture is ready. Acknowledgments The author thanks the ARC Prize Foundation for creating the ARC-AGI-3 challenge. The Sensi agent code is built on the ARC-AGI-3-Agents framework.111https://github.com/arcprize/ARC-AGI-3-Agents This work was conducted independently without institutional funding. References [1] ARC Prize Foundation (2024) ARC prize 2024: technical report. Technical report ARC Prize Foundation. External Links: Link Cited by: §1, §2.1, §8. [2] Y. Bengio, J. Louradour, R. Collobert, and J. Weston (2009) Curriculum learning. Proceedings of the 26th International Conference on Machine Learning, p. 41–48. Cited by: §8. [3] F. Chollet (2019) On the measure of intelligence. arXiv preprint arXiv:1911.01547. Cited by: §8. [4] Y. He, J. Liu, Y. Liu, Y. Li, T. Cao, Z. Hu, X. Xu, and B. Hooi (2025) EvoTest: evolutionary test-time learning for self-improving agentic systems. arXiv preprint arXiv:2510.13220. Cited by: §2.3, §8. [5] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra (1998) Planning and acting in partially observable stochastic domains. Artificial Intelligence 101 (1-2), p. 99–134. Cited by: §2.2. [6] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Mober, et al. (2023) DSPy: compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714. Cited by: §5.1. [7] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. (2024) Self-refine: iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §8. [8] D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell (2017) Curiosity-driven exploration by self-supervised prediction. Proceedings of the 34th International Conference on Machine Learning, p. 2778–2787. Cited by: §3.2, §4.5. [9] R. Portelas, C. Colas, L. Weng, K. Hofmann, and P. Oudeyer (2020) Automatic curriculum learning for deep RL: a short survey. In International Joint Conference on Artificial Intelligence, p. 4819–4825. Cited by: §8. [10] J. Schmidhuber (1991) A possibility for implementing curiosity and boredom in model-building neural controllers. Proceedings of the International Conference on Simulation of Adaptive Behavior, p. 222–227. Cited by: §3.2, §4.5. [11] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2024) Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §8. [12] Y. Sun, X. Li, K. Dalal, J. Xu, A. Vikram, G. Zhang, Y. Dubois, X. Chen, X. Wang, S. Kumbhar, et al. (2024) Learning to (learn at test time): RNNs with expressive hidden states. arXiv preprint arXiv:2407.04620. Cited by: §2.3, §8. [13] R. S. Sutton and A. G. Barto (2018) Reinforcement learning: an introduction. 2nd edition, MIT Press. Cited by: §4.5. [14] G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar (2024) Voyager: an open-ended embodied agent with large language models. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §8. [15] X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou (2023) Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations, Cited by: §1, §2.3, §8. [16] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V. Le, and D. Zhou (2022) Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, Vol. 35, p. 24824–24837. Cited by: §1, §2.3, §8. [17] Y. Wu, Y. Xie, S. Shi, X. Yu, F. Liu, Y. Liu, J. N. Foerster, and W. Y. Clark (2023) SPRING: studying the paper and reasoning to play games. arXiv preprint arXiv:2305.15486. Cited by: §8. [18] Y. Xu, E. B. Khalil, and S. Sanner (2023) Program synthesis for ARC. arXiv preprint arXiv:2306.03553. Cited by: §8. [19] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, Cited by: §8. [20] M. Yuksekgonul, D. Koceja, X. Li, F. Bianchi, J. McCaleb, X. Wang, J. Kautz, Y. Choi, J. Zou, C. Guestrin, and Y. Sun (2025) Learning to discover at test time. arXiv preprint arXiv:2503.07432. Cited by: §1, §2.3, §6.3, §8. [21] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, et al. (2024) Judging LLM-as-a-Judge with MT-Bench and chatbot arena. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §1, §8. Appendix A Sensi v1: Discovered Game Knowledge For completeness, we list the 15 figured-out items that Sensi v1 discovered for game LS20, Level 1. These items were accumulated over the course of gameplay and represent the agent’s complete model of the game mechanics: 1. RESET starts the game. 2. Blue platform (with red top) is our player. 3. ACTION1 moves the player 1 pixel up. 4. ACTION2 moves the player 1 pixel down. 5. ACTION3 moves the player 1 pixel left. 6. ACTION4 moves the player 1 pixel right. 7. Each action consumes 1 unit of energy shown as dots. 8. Key generator produces matching keys for doors. 9. Walking into a key generator creates a key of that color. 10. Keys are consumed when used to open matching doors. 11. Doors disappear when opened with the correct key. 12. Energy dots provide additional energy when collected. 13. Game over occurs when energy reaches zero. 14. Stars are collectible items. 15. Collecting all stars completes the level. These items were discovered purely through interaction, without any prior game knowledge. The accuracy of these items (verified against actual game mechanics) demonstrates that the two-player architecture can produce correct game models when the perception layer functions adequately. Appendix B Algorithm: Sensi v2 Turn Execution Algorithm 1 Sensi v2: Single Turn Execution 1:Game frame ℱtF_t, previous frame ℱt−1F_t-1, database DB 2:Action ata_t 3:Δt←FrameDiff(ℱt−1,ℱt) _t (F_t-1,F_t) ⊳ LLM1: visual differencing 4:I∗←DB.get_active_item()I^* .get\_active\_item() ⊳ Current learning item 5:t←DB.get_facts()K_t .get\_facts() ⊳ Accumulated facts 6:if σI∗=not_reached _I^*= not\_reached then 7: σI∗←learning _I^*← learning 8: μI∗←MetricGen(I∗) _I^* (I^*) ⊳ LLM2: generate rubric 9: DB.store_metric(I∗,μI∗)DB.store\_metric(I^*, _I^*) 10:end if 11:ϕt,rt←SenseScore(I∗,μI∗,t,Kt−1) _t,r_t (I^*, _I^*,K_t,K_t-1) ⊳ LLM3: evaluate 12:if ϕt≥τI∗ _t≥ _I^* then 13: σI∗←completed _I^*← completed 14: DB.promote_to_facts(Kt−1)DB.promote\_to\_facts(K_t-1) ⊳ Knowledge accumulation 15:end if 16:Gt,Kt←Player1(ℱt,Δt,t,Gt−1,Kt−1,I∗,rt)G_t,K_t _1(F_t, _t,K_t,G_t-1,K_t-1,I^*,r_t) ⊳ LLM4 17:DB.store(Gt,Kt)DB.store(G_t,K_t) 18:dt,at←Player2(Gt,Kt,t,I∗)d_t,a_t _2(G_t,K_t,K_t,I^*) ⊳ LLM5 19:DB.log_action(at,dt)DB.log\_action(a_t,d_t) 20:return ata_t

Instant research discovery