Paper deep dive
Online Monitoring and Corrective Steering of Programming Agents
Shuyang Liu, Saman Dehghan, Ji Young Kim, Jatin Ganhotra, Martin Hirzel, Reyhaneh Jabbarvand
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/10/2026, 2:59:13 AM
Summary
The paper introduces LivePlan, an online monitoring and corrective steering framework for LLM-based programming agents. It decouples trajectory monitoring (judging) from advice generation (advising) by using a deterministic, rule-based monitor to detect behavioral drifts (e.g., oscillation, stagnation, plan violations) without invoking an LLM. Only when a drift is detected does it consult an LLM advisor for high-level corrective steering. Evaluated on SWE-bench Verified and Pro, LivePlan improves issue resolution rates by up to 15.2% compared to vanilla SWE-agent with minimal cost increase.
Entities (10)
Relation Signals (10)
LivePlan → implements → decoupled judging and advising
confidence 95% · LivePlan decouples judging from advising: a deterministic, rule-based monitor... and only when an issue is detected does it consult an advisor LLM
LivePlan → improves → issue resolution rates
confidence 95% · LivePlan notably improves issue resolution rates, achieving consistent gains of up to 15.2%
LivePlan → outperforms → SWE-agent
confidence 95% · Compared to vanilla SWE-agent, LivePlan notably improves issue resolution rates
Deterministic Monitor → detects → Behavioral Drift
confidence 92% · deterministic, rule-based monitor examines general signals over the trajectory to detect issues
LivePlan → evaluatedon → SWE-bench Pro
confidence 92% · evaluate it using five LLMs ... across SWE-bench Verified and SWE-bench Pro
LivePlan → evaluatedon → SWE-bench Verified
confidence 92% · evaluate it using five LLMs ... across SWE-bench Verified and SWE-bench Pro
LLM Advisor → provides → corrective steering
confidence 92% · consult an advisor LLM for a high-level, next-step correction
LivePlan → outperforms → SAGE
confidence 90% · LivePlan consistently outperforms alternative approaches in resolution rate
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Fixing GitHub issues in large-scale projects is a long-horizon task, especially when a fix requires changes across multiple locations or the issue description lacks the information needed to localize and repair it. As a result, agents traverse long trajectories that are prone to inefficiency and error: they drift away from their intended plan, repeat failed actions, or terminate without a working patch. This paper proposes LivePlan to monitor, detect, and correct such behavioral inefficiencies and drifts in real time. LivePlan decouples judging from advising: a deterministic, rule-based monitor examines general signals over the trajectory to detect issues without invoking an LLM, and only when an issue is detected does it consult an advisor LLM for a high-level, next-step correction. This design avoids the misleading re-planning and costly interventions of prior approaches. We implement LivePlan on top of SWE-agent and evaluate it using five LLMs (three as executor agents and two as advisors) across SWE-bench Verified and SWE-bench Pro. Compared to vanilla SWE-agent, LivePlan notably improves issue resolution rates, achieving consistent gains of up to 15.2% (average: 9.9%), while incurring only an additional cost of $0.08 per instance. The additional solutions concentrate on medium and hard instances. LivePlan consistently outperforms alternative approaches in resolution rate, with minimal regression on already successful runs and new successes on problems that no baseline solves.
Tags
Links
- Source: https://arxiv.org/abs/2608.06701v1
- Canonical: https://arxiv.org/abs/2608.06701v1
Trouble viewing inline? Open PDF directly →
Full Text
66,771 characters extracted from source content.
Expand or collapse full text
Online Monitoring and Corrective Steering of Programming Agents Shuyang Liu † , Saman Dehghan † , Ji Young Kim † , Jatin Ganhotra ∗ , Martin Hirzel ∗ , and Reyhaneh Jabbarvand † † University of Illinois Urbana-Champaign, USA, sl225, samand2, jyk14, reyhaneh@illinois.edu ∗ IBM, USA, jatinganhotra, hirzel@us.ibm.com Abstract—Fixing GitHub issues in large-scale projects is a long-horizon task, especially when a fix requires changes across multiple locations or the issue description lacks the information needed to localize and repair it. As a result, agents traverse long trajectories that are prone to inefficiency and error: they drift away from their intended plan, repeat failed actions, or terminate without a working patch. This paper proposes LIVEPLAN to monitor, detect, and correct such behavioral inefficiencies and drifts in real time. LIVEPLAN decouples judging from advising: a deterministic, rule-based monitor examines general signals over the trajectory to detect issues without invoking an LLM, and only when an issue is detected does it consult an advisor LLM for a high-level, next-step correction. This design avoids the misleading re-planning and costly interventions of prior approaches. We implement LIVEPLAN on top of SWE-agent and evaluate it using five LLMs (three as executor agents and two as advisors) across SWE-bench Verified and SWE-bench Pro. Compared to vanilla SWE-agent, LIVEPLAN notably improves issue resolution rates, achieving consistent gains of up to 15.2% (average: 9.9%), while incurring only an additional cost of $0.08 per instance. The additional solutions concentrate on medium and hard instances. LIVEPLAN consistently outperforms alternative approaches in resolution rate, with minimal regression on already successful runs and new successes on problems that no baseline solves. I. INTRODUCTION LLM-based agents solve complex software-engineering tasks by following a plan: the sequence of issue-resolution phases prescribed in their system prompt, e.g., localize, repro- duce, patch, validate [37]. Yet an agent may drift away from this plan during execution [18], e.g., skipping validation, or exhibit inefficiencies [8, 19] such as repeated actions and tool call failures, which cause failures or inflate costs even when the agent succeeds [17]. Existing attempts to correct such drift suffer from three main limitations: (1) They fold judging (is the trajectory in trouble) and advising (what should the agent do to fix the trouble) into a single mechanism: an advisor LLM serves as the judge to eval- uate the trajectory and, based on that evaluation, recommends corrective actions for the remainder of the trajectory [31]. Judging and advising are inherently conflicting tasks: an advisor prompted to diagnose and provide corrective advice is incentivized to find a problem, a tendency related to the documented sycophantic bias of LLMs toward framing their prompts [30]. The advisor may surface a problem that does not exist and impose misleading advice that derails a run that was, in fact, on track. Overcoming this limitation requires separating judging and advising. Figure 1 shows an example where SAGE [11] re-plans a trajectory for an issue originally resolved by SWE-agent [37] Action: view lib/ansible/plugins/callback/__init__.py Observation: The path lib/ansible/plugins/callback/__init__.py is not an absolute path... Thought: The user wants to implement the `host_label` static method in `CallbackBase` Thought: I made a mistake in the path; it should be absolute. I will correct it and retry viewing the file. Thought: I'm still not finding `CallbackBase`. I will try to be more strategic and use `rg` to find the exact line numbers of `class CallbackBase` view /app/lib/ansible/plugins/callback/__init__.py --view_range -100 -1 Observation: Invalid `view_range`: [-100, -1]. Its first element `-100` 43 ... (Repeat Failed view) Submit 2 10 1 SAGE, ansible-185d41031 in SWE-bench Pro run with Gemini-2.5-Flash New Plan Customized High-Level Plan 1. Identify all locations in the default stdout callback (and related callback methods) where host labels are constructed, especially for delegated tasks. 2. Implement a robust `CallbackBase.host_label(result)` in `lib/ansible/plugins/callback/__init_ _.py` that mirrors existing formatting and handles missing `_result` or delegation metadata safely. 3. Replace each duplicated host-label construction with calls to `CallbackBase.host_label(result)` while preserving any surrounding formatting (coloring, prefixes). 4. Run targeted checks or minimal validation to ensure delegated and non-delegated outputs are consistent with previous behavior. ... (Change strategy) Fig. 1: An example of SAGE [11] incorrect advice (a new global plan), causing SWE-agent to fail on a SWE-bench Pro [7] issue it had originally resolved in a vanilla run. with Gemini-2.5-Flash. SAGE uses GPT-5.2-Codex as an advisor to analyze an existing trajectory—blind to the res- olution status—and generates a new plan for re-execution to resolve possible trajectory problems. The advisor’s plan incorrectly directs the agent to implement CallbackBase in lib/ansible/plugins/callback/__init__.py, which does not exist in the repository. This causes the agent to struggle through its first 10 steps, trying to locate a file that does not exist, ultimately terminating without patching. (2) Using an LLM-as-a-judge requires determining how often to trigger it for analysis. Some techniques wait until the end of the trajectory to perform a post-hoc analysis and rerun the agent with advice (new plan) based on the analysis [8, 11, 27]. However, although the advisor can provide a global plan to guide the agent from the beginning, the benefit comes with caveats: if the plan is incorrect, similar to Fig. 1, the agent may not be able to understand and recover, given it is instructed to follow the plan. This is also costly, as it feeds the entire trajectory to the advisor LLM and reruns the agent. Other techniques periodically ask the advisor to evaluate a sliding window of the trajectory [9, 10, 25]. However, if the frequency is too high [9, 25], the approach remains costly; if it is too low [10], the steering feedback arrives too late, making it hard for the agent to recover from the failure mode. Overcoming this limitation entails triggering trajectory analysis only when necessary. Figure 2 shows the execution of SWE-agent with DeepSeek- V3 on Openlibrary-ba3abfb6a from SWE-bench Pro [7]. arXiv:2608.06701v1 [cs.SE] 7 Aug 2026 Steps 1 - 5 Navigate the directory structure, read relevant files Steps 6 - 9 Continue exploring, search for and read functions Understood! Let's inspect the related functions to understand their current logic view __init__.py --view_range 760 950 11 openlibrary-ba3abfb6a, SWE-bench Pro, Deepseek-V3 Locate the actual validate_record implementation... then inspect the validation checks Open the relevant portion, and load ... to inspect the validation checks 1 2 Steps 12 - 15 Read broader context Make an initial submission (no edit) Update the core import validation logic to accept override validation 3 submit 16 Step 5th Step 10th Step 15th Found it! The ‘load’ function is defined in ‘__init__.py”. Let’s inspect this file and implement the changes. view __init__.py 10 Fig. 2: An advisor periodically checking the trajectory fails to resolve an issue due to late feedback. A periodic advisor, similar to SWE-PRM [10], evaluates the trajectory every five steps. During the first intervention after step 5, the executor is advised to continue exploring the repository to locate the buggy file. At step 10, it views the exact bug location. However, before taking any repair action in the next step, the advisor incorrectly concludes that the buggy location has not been found and that the agent should keep exploring, causing the agent to continue exploring without patching for the next five steps. After step 15, the advisor offers a patching suggestion, but the guidance arrives too late: the executor has already decided to terminate the run and resubmits, leaving the issue unresolved. (3) While LLM-as-a-judge offers generalizability, i.e., the LLM draws on its internal knowledge to flag problems that are not necessarily encoded, it comes at the cost of reliability: LLMs are known to be unreliable evaluators that both miss real problems and flag spurious ones [8]. Overcoming this limitation entails a deterministic judge whose approach can flag a broad class of behavioral drifts. We propose LIVEPLAN for online monitoring of agent trajectories to detect behavioral drifts and intervene with cor- rective steering. LIVEPLAN separates judgment from advising: it relies on a deterministic, rule-based monitor that examines a set of general signals over the trajectory to determine behavioral drifts (§I-A). These signals are general rather than tied to a specific benchmark or repository, so they flag a broad class of behavioral drift. Upon detection of a behavioral drift, LIVEPLAN generates predefined or custom high-level, corrective, next-step advice to steer the trajectory (§I-B). By separating judgment from advising and planning, judging with a deterministic monitor that does not hallucinate a non- existent problem, and advising only when that monitor fires, LIVEPLAN addresses the limitations of prior work. We implement LIVEPLAN on top of SWE-agent [37] and compare its performance against four baselines and ablated versions on the SWE-bench Verified [12] and SWE-bench Propose action Think Observation /Advice Incrementally update Trajectory Representation Configurable Signal Database Generate a custom advice (based on: issue description, last guidance, recent trajectory, pre-definedadvice) Use pre-defined advice yes no no yes Execute action Keep action, keep advice yes no Any signal detected? Still cooling down? Block action? Drop action, keep advice Data LLM-basedRule-based Legend: Gate vanilla Advisor Monitor Executor Agent Intervention Processor Fig. 3: Overview of LIVEPLAN workflow. Pro [7] datasets, using five diverse general and reasoning LLMs (three as executors and two as advisors). The compre- hensive evaluation confirms the effectiveness of LIVEPLAN in consistently improving the resolution rate of SWE-agent across all models and benchmarks, with consistent gains of up to 15.2% (average: 9.9%). LIVEPLAN not only consistently resolves more GitHub issues than the baselines but also guides the agent more effectively toward solving medium and hard issues. Besides improving the overall resolution rate, LIVEPLAN yields trajectories that are more compliant with the predefined instruction plan in SWE-agent [18], i.e., explore, localize, repair, and validate [33]. Our contributions are: • Technique. We propose LIVEPLAN, an online monitoring and intervention technique that decouples judging from advising for agent trajectory steering. LIVEPLAN pairs a deterministic, rule-based monitor to detect whether and where a trajectory is in trouble without invoking an LLM, and thus without the risk of hallucinating a non-existent problem (§I-A). The LLM advisor is consulted only when the monitor fires and only for a high-level, next-step correc- tion (§I-B). • Empirical Evaluation. We perform a large-scale evaluation to assess the effectiveness of LIVEPLAN and other tech- niques on two widely used GitHub issue datasets. The study results in 7752 trajectories, 4668 of which were generated by intervention approaches, providing a valuable dataset for future research. We conducted a manual analysis of successful and unsuccessful interventions by LIVEPLAN and other baselines, discussing major root causes for intervention failures and guidelines for future research. I. LIVEPLAN APPROACH LIVEPLAN consists of three components (Figure 3): (1) Monitor incrementally constructs process-centric represen- tations of the trajectory, evaluates them against a configurable set of general signals to determine behavioral drifts, and invokes the Advisor upon drift detection (§I-A); (2) Advisor 2 TABLE I. Blocking and non-blocking behavioral drifts detected by LIVEPLAN. TypeCategorySignalBehavioral DriftDescription Blocking Plan Violation Skipping phase(s) in LANGUTORY. Premature PatchingPatch before Localization. Skip PatchingFinalize without producing a patch. Skip ValidationSubmit before validating the patch. Oscillation Repeated back-edges in GRAPHECTORY Thought OscillationRepeated reasoning pattern without mean- ingful progress. Action OscillationRepeated execution loops, such as self- loops or cyclic action sequences. Non-blocking Long Stagnation Longsame-phase sequencein LANGUTORY Prolonged NavigationSpends many steps in navigation. Prolonged ReproductionProlonged bug reproduction. Prolonged PatchingPerforms many consecutive edits. Prolonged ValidationSpends excessive effort validating. Repeated Action A back edge in GRAPHECTORY Action RevisitRepeatedly performs the same action, such as revisiting previously viewed locations. returns a predefined advice corresponding to the drift, or prompts an LLM to generate a custom advice based on issue description, trajectory context, Monitor status, and latest advice, if available (§I-B); depending on the latest action and Monitor observation, (3) Intervention Processor interacts with the executor agent for corrective steering (§I-C). A. Monitor Algorithm 1 shows the workflow of Monitor along with other components. At each step, after the executor reasons about the next action but before executing it (Line 7), Monitor tentatively updates the trajectory representations and checks them for behavioral drift (Lines 8–9). If none is observed, ex- ecution proceeds (Lines 10–13). Otherwise, Monitor consults the Advisor for corrective steering advice (Lines 14–22). A common strategy is to feed an LLM the entire trajectory, or a slice of it, to review and surface issues. Using an LLM as a judge is prone to hallucination and error [8, 30]. More critically, when a single LLM performs both judging and advising, its steering suggestion is subject to its own biased as- sessment [6, 14, 20, 30]. To mitigate this, recent work proposes rubric-based judging, directing the LLM to score trajectories against predefined signals [29]. While this constrains the judge and reduces hallucination and bias, it does not eliminate the deeper problem: evaluation still depends on the LLM applying the rubric correctly and, crucially, deterministically. Layering a non-deterministic judge onto an already non-deterministic agent thus compounds rather than contains this instability. Also, a single-prompt rubric is often insufficient: reliable judging may require a tool-equipped sub-agent to actively inspect the environment, substantially increasing cost [3]. Monitor uses two process-centric trajectory representations, GRAPHECTORY and LANGUTORY [17], to generically encode behavioral drifts and detect them via deterministic algorithms. GRAPHECTORY converts the trajectory into an enriched graph, with nodes as distinct agent actions and edges as chronolog- ical execution order; LANGUTORY abstracts low-level actions into a sequence of problem-solving phases. The original GRAPHECTORY treats only actions as nodes, with thought included as a node property. LIVEPLAN augments GRAPHEC- Algorithm 1 LIVEPLAN Runtime Monitoring and Intervention Require: Issue description I , advisor cooldown threshold θ c , max consecu- tive blocking interventions θ i , long stagnation threshold θ p 1: T ←∅▷ committed trajectory 2: H ← BUILDINITIALHISTORY(I)▷ executor-visible history 3: G←∅; L←∅ ▷ GRAPHECTORY and LANGUTORY monitor states 4: lastTrigger ← 0; lastAdvisor ← 0; latestAdvice←∅ 5: blockCount← 0▷ consecutive blocking interventions 6: while task not finished do 7: step← PROPOSESTEP(H) 8: (G ′ ,L ′ )← EXTENDMONITORSTATE(G,L,step) 9: rules← CHECKRULES(G ′ ,L ′ ,θ p ) 10:if rules =∅ then 11:(T,H)← EXECUTEANDRECORD(step,T,H) 12:(G,L)← (G ′ ,L ′ ) 13:blockCount← 0 14:else 15:preDefinedAdvice← MONITORADVICE(rules) 16:advice← preDefinedAdvice 17:if |T|− lastAdvisor ≥ θ c then ▷ advisor cooldown elapsed 18:recentSteps← STEPSSINCE(T,lastTrigger) 19:customAdvice← CALLADVISOR 20:advice← customAdvice 21:lastAdvisor ←|T| 22:end if 23:if CONTAINBLOCKINGRULE(rules) and blockCount < θ i then 24:H ← APPENDADVICE(H,advice) ▷ do not execute action 25:blockCount← blockCount + 1 ▷ rollback: discard G ′ ,L ′ 26:else 27:(T,H)← EXECUTEANDRECORD(step,T,H) 28:(G,L)← (G ′ ,L ′ ) 29:H ← APPENDADVICE(H,advice) 30:blockCount← 0 31:end if 32:latestAdvice← advice 33:lastTrigger ←|T| 34:end if 35: end while 36: function CALLADVISOR 37:Input: I , recentSteps, latestAdvice, preDefinedAdvice 38:Return: one LLM-generated next-step advice 39: end function TORY with thought nodes and edges to also monitor ineffective thinking that manifests as behavioral drift. It also splits the GRAPHECTORY’s localization phase into navigation (locating the potential bug) and reproduction (confirming it), extending both GRAPHECTORY nodes and LANGUTORY alphabets with these new phases. This lets Monitor and Advisor detect be- havioral drifts and issue corrective steering more specifically. 3 Navigation (N) Patch (P) Validation (V) General python Reproduction (R) ... 18 str_replace parser.py × <str3, str4> 17 (b) Oscillation test_parser.py 24 (c) Long Stagnation (d) Plan Violation /testbed view /testbed/html.py view grep str_replace /testbed/html.py × <str1, str2> 1 2 format 3 4 (a) Repeated Action python /testbed/test_html.py 5 20 19 21 22 23 submit /testbed view 1 /testbed/io view 2 /testbed/ascii × view input grep view 5 3 4 /testbed/io/ascii/core.py –-view_range [128,300] view 6 /testbed/io/ascii/core.py ... Langutory: N /testbed/manager.py view 1 reproduce.py create 2 reproduce.py python 3 str_replace /testbed/manager.py × <str5, str6> 4 submit Langutory: NRRP Fig. 4: Examples of behavioral drifts captured by GRAPHECTORY and LANGUTORY. We define behavioral drift as a condition in a trajectory that deviates from common expectations. Under this definition, deviation from the specified plan, repetitive thoughts or actions that fail to advance the agent, and stagnation in any problem- solving phase are behavioral drifts. While signals defined this way are general, they may be incomplete. To account for that, LIVEPLAN maintains a configurable signal database that users can augment. Furthermore, as we discuss in §I-B, the LLM Advisor can still use its general knowledge to guide corrective steering. Some behavioral drifts are alarming, e.g., skipping critical problem-solving phases, such as patch validation before sub- mission, which may result in an incorrect patch. Therefore, the execution of actions that manifest such drifts should be blocked. Others represent inefficiencies that can be improved but are not necessarily harmful. For example, prolonged nav- igation through the repository to localize a bug could be due to ineffective reasoning/tooling, or a legitimate need to gather sufficient context to repair a multi-hunk bug. Therefore, execu- tion of the corresponding actions can be allowed, accompanied by corrective steering advice. Table I lists ten blocking and non-blocking behavioral drifts that Monitor can detect. They can be detected in GRAPHEC- TORY and LANGUTORY through generic signals: a back-edge in GRAPHECTORY, regardless of the nature of the action, indicates Repeated Action (Figure 4a); repeated self-loops or multi-node cycles in GRAPHECTORY indicate Thought or Action Oscillation (Figure 4b); a sequence of the same phase in LANGUTORY, whose length exceeds the stagnation threshold θ p , indicates Long Stagnation (Figure 4c); and a missing phase symbol in LANGUTORY indicates skipping a plan phase, hence Plan Violation (Figure 4d). B. Advisor LIVEPLAN either provides predefined advice or invokes the LLM-based Advisor to generate custom advice (Lines 14– 22). Predefined advice offers only high-level, problem-agnostic guidance; Figure 5 shows the predefined messages for Pro- longed Navigation and Action Oscillation drifts. Although relevant to the detected drift, it may be too general to help the executor recover, so Advisor is also equipped with an LLM to generate advice specific to the problem. Long Stagnation. Phase-specific messages are used for Navi- gation, Reproduction, Patching, and Validation. • Prolonged Navigation. You have the following options to explore next, which increase your chance to solve the prob- lem: (1) keep exploring the code base as you do, but focus on the relevant code only; (2) create and run a reproduction test to better localize the bug; (3) if you already have enough information, edit the source code to implement the fix. Oscillation. You are repeating action action in the last X trajectory steps. The observation has likely not changed. Concretely reason if this action can help you resolve the issue. If not, think about a better action in the next step so that you can resolve the problem. Fig. 5: Examples of predefined advice in LIVEPLAN. When invoked, the LLM-based Advisor receives four inputs: the issue description, the committed trajectory slice since the last advice (to minimize context), the latest advice if available, and the predefined advice for the detected drift. The predefined advice serves as a high-level hint to help it focus, while the recent trajectory remains the primary evidence for grounding the recommendation. Advisor outputs a next-step recommendation, rather than a long-horizon plan, keeping the intervention focused, avoiding over-constraining the executor, and preventing context clutter and degradation [16, 34, 35]. Since even minimal context can increase cost and latency, LIVEPLAN implements a cooling mechanism: Advisor invokes the LLM only if at least θ c trajectory steps have elapsed since the last LLM-generated advice (Line 17). C. Intervention Processor The Intervention Processor component takes the advice from Advisor and, depending on the drift type, performs one of two actions: (1) for a blocking drift, it rejects the culprit action before execution, discards the tentative GRAPHECTORY and LANGUTORY updates, and appends the advice to the executor history for corrective steering, up to θ i consecutive times to avoid stalling execution (Lines 23–25); (2) for a non-blocking drift, or once the blocking-intervention limit is reached, it lets the executor proceed and appends the advice to steer subsequent decisions (Lines 26–31). 4 TABLE I. Effectiveness on SWE-bench Pro-Python. Model Method Success Rate (%) Average Cost AdvisorExecutor - DeepSeek (V3) Vanilla21.760.05 GPT-5.2SAGE18.79 (-2.97)0.23 -Predefined Advice 25.00 (+3.24)0.11 GPT-5.2Periodic Advisor28.79 (+7.03)0.12 GPT-5.2LIVEPLAN34.09 (+12.33)0.15 - Gemini (2.5-flash) Vanilla13.170.78 GPT-5.2SAGE18.18 (+5.01)1.16 -Predefined Advice 18.94 (+5.77)0.98 GPT-5.2Periodic Advisor26.14 (+12.97)0.84 GPT-5.2LIVEPLAN28.41 (+15.24)1.04 - MiniMax (M2.5) Vanilla52.50.24 MiniMax-M3SAGE50.38 (-2.12)0.34 -Predefined Advice 54.17 (+1.67)0.23 MiniMax-M3Periodic Advisor54.54 (+2.04)0.32 MiniMax-M3LIVEPLAN57.95 (+5.45)0.24 I. EXPERIMENTAL EVALUATION LIVEPLAN is pluggable to any open-source ReAct- based [38] agent. We build LIVEPLAN on top of SWE-agent as it offers a stable, well-tested implementation. Benchmarks. We evaluate on two repository-level issue reso- lution benchmarks: SWE-bench Verified [12] and SWE-bench Pro 1 [7], a more challenging benchmark designed to be resistant to contamination. Baselines. We compare LIVEPLAN against (1) SWE-agent using its original, predefined plan with the executor LLM, and (2) SAGE [11], which replaces this original plan with one generated by the advisor LLM given a previous trajectory. We also evaluate two ablated variants of LIVEPLAN: (3) Peri- odic Advisor, which removes Monitor so Advisor periodically evaluates and intervenes on the trajectory, similar to SWE- PRM [10] and Wink [25] 2 , and (4) Predefined Advice, which removes Advisor and monitors the trajectory using only pre- defined message interventions, similar to [17] 3 . LLMs. We use three representative LLMs as executors: DeepSeek-V3 [15], an open-source general-purpose model; Gemini-2.5-Flash [5], a lightweight reasoning model; and MiniMax-M2.5 [23], a model optimized for coding and agen- tic workflows. Each executor is paired with a stronger but affordable advisor: GPT-5.2-Codex [26] (medium reasoning effort) pairs with the general-purpose executors DeepSeek- V3 and Gemini-2.5-Flash, while MiniMax-M3 [24] pairs with the coding-specialized MiniMax-M2.5. A stronger advisor is a prerequisite for meaningful intervention: an advisor no more capable than the executor offers no guidance beyond what the executor could already reach through its own reasoning. We apply the same executor-advisor pairing across all intervention approaches, so that differences in resolution rate reflect the intervention mechanism rather than advisor capability. Hyperparameters. Following the default configurations of SWE-agent and SWE-bench Pro [7, 33], we use a per- instance cost limit of $2 and a model temperature of 0 1 To account for the cost, we use SWE-bench Pro-Python. 2 These techniques have no runnable artifacts available, so this ablation serves as the closest reproducible analog. 3 Artifacts for [17] exist but do not implement all LIVEPLAN monitoring rules and signals; we use our ablated version instead to isolate the impact of the intervention technique. TABLE I. Effectiveness on SWE-bench Verified. Model Method Success Rate (%) Average Cost AdvisorExecutor - DeepSeek (V3) Vanilla38.200.04 GPT-5.2LIVEPLAN49.40 (+11.20)0.08 - Gemini (2.5-flash) Vanilla37.800.46 GPT-5.2LIVEPLAN48.40 (+10.60)0.58 - MiniMax (M2.5) Vanilla74.200.30 MiniMax-M3LIVEPLAN79.20 (+5.00)0.25 for all experiments. In the ablated Periodic Advisor variant, Advisor is triggered every five steps, consistent with the SWE-PRM setting for fair comparison. Similarly, we set the cooling period θ c to five steps. For long stagnation detection, we set the threshold θ p to seven consecutive steps in the same phase, computed from Vanilla SWE-agent runs prior to any intervention, yielding average maximum consecutive phase lengths of 5.64 for resolved instances and 7.61 for unresolved instances across all models. This threshold thus separates normal progress from likely stagnation without being tuned to any intervention method. For blocking behavioral drifts, LIVEPLAN allows at most five consecutive interventions (θ i = 5); beyond this, the agent prevents repeated blocking from stalling execution indefinitely. We run SAGE using their provided prompts and pipeline without modification. A. RQ1: Effectiveness in Improving Resolution Rate We run Vanilla SWE-agent and LIVEPLAN with all advisor and executor LLMs on both benchmarks. To manage experi- mental costs, we run the ablated versions and SAGE on SWE- bench Pro only across all three executor models. We prioritize SWE-bench Pro as it has emerged as the dominant benchmark for long-horizon, agentic software engineering, with SWE- bench Verified increasingly retired as a primary measure of progress. Tables I-I summarize the results. 1) Comparison Against Vanilla Runs: LIVEPLAN con- sistently achieves a higher resolution rate than Vanilla SWE-agent across both benchmarks, demonstrating the effectiveness of real-time monitoring and corrective steer- ing. Gains are greater on SWE-bench Pro, with improve- ments of +12.33 and +15.24 percentage points for DeepSeek- V3 and Gemini-2.5-Flash, respectively, versus +11.20 and +10.60 points on SWE-bench Verified. Even for MiniMax- M2.5, which is a strong model, LIVEPLAN still improves the success rates by +5.45 points on SWE-bench Pro and +5.00 points on SWE-bench Verified. These gains come with only modest cost increases for DeepSeek-V3 and Gemini-2.5-Flash, while simultaneously reducing costs for MiniMax-M2.5. Figure 6 compares the overlap of resolved instances between LIVEPLAN and Vanilla, grouped by difficulty. Black dots indicate the method(s) that resolve the corresponding instances to a bar, and gray dots mark those that do not. Colors indicate instance difficulty. For SWE-bench Verified, we use the difficulty labels provided by the benchmark based on the estimated developer effort to resolve the issue. For SWE-bench Pro, following prior work [3], we estimate difficulty using the number of files modified in the reference patch: easy (1 file), 5 SWE-bench Pro MiniMax-M2.5 Gemini-2.5-Flash DeepSeek-V3 SWE-bench Verified Easy Medium Hard Very Hard Fig. 6: Resolution overlap between LIVEPLAN and Vanilla. TABLE IV. Comparison of trigger frequency of the Advisor by different techniques. Model Method Average Intervention Trigger Rate (%) AdvisorExecutor GPT-5.2 DeepSeek (V3) Periodic Advisor3.2599.6 LIVEPLAN2.3393.9 Gemini (2.5-flash) Periodic Advisor7.3797.7 LIVEPLAN2.9993.9 MiniMax (M3) MiniMax (M2.5) Periodic Advisor7.3899.6 LIVEPLAN1.4783.3 medium (2–3 files), hard (4–10 files), and very hard (> 10 files). Across all model-benchmark pairs, LIVEPLAN consis- tently resolves a substantial number of instances beyond those already solved by the vanilla. More importantly, these gains are concentrated on harder tasks: on SWE-bench Verified, more than half of the instances uniquely resolved by LIVEPLAN belong to the hard category; on SWE-bench Pro, nearly all additional resolutions fall into the medium and hard categories. These results suggest that online monitoring and steering are particularly beneficial for more challenging and less contamination-prone software engineering tasks. 2) Comparison Against SAGE: LIVEPLAN consistently outperforms SAGE at even lower cost. SAGE’s notable cost overhead comes from running Vanilla SWE-agent once, feeding its entire trajectory to Advisor for re-planning, and re- executing the agent with the new plan. SAGE even underper- forms the Vanilla run under DeepSeek-V3 and MiniMax-M2.5. Although SAGE produces highly specific guidance, such as exact line numbers or concrete code snippets to insert, this detail is often factually incorrect (Figure 1). Executor LLMs, being generally less capable, often cannot critically evaluate or recover from such erroneous instructions and instead follow the hallucinated guidance faithfully, causing task failures. 3) Comparison Against Ablated Versions and Similar Ap- proaches: Removing Monitor or Advisor degrades the per- formance of LIVEPLAN, demonstrating the effectiveness of SWE-bench Pro SWE-bench Verified Deepseek-V3Gemini-2.5-FlashMiniMax-M2.5 Fig. 7: Overlapping resolved instances among techniques. both and the superiority of LIVEPLAN over similar alter- natives: SWE-PRM (every five steps) and Wink (every step) periodically analyze the trajectory and intervene, similar to the Periodic Advisor variant. Such techniques are guaranteed to trigger the Advisor, whereas LIVEPLAN does so when needed. Table IV shows that the Periodic Advisor triggers Advisor in almost all trajectories 4 , while LIVEPLAN triggers it as needed. The average number of interventions in Periodic Advisor is also notably higher than LIVEPLAN. As we confirm through manual analysis, the lower performance despite the higher and denser triggering rate is due to the Advisor providing misleading advice. Using DeepSeek-V3 and Gemini-2.5-Flash, the average cost of LIVEPLAN per instance is higher than that of Periodic Advisor, despite triggering Advisor less often. The additional cost reflects longer successful trajectories rather than more expensive interventions (details in §I-B). Compared to the Predefined Advice variant, which is similar to the proposed technique in [17], LIVEPLAN consistently outperforms it with a considerable margin. This is because this variant only provides predefined, high-level messages, which may be insufficient to help the agent recover from behavioral drifts. Since Predefined Advice variant detects a broader set of issues than [17], it is an upper bound on that technique’s performance, i.e., LIVEPLAN’s margin over that technique would be even larger. Figure 7 illustrates the overlap of resolved instances by all approaches. Across most model-benchmark pairs, LIVEPLAN contributes the largest set of uniquely resolved instances, e.g., 73 for Gemini-2.5-Flash and 32 for MiniMax-M2.5 on SWE- bench Verified. The distribution of resolved instances suggests that different intervention strategies exhibit complementary strengths across task subsets. Overall, LIVEPLAN offers the strongest balance between effectiveness and cost. B. RQ2: Detailed Cost Analysis of LIVEPLAN LIVEPLAN’s gains come at minimal cost and overhead. Table V reports that across all settings, the advisor costs only $0.01–$0.06 per instance on average. Rule-based monitoring incurs nearly zero runtime overhead, i.e., a few milliseconds. The Advisor latency is modest relative to the 4 The trajectories in which the Advisor is not triggered are shorter than the five-step threshold due to unsuccessful early termination. 6 TABLE V. Cost and latency analysis. Exec: Executor; Adv: Advisor; Mon: Monitor. “—” indicates not applicable. Setting Method Steps Cost ($) Latency (s) Dataset AdvisorExecutorExec. Adv. Mon. Adv. Pro - DeepSeek (V3) Vanilla160.04— GPT-5.2LIVEPLAN 300.12 0.04 0.010.9 - Gemini (2.5-flash) Vanilla750.78— GPT-5.2LIVEPLAN 470.98 0.06 0.09.6 - MiniMax (M2.5) Vanilla470.24— MiniMax-M3LIVEPLAN 470.24 0.01 0.029.1 Verified - DeepSeek (V3) Vanilla210.04— GPT-5.2LIVEPLAN 240.06 0.02 0.05.8 - Gemini (2.5-flash) Vanilla520.46— GPT-5.2LIVEPLAN 380.55 0.04 0.08.1 - MiniMax (M2.5) Vanilla700.30— MiniMax-M3LIVEPLAN 580.23 0.02 0.039.5 Vanilla LivePlan Vanilla LivePlan Vanilla LivePlan [B] Plan Violation [B] Oscillation Navigation Reproduction Patching Validation [N] Repeated Action [B] Plan Violation [B] Oscillation Navigation Reproduction Patching Validation [N] Repeated Action [N] Long Stagnation [N] Long Stagnation 86.759.535.732.70.40.0 86.717.462.013.57.60.8 21.625.827.133.151.148.9 0.81.58.65.66.15.3 4.511.721.120.70.41.1 1.911.410.920.718.221.2 15.926.153.460.542.443.9 25.414.66.24.66.21.4 50.42.054.221.05.00.8 15.822.835.231.845.642.4 7.87.613.612.824.221.6 1.60.68.67.80.20.6 7.214.613.014.651.648.0 10.212.834.833.058.846.8 DeepSeek-V3Gemini-2.5-FlashMiniMax-M2.5 Pro Verified 0 20 40 60 80 100 Fig. 8: Behavioral drifts in final trajectories. runtime of long-horizon repository-level repair tasks. Beyond cost, LIVEPLAN often reduces execution effort by decreasing inefficient exploration. For Gemini-2.5-Flash and MiniMax- M2.5, interventions generally shorten trajectories and, in some cases, even reduce the overall cost. In contrast, trajectories become longer for DeepSeek-V3 because LIVEPLAN dis- courages common shortcuts such as skipping validation [18], encouraging a more complete and reliable repair process. Regardless, the resulting cost overhead remains small. C. RQ3: Process-Centric Analysis of Trajectories In addition to outcome-centric analysis, we conduct three types of automated process-centric analyses to better under- stand the notable improvement of LIVEPLAN over Vanilla. 1) Prevalence of Triggering Signals: We measure the prevalence of behavioral drifts in completed trajectories of Vanilla and LIVEPLAN (Figure 8), and the percentage of instances in which LIVEPLAN observes each drift during execution (Table VI). Drifts observed by LIVEPLAN may not remain in the final trajectory, since LIVEPLAN steers the agent away from them. On average, 86.16% and 78.33% of Vanilla trajectories in SWE-bench Pro and SWE-bench Verified show behavioral drifts. They are more prevalent in SWE- bench Pro, whose more complex problems require longer, drift-prone trajectories. The exact behavioral drifts in Vanilla runs may not be observed in LIVEPLAN. However, Table VI shows that LIVE- PLAN observes many similar behavioral drifts during exe- cution and attempts to steer the agent away. Specifically, 0.550.540.740.630.850.81 0.550.540.740.630.850.81 1.000.990.950.970.800.81 0.670.660.800.720.830.81 0.630.610.710.650.830.80 0.630.610.710.650.830.80 0.980.970.970.970.800.80 0.730.710.780.730.820.80 0.790.780.890.830.970.94 0.750.730.890.830.950.93 0.990.980.980.960.800.82 0.830.810.920.860.900.89 0.810.840.890.880.970.97 0.770.790.890.880.970.96 0.950.960.970.960.800.80 0.840.850.910.900.910.90 RURURU DeepSeek-V3Gemini-2.5-FlashMiniMax-M2.5 PPC POC PPF PC PPC POC PPF PC PPC POC PPF PC PPC POC PPF PC Vanilla LivePlan Vanilla LivePlan SWE-bench Pro SWE-bench Verified 0.5 0.6 0.7 0.8 0.9 1.0 Score Fig. 9: Average plan compliance score for Resolved (R) and Unresolved (U) instances of Vanilla and LIVEPLAN. LIVEPLAN significantly reduces blocking signals. Many of the high-risk behavioral drifts, namely Plan Violation and Action/Thought Oscillation, that are prevalent in the final Vanilla trajectories and are notably observed during LIVE- PLAN runs do not appear in final LIVEPLAN trajectories. 55.63% and 40.27% of blocking behavioral drifts in Vanilla trajectories under SWE-bench Pro and SWE-bench Verified drops to 34.37% and 13.67% in LIVEPLAN, respectively. As our manual analysis confirms (§I-D), the remaining blocking drifts are due to the executor either being unable to follow the advice or choosing not to, repeating the drift after exceeding the consecutive-intervention threshold θ i . Non- blocking signals may remain in completed LIVEPLAN trajec- tories. This is because they can signal either an inefficiency or a legitimate, prolonged attempt necessary to solve hard prob- lems. These results show that LIVEPLAN improves execution quality by minimizing high-risk behavioral drifts while preserving the flexibility needed for long-horizon tasks. 2) Plan Compliance Analysis: LIVEPLAN’s final trajec- tories, specifically resolved SWE-bench Pro instances by DeepSeek-V3 and Gemini-2.5-Flash, exhibited Plan Violation, albeit to a lesser extent than Vanilla, motivating deeper plan- compliance analysis. Following prior work [18], we measure trajectory adherence to SWE-agent’s repair plan, i.e., Navigate, Reproduce, Patch, and Validate, via four metrics: Plan Phase Compliance (PPC) penalizes skipped phases; Plan Order Compliance (POC) penalizes out-of-order execution; and Plan Phase Fidelity (PPF) penalizes behaviors outside the plan. The overall Plan Compliance (PC) is the geometric mean of PPC, POC, and PPF. PC=1 only when a trajectory executes all and only the specified phases in the correct order. Figure 9 reports average compliance scores for resolved and unresolved instances of Vanilla and LIVEPLAN. Consistent with prior work [18], resolved trajectories achieve higher PC than unresolved ones, indicating that successful repairs mostly followed the intended workflow. LIVEPLAN improves PC mainly through higher PPC and POC, suggesting online intervention helps agents complete the required phases in the intended order. PPF is occasionally lower under LIVE- 7 TABLE VI. Observed behavioral drifts by LIVEPLAN. Setting Plan Violation Long Stagnation Oscillation Repeated Action DatasetAdvisorExecutorNavigationReproductionPatchingValidation SWE-bench Pro GPT-5.2 DeepSeek-V380.726.11.511.411.095.528.8 Gemini-2.5-Flash43.931.85.320.520.873.559.5 MiniMax-M3MiniMax-M2.51.948.75.31.121.310.344.1 SWE-bench Verified GPT-5.2 DeepSeek-V328.222.87.40.614.637.012.8 Gemini-2.5-Flash7.734.913.88.415.778.337.8 MiniMax-M3MiniMax-M2.56.242.421.20.647.86.847.0 0.0 0.5 1.0 LCS Similarity SWE-bench Pro DeepSeek-V3Gemini-2.5-FlashMiniMax-M2.5 0.0 0.5 1.0 LCS Similarity SWE-bench Verified Prefix PhaseFull Phase Fig. 10: Trajectory similarity between Vanilla and LIVEPLAN before the first intervention and over complete trajectories. PLAN, which we attribute to Advisor suggesting beneficial out- of-plan actions, e.g., additional regression testing to validate the patch. Since these trajectories succeed, the out-of-plan behavior appears productive rather than aimless. 3) Strategy Divergence Analysis: The plan-compliance analysis showed that Advisor interventions affect how closely the executor follows its prescribed plan. Independent of that plan, we next perform an apples-to-apples comparison between Vanilla and LIVEPLAN trajectories to understand whether they are similar before the intervention and how they change after it. We compute normalized longest common subsequence (LCS) similarity between LANGUTORY of Vanilla (X) and LIVEPLAN (Y ) for each instance as 2×|LCS(X,Y )| |X|+|Y| . Larger values indicate more similar execution behavior. We evaluate similarity on (1) the LANGUTORY prefixes before the first intervention and (2) the LANGUTORY of complete trajecto- ries. The former captures divergence from execution non- determinism, since both runs evolve independently before any intervention. The latter captures the combined effect of non- determinism and online intervention. Figure 10 reports the results. Prefix similarity is high, often 0.8–0.9, indicating agents initially follow similar strate- gies despite execution non-determinism. After intervention, similarity consistently decreases, showing that LIVEPLAN redirects the subsequent problem-solving strategy. A paired one-sided Wilcoxon signed-rank test [36] comparing prefix and full-trajectory similarity per instance confirms the sig- nificance: in every model-benchmark pair, prefix similarity is significantly higher (median paired difference > 0, p < 1e −10 ). Since prefix and full-trajectory similarities are paired by instance, and difficulty is held fixed across the two measure- TABLE VII. Resolution transition matrix relative to Vanilla. R: Resolved, U: Unresolved, DS: DeepSeek-V3, GF: Gemini- 2.5-Flash, and M: MiniMax-M2.5. DatasetMethod R→U ↓U→R ↑ DS GF M DS GF M SWE-bench Pro SAGE26 122219 2816 Predefined Advice 16 131525 2815 Periodic Advisor16 111735 4222 LIVEPLAN22733 3821 SWE-bench Verified LIVEPLAN127756 6828 ments, the post-intervention drop reflects the intervention itself rather than task variation. While we discuss the impact of non- determinism in detail later (§I-E), these results indicate that LIVEPLAN’s performance gains stem from targeted behavioral corrections rather than run-to-run variation. D. RQ4: Analysis of Improvements and Regressions Table VII reports resolution transitions relative to Vanilla, computed over instances where each method is triggered. LIVEPLAN consistently achieves the fewest R→U regressions while producing many U→R improvements 5 , particularly on SWE-bench Pro. These results show that LIVEPLAN not only resolves notable previously unsuccessful instances but also preserves the Vanilla successes. We manually analyzed improvement (U→R) and regression R→U cases to under- stand root causes. Two authors independently inspected the trajectory, intervention(s), resulting patch, and golden patch to inductively identify root causes, with categories emerging from and refined during analysis rather than defined in advance. The authors then discussed findings to reach consensus on final root-cause labels 6 . 1) Improvements (U→R): Our analysis identifies the fol- lowing reasons for LIVEPLAN’s success in converting pre- viously unsuccessful Vanilla runs: (1) Correcting behav- ioral drift. Weaker executors, particularly DeepSeek-V3 and Gemini-2.5-Flash, frequently exhibit blocking behavioral drifts (Table VI); LIVEPLAN detects them promptly and redirects ex- ecution before they persist, eliminating Oscillations in 104/128 (81.3%) of affected trajectories and Plan Violation cases in 37/72 (51.4%). (2) Refocusing on the repair task. As trajec- tories lengthen, executors may drift from the main repair task due to accumulated context or intermediate failures (context poisoning [3]); by conditioning on the issue description and re- cent trajectory steps, Advisor redirects the executor away from 5 Although Periodic Advisor achieves slightly more U→R transitions in some settings, it also introduces substantially more R→U regressions, leading to a lower overall success rate than LIVEPLAN. 6 Identified root causes and corresponding instances are available in artifacts [21]. 8 Navigation Patch Validation General python Exit ... (Repeat Failed Edit-Test Cycles) (a) Vanilla (Unresolved) ansible-0fd88717, SWE-bench Pro, Deepseek-V3 Reproduction Pause ad-hoc tests. Manually inspect password.py::run() ... Adjust the control flow... ... ⚠Prolonged Validation password.py::run() view str_replace Submit 23 32 password.py::run() √ <str3, str4> 22 Advisor redirect ... str_replace cat test_password _lookup.py Steps 1 - 14 Initial incomplete patch Thought: Let’s execute the test to verify the fix 18 python /app/test_password_lookup.py Observation: ValueError 18 echo test_script.py python cat Prolonged Validation ... ... str_replace password.py × <str1, str2> 15 16 16 17 ... ... 55 22 21 create test_parse_format.py 24 (b) LivePlan (Resolved) test_script.py √ test_password _lookup.py test_password _lookup.py × Thought: Let’s create a script to test the look up plugin 15 cat << ‘EOF’ > /app/test_password_lookup.py TEST_SCRIPT_1 EOF Thought: There was an issue. Let me rephrase and execute the test script properly to verify the fix 16 echo “TEST_CONTENT” > test_script.py && python test_script.py Thought: Here's an updated test script that includes the required setup: 17 cat << ‘EOF’ > /app/test_password_lookup.py TEST_SCRIPT_2 EOF Repeat the same test execution 19 - 20 python /app/test_password_lookup.py Observation: ValueError Thought: fix parsing logic to separate salt and ident 23 str_replace password.py::parser_content() -salt=password[salt_sep+len(salt_slug):] +remain=password[salt_sep+len(salt_slug):] Exit (auto submitted) 55 Steps 1 - 22 -Initial incomplete patch -Spends 7 consecutive steps editing and rerunning test_passwor_lookup.py (Similar to Vanilla run) Action cycle: edit test_password_lookup.py python test_password_lookup.py Thought: Let’s carefully review the ‘run’ method logic 23 view /app/password.txt Observation: salt=... ident=2b Thought: Follow Guidance: edit run() control flow 24 str_replace password.py::run() + if file_ident is not None and ident is None: + ident = file_ident + elif ident is not None and (..): + changed = True Thought: The changes looks correct. Now let's create a simpler test script that doesn't require mocking the entire Ansible environment. 25 create test_parse_format.py Steps 27 - 33 Focused validation + clean-up + submit test_password _lookup.py python test_password _lookup.py × 19 Thought: Let’s run the test: 26 python test_parse_format.py python test_parse_format.py √ 25 Fig. 11: Case study of a U→R transition. Intervention redirects the executor from ineffective validation to successful repair. unproductive exploration and back toward the core problem. (3) Progressive online correction. 182/244 (74.6%) of U→R cases receive multiple interventions, averaging 3.70 across all 244 cases, letting LIVEPLAN continually correct newly emerging drifts rather than relying on a one-time redirection. LIVEPLAN consistently intervenes before recovery becomes difficult. The first intervention occurs within the first third of the trajectory in all 244 U→R cases, leaving sufficient oppor- tunity to steer the remaining execution. Figure 11 illustrates such a case (ansible-0fd88717 from SWE-bench Pro under DeepSeek-V3). Both Vanilla and LIVEPLAN runs initially generate an incomplete patch and then enter validation. As shown in Figure 11a, the Vanilla run gets stuck in ad-hoc test script creation and execution after several test setup failures, drifting away from the main repair task. The trajectory therefore exhibits Prolonged Validation (steps 15–21) and Action Oscillation (self-loop edges at step 18 and 19) before terminating unsuccessfully at step 55. In contrast, Figure 11b shows that LIVEPLAN detects the Prolonged Validation at step 22 7 and invokes Advisor, which recommends pausing the ad- hoc tests and instead inspecting and modifying the core logic directly. The executor follows this guidance, edits the correct control flow, performs focused validation, and successfully submits the final patch at step 33. 2) Regressions (R→U): Regressions across all techniques arise from several sources: (1) Misleading or low-quality guidance. Each method fails differently. SAGE may generate an incorrect global plan that redirects the executor toward a non-existent problem (Figure 1) Predefined Advice is often insufficient for task-specific repair. Periodic Advisor may misdiagnose the trajectory state and generate unhelpful guid- ance (Figure 2). (2) Execution non-determinism. The rerun may diverge from Vanilla execution before any intervention, causing failures absent in the original trajectory. (3) Brittle Vanilla successes. Some Vanilla runs succeed by chance despite limited or missing navigation and validation—likely 7 Note that trajectories are not identical and similar behavioral drifts may occur at different steps. due to memorization/data contamination [1] rather than proper reasoning, making corrective intervention confusing. (4) In- complete compliance. The executor may fail to faithfully follow or fully implement a correct advice. (5) Secondary- task distraction. Advice may intentionally redirect the execu- tor toward secondary issues (e.g., environment configuration), distracting it from the primary repair task. In nearly all LIVEPLAN R→Us, the root cause is not incorrect advice or a failure to intervene on time, but rather the executor’s inability to follow the correct guidance. We confirmed the advice’s accuracy by referring to the golden patch. Figure 12 shows such a case (ansible-8127abbc from SWE-bench Pro under MiniMax-M2.5). Monitor detects a prolonged navigation without generating a patch and triggers Advisor (step 14). The Advisor correctly identifies all five key repair elements (GP1–GP5), consistent with the gold patch, and directs the executor to begin editing. The executor follows part of the advice, e.g., modifying GP5 at step 14. Later, it (incorrectly) reasons that the GP3 edit was incorrect, reverting it at step 56, and ultimately submits a partial patch. In contrast, the Vanilla run remains in navigation until step 24, but collects all the context to generate a successful patch. Guidelines for Future Research. Inference-time intervention should be complemented by intervention-aware post-training. While LIVEPLAN identifies when and how to steer execution, successful recovery depends on the executor faithfully carrying out the advice. Future work could train on intervention- conditioned trajectories using process-level rewards that en- courage executors to satisfy explicit guidance constraints across following steps, since optimizing only for final task success may encourage brittle, under-validated solutions. Post- training complements this work: LIVEPLAN determines when and what advice to give, while the trained executor learns to reliably translate that advice into a correct solution. Given the execution non-determinism observed here, the evaluation should use repeated paired rollouts to estimate how an in- tervention changes the probability of resolution, rather than 9 Navigation Patch Validation General LivePlan (Unresolved) ansible-8127abbc, SWE-bench Pro, MiniMax-M2.5 Reproduction str_replace connection/ __init__.py Partial compliance. Let me implement the changes.. (GP5 ✔) ... str_replace worker.py Drops plugin-loader setup keeps only CLI-context initialization. (GP3 ✗) Exit Golden Patch Diff (relevant excerpt) worker.py +display.set_queue(self._final_q) # GP1: queue+detach +self._detach() # GP1: queue+detach +context.CLIARGS = self._cliargs # GP2:restore CLI context +paths = context.CLIARGS.get('collections_path') or [] +paths = paths if is_sequence(paths) else [paths] +init_plugin_loader(paths) # GP3: initialize plugin loader connection/__init__.py +class ConnectionKwargs(t.TypedDict): ... # GP5: apply ConnectionKwargs -new_stdin: io.TextIOWrapper | None = None # GP4: remove new_stdin +**kwargs: t.Unpack[ConnectionKwargs] # GP5: apply ConnectionKwargs Prolonged navigation, no patch Advisor correctly identifies GP1-GP5 ... view connection/ __init__.py Incorrect reasoning The new_stdin is optional and there is no need to change it (GP4 ✗) ... Implements only part of the worker setup (GP1 ✔,GP2 ✔, GP3 ✔, GP4 ✗) Unresolved (GP1 ✔, GP2 ✔, GP3 ✗, GP4 ✗, GP5 ✔) 56 16 14 ... (GP1 ✗, GP2 ✗, GP3 ✗, GP4 ✗, GP5 ✗) Fig. 12: Case study of a R→U transition under LIVEPLAN. TABLE VIII. Effectiveness in resolving instances of SWE- bench Pro that deterministically pass or fail in Vanilla runs. Model #InstancesMethod Success Rate (%) AdvisorExecutor - DeepSeek (V3) 222 Vanilla19.82 GPT-5.2SAGE16.67 (-3.15) -Predefined Advice 20.27 (+0.45) GPT-5.2Periodic Advisor23.87 (+4.05) GPT-5.2LIVEPLAN27.03 (+7.21) - Gemini (2.5-flash) 213 Vanilla8.92 GPT-5.2SAGE11.74 (+2.82) -Predefined Advice 12.68 (+3.76) GPT-5.2Periodic Advisor18.31 (+9.39) GPT-5.2LIVEPLAN18.78 (+9.86) - MiniMax (M2.5) 234 Vanilla52.99 MiniMax-M3SAGE50.00 (-2.99) -Predefined Advice 52.99 (+0.00) MiniMax-M3Periodic Advisor54.27 (+1.28) MiniMax-M3LIVEPLAN56.41 (+3.42) drawing conclusions from a single trajectory. E. RQ5: Impact of Non-determinism We re-evaluate all methods on the subset of SWE-bench Pro with deterministic outcomes across two repeated Vanilla runs. This reduces the number of instances but retains the majority of instances: 84%, 80%, and 88% of the study in §I-A for each model. The distribution of instances with consistent outcomes is similar to that of the complete set (20.3% easy, 41.0% medium, 36.8% hard, 1.9% very hard): DeepSeek-V3 (19.4% easy, 40.1% medium, 38.3% hard, 2.3% very hard); Gemini-2.5-Flash (21.1% easy, 39.4% medium, 37.6% hard, 1.9% very hard); and MiniMax-M2.5 (21.8% easy, 40.2% medium, 36.3% hard, 1.7% very hard). Because this subset is defined solely by Vanilla’s consistency, it cannot selectively favor or penalize any. As shown in Table VIII, LIVEPLAN preserves the method ranking from Table I, achieving the highest success rate across all evaluated models, followed by Periodic Advisor, Predefined Advice, and Vanilla. It does so with an advantage comparable in magnitude to that of the full study, indicating that the gain is not an artifact of baseline volatility. We further repeat the SWE-bench Verified Vanilla runs for DeepSeek-V3 (limiting this validation to one model to contain experimental cost), where LIVEPLAN again substantially outperforms Vanilla, improving the success rate from 36.58% to 46.56%, a margin consistent with its full-set gain. These results confirm that the gains of LIVEPLAN are robust to execution non- determinism rather than favorable run-to-run variation. IV. RELATED WORK Trajectory analysis and failure diagnosis. Liu et al. [17] propose process-centric trajectory representations via struc- tured graphs and language abstractions. Chen et al. [4] and Liu et al. [19] conduct process-oriented error analysis on GitHub issue-resolution trajectories. TRAIL [8] and MAST [27] de- velop failure taxonomies, while FALAT [28] and AgentRx [2] offer step-wise failure localization frameworks. These provide post-hoc analysis but do not intervene during execution; LIVE- PLAN instead monitors execution continuously and intervenes upon detecting drift, correcting behavior before it propagates. Online steering, planning, and self-correction. Self- Refine [22] and LATS [40] use iterative self-reflection but target stateless reasoning rather than long-horizon agent ex- ecution. Planning-based approaches, Plan-and-Act [9], EA- GLET [32], and ReCAP [39], decouple planning from exe- cution or learn stronger planners. For software engineering agents, SAGE [11] regenerates a plan from a completed trajec- tory and reruns the executor; SWE-PRM [10] and Wink [25] periodically invoke an LLM evaluator over recent windows; TrajEval [13] compares fine-grained trajectories against refer- ence patches. LIVEPLAN instead avoids periodic LLM judg- ment and unnecessary interventions while providing targeted next-step advice, achieving the highest resolution rate across all settings, with substantially fewer Advisor interventions. V. THREATS TO THE VALIDITY External Validity. To ensure the generalizability of the results, we evaluate LIVEPLAN across three executors and two advisor LLMs on two widely used benchmarks of real- world GitHub issues. We compare LIVEPLAN with four baselines and ablated versions that resemble related work. To account for execution non-determinism, we further reevaluate LIVEPLAN on a subset of instances with consistent outcomes across two repeated Vanilla executions. Similar improvements are observed over the Vanilla baseline, suggesting that our approach is robust to run-to-run variation. Internal Validity. We investigate the impact of the proposed technique not just on the outcome but also on the process. Construct Validity. Our pipeline is built on peer-reviewed artifacts and validated with well-vetted tools. We analyze the results quantitatively and qualitatively, aligning the two analyses and avoid incorrect metric implementation or mea- surement. 10 VI. CONCLUDING REMARKS We propose LIVEPLAN for lightweight monitoring and corrective steering of programming agents. LIVEPLAN relies on two abstract representations of raw trajectories, namely GRAPHECTORY and LANGUTORY, to monitor and determine behavioral drifts. It benefits from predefined advice relevant to the drift or custom LLM-generated advice when needed. In the next step, we pursue intervention-aware post-training. VII. DATA AVAILABILITY The artifacts of this paper are publicly available at [21]. ACKNOWLEDGMENTS This work is supported by NSF CCF-2238045 and IBM- Illinois Discovery Accelerator Institute (IIDAI) grants. REFERENCES [1] Anthropic, “Introducing claude opus 4.7,” https://w. anthropic.com/news/claude-opus-4-7, 2026. [2] S. Barke, A. Goyal, A. Khare, A. Singh, S. Nath, and C. Bansal, “Agentrx: Diagnosing ai agent failures from execution trajectories,” arXiv preprint arXiv:2602.02475, 2026. [3] Y. Chen, A. Ahmad, Y. Zhou, and R. Jabbarvand, “Un- locking model potentials through adaptive multi-agent scaffolding for efficient issue resolution,” arXiv preprint arXiv:2606.25514, 2026. [4] Z. Chen, W. Ma, and L. Jiang, “Beyond final code: A process-oriented error analysis of software development agents in real-world github scenarios,” arXiv preprint arXiv:2503.12374, 2025. [5] G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen et al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,” arXiv preprint arXiv:2507.06261, 2025. [6] H. Deng, H. Zhang, J. Ou, and C. Feng, “Can llm be a good path planner based on prompt engineering? mitigat- ing the hallucination for path planning,” in International Conference on Intelligent Computing.Springer, 2025, p. 3–15. [7] X. Deng, J. Da, E. Pan, Y. Y. He, C. Ide, K. Garg, N. Lauffer, A. Park, N. Pasari, C. Rane, K. Sampath, M. Krishnan, S. Kundurthy, S. Hendryx, Z. Wang, C. B. C. Zhang, N. Jacobson, B. Liu, and B. Kenstler, “SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks?” Sep. 2025. [Online]. Available: https://arxiv.org/abs/2509.16941 [8] D. Deshpande, V. Gangal, H. Mehta, J. Krishnan, A. Kan- nappan, and R. Qian, “Trail: Trace reasoning and agen- tic issue localization,” arXiv preprint arXiv:2505.08638, 2025. [9] L. E. Erdogan, N. Lee, S. Kim, S. Moon, H. Furuta, G. Anumanchipalli, K. Keutzer, and A. Gholami, “Plan-and-act: Improving planning of agents for long- horizon tasks,” in International Conference on Machine Learning (ICML), Jul. 2025. [Online]. Available: https://openreview.net/forum?id=ybA4EcMmUZ [10] S. Gandhi, J. Tsay, J. Ganhotra, K. Kate, and Y. Rizk, “When agents go astray: Course-correcting SWE agents with PRMs,” in Workshop on Scaling Environments for Agents (SEA@NeurIPS), Dec. 2025. [Online]. Available: https://openreview.net/forum?id=wyrcoDNaGO [11] H. Hayashi, B. Pang, W. Zhao, Y. Liu, A. Gokul, S. Bansal, C. Xiong, S. Yavuz, and Y. Zhou, “Self- abstraction from grounded experience for plan-guided policy refinement,” arXiv preprint arXiv:2511.05931, 2025. [12] C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan, “SWE-bench: Can language models resolve real-world github issues?” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https:// openreview.net/forum?id=VTF8yNQM66 [13] M. Kim, D. Wang, S. Cui, F. Farmahinifarahani, S. Garg, B. Ray, T. Y. Zhuo, R. Mukherjee, and V. Kumar, “Traje- val: Decomposing code agent trajectories for fine-grained diagnosis,” arXiv preprint arXiv:2603.24631, 2026. [14] X. Lin, Y. Ning, J. Zhang, Y. Dong, Y. Liu, Y. Wu, X. Qi, N. Sun, Y. Shang, K. Wang et al., “Llm-based agents suf- fer from hallucinations: A survey of taxonomy, methods, and directions,” arXiv preprint arXiv:2509.18970, 2025. [15] A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek- v3 technical report,” arXiv preprint arXiv:2412.19437, 2024. [16] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Transactions of the Association for Computational Linguistics, vol. 12, p. 157–173, 02 2024. [Online]. Available: https://doi.org/10.1162/tacl a00638 [17] S. Liu, Y. Chen, R. Krishna, S. Sinha, J. Ganhotra, and R. Jabbarvand, “Process-centric analysis of agentic software systems,” Proc. ACM Program. Lang., vol. 10, no. OOPSLA1, Apr. 2026. [Online]. Available: https: //doi.org/10.1145/3798271 [18] S. Liu, S. Dehghan, J. Ganhotra, M. Hirzel, and R. Jab- barvand, “Evaluating plan compliance in autonomous programming agents,” arXiv preprint arXiv:2604.12147, 2026. [19] S. Liu, F. Liu, L. Li, X. Tan, Y. Zhu, X. Lian, and L. Zhang, “An empirical study on failures in automated issue solving,” arXiv preprint arXiv:2509.13941, 2025. [20] X. Liu, X. Yang, Z. Li, P. Li, and R. He, “Agen- thallu: Benchmarking automated hallucination attribution of llm-based agents,” arXiv preprint arXiv:2601.06818, 2026. [21] Liu, Shuyang and Dehghan, Saman and Kim, Jiyoung and Ganhotra, Jatin and Hirzel, Martin and Jabbar- 11 vand, Reyhaneh, “Artifact repository,” https://github.com/ Intelligent-CAT-Lab/Agent-Planner.git, 2026. [22] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S.Welleck,A.Yazdanbakhsh,andP.Clark, “Self-refine: Iterative refinement with self-feedback,” in Conference on Neural Information Processing Systems (NeurIPS), Dec. 2023. [Online]. Available: https: //proceedings.neurips.c/paper files/paper/2023/hash/ 91edff07232fb1b55a505a9e9f6c0f3-Abstract-Conference. html [23] MiniMax, “Minimax m2.5,” https://w.minimax.io/ models/text, 2025. [24] —, “Minimax m3,” https://w.minimax.io/blog/ minimax-m3, 2026. [25] R. Nanda, C. Maddila, S. Jha, E. M. Khan, M. Paltenghi, and S. Chandra, “Wink: Recovering from misbehaviors in coding agents,” arXiv preprint arXiv:2602.17037, 2026. [26] OpenAI, “Introducing gpt-5.2-codex,” https://openai. com/index/introducing-gpt-5-2-codex/, 2025. [27] M. Z. Pan, M. Cemri, L. A. Agrawal, S. Yang, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, K. Ramchandran, D. Klein, J. E. Gonzalez, M. Zaharia, and I. Stoica, “Why do multiagent systems fail?” in ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025. [Online]. Available: https://openreview.net/forum?id=wM521FqPvI [28] M. N. Rafi, M. Ahasanuzzaman, D. J. Kim, Z. Wang, and T.-H. Chen, “Falat: Tracing failures in llm agent tra- jectories via dependency-guided search,” arXiv preprint arXiv:2606.00765, 2026. [29] D. Rao and C. Callison-Burch, “Autorubric: Uni- fying rubric-based llm evaluation,” arXiv preprint arXiv:2603.00077, 2026. [30] M. Sharma, M. Tong, T. Korbak, D. Duvenaud, A. Askell, S. Bowman, E. Durmus, Z. Hatfield-Dodds, S. Johnston, S. Kravec et al., “Towards understanding sycophancy in language models,” in International Con- ference on Learning Representations, vol. 2024, 2024, p. 110–144. [31] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” Advances in neural information processing systems, vol. 36, p. 8634–8652, 2023. [32] S. Si, H. Zhao, K. Luo, G. Chen, F. Qi, M. Zhang, B. Chang, and M. Sun, “A goal without a plan is just a wish: Efficient and effective global planner training for long-horizon agent tasks,” Oct. 2025. [Online]. Available: https://arxiv.org/abs/2510.05608 [33] SWE-agent,“Swe-agentdocumentation,” https://swe-agent.com/latest/usage/batch mode/, 2025. [34] Teresa Torres, “Context rot: Why ai gets worse the longer you chat,” https://w.producttalk.org/ context-rot/, 2026. [35] X. J. Wang, H. Bai, Y. Sun, H. Wang, S. Zhang, W. Hu, M. Schroder, B. Mutlu, D. Song, and R. D. Nowak, “The long-horizon task mirage? diagnosing where and why agentic systems break,” arXiv preprint arXiv:2604.11978, 2026. [36] F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics Bulletin, vol. 1, no. 6, p. 80–83, 1945. [Online]. Available: http://w.jstor.org/stable/ 3001968 [37] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “Swe-agent: agent- computer interfaces enable automated software engineer- ing,” in Proceedings of the 38th International Conference on Neural Information Processing Systems, ser. NIPS ’24. Red Hook, NY, USA: Curran Associates Inc., 2024. [38] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), May 2023. [Online]. Available: https://openreview.net/forum? id=WE vluYUL-X [39] Z. Zhang, T. Chen, W. Xu, A. Pentland, and J. Pei, “ReCAP: Recursive context-aware reasoning and planning for large language model agents,” in Conference on Neural Information Processing Systems (NeurIPS), Dec. 2025. [Online]. Available: https://openreview.net/forum?id=r2ykUnzuGt [40] A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, and Y.-X. Wang, “Language agent tree search unifies reasoning, acting, and planning in language models,” in International Conference on Machine Learning (ICML), Jul. 2024, p. 62 138–62 160. [Online]. Available: https://proceedings.mlr.press/v235/zhou24r.html 12