Paper deep dive

When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict

Lu Yang, Shusheng Xu, Zhuoran Li, Tongkai Yang, Longbo Huang

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 147

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 8/17/2026, 4:35:56 AM

Summary

The paper introduces TANGLE, a benchmark for evaluating LLM agents' handling of irreducible personal memory conflicts. It identifies three conflict types: Context-Partitioned (CPC), Behavior-Oscillation (BOC), and Source-Contradiction (SCC). The benchmark uses oracle and pipeline tracks to separate memory extraction errors from reasoning failures. Results show that while models can recognize conflicts with curated memory, they struggle with action calibration and clarification. The paper proposes Conflict-Aware Action Policy (CAAP) as a superior approach to fixed resolution rules.

Entities (9)

TANGLE · benchmark · 98%BOC · conflict-type · 95%CPC · conflict-type · 95%SCC · conflict-type · 95%CAAP · method · 92%Oracle Track · evaluation-method · 90%Pipeline Track · evaluation-method · 90%Ant Group · organization · 85%Tsinghua University · organization · 85%

Relation Signals (8)

TANGLE → hasconflicttype → BOC

confidence 95% · ...Behavior-Oscillation Conflict (BOC), and Source-Contradiction Conflict (SCC).

TANGLE → hasconflicttype → CPC

confidence 95% · It comprises 541 instances across 40 personas and three types: Context-Partitioned Conflict (CPC)...

TANGLE → hasconflicttype → SCC

confidence 95% · ...Source-Contradiction Conflict (SCC).

TANGLE → usesevaluationtrack → Pipeline Track

confidence 90% · ...and a pipeline track that extracts memory from multi-session dialogues...

TANGLE → usesevaluationtrack → Oracle Track

confidence 90% · We evaluate two tracks---an oracle track with curated memory...

Lu Yang → affiliatedwith → Tsinghua University

confidence 85% · Lu Yang 1 ... 1 Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University

Shusheng Xu → affiliatedwith → Ant Group

confidence 85% · Shusheng Xu 2 ... 2 Ant Group

CAAP → proposedby → TANGLE

confidence 85% · These findings motivate Conflict-Aware Action Policy (CAAP)...

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:LLM agents increasingly maintain personal memory across sessions, but it can conflict. Preferences depend on context, behavior evolves, and sources can conflict. When a query lacks context, time, or source authority to interpret conflict, treating one memory as definitive converts unresolved conflict into an unjustified, overconfident action. Existing benchmarks recover one answer from conflicting evidence, overlooking whether agents recognize underdetermination, preserve alternatives, seek missing information, and choose appropriate actions. We introduce \underline{T}esting \underline{A}gents' \underline{N}avigation of \underline{G}enuine, \underline{L}atent, and \underline{E}ntangled Memory Conflicts (\textsc{TANGLE}), a benchmark for genuinely unresolvable memory conflicts. It comprises 541 instances across 40 personas and three types: Context-Partitioned Conflict (CPC), Behavior-Oscillation Conflict (BOC), and Source-Contradiction Conflict (SCC). We evaluate two tracks---an oracle track with curated memory and a pipeline track that extracts memory from multi-session dialogues---on five dimensions: conflict perception, causal reasoning, confidence calibration, clarification seeking, and memory faithfulness. Experiments reveal pipeline challenges. With curated memory, models recognize conflicts more reliably than they calibrate actions or seek targeted clarification. With end-to-end pipeline memory, extraction fails to preserve conflict-bearing relations needed for downstream reasoning. Policy comparisons show fixed rules are insufficient when actions must reflect conflict. These findings motivate Conflict-Aware Action Policy (CAAP), which adapts actions to each conflict using available evidence. \textsc{TANGLE} frames conflict handling as recognizing underdetermination, retaining conflicting evidence, and acting without forcing a definitive answer.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

143,945 characters extracted from source content.

Expand or collapse full text

arXiv preprint, August 17, 2026 When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict Lu Yang 1 , Shusheng Xu 2 , Zhuoran Li 1 , Tongkai Yang 2 , and Longbo Huang 1 1 Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University 2 Ant Group Abstract. Large Language Model (LLM)-based agents increasingly maintain persistent personal memory across sessions, but such memory is not always consistent. Preferences depend on context, behaviors evolve over time, and information from different sources can be contradictory. When a query omits the context, time, or source authority needed to interpret conflicting memories, treating one memory as definitive turns an unresolved conflict into an unjustified answer and leads to overconfident actions. Existing benchmarks are built around a single recoverable answer: they reward selecting or reconstructing one value from conflicting evidence, while overlooking whether an agent recognizes underdetermination, preserves alternatives, seeks missing information, and chooses an appropriate action. We introduceTestingAgents’Navigation ofGenuine,Latent, andEntangled Memory Conflicts (TANGLE), a benchmark for evaluating cognitive behavior when agents face genuinely unresolvable conflicts in personal memory. TANGLE comprises 541 instances that span 40 diverse personas and three conflict types: Context-Partitioned Conflict (CPC), Behavior-Oscillation Conflict (BOC), and Source-Contradiction Conflict (SCC). We evaluate two tracks—an oracle track that provides curated memory directly and a pipeline track in which memory systems extract memory from multi-session dialogues—to assess five cognitive dimensions required for conflict handling: conflict perception, causal reasoning, confidence calibration, clarification seeking, and memory faithfulness. Our experiments reveal challenges across the agent pipeline. With complete curated memory, models recognize conflicts more reliably than they calibrate actions or seek targeted clarification. With end-to-end pipeline memory, an upstream bottleneck emerges: memory extraction fails to preserve the conflict-bearing relations required for downstream reasoning. At the downstream action stage, policy comparisons show that fixed resolution rules are insufficient when the action must reflect the specific conflict. These findings motivate our Conflict-Aware Action Policy (CAAP), which adapts its action to each conflict according to the available evidence. Together, TANGLE and these findings frame conflict handling as recognizing underdetermination, retaining conflicting evidence, and selecting an action without forcing a definitive answer. 1. Introduction Large language model (LLM) agents are evolving from stateless assistants into persistent systems that retain personal memory across dialogue sessions (Packer et al., 2023; Chhikara et al., 2025; Li et al., 2025). As memory grows, agents accumulate records about different aspects of a user’s life across contexts, over time, and across sources. For a given aspect, conflict arises without any one record being false: a preference is appropriate in one context but not another (Tversky & Simonson, 1993); a behavior reflects a prior or transient state rather than a stable trait as user information evolves over long-term interaction (Ong et al., 2025; Bae et al., 2022; Jiang et al., 2025a; Chao et al., 2026); and a source is informative within its own scope without being authoritative for every decision (Li et al., 2014; Tian et al., 2020). The challenge is that conflict memory preserves information whose applicability depends on context, time, or source, while the current query omits the variable needed to determine which piece of information applies. The memory bank therefore does not support a unique answer. If an agent nevertheless treats one piece of information as the definitive answer, it removes the conditions under which each piece of information applies, 1 arXiv:2608.13921v1 [cs.AI] 14 Aug 2026 Evaluating LLM Agents under Irreducible Conflict misclassifies the user’s current state, gives advice that is inappropriate for the present situation, or executes a choice that has not been confirmed. This makes conflict handling a central reliability and safety problem for memory agents (Pulipaka et al., 2026; Xie et al., 2026): consequential settings require cautious verification, while lower-stakes settings call for conditional or reversible assistance. Existing conflict benchmarks primarily focus on recovering a unique answer when the available evidence supports one, leaving behavior under genuinely unresolvable memory conflict outside their evaluation target. Their task definitions formulate conflict as a problem with a single recoverable answer, treating disagreement as something to settle rather than structure to preserve. This answer-centered design encourages agents to force conflicting memories into one definitive answer, producing overconfident or inappropriate actions when the relevant context, temporal state, or source authority remains unresolved. In addition, prior benchmarks do not disentangle memory extraction from conflict resolution. A single, undifferentiated track conflates failures in memory construction with failures in conflict reasoning, leaving unclear whether an agent lacks conflict-bearing evidence or fails to act appropriately on the evidence it receives. The comparison in Table 1 makes the resulting gap concrete: prior benchmarks do not jointly support multiple valid responses, qualitative response grading, multiple evaluation tracks, and genuine ambiguity. Table 1|Comparison with prior conflict benchmarks. Ctx: average/maximum context length; #Q: queries; MV/QL/M- S/MT/Amb.: multiple valid responses, graded quality, multi-session data, multi-track inputs, and genuine ambiguity. Benchmark Ctx (avg./max.) #Q Conflict focus Response format Evaluation coverage MV QL MS MT Amb. MemConflict (Tao et al., 2026)3.9K/204K 1,492 Temporality / fact / contextShort answer✗✓✗ STALE (Chao et al., 2026)152K/165K 1,200 Implicit temporalityOpen-form✗✓✗ HaluMem (Chen et al., 2026a)2.3K/1M 3,467 Erroneous premiseShort answer✗✓✗ ConflictBank (Su et al., 2024)N/A † 553K FactMultiple choice✗ CONFLICTINGQA (Wan et al., 2024)512/1K238 OpinionBinary choice✗ CONFLICTS (Cattan et al., 2025)512/5K458 Temporality / fact / opinion Open-form✗ SelectiveQA (Yang et al., 2026)N/A ‡ 34.6K Source biasShort answer + abstain✗ TANGLE744/48K541Context / behavior / sourceOpen-form✓ † Sentence-level claim–evidence pairs. ‡ Structured tabular input. To address this gap, we introduceTestingAgents’Navigation ofGenuine,Latent, andEntangled Memory Conflicts (TANGLE), a benchmark that evaluates how an agent shall reason and act when personal memory contains a genuinely unresolvable conflict. TANGLE comprises 541 instances from 40 personas, covering 46 life aspects in 10 domains and three conflict types: Context-Partitioned Conflict (CPC), in which a preference depends on context; Behavior-Oscillation Conflict (BOC), in which preferences change without stabilizing; and Source-Contradiction Conflict (SCC), in which sources disagree. These conflicts are unresolvable because context, behavioral dynamics, or source reliability is unknown. We evaluate TANGLE through two matched tracks. The oracle track provides curated memory directly and isolates conflict reasoning. The pipeline track evaluates end-to-end behavior after a memory system extracts memory from multi-session dialogue (Maharana et al., 2024; Wu et al., 2025). Because both tracks encode the same underlying conflicts, their comparison separates information loss during memory construction from failures in conflict reasoning. Since no response is uniquely correct, our rubric assesses conflict perception, causal reasoning, confidence calibration (Matsnev, 2026), clarification seeking, and memory faithfulness, and flags high-risk cases where overconfident resolution causes harm. Representative instances are provided in Appendix K. Across five response models, oracle results reveal a recognition-to-action gap: even with complete curated memory, models recognize conflicts more reliably than they calibrate recommendations or seek targeted clarification. Comparing the oracle and pipeline tracks identifies memory extraction as an upstream bottleneck, as systems often retain topical facts while losing the context, temporal, and source relations required to interpret the conflict; partial retrieval consequently lowers perception and diagnosis. The loss of conflict structure, together with same-domain distractors, further weakens subsequent reasoning and action. Policy comparisons show that deterministic selectors and conservative templates impose predetermined response patterns on unresolved conflicts. These findings motivate Conflict-Aware Action Policy (CAAP), which adapts its action to each conflict according to the available evidence. 2 Evaluating LLM Agents under Irreducible Conflict Our contributions are: •We expose two limitations in prior memory-conflict benchmarks: they formulate conflict as a single-answer recovery task and evaluate agents in a single, undifferentiated track. The single-answer formulation encourages agents to force conflicting memories into one definitive answer, producing overconfident or inappropriate actions when the relevant context, temporal state, or source authority is unresolved. The single-track design conflates failures in memory construction with failures in conflict reasoning, leaving unclear whether an agent lacks conflict-bearing evidence or fails to act appropriately on the evidence it receives. •We establish TANGLE, a benchmark that shifts personal-memory conflict evaluation from recovering a single answer to assessing how agents recognize underdetermination, retain conflicting evidence, and choose an appropriate action. It contains 541 instances across 40 personas, 46 life aspects, 10 domains, and three conflict types. We provide oracle and pipeline tracks that separate conflict reasoning from memory extraction, together with a behavior-centered rubric for cases without a unique gold answer. •We show that fixed resolution rules provide an incomplete treatment of unresolved conflict, and introduce Conflict-Aware Action Policy (CAAP), which selects an action for each conflict from the visible evidence rather than applying a single resolution rule. 2. Related Work 2.1. Conflict Benchmarks Work on conflict evaluation spans settings with deterministic resolution and settings that characterize model behavior under disagreement. Many benchmarks formulate conflict resolution as recovering a single answer, whether through temporal recency (Tao et al., 2026; Tan et al., 2025), implicit state invalidation (Chao et al., 2026), premise correction (Chen et al., 2026a), explicit resolution mandates (Hu et al., 2026), pre- construction conflict elimination (Shen et al., 2026), or knowledge-grounded contradiction handling (Hou et al., 2024; Zhao et al., 2026). A parallel line instead examines how models arbitrate among competing evidence, including memorization tendencies under forced choice (Su et al., 2024; Xie et al., 2024), source and evidence sensitivity (Wan et al., 2024; Schuster et al., 2026), dynamic fact updates and misinformation effects (Marjanović et al., 2024; Peng et al., 2025), conflict-type-aware response generation (Cattan et al., 2025), multi-hop conflict localization (Lee et al., 2025), and selective question answering (QA) with abstention over conflicting multi-source personal memory (Yang et al., 2026). For broader context, Xu et al. (2024) provide a comprehensive synthesis. Our evaluation focuses specifically on the quality of model behavior when conflicts are intentionally unresolvable by design. 2.2. Long-Term Memory Benchmarks Long-term memory benchmarks have largely emphasized stable recall, personalization fidelity, and memory use under evolving interaction histories. Prior datasets test factual and temporal recall (Maharana et al., 2024; Wu et al., 2025), preference consistency and persona adherence (Zhao et al., 2025; Jiang et al., 2025a,b), and extend to real-dialogue personalization (Xiao et al., 2026) and emotionally supportive conversations (Chen et al., 2026b). Recent work increasingly stresses dynamic memory conditions, including forgetting-aware eval- uation (Uddin et al., 2026), memory-to-action grounding (Shen et al., 2026), multi-session state tracking (He et al., 2026; Deshpande et al., 2025), and self-evolving memory (Wang et al., 2026; Wei et al., 2026; Xie et al., 2026), while Pulipaka et al. (2026) highlights downstream safety concerns such as memory-induced sycophancy. 2.3. Agent Memory Systems Agent memory architectures span virtual context management and scalable external memory layers (Packer et al., 2023; Chhikara et al., 2025; Li et al., 2025; Zhong et al., 2023), reflective and self-updating memory mechanisms (Xu et al., 2026; Salama et al., 2025), and procedural memory for workflow execution (Fang et al., 2025; Wang et al., 2024). Optimization-oriented approaches further treat memory use as a policy-learning 3 Evaluating LLM Agents under Irreducible Conflict Irreducible Conflict Cases CPC - work communication mode "What is the fastest way to reach Greg about a work update?" active deployment in person station / contract detailed text ? hidden active work context BOC - sleep schedule "How should I handle sleep across off-season and deployment?" missed obligation strict reset remission decay deployment external structure SCC - insurance claim evidence "What documents should I gather before filing an equipment-damage claim?" WRITTEN POLICY individual filing VERBAL EXCEPTION batch / partner ? preserve evidence or verify in writing exception authority is unverified Two Evaluation Tracks Oracle track memory bank user query response model response Pipeline track Persona timeline (10 - 18 months) conflict episode filler session conflict episode filler session multi-session dialogue memory system extracted memoryuser query FULL / PARTIAL / NONE observability stage 1 response model response stage 2 Evaluating Conflict Handling Evaluation protocol MemoryUser queryResponse Judge D1-D5 rubric D1 perceive D2 reason D3 calibrate D4 clarify D5 faithful Conflict-Aware Action Policy memory bank + query infer what is unresolved choose action Action Space commitconditionalizeclarify verifydeferreversible trial action-aware, evidence-grounded response Figure 1|Overview of the TANGLE benchmark. TANGLE (TestingAgents’Navigation ofGenuine,Latent, andEntangled Memory Conflicts) covers Context-Partitioned Conflict (CPC), Behavior-Oscillation Conflict (BOC), and Source-Contradiction Conflict (SCC), with oracle and pipeline tracks evaluating agents’ capabilities in handling memory conflicts. The Conflict- Aware Action Policy (CAAP) selects evidence-grounded actions rather than forcing a single value. problem, including reinforcement-based methods for memory control (Yan et al., 2025; Zhang et al., 2026; Zhou et al., 2025). 3. The TANGLE Benchmark 3.1. Problem Formulation The goal of our benchmark is to evaluate how effectively LLM agents resolve conflicts in personal memory where the available evidence does not support a unique deterministic answer. LetM=푚 1 , . . . , 푚 푛 denote a memory bank comprising conflict-bearing memories and background entries, and letD=푑 1 , . . . , 푑 푘 denote same-domain distractor memories that are topically related but non-diagnostic for the conflict. Given the combined input(M∪D, 푞)—where푞is a natural user request that does not indicate which information is currently relevant—the agent produces a response 푟. In deployed memory agents, conflict resolution failures stem from two distinct sources: the memory system fails to extract or retrieve relevant evidence, or the model fails to reason appropriately over the evidence it receives. We therefore define two evaluation tracks. Under the oracle track,푟= 푓(M ∪D, 푞): the model directly receives the curated memory set and must resolve the conflict from the evidence alone. Under the pipeline track, a memory system푔first processes multi-session dialogue푆=(푠 1 , . . . , 푠 푇 )to produce an extracted memory set ˆ M= 푔(푆); the model then responds as푟= 푓( ˆ M, 푞). Comparing oracle and pipeline performance isolates conflict resolution capability from extraction quality. Unlike prior benchmarks where a unique gold output 푦 ∗ exists, our conflicts are constructed so that multiple response strategies can all be appropriate. Effective resolution demands the following cognitive capabilities: 1) Conflict awareness. Recognizing that the memory bank contains contradictory or incompatible information relevant to the query. 2) Causal reasoning. Analyzing why the conflict exists—comparing hypotheses and linking reasoning to specific memory evidence. 3) Epistemic calibration. Expressing certainty proportional to 4 Evaluating LLM Agents under Irreducible Conflict Audience (13%) Risk (12%) Goal phase (11%) Resource (9%) Rule governance (8%) Social visibility (7%) Relational (7%) Commitment level (7%) Physiological (6%) Competence (6%) Beneficiary (4%) Location (4%) Context-Partitioned (n=193) Method alternation (25%) Partial retention (18%) Threshold toggling (17%) Burst decay (13%) Amplitude drift (11%) Escalation ladder (10%) Channel substitution (6%) Behavior-Oscillation (n=121) Interpretation conflict (22%) Record vs. reality (15%) Policy vs. exception (13%) Multi-source (13%) Evidence quality (11%) Jurisdiction collision (9%) Identity linkage (7%) Source-Contradiction (n=227) Figure 2 | TANGLE structural diversity by conflict type. The three panels show the distributions of CPC context-partition types, BOC oscillation shapes, and SCC source-conflict types; each labeled sector reports its share within that conflict type. evidence strength—committing when evidence is clear, hedging when genuinely ambiguous. 4) Information seeking. Identifying what missing information would resolve the conflict and asking targeted questions. 5) Evidential grounding. Basing all factual claims on the memory bank without fabricating information beyond what is stored. 3.2. Data Statistics and Conflict Taxonomy The benchmark contains 541 instances spanning 40 personas, 46 life aspects (recurring facets of a user’s life—behaviors, preferences, and factual records such as transport mode, exercise routine, and financial records) across 10 life domains and three conflict types (Figure 2): Context-Partitioned Conflict (CPC, 193 instances), Behavior-Oscillation Conflict (BOC, 121 instances), and Source-Contradiction Conflict (SCC, 227 instances). Under the oracle track, each instance contains 8–11 core memory items plus 6 same-domain distractor memories and a natural user query, totaling 4,893 core memories and 3,246 distractors across the benchmark. Under the pipeline track, we provide 2,580 multi-session dialogues (mean 64.5 sessions per persona, mean 744 tokens per session,∼48K tokens per persona) from which memory systems extract user information before receiving the same queries. Each conflict type is defined by a distinct latent variable that makes deterministic resolution impossible. (1) Context-Partitioned Conflict (CPC). LetM=푚 푖 be the set of retrieved memories, where each memory푚 푖 states a value푣 푖 for the same target life aspect푎(e.g., preferred restaurant type). In CPC, each statement is valid only under an unobserved context푐(such as audience or risk level):푚 푖 holds ⇐⇒ 푐 ∈ 퐶 푖 , where the context regions퐶 푖 are disjoint and support incompatible values for푎. If the query푞does not specify which context applies, no single value is entailed. Our CPC subset covers 17 context-partition types (including audience, goal phase, risk level, and social visibility). (2) Behavior-Oscillation Conflict (BOC). Memories in Mreport the user’s value on life aspect푎at different times, denoted푣 푡 1 , 푣 푡 2 , . . .(the observed value of푎at successive time points 푡 1 , 푡 2 , . . . ), but the trajectory is non-convergent—exhibiting repeated reversals, partial retention, or amplitude drift rather than settling to a stable state. The latent factor is the user’s oscillation phase푧 푡 ∈ adopt, friction, abandon, retry, . . .; when푞does not anchor the relevant phase, the conflict is unresolvable from any single memory. Our BOC subset includes 13 oscillation drivers and 8 oscillation shapes. (3) Source-Contradiction Conflict (SCC). Memories report conflicting values for the same life aspect푎from 퐾 ≥2 different sources푠 1 , . . . , 푠 퐾 (e.g., self-report, behavioral inference, official record): each memory푚 푗 reports a value푣 푗 from source푠 푗 , and these values disagree (푣 푗 ≠ 푣 푘 for푗≠ 푘). The latent variable is source reliability푤(푠); if푞does not indicate which source governs and the reliability ordering is itself uncertain, no fixed priority rule yields a deterministic answer. Our SCC subset contains 10 source-conflict types, with source-count distribution of 2 sources (54%), 3 sources (37%), and 4 sources (10%). 5 Evaluating LLM Agents under Irreducible Conflict 3.3. Data Construction Persona–aspect schema. Starting from Persona Hub (Ge et al., 2025) seeds, we sample and curate a diverse pool of 40 personas spanning age, occupation, income, family structure, health conditions, and geographic setting. Persona Hub supplies initial profile material rather than fixed benchmark instances; we normalize the sampled profiles into a common schema and use internal benchmark identifiers because no formal source IDs are retained. Each persona is mapped to a sparse subset of relevant life aspects rather than a full Cartesian assignment, preserving realism under life constraints (e.g., a rotating-shift nurse receives sleep_schedule but not career_commitment). Full schema definitions are in Appendix A–B. Instance synthesis. For each selected persona–aspect pair, we generate conflict instances with same-domain distractors using a constrained generator. Memories are written as third-person episodic statements compressed to 18–32 words, matching the output format of deployed memory systems. We enforce two integrity constraints: (i) a non-leakage rule guarantees no single memory states the conclusion—the conflict is only recoverable from the aggregate evidence; (i) pattern balancing ensures no single conflict subtype dominates through targeted diversity rewrites. Generation details and quality control procedures are in Appendix F. Query protocol. Each instance is paired with a natural user request that reads as an ordinary task yet withholds the information needed to resolve the conflict. CPC queries hide the context variable entirely, forcing the agent to ask or conditionalize. BOC and SCC queries use open-ended task delegation (e.g., “help me get my exercise routine working well”) without temporal anchors or explicit conflict cues, requiring proactive conflict discovery. Multi-session dialogue synthesis (pipeline track). The pipeline track is derived from the same curated oracle memory set: for each instance we verbalize its memories into longitudinal multi-session dialogue, rendering every memory as user utterances within natural multi-turn conversations that stay consistent with the persona’s profile and speaking style. Conflict-bearing memories are staggered across sessions to avoid clustering, and filler sessions containing no conflict-relevant information are interleaved. Because both tracks encode identical underlying conflicts, comparing them isolates extraction quality from conflict resolution. Full generation algorithm and prompts are in Appendix H. 3.4. Rubric Design Because our conflicts admit no single gold answer, we score the quality of an agent’s cognitive behavior rather than the correctness of a final value. We operationalize the five capabilities required for effective resolution into five scoring dimensions (D1–D5), each rated on a 0–4 scale with conflict-type-specific anchors. Dimensions are scored independently, so a response may score high on one and low on another. D1 (Conflict Perception) captures whether the response identifies and articulates the conflict. D2 (Causal Reasoning) captures the depth of causal or source-reliability reasoning linked to specific memory evidence rather than generic explanation. D3 (Confidence Calibration) captures whether expressed certainty matches evidence strength, rewarding conditional recommendations over overcommitment. D4 (Clarification Seeking) captures the quality of information-seeking targeted at the unresolved conflict variable. D5 (Memory Faithfulness) captures whether factual claims are traceable to the memory bank without fabrication. For the 62 high-risk SCC instances in medical and authorization domains, we additionally assign a ternary D6 (Commitment Appropriateness) flag—safe, partial, or unsafe—reported separately from the composite score. Full dimension definitions with anchoring examples are in Appendix G. 4. Experiments We use TANGLE to study how well current LLM agents handle genuinely unresolvable conflicts in personal memory. Rather than asking whether a model recovers a single correct value, we measure the quality of the cognitive behavior it exhibits: whether it perceives the conflict, reasons about its source, calibrates its confidence, seeks the information needed to resolve the conflict, and stays faithful to the memory bank. Our experiments are organized around four questions that move from model capability to system reliability and policy design: 6 Evaluating LLM Agents under Irreducible Conflict 1112131415 Total score (0--20) Sonnet 5 Gemini 3.1 Pro GLM-4.7 DeepSeek V3.2 GPT-4o 14.37 13.01 12.43 11.92 11.76 15.07 12.83 12.36 11.29 10.76 (a) Overall performance GPT-5.4 Claude Opus 4.7 D1D2D3D4D5 Cognitive dimension Sonnet 5 Gemini 3.1 Pro GLM-4.7 DeepSeek V3.2 GPT-4o 3.623.162.611.853.13 3.332.752.041.743.14 3.192.772.071.432.96 2.982.592.141.272.93 2.882.332.311.123.12 (b) GPT-5.4 D1D2D3D4D5 Cognitive dimension 3.482.932.682.203.78 3.112.402.001.873.45 3.062.382.021.483.41 2.692.031.911.393.27 2.591.621.931.383.24 (c) Claude Opus 4.7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Mean score (0--4) Oracle-track model performance and cognitive profiles (541 instances) Figure 3|Oracle-track performance and cognitive profiles. (a) Mean D1–D5 totals with 95% confidence intervals. (b,c) Judge-specific model–dimension profiles (0–4); D4 clarification is the main bottleneck. •What profiles do current LLM agents exhibit when handling genuinely unresolved conflicts, and how do these profiles vary across conflict types, cognitive dimensions, and models? (§4.2) • How does memory construction shape the availability and quality of conflict-sensitive reasoning in an end-to-end system? (§4.3) • How does conflict handling change as memory noise increases? (§4.4) • What do different conflict-handling policies reveal about resolving unresolved conflict? (§4.5) 4.1. Setup We evaluate Claude Sonnet 5, Gemini 3.1 Pro, DeepSeek-V3.2, GLM-4.7, and GPT-4o in the oracle and pipeline tracks. Unless otherwise stated, models generate responses with temperature 0 and a 4,096-token limit. GPT-5.4 and Claude Opus 4.7 independently score each response on D1–D5 using the same 0–4 rubric, and we report their results separately throughout; their total scores agree strongly across the 2,705 shared responses (Pearson푟=0.838; Spearman휌=0.820). To assess rubric reliability, two human annotators independently score a 556-record reference set and agree within one point on 85.8% of dimension scores overall, 89.4% for BOC, 80.5% for SCC, and 91.3% for CPC. 4.2. Response Quality in the Oracle Track We first evaluate response quality when every model receives the complete oracle memory bank: all canonical core memories and six same-domain distractors for each of the 541 instances. The aggregate totals indicate substantial room for improvement even under complete oracle access: the strongest model remains below 15/20 under both Judges. Figure 3 reports the overall model ranking and the model–dimension profiles under both Judges; no separate oracle table is needed. Conflict structure determines difficulty. Performance varies systematically with the unresolved variable: CPC is easiest (14.79 under GPT-5.4; 14.96 under Opus), SCC is intermediate (11.92; 11.58), and BOC is hardest (10.81; 10.14). CPC requires reasoning about which context–preference mapping applies when the query omits the relevant context. SCC requires comparing conflicting claims whose source scope, authority, or evidence quality differ. BOC requires reasoning about temporal stability rather than selecting among competing propositions: the model must determine whether the current behavioral state will persist. The conflict-type comparison is summarized numerically above, while Figure 3(b,c) shows the corresponding model–dimension profiles under the two Judges. Models recognize conflict more reliably than they act under it. Across both Judges, conflict perception, diagnosis, and memory faithfulness are stronger than confidence calibration and clarification seeking, with D4 the lowest-scoring dimension for every response model. The common bottleneck is therefore not simply 7 Evaluating LLM Agents under Irreducible Conflict 0255075100 Share of pipeline cases (%) Letta Mem0 A-mem MemOS 8.3% loss 30.9% loss 53.1% loss 53.8% loss FULLPARTIALNONE D3D4D5D3D4D5 Letta Mem0 A-mem MemOS +0.07+0.22-0.36+0.14+0.05+0.30 +0.20+0.04-0.12+0.13-0.07+0.29 +0.03+0.13-0.26+0.06-0.18+0.34 +0.23+0.55-0.11+0.18+0.42+0.29 GPT-5.4Claude Opus 4.7 (a) Conflict observability(b) Pipeline--oracle gap Figure 4|Memory construction shapes observability and action. (a) Pipeline extraction yields full, partial, or no target-conflict observability. (b) Conditional pipeline-minus-oracle gaps for D3–D5; negative values favor oracle inputs. detecting contradictory evidence or avoiding fabrication. Models often describe the tension without expressing calibrated uncertainty, asking the question that would change the recommendation, or selecting a reversible action appropriate to the unresolved conflict. BOC provides the clearest evidence of this gap: D5 remains high (3.52–3.55), while D3 and D4 remain low (D3: 0.65–1.03; D4: 0.65–0.69). Models faithfully recount an oscillatory history without treating that history as evidence that a fixed recommendation does not persist. Detecting behavioral change does not by itself tell the model whether the current state will persist, whether a proposed intervention will work, or which unresolved trigger it needs to clarify. Figure 3 summarizes the model-level dimension profile. Model profiles differ within this shared bottleneck. Although all models exhibit the recognition-to-action gap, its severity varies. Sonnet 5 is the most balanced model across dimensions, whereas Gemini 3.1 Pro and GLM-4.7 retain relatively strong conflict perception but are less consistent in translating that recognition into calibrated action. DeepSeek V3.2 and GPT-4o show the weakest clarification behavior, with GPT-4o also exhibiting a diagnosis deficit on source-contradiction cases. Thus, the models share the same broad recognition-to-action gap, but differ in which stage of conflict resolution is most affected. 4.3. Oracle vs. Pipeline Degradation The oracle track supplies the complete curated memory bank, whereas the pipeline track uses the ordered memories returned by a system-native extractor. We first quantify how often each system preserves the target conflict using three observability states: FULL, when the complete conflict structure is visible; PARTIAL, when target-specific tension remains visible but at least one conflict-bearing relation is missing or degraded; and NONE, when the retrieved bank does not expose the target conflict, even if it contains useful memories for answering the query. We then evaluate downstream response quality with D1–D2 conditioned on the observability state and D3–D5 computed over all retrieved banks. This keeps conflict perception and diagnosis separate from calibration, clarification, and faithfulness, which remain applicable even when the target conflict is not fully exposed. Memory systems differ sharply in conflict observability. Letta preserves the complete target conflict in 91.7% of cases, compared with 69.1% for Mem0, 47.0% for A-mem, and 46.2% for MemOS (Figure 4(a)). The resulting information loss ranges from 8.3% to 53.8%. This is a retrieval-level result rather than a downstream quality ranking: a non-FULL bank still supports a useful answer while omitting the trajectory, context partition, or source comparison that defines the benchmark conflict. Complete and partial retrieval reveal different downstream limits. Figure 4(b) shows that pipeline responses do not simply incur a uniform penalty relative to the complete-evidence oracle. Across systems, D5 remains comparatively robust, whereas D3 and especially D4 are more variable; this indicates that models remain faithful to the memories they receive while still lacking the calibration or targeted inquiry needed 8 Evaluating LLM Agents under Irreducible Conflict 0246 Same-domain distractors, N 11.5 12.0 12.5 13.0 13.5 14.0 Mean total score (0--20) GPT-5.4: -0.62 Claude Opus 4.7: -0.87 (a) Overall noise effect GPT-5.4 Claude Opus 4.7 D1 Perception D2 Diagnosis D3 Calibration D4 Clarification D5 Faithfulness 0.00 0.10 0.20 0.30 Mean score decrease from N = 0 to N = 6 (b) Dimension-level noise effects 0.09 0.15 0.10 0.20 0.09 0.15 0.19 0.17 0.13 0.23 GPT-5.4 Claude Opus 4.7 Figure 5|Distractor noise weakens conflict-sensitive reasoning. (a) Mean total quality from푁=0 to 6 distractors, with 95% intervals. (b) Dimension-wise score drops; larger values indicate greater degradation. to resolve an incompletely observed conflict. The oracle therefore provides an upper-condition reference for response behavior, not a single expected score that every pipeline system matchs. Conditional on FULL retrieval, D1 and D2 are relatively similar across memory systems under both Judges; the range across systems is much smaller than the corresponding gap in FULL rates. Once the target conflict has been reconstructed, the response models therefore receive broadly comparable evidence for perception and diagnosis. PARTIAL retrieval provides a useful boundary condition: although D1 and D2 are scored only on the tension that remains visible, both Judges report lower values than under FULL retrieval for every system. The lower PARTIAL values are explained by a structural loss: the model must interpret the surviving memories without the relation that links them to the omitted evidence, making both conflict mapping and diagnosis less determinate. The axis-level breakdown makes the consequence concrete. BOC requires a multi-phase behavioral trajectory, CPC requires both sides of a context partition, and SCC requires an attributed map of competing claims and their sources. Partial retrieval preserves answer-relevant content while destroying the relation that determines how that content is interpreted. If an SCC bank retains one claim but omits the competing source, for example, the model produces a locally coherent recommendation based on the surviving claim but cannot compare the claims or determine whether their reliability or scope differs. Similarly, omitting earlier BOC phases hides the trajectory that makes the current state uncertain, while omitting one CPC context hides the condition under which the alternative preference applies. The consistency of this pattern across response models indicates that the missing relations change the reasoning problem itself: each model must interpret a fragment whose meaning is underdetermined by the retrieved bank. Extraction is the main pipeline bottleneck. The system-level gaps are driven primarily by whether the memory bank preserves the relations that define the target conflict. Letta reaches the FULL condition far more often than the other systems, whereas the conditional D1/D2 differences among FULL cases are comparatively modest. Thus, improving end-to-end conflict handling requires more than retrieving topically relevant memories. It requires preserving the conflict-bearing relations, dependencies, and evidence structure needed for the response model to reconstruct, explain, and act on unresolved tension. The response model is unchanged across systems, but the evidence available to it is not. The dominant failure point therefore lies in memory construction and conflict observability. 4.4. Conflict-Sensitive Reasoning under Distractor Memories We next compare response quality across distractor-load conditions using a paired cohort of 60 instances, balanced across BOC, CPC, and SCC. For each instance, we retain the conflict-bearing memories and vary the number of same-domain distractors included in the response-model input, with푁 ∈ 0,2,4,6. The푁=6 condition is the standard oracle input, while푁 <6 conditions are lower-load ablations. Each condition contains five response models, and all comparisons are paired within instance. We report Judge-specific estimates and 9 Evaluating LLM Agents under Irreducible Conflict D1D2D3D4D5 Recency (BOC) Majority (CPC) Source priority (SCC) Always ask Always conditionalize Lexical detector 1.020.480.020.003.96 1.070.850.110.003.12 0.941.050.130.003.83 2.000.023.752.214.00 2.000.732.880.263.91 1.080.543.312.133.92 GPT-5.4 D1D2D3D4D5 1.320.050.040.003.34 1.050.400.180.002.74 0.440.920.040.002.78 1.610.032.432.204.00 1.760.482.250.453.99 1.050.212.131.753.96 Claude Opus 4.7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 D3D4D5Total GPT-4o Sonnet 5 Gemini 3.1 Pro +0.50 2.31 2.81 +0.63 1.12 1.75 +0.55 3.12 3.67 +1.16 11.76 12.92 +0.83 2.61 3.44 +0.75 1.85 2.60 +0.00 3.13 3.13 +2.01 14.37 16.38 +0.60 2.04 2.64 -0.60 1.74 1.14 +0.02 3.14 3.16 +0.23 13.01 13.24 CAAP effects 0.8 0.0 0.8 1.6 CAAP Direct Figure 6|Fixed policies fail structurally; CAAP improves behavior. Left: fixed-policy profiles (0–4) under GPT-5.4 and Claude Opus 4.7. Right: CAAP-minus-direct-oracle changes under identical inputs, evaluated by GPT-5.4. use paired instance-cluster bootstrap intervals to quantify uncertainty in the within-instance changes. Distractors impose a cumulative cost on conflict handling. The aggregate D1–D5 score total declines monotonically with distractor load under both evaluations: the mean total decreases from 13.26 to 12.63 under GPT-5.4 and from 13.35 to 12.48 under Opus when moving from푁=0 to푁=6. The corresponding paired intervals exclude zero for both evaluations. We therefore examine the dimension-level profiles separately to distinguish degradation in conflict-sensitive reasoning from changes in faithfulness to the memory bank. Figure 5(a) shows a gradual decline rather than a sharp threshold at one particular load. Increasing the memory set therefore creates a competition problem: topically related memories remain plausible user evidence, but make it harder to preserve the distinction between information that defines the conflict and information that only describes the broader user profile. Noise weakens post-detection reasoning more consistently than initial conflict perception. We separate the conflict-sensitive dimensions (D1–D4) from faithfulness to the visible memory bank (D5). D1 changes only modestly as distractors increase, whereas D2–D4 show larger declines. Under GPT-5.4, D1 decreases from 3.357 to 3.270, while D2, D3, and D4 decrease from 2.880, 2.117, and 1.743 to 2.730, 2.017, and 1.543, respectively. Under Opus, the corresponding changes are from 3.173 to 3.020 for D1 and from 2.467, 2.220, and 1.823 to 2.273, 2.053, and 1.693 for D2–D4. The largest change is in D4 under GPT-5.4, while Opus shows a broader reduction across the post-detection dimensions. D5 is comparatively stable in the aggregate, although its conflict-type trajectory differs because additional distractors provide traceable user details without improving diagnosis. Figure 5(a,b) reports the aggregate trajectory and the dimension-level decreases on a positive-loss scale. The qualitative pattern is shared across conflict types, with different manifestations in the score profiles. Across conflict types, the degradation is heterogeneous rather than uniform. In this paired cohort, CPC exhibits the clearest and most consistent decline in conflict-sensitive dimensions under both evaluations, particularly in diagnosis, calibration, and clarification; BOC shows a weaker D1–D4 decline with a partly separate faithfulness pattern, while SCC is comparatively stable. This variation is consistent with the structural demands of the axes: CPC requires the model to retain and use a context partition despite plausible same-domain alternatives. The matched-instance results support the aggregate interpretation: the evaluations are strongly associated overall (Pearson푟=0.853), with lower agreement for BOC because its behavioral episodes require more interpretive judgments about whether a response sufficiently explains the underlying pattern. The complete type-level breakdown and case audit appear in Appendix I. 4.5. Heuristic and Policy Baselines Can simple heuristics resolve memory conflicts? We compare three axis-specific value selectors—strict recency on 121 BOC instances, strict majority vote on 193 CPC instances, and fixed source priority on 227 SCC instances—with three cross-axis conservative controls evaluated on all 541 instances: Ask-only, which requests further information before recommending; Conditionalize-only, which retains alternatives without selecting one; and a lexical conflict detector, which uses surface opposition in the memory text to trigger a conservative response. 10 Evaluating LLM Agents under Irreducible Conflict The six policies introduced above serve two different diagnostic purposes. The axis-specific selectors test whether an unresolved conflict is reduced to a deterministic choice of behavioral phase, preference, or source, whereas the cross-axis controls test simpler strategies that avoid commitment, preserve alternatives, or react to surface-level opposition. The mechanism statistics already show what is lost by these fixed rules: strict recency, strict majority, and fixed source priority commit to one value on every applicable instance, while majority voting is tied in 28.0% of CPC cases and has a margin of at most one in 71.5% of cases. The lexical detector likewise identifies only a small subset of conflict-positive banks (recall 0.181), because surface opposition does not reliably expose temporal, contextual, or source-dependent conflict. The selectors therefore serve as structural ablations, and the controls as reference behaviors rather than interchangeable policy competitors: the former discard conflict-bearing relations, while the latter avoids or delays commitment without necessarily explaining the underlying conflict. Figure 6(a) summarizes these cross-judge failure profiles. Because the rubric explicitly evaluates calibrated action, information seeking, and memory faithfulness, the figures are descriptive rather than a universal policy ranking. We therefore design the Conflict-Aware Action Policy (CAAP) to move beyond fixed value selection and fixed response templates. Rather than always selecting one value, always asking, or always conditionalizing, CAAP first decides how to handle the conflict and then writes a response that follows that decision. Given a visible memory bank 푀 and user query 푞, it uses two stages: (푧, 푎)= 휋 dec 휃 (푀, 푞), 푟= 휋 real 휃 (푀, 푞, 푧, 푎). Here,휋 dec simply denotes the first-stage decision module. It produces a brief internal decision record푧(for example, what is uncertain and what information is missing) and selects an action푎∈ A.휋 real denotes the second-stage response-writing module: it uses the visible memories, query, and selected action to produce the final response 푟. Its action space is A=commit, conditionalize, clarify, verify, defer, reversible_trial. Both stages use the same response-model backbone within each matched evaluation. CAAP receives only the visible memory texts and the user query; it is not given definition-level information such as the conflict type. The decision stage is instructed to choose the least cautious action that is still justified by the visible evidence, while becoming more cautious when an incorrect recommendation would have greater consequences. The realization stage then turns that decision into a natural user-facing response. We compare CAAP with direct oracle responses for Claude Sonnet 5, Gemini 3.1 Pro, and GPT-4o while holding the response model, visible memory bank, query, and Judge constant. Figure 6(b) reports the matched CAAP comparison. CAAP is designed to select an appropriate conflict-handling action for each instance, so D3–D5 are the primary outcomes, while D1–D2 provide diagnostic context. The three model evaluations show that CAAP’s flexibility is useful but not uniformly realized. Sonnet 5 improves in both action calibration and decision-relevant coverage while preserving faithfulness, with the largest gains on BOC and SCC. Gemini 3.1 Pro also improves action calibration, but its coverage decreases, leaving little overall change. GPT-4o improves in action calibration, clarification, and faithfulness under CAAP, although BOC remains its weakest conflict type. Across the three models, CAAP supports evidence-sensitive action selection, but the magnitude and composition of its benefit depend on how reliably each response backbone realizes the selected action and supplies the evidence required by the case. CAAP action distributions provide a mechanism-level audit rather than the primary outcome: without explicit axis labels, it tends toward reversible trials for BOC, conditionalization or clarification for CPC, and verification or source-sensitive caution for SCC. The model-specific distributions and the full D1–D5 comparison are reported in Appendix J. CAAP is therefore a positive action-policy baseline rather than an upper bound: it moves beyond a single fixed response template, while temporal-pattern reconstruction and detailed source reconciliation remain open challenges. 5. Conclusion We introduced TANGLE, a benchmark for evaluating memory agents under genuinely unresolvable conflict. Across 541 instances, five response models, and four memory systems, we find a consistent recognition-to-action 11 Evaluating LLM Agents under Irreducible Conflict gap: models detect and describe conflict more reliably than they calibrate actions or seek clarification, while memory extraction and distractor noise limit the conflict structure available for reasoning. Fixed policies further compress or sidestep this structure, motivating CAAP as a flexible, evidence-sensitive action policy. These findings suggest that reliable memory agents needs to preserve conflicting evidence and its relations, represent what remains unresolved, and choose actions that are calibrated to both the available evidence and the consequences of error. References Bae, S., Kwak, D., Kang, S., Lee, M. Y., Kim, S., Jeong, Y., Kim, H., Lee, S.-W., Park, W., and Sung, N. Keep me updated! memory management in long-term conversations. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, p. 3769–3787, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. findings-emnlp.276. URL https://aclanthology.org/2022.findings-emnlp.276/. Cattan, A., Jacovi, A., Ram, O., Herzig, J., Aharoni, R., Goldshtein, S., Ofek, E., Szpektor, I., and Caciularu, A. Dragged into conflicts: Detecting and addressing conflicting sources in search-augmented llms, 2025. URL https://arxiv.org/abs/2506.08500. Chao, H., Bai, Y., Sheng, R., Li, T., and Sun, Y. Stale: Can llm agents know when their memories are no longer valid?, 2026. URL https://arxiv.org/abs/2605.06527. Chen, D., Niu, S., Li, K., Liu, P., Zheng, X., Tang, B., Li, X., Xiong, F., and Li, Z. Halumem: Evaluating hallucinations in memory systems of agents, 2026a. URL https://arxiv.org/abs/2511.03506. Chen, T., Lu, J., Shen, Y., and Zhang, L. Es-memeval: Benchmarking conversational agents on personalized long-term emotional support. Proceedings of the ACM Web Conference 2026, 2026b. URLhttps://api. semanticscholar.org/CorpusID:285269543. Chhikara, P., Khant, D., Aryan, S., Singh, T., and Yadav, D. Mem0: Building production-ready ai agents with scalable long-term memory, 2025. URL https://arxiv.org/abs/2504.19413. Deshpande, D., Gangal, V., Mehta, H., Kannappan, A., Qian, R., and Wang, P. Memtrack: Evaluating long-term memory and state tracking in multi-platform dynamic agent environments, 2025. URLhttps: //arxiv.org/abs/2510.01353. Fang, R., Liang, Y., Wang, X., Wu, J., Qiao, S., Xie, P., Huang, F., Chen, H., and Zhang, N. Memp: Exploring agent procedural memory. ArXiv, abs/2508.06433, 2025. URLhttps://api.semanticscholar.org/CorpusID: 280561810. Ge, T., Chan, X., Wang, X., Yu, D., Mi, H., and Yu, D. Scaling synthetic data creation with 1,000,000,000 personas, 2025. URL https://arxiv.org/abs/2406.20094. He, Z., Wang, Y., Zhi, C., Hu, Y., Chen, T.-P., Yin, L., Chen, Z., Wu, T. A., Ouyang, S., Wang, Z., Pei, J., McAuley, J., Choi, Y., and Pentland, A. Memoryarena: Benchmarking agent memory in interdependent multi-session agentic tasks, 2026. URL https://arxiv.org/abs/2602.16313. Hou, Y., Pascale, A., Carnerero-Cano, J., Tchrakian, T., Marinescu, R., Daly, E., Padhi, I., and Sattigeri, P. Wikicontradict: A benchmark for evaluating llms on real-world knowledge conflicts from wikipedia, 2024. URL https://arxiv.org/abs/2406.13805. Hu, Y., Wang, Y., and McAuley, J. Evaluating memory in LLM agents via incremental multi-turn interactions. In The Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview. net/forum?id=DT7JyQC3MR. Jiang, B., Hao, Z., Cho, Y.-M., Li, B., Yuan, Y., Chen, S., Ungar, L., Taylor, C. J., and Roth, D. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale, 2025a. URL https://arxiv.org/abs/2504.14225. Jiang, B., Yuan, Y., Shen, M., Hao, Z., Xu, Z., Chen, Z., Liu, Z., Vijjini, A. R., He, J., Yu, H., Poovendran, R., Wornell, G., Ungar, L., Roth, D., Chen, S., and Taylor, C. J. Personamem-v2: Towards personalized 12 Evaluating LLM Agents under Irreducible Conflict intelligence via learning implicit user personas and agentic memory, 2025b. URLhttps://arxiv.org/ abs/2512.06688. Lee, J., Lee, K., and Kim, T. Magic: A multi-hop and graph-based benchmark for inter-context conflicts in retrieval-augmented generation. ArXiv, abs/2507.21544, 2025. URLhttps://api.semanticscholar. org/CorpusID:280337652. Li, Q., Li, Y., Gao, J., Zhao, B., Fan, W., and Han, J. Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, SIGMOD ’14, p. 1187–1198, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450323765. doi: 10.1145/2588555.2610509. URLhttps://doi.org/10.1145/ 2588555.2610509. Li, Z., Song, S., Wang, H., Niu, S., Chen, D., Yang, J., Xi, C., Lai, H., Zhao, J., Wang, Y., Ren, J., Lin, Z., Huo, J., Chen, T., Chen, K., Li, K., Yin, Z., Yu, Q., Tang, B., Yang, H., Xu, Z.-Q. J., and Xiong, F. Memos: An operating system for memory-augmented generation (mag) in large language models, 2025. URL https://arxiv.org/abs/2505.22101. Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., and Fang, Y. Evaluating very long-term conversational memory of llm agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 13851–13870, 2024. Marjanović, S. V., Yu, H., Atanasova, P., Maistro, M., Lioma, C., and Augenstein, I. Dynamicqa: Tracing internal knowledge conflicts in language models. In Conference on Empirical Methods in Natural Language Processing, 2024. URL https://api.semanticscholar.org/CorpusID:271404307. Matsnev, G. Uncertainty decomposition for clarification seeking in llm agents, 2026. URLhttps://arxiv. org/abs/2606.19559. Ong, K. T.-i., Kim, N., Gwak, M., Chae, H., Kwon, T., Jo, Y., Hwang, S.-w., Lee, D., and Yeo, J. Towards lifelong dialogue agents via timeline-based memory management. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 8631–8661, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long.435. URL https://aclanthology.org/2025.naacl-long.435/. Packer, C., Fang, V., Patil, S. G., Lin, K., Wooders, S., and Gonzalez, J. Memgpt: Towards llms as operating sys- tems. ArXiv, abs/2310.08560, 2023. URLhttps://api.semanticscholar.org/CorpusID:263909014. Peng, M., Chen, N., Tang, J., and Li, J. How does misinformation affect large language model behaviors and preferences?, 2025. URL https://arxiv.org/abs/2505.21608. Pulipaka, S., Chen, O., Sharma, M., Bajwa, T. S., Raina, V., and Sheth, I. Persistbench: When should long-term memories be forgotten by llms?, 2026. URL https://arxiv.org/abs/2602.01146. Salama, R. K. A., Cai, J., Yuan, M., Currey, A., Sunkara, M., Zhang, Y. E., and Benajiba, Y. Meminsight: Autonomous memory augmentation for llm agents. ArXiv, abs/2503.21760, 2025. URLhttps://api. semanticscholar.org/CorpusID:277349587. Schuster, J., Gautam, V., and Markert, K. Whose facts win? llm source preferences under knowledge conflicts, 2026. URL https://arxiv.org/abs/2601.03746. Shen, Y., Li, K., Zhou, W., and Hu, S. Mem2actbench: A benchmark for evaluating long-term memory utilization in task-oriented autonomous agents, 2026. URL https://arxiv.org/abs/2601.19935. Su, Z., Zhang, J., Qu, X., Zhu, T., Li, Y., Sun, J., Li, J., Zhang, M., and Cheng, Y. ConflictBank: A benchmark for evaluating the influence of knowledge conflicts in llms. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., and Zhang, C. (eds.), Advances in Neural In- formation Processing Systems, volume 37, p. 103242–103268. Curran Associates, Inc., 2024. doi: 10.52202/079017-3280. URLhttps://proceedings.neurips.c/paper_files/paper/2024/file/ baf4b960d118f838ad0b2c08247a9ebe-Paper-Datasets_and_Benchmarks_Track.pdf. 13 Evaluating LLM Agents under Irreducible Conflict Tan, H., Zhang, Z., Ma, C., Chen, X., Dai, Q., and Dong, Z. Membench: Towards more comprehensive evaluation on the memory of llm-based agents. In Findings of the Association for Computational Linguistics: ACL 2025, p. 19336–19352, 2025. Tao, Z., Zhao, J., Liu, P., Xi, D., Chen, Y., Xu, W., and Li, Z. Memconflict: Evaluating long-term memory systems under memory conflicts. 2026. URL https://api.semanticscholar.org/CorpusID:288655900. Tian, H., Sheng, W., Shen, H., and Wang, C. Truth finding by reliability estimation on inconsistent entities for heterogeneous data sets. Knowledge-Based Systems, 187:104828, 2020. ISSN 0950-7051. doi: https: //doi.org/10.1016/j.knosys.2019.06.036. URLhttps://w.sciencedirect.com/science/article/ pii/S095070511930303X. Tversky, A. and Simonson, I. Context-dependent preferences. Management Science, 39(10):1179–1189, 1993. ISSN 00251909, 15265501. URL http://w.jstor.org/stable/2632953. Uddin, M. N., Shubham, K., Blanco, E., Baral, C., and Wang, G. From recall to forgetting: Benchmarking long-term memory for personalized agents, 2026. URL https://arxiv.org/abs/2604.20006. Wan, A., Wallace, E., and Klein, D. What evidence do language models find convincing? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7468–7484, 2024. Wang, Y., Zhang, Z., Chi, M., Yu, K., Li, Y., Peng, M., Tong, B., Zhang, C., Zhou, Y., and Li, J. Evomembench: Benchmarking agent memory from a self-evolving perspective, 2026. URLhttps://arxiv.org/abs/2605. 18421. Wang, Z. Z., Mao, J., Fried, D., and Neubig, G. Agent workflow memory, 2024. URLhttps://arxiv.org/ abs/2409.07429. Wei, T., Sachdeva, N., Coleman, B., He, Z., Bei, Y., Ning, X., Ai, M., Li, Y., He, J., Chi, E. H., Wang, C., Chen, S., Pereira, F., Kang, W.-C., and Cheng, D. Z. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory, 2026. URL https://arxiv.org/abs/2511.20857. Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., and Yu, D. Longmemeval: Benchmarking chat assistants on long-term interactive memory, 2025. URL https://arxiv.org/abs/2410.10813. Xiao, J., Yu, X., Wang, C., Zheng, W., Lin, X., Liu, K., Ding, H., Zhang, Y., Wang, W., Feng, F., and He, X. Alpsbench: An llm personalization benchmark for real-dialogue memorization and preference alignment. ArXiv, abs/2603.26680, 2026. URL https://api.semanticscholar.org/CorpusID:286961600. Xie, J., Zhang, K., Chen, J., Lou, R., and Su, Y. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In International Conference on Learning Representations, volume 2024, p. 35623–35646, 2024. Xie, W., Guo, S., Zhang, F., Xia, T., Yang, X., Ma, L., Yan, J., and Ren, Q. Memevobench: Benchmarking safety risks from memory misevolution in llm agents, 2026. URL https://arxiv.org/abs/2604.15774. Xu, R., Qi, Z., Guo, Z., Wang, C., Wang, H., Wang, H., and Xu, W. Knowledge conflicts for llms: A survey. ArXiv, abs/2403.08319, 2024. URL https://api.semanticscholar.org/CorpusID:268379757. Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., and Zhang, Y. A-mem: Agentic memory for llm agents. Advances in Neural Information Processing Systems, 38:17577–17604, 2026. Yan, S., Yang, X., Huang, Z., Nie, E., Ding, Z., Li, Z., Ma, X., Schutze, H., Tresp, V., and Ma, Y. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. ArXiv, abs/2508.19828, 2025. URL https://api.semanticscholar.org/CorpusID:280918480. Yang, T., Schonlau, M., and Sucholutsky, I. Selective qa over conflicting multi-source personal memory: A diagnostic testbed and method comparison, 2026. URL https://arxiv.org/abs/2605.30087. 14 Evaluating LLM Agents under Irreducible Conflict Zhang, S., Wang, J., Zhou, R., Liao, J., Feng, Y., Li, Z., Zheng, Y., Zhang, W., Wen, Y., Li, Z., Xiong, F., Qi, Y., Tang, B., and Wen, M. Memrl: Self-evolving agents via runtime reinforcement learning on episodic memory, 2026. URL https://arxiv.org/abs/2601.03192. Zhao, S., Hong, M., Liu, Y., Hazarika, D., and Lin, K. Do llms recognize your preferences? evaluating personalized preference following in llms, 2025. URL https://arxiv.org/abs/2502.09597. Zhao, T., Chen, J., Zhang, S., Zhu, H., Lin, Q., and Liu, J. Exploring knowledge conflicts for faithful llm reasoning: Benchmark and method, 2026. URL https://arxiv.org/abs/2604.11209. Zhong, W., Guo, L., Gao, Q.-F., Ye, H., and Wang, Y. Memorybank: Enhancing large language models with long-term memory. ArXiv, abs/2305.10250, 2023. URLhttps://api.semanticscholar.org/CorpusID: 258741194. Zhou, H., Chen, Y., Guo, S., Yan, X., Lee, K.-H., Wang, Z., Lee, K. Y., Zhang, G., Shao, K., Yang, L., and Wang, J. Memento: Fine-tuning llm agents without fine-tuning llms. ArXiv, abs/2508.16153, 2025. URL https://api.semanticscholar.org/CorpusID:280708996. 15 Evaluating LLM Agents under Irreducible Conflict Appendix A Persona Definitions17 B Attribute Schema19 C Conflict-Structure Schema20 D Persona–Attribute Coverage22 E Dataset Composition and Diversity26 F Generation, Session Synthesis, and Quality Control26 G Scoring Rubric and Judge Protocol27 H Response and Evaluation Prompts29 I RQ3 Noise Interpretation Audit31 I.1 Case A: evaluator-sensitive BOC response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 J RQ4 Policy Diagnostics32 K Illustrative Instances32 K.1 Behavior-Oscillation Conflict (BOC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 K.2 Context-Partitioned Conflict (CPC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 K.3 Source-Contradiction Conflict (SCC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 K.4 Pipeline Observability Contrasts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 16 Evaluating LLM Agents under Irreducible Conflict A. Persona Definitions Starting from Persona Hub (Ge et al., 2025) seeds, we sample and curate 40 personas to cover a broad range of life circumstances. Persona Hub provides the initial profile material; it is not treated as a fixed set of benchmark instances. Because the source profiles do not have formal identifiers that we retain, the IDs in Table 2 are internal row identifiers assigned for this benchmark, not Persona Hub IDs. We normalize the sampled profiles into a common schema and lightly revise them where needed to make the life circumstances coherent and to support the benchmark’s coverage objectives. The resulting personas span age (20–78), occupation (e.g., student, nurse, freelancer, warehouse associate, attorney, firefighter, retiree), income (low to high), family structure (single, roommates, partnered, single-parent, married-with-children, multi-generational, blended), health status (healthy, chronic conditions such as lupus, CKD, or diabetes, disabilities including wheelchair use and Deafness, and ADHD), and location (urban, suburban, rural, expatriate). Each persona is mapped to only 13–15 relevant attributes (of 46 distinct serialized names; 48 axis-specific labels; see §B), rather than the full persona×attribute product; the complete persona–attribute mapping is maintained as part of the dataset metadata. Table 2 reports the normalized persona profiles and the life constraints used to ground their conflict instances, documenting the profile grounding and preserving plausible tensions rather than arbitrary contradictions. Table 2|Persona roster and conflict-relevant grounding. IDs are internal benchmark identifiers (p01–p40), not source identifiers. We deliberately assign each persona only 13–15 life-relevant instances rather than the full persona×attribute product. The final column records recurring pressures used to make conflicts ecologically plausible. ID Name textbfAge Work / life stageHousehold and settingConflict-relevant anchors Early career, education, and entry-level work p01 Maya (19–22) College student and baristaThree roommates in a cityIrregular work and class schedule; tight budget; shared-space coordination p02 Jordan (24–29) Junior software engineer, hybrid Lives alone in an expensive city Cross-time-zone collaboration; sustained cognitive load; “always-on” work expectations p03 Elena (31–38) Registered nurse on rotating shifts Partner and young child Night and weekend shifts; sleep disruption; childcare handoffs p04 DeShawn (34–42) Warehouse associate on shiftsMulti-generational householdPhysically demanding work; little schedule control; financial support for family p05 Priya (36–44) Middle-school teacherMarried with two childrenEvening spillover from work; emotional labor; rigid school-day schedule p06 Luis (28–37) Freelance graphic designerWorks from home with partner Volatile income; client deadlines; self-managed work boundaries p07 Amina (40–50) Small grocery-store ownerSpouse and extended familyLong operating hours; staffing shortfalls; intertwined household and business finances p08 Noah (45–55) Mid-level corporate managerSpouse and eldercare responsibilities Competing work and caregiving demands; medical coordination; decision fatigue p09 Grace (52–60) Public-transit bus driverLives aloneFixed routes and breaks; minimal flexibility; strict punctuality requirements p10 Ethan (61–70) Recently retired accountantSpouse in a suburbIdentity transition after retirement; fixed income; rising medical needs Precarious, care-intensive, and high-demand work continued on next page 17 Evaluating LLM Agents under Irreducible Conflict Table 2 (continued) ID Name textbfAge Work / life stageHousehold and settingConflict-relevant anchors p11 Sofia (27–35) Hospitality worker and immigrant single parent Relatives and one childLanguage friction; variable shifts; limited childcare options p12 Caleb (30–40) Gig driver and online learnerLives with a roommateAlgorithm-driven income; no paid leave; self-funded reskilling p13 Riley (18–21) Retail cashier and community-college student Single parent with siblings in a small town Unreliable transit; contributes to bills; limited study space p14 Owen (22–27) Line cookShares an apartment with coworkers Late-night shifts; volatile tips; sleep and noise disruption p15 Zoe (23–30) Social-media coordinator at a startup Partner in an expensive cityAlways-on culture; high rent; engagement-metric pressure p16 Malik (26–33) Paramedic / first responderFiancé, infant, and outer-suburban home Trauma exposure; unpredictable overtime; childcare gaps p17 Hannah (29–36) Attorney at a large firmMarried, no children, downtown Billable-hour pressure; frequent travel; financial and practical support for a parent p18 Victor (32–40) Union electricianCo-parents one child across two households Weather-sensitive jobs; custody coordination; injury risk p19 Leah (35–43) HR directorDivorced, two children, suburban Dual-household logistics; workplace conflict; little personal time p20 Samir (38–46) Long-haul truck driverSpouse in a rural areaLong absences; chronic back pain; limited healthcare access on the road p21 Nadia (41–49) City police sergeantSingle; caregiver for an elderly aunt Rotating shifts; public scrutiny; caregiver burnout p22 Trevor (44–52) Regional sales executiveRemarried, blended familyHeavy travel; stepfamily tensions; quota-linked compensation p23 Mei (47–55) Home-health aideRents a room in a shared house Low wages; emotional labor; remittance obligations p24 Andre (50–58) Manufacturing-plant supervisorMarried; adult son has moved home Sandwich-generation demands; production deadlines; hypertension management Later career, retirement transition, and health-related constraints p25 Kendra (53–61) State corrections officerLives alone in an exurban area High-stress safety work; long commute; limited local support p26 Farah (55–63) PharmacistMarried; helps with family business Medication liability; long periods standing; competing household and business finances p27 Ben (58–66) FarmerSpouse in a rural, multi-generational household Weather volatility; equipment debt; limited local services p28 Carla (60–68) Recently widowed hotel housekeeper Senior apartmentFixed income; grief and isolation; physically demanding work p29 Dmitri (62–70) Post-retirement rideshare driver and immigrant SpouseLanguage barriers; variable earnings; diabetes management p30 Janice (64–72) Part-time librarianCaregiver for spouse with dementia Caregiver fatigue; medical-system navigation; reduced social contact continued on next page 18 Evaluating LLM Agents under Irreducible Conflict Table 2 (continued) ID Name textbfAge Work / life stageHousehold and settingConflict-relevant anchors p31 Haruto (67–75) Retired engineer and expatriateSplits time across two countriesVisa and tax complexity; fragmented health records; cross-border coordination p32 Lila (70–78) Retired teacher and wheelchair user Alone in an older buildingAccessibility barriers; fixed income; dependence on paratransit Disability, chronic illness, mobility, and nonstandard family arrangements p33 Miguel (24–32) Deaf customer-support specialist, remote Lives with partnerInaccessible meetings; captioning failures; career bias p34 Avery (27–34) UX researcher with ADHDCo-lives with two friendsExecutive-function demands; time blindness; meeting overload p35 Rosa (33–41) Municipal clerk with lupusSingle parent of oneUnpredictable flare-ups; strict attendance rules; fragile childcare arrangements p36 Imani (30–38) Army reservist and logistics coordinator Married with a toddlerDeployment interruptions; spouse’s career sacrifices; childcare continuity p37 Chen (39–47) International-school teacher and expatriate Abroad with spouse and teenager Visa insecurity; cultural friction; distant eldercare p38 Paula (42–50) Restaurant-franchise ownerMarried; supports a college-age child Debt and cash-flow pressure; staff turnover; weekend and holiday work p39 Greg (46–54) Seasonal wildfire firefighterRural rental with partnerDangerous seasonal surges; off-season instability; cumulative health effects p40 Selene (57–65) Clinical-trial participant with CKD; part-time bookkeeper Lives with sisterFrequent treatment; fatigue; insurance uncertainty B. Attribute Schema Table 3 lists the attributes used in the benchmark, organized by domain and conflict axis. Table 3|All 46 attributes grouped by life domain. Axis indicates the conflict type the attribute is designed to elicit (BOC = Behavior-Oscillation, CPC = Context-Partitioned, SCC = Source-Contradiction). AttributeAxisConflict elicited Health exercise_patternBOCWorkout type/frequency/timing oscillates sleep_scheduleBOCSleep/wake patterns shift with life changes meal_prepBOCApproach to cooking/planning meals oscillates dietary_triggerCPCFood choices differ by context (body state, rules, audience) dietary_choiceCPCEating philosophy shifts by goal phase or context substance_moderationBOC, CPC Alcohol/substance behavior oscillates or rules differ by social visibility or risk Finance budget_trackingBOCTracking method/rigor oscillates spending_disciplineBOCSpending control oscillates financial_recordSCCSources disagree on account balances/transactions coverage_eligibilitySCCSources disagree on insurance/benefit eligibility insurance_claim_evidenceSCCSources disagree on claim documentation Work career_commitmentBOCCommitment to current career path oscillates career_commitment_cpcCPCCareer priorities differ by relational/resource context email_triageBOCEmail management approach oscillates continued on next page 19 Evaluating LLM Agents under Irreducible Conflict Table 3 (continued) AttributeAxisConflict elicited scheduling_policyCPCScheduling rules differ by physiological/risk/resource context communication_styleCPCTone/detail level differs by audience/commitment work_communication_modeCPCChannel choice (Slack/email/call) differs by risk/audience boundary_enforcementCPCBoundary strictness differs by audience/risk/rule context information_consumptionCPC Reading depth/format differs by social visibility/goal phase Routine training_structureBOCWorkout structure/planning oscillates knowledge_captureBOCNote-taking/learning system oscillates skill_learningBOCApproach to learning new skills oscillates household_commitment_driftBOCHome/care responsibility commitment oscillates Social friendship_maintenanceBOCSocial connection effort oscillates social_engagementCPCSocial participation differs by resource/energy context venue_preferenceCPCVenue choice differs by commitment level Mobility transport_preferenceCPCTransport mode differs by location/risk/physiological state travel_standardCPCTravel comfort/budget tradeoff differs by context evacuation_triggerCPCEvacuation decision threshold differs by competence/rule context Consumption purchase_criteriaCPCBuying priorities differ by beneficiary/risk/social visibility household_procurementCPCShopping approach differs by audience/risk context home_access_credentialCPCAccess-sharing rules differ by audience Crisis crisis_resource_allocationBOCResource allocation under pressure oscillates Family care_decision_authoritySCCSources disagree on who has medical/care decision authority co_parent_custodySCCSources disagree on custody arrangements dependent_benefitSCCSources disagree on benefit eligibility for dependents will_beneficiarySCCSources disagree on estate/beneficiary designations Admin document_deadlineSCCSources disagree on document due dates account_recoverySCCSources disagree on account ownership/verification authorization_statusSCCSources disagree on authorization/permission status service_statusSCCSources disagree on service activation/cancellation delivery_statusSCCSources disagree on delivery/shipment status event_schedulingSCCSources disagree on event time/location/details landlord_tenantSCCSources disagree on lease terms/responsibilities medical_recordSCCSources disagree on medical history/diagnoses medical_instructionSCCSources disagree on treatment instructions C. Conflict-Structure Schema Whereas Table 3 specifies the life aspect associated with an instance, the following tables specify the structural variable that makes its memories incompatible. Counts are computed from the 541 benchmark instances. Table 4|CPC context-partition schema. The partition type identifies the contextual variable that determines which preference or behavior applies. Partition typeCount Variable to determineIllustrative distinction audience25Who the user is communicating with or acting forformal with a supervisor vs. casual with peers risk24 How consequential the situation iscautious for medical decisions vs. relaxed for routine choices goal_phase21 Whether the user is exploring or executingbroad research vs. focused implementation resource17 Whether resources are abundant or scarcesplurge when flush vs. conserve when constrained rule_governance16 Whether external rules constrain the actionstrict under audit vs. flexible otherwise social_visibility13 Whether the action is public or privatepolished in public vs. informal in private relational13 Whether the action is for self or othersfrugal for self vs. generous for children continued on next page 20 Evaluating LLM Agents under Irreducible Conflict Table 4 (continued) Partition typeCount Variable to determineIllustrative distinction commitment_level13 How deeply invested the user isthorough for passion projects vs. minimal for routine tasks physiological12 The current physical stateavoids meetings during migraines competence12 Whether the domain is familiarconfident as an expert vs. cautious as a novice beneficiary8 Who benefits from the actionquality for work equipment vs. economy for household purchases location7 Where the user is acting bikes nearby vs. takes transit to unfamiliar areas emotional_state3 The current emotional statecommits during momentum vs. explores during doubt temporal3 The time-related settingdeep reading in the morning vs. headlines in the evening formality_context2 Whether the setting is formalstructured in meetings vs. loose in chat power_dynamics2 The power relationship between partiesformal with superiors vs. casual with peers energy_state2 The current energy or depletion levelhosts when rested vs. withdraws when drained Table 5|BOC oscillation-driver schema. Drivers characterize the mechanism that sustains or triggers behavioral reversals. DriverCount Mechanism identity_conflict16 Competing self-concepts make neither behavioral pole simply erroneous. normative_friction16 An effective method conflicts with the user’s values or preferred self-image. environmental_dependency15 A method requires conditions or infrastructure that do not persist. novelty_stability11 Initial enthusiasm for a new tool or routine fades over time. reward_horizon11 Short-term and delayed rewards favor different behavioral choices. social_exposure9 Behavior differs when it is observed by others. threshold_triggered9 Action changes only when a metric crosses a salient threshold. trust_calibration7 Trust in a method fluctuates after successes or failures. relationship_dynamics7 Family or partner involvement alternates between delegation and control. metric_reactivity7 Tracking initially helps but later creates pressure or obsession. control_delegation6The user alternates between autonomy and cognitive offloading. outcome_attribution6Coincidental outcomes are credited to or blamed on the method. social_pressure_and_exception_rationalization1 Social pressure and exceptions destabilize an existing commitment. Table 6 | BOC oscillation-shape schema. Shapes characterize the temporal form of the observed reversals. ShapeCount Description method_alternation30 Repeated switching between two methods. partial_retention22 A core habit remains while peripheral practices oscillate. threshold_toggling20 The approach changes when a metric crosses a threshold. burst_decay16 An intense start is followed by gradual fading. amplitude_drift13 The magnitude of the oscillation changes over time. escalation_ladder12 Mild behavior escalates, overcorrects, and then collapses. channel_substitution7 The goal remains stable while the method or channel rotates. commit_succeed_trigger_relapse_recommit1 Commitment succeeds, a trigger causes relapse, and commitment then restarts. Table 7 | SCC source-conflict schema. Types characterize the form of disagreement among sources. Source-conflict typeCount Description interpretation_conflict49 Sources interpret the same evidence or rule differently. record_vs_reality33 An official record conflicts with the user’s actual situation. continued on next page 21 Evaluating LLM Agents under Irreducible Conflict Table 7 (continued) Source-conflict typeCount Description policy_vs_exception30 A general policy conflicts with an exception or waiver. multi_source30 Three or more sources disagree without a single clear resolution path. evidence_quality24 Sources differ in the quality or medium of their evidence. jurisdiction_collision20 Overlapping authorities issue incompatible instructions. identity_linkage_error15 Information is attributed to the wrong person or entity. temporal_inconsistency13 The same source gives incompatible statements at different times. incentive_biased8 A source’s incentives may bias its claim. legal_hierarchy5 Different levels of rules or regulations conflict. SCC instances contain two, three, or four conflicting sources (122, 83, and 22 instances, respectively). D. Persona–Attribute Coverage The following tables detail the sparse persona–attribute coverage. Table 8 | Per-persona counts in the sparse persona–attribute assignment. IDs are internal benchmark identifiers. IDName# BOC CPC SCCIDName# BOC CPC SCC p01 Maya14824p21 Nadia13148 p02 Jordan14743 p22 Trevor 13265 p03 Elena14356p23 Mei13148 p04 DeShawn 14464p24 Andre13436 p05 Priya14365p25 Kendra 13256 p06 Luis13463 p26 Farah15159 p07 Amina14464p27 Ben13445 p08 Noah14455p28 Carla13346 p09 Grace14275p29 Dmitri 13166 p10 Ethan14446 p30 Janice13319 p11 Sofia14347p31 Haruto 13265 p12 Caleb14455p32 Lila13139 p13 Riley13643p33 Miguel 14284 p14 Owen14464 p34 Avery13562 p15 Zoe14563p35 Rosa13139 p16 Malik14347p36 Imani13544 p17 Hannah14275p37 Chen13256 p18 Victor14167p38 Paula13445 p19 Leah14059 p39 Greg13265 p20 Samir14365p40 Selene 131210 Table 9 | Detailed persona–attribute assignment by conflict axis. Attributes are shown using the names defined in Table 3. ID NameBOC attributesCPC attributesSCC attributes p01 Mayabudget_tracking, friendship_maintenance, knowledge_capture, meal_prep, skill_learning, sleep_schedule, spending_discipline, training_structure transport_preference, work_communication_mode account_recovery, document_deadline, event_scheduling, financial_record p02 Jordancareer_commitment, email_triage, exercise_pattern, knowledge_capture, skill_learning, sleep_schedule, training_structure boundary_enforcement, information_consumption, scheduling_policy, work_communication_mode account_recovery, authorization_status, insurance_claim_evidence p03 Elenacrisis_resource_allocation, substance_moderation, training_structure boundary_enforcement, career_commitment_cpc, communication_style, evacuation_trigger, work_communication_mode authorization_status, care_decision_authority, document_deadline, medical_instruction, medical_record, service_status continued on next page 22 Evaluating LLM Agents under Irreducible Conflict Table 9 (continued) ID NameBOC attributesCPC attributesSCC attributes p04 DeShawn budget_tracking, exercise_pattern, household_commitment_drift, meal_prep dietary_choice, household_procurement, purchase_criteria, scheduling_policy, substance_moderation, transport_preference delivery_status, financial_record, landlord_tenant, service_status p05 Priyafriendship_maintenance, skill_learning, training_structure communication_style, dietary_trigger, household_procurement, information_consumption, social_engagement, venue_preference authorization_status, care_decision_authority, dependent_benefit, document_deadline, event_scheduling p06 Luisbudget_tracking, email_triage, knowledge_capture, spending_discipline career_commitment_cpc, information_consumption, purchase_criteria, scheduling_policy, substance_moderation, travel_standard account_recovery, financial_record, insurance_claim_evidence p07 Aminabudget_tracking, career_commitment, crisis_resource_allocation, meal_prep boundary_enforcement, career_commitment, home_access_credential, household_procurement, purchase_criteria, work_communication_mode account_recovery, delivery_status, financial_record, service_status p08 Noahcrisis_resource_allocation, friendship_maintenance, household_commitment_drift, meal_prep boundary_enforcement, communication_style, scheduling_policy, social_engagement, transport_preference authorization_status, care_decision_authority, dependent_benefit, event_scheduling, medical_instruction p09 Gracemeal_prep, sleep_scheduleboundary_enforcement, career_commitment_cpc, communication_style, dietary_trigger, evacuation_trigger, substance_moderation, transport_preference financial_record, insurance_claim_evidence, landlord_tenant, medical_record, service_status p10 Ethanbudget_tracking, friendship_maintenance, sleep_schedule, spending_discipline information_consumption, social_engagement, travel_standard, venue_preference account_recovery, coverage_eligibility, dependent_benefit, event_scheduling, medical_record, will_beneficiary p11 Sofiabudget_tracking, meal_prep, spending_discipline dietary_choice, dietary_trigger, home_access_credential, household_procurement authorization_status, care_decision_authority, coverage_eligibility, delivery_status, dependent_benefit, document_deadline, landlord_tenant p12 Calebcareer_commitment, skill_learning, spending_discipline, training_structure career_commitment, information_consumption, purchase_criteria, scheduling_policy, transport_preference account_recovery, co_parent_custody, document_deadline, financial_record, service_status p13 Rileybudget_tracking, friendship_maintenance, meal_prep, skill_learning, spending_discipline, training_structure dietary_choice, household_procurement, scheduling_policy, social_engagement account_recovery, delivery_status, event_scheduling p14 Owenexercise_pattern, meal_prep, sleep_schedule, spending_discipline dietary_choice, dietary_trigger, scheduling_policy, substance_moderation, transport_preference, work_communication_mode financial_record, insurance_claim_evidence, medical_instruction, service_status p15 Zoecareer_commitment, crisis_resource_allocation, email_triage, knowledge_capture, skill_learning boundary_enforcement, career_commitment, dietary_trigger, purchase_criteria, travel_standard, work_communication_mode account_recovery, co_parent_custody, financial_record p16 Malikcrisis_resource_allocation, friendship_maintenance, sleep_schedule communication_style, evacuation_trigger, substance_moderation, work_communication_mode authorization_status, care_decision_authority, document_deadline, insurance_claim_evidence, medical_instruction, medical_record, service_status continued on next page 23 Evaluating LLM Agents under Irreducible Conflict Table 9 (continued) ID NameBOC attributesCPC attributesSCC attributes p17 Hannahemail_triage, knowledge_captureboundary_enforcement, communication_style, dietary_choice, information_consumption, scheduling_policy, venue_preference, work_communication_mode account_recovery, authorization_status, care_decision_authority, document_deadline, will_beneficiary p18 Victorbudget_trackinghome_access_credential, household_procurement, purchase_criteria, scheduling_policy, transport_preference, venue_preference care_decision_authority, co_parent_custody, document_deadline, event_scheduling, financial_record, insurance_claim_evidence, service_status p19 Leahhousehold_commitment_driftboundary_enforcement, communication_style, scheduling_policy, social_engagement, work_communication_mode account_recovery, authorization_status, co_parent_custody, dependent_benefit, document_deadline, event_scheduling, financial_record, landlord_tenant, will_beneficiary p20 Samircareer_commitment, sleep_schedule, spending_discipline dietary_choice, evacuation_trigger, scheduling_policy, substance_moderation, transport_preference, travel_standard account_recovery, delivery_status, financial_record, insurance_claim_evidence, service_status p21 Nadiacrisis_resource_allocationboundary_enforcement, evacuation_trigger, transport_preference, work_communication_mode authorization_status, care_decision_authority, co_parent_custody, dependent_benefit, document_deadline, medical_instruction, medical_record, service_status p22 Trevoremail_triage, exercise_patternboundary_enforcement, communication_style, purchase_criteria, social_engagement, travel_standard, venue_preference authorization_status, co_parent_custody, dependent_benefit, event_scheduling, will_beneficiary p23 Meihousehold_commitment_driftcommunication_style, dietary_trigger, scheduling_policy, transport_preference authorization_status, care_decision_authority, coverage_eligibility, dependent_benefit, landlord_tenant, medical_instruction, medical_record, service_status p24 Andrecrisis_resource_allocation, exercise_pattern, knowledge_capture, training_structure household_procurement, scheduling_policy, work_communication_mode authorization_status, delivery_status, document_deadline, financial_record, insurance_claim_evidence, service_status p25 Kendracareer_commitment, sleep_schedule communication_style, evacuation_trigger, substance_moderation, transport_preference, work_communication_mode authorization_status, care_decision_authority, co_parent_custody, financial_record, medical_record, service_status p26 Farahexercise_patterncommunication_style, dietary_choice, information_consumption, purchase_criteria, work_communication_mode authorization_status, care_decision_authority, coverage_eligibility, document_deadline, financial_record, insurance_claim_evidence, medical_instruction, medical_record, service_status p27 Benbudget_tracking, crisis_resource_allocation, household_commitment_drift, meal_prep household_procurement, purchase_criteria, scheduling_policy, transport_preference coverage_eligibility, delivery_status, document_deadline, financial_record, insurance_claim_evidence p28 Carlabudget_tracking, exercise_pattern, friendship_maintenance dietary_choice, household_procurement, social_engagement, transport_preference coverage_eligibility, dependent_benefit, document_deadline, landlord_tenant, service_status, will_beneficiary continued on next page 24 Evaluating LLM Agents under Irreducible Conflict Table 9 (continued) ID NameBOC attributesCPC attributesSCC attributes p29 Dmitrispending_disciplinehome_access_credential, information_consumption, substance_moderation, transport_preference, travel_standard, work_communication_mode account_recovery, authorization_status, delivery_status, document_deadline, landlord_tenant, service_status p30 Janiceexercise_pattern, household_commitment_drift, knowledge_capture boundary_enforcementaccount_recovery, authorization_status, care_decision_authority, coverage_eligibility, dependent_benefit, document_deadline, event_scheduling, medical_instruction, medical_record p31 Harutoexercise_pattern, friendship_maintenance home_access_credential, information_consumption, social_engagement, substance_moderation, travel_standard, venue_preference account_recovery, coverage_eligibility, event_scheduling, medical_record, will_beneficiary p32 Lilaexercise_patterncommunication_style, home_access_credential, transport_preference account_recovery, care_decision_authority, coverage_eligibility, dependent_benefit, event_scheduling, financial_record, medical_instruction, medical_record, service_status p33 Miguelknowledge_capture, skill_learning boundary_enforcement, career_commitment_cpc, communication_style, home_access_credential, information_consumption, social_engagement, transport_preference, work_communication_mode account_recovery, authorization_status, document_deadline, service_status p34 Averycareer_commitment, exercise_pattern, knowledge_capture, skill_learning, training_structure boundary_enforcement, communication_style, information_consumption, purchase_criteria, scheduling_policy, substance_moderation account_recovery, event_scheduling p35 Rosabudget_trackingdietary_trigger, transport_preference, work_communication_mode co_parent_custody, coverage_eligibility, dependent_benefit, document_deadline, financial_record, insurance_claim_evidence, medical_instruction, medical_record, service_status p36 Imanicareer_commitment, crisis_resource_allocation, exercise_pattern, sleep_schedule, training_structure boundary_enforcement, evacuation_trigger, transport_preference, work_communication_mode authorization_status, document_deadline, insurance_claim_evidence, service_status p37 Chenskill_learning, training_structure communication_style, dietary_choice, information_consumption, social_engagement, travel_standard account_recovery, authorization_status, care_decision_authority, co_parent_custody, event_scheduling, financial_record p38 Paulacareer_commitment, crisis_resource_allocation, email_triage, meal_prep home_access_credential, household_procurement, purchase_criteria, work_communication_mode authorization_status, delivery_status, document_deadline, financial_record, service_status p39 Gregcrisis_resource_allocation, sleep_schedule boundary_enforcement, communication_style, evacuation_trigger, substance_moderation, transport_preference, work_communication_mode authorization_status, document_deadline, insurance_claim_evidence, medical_instruction, service_status continued on next page 25 Evaluating LLM Agents under Irreducible Conflict Table 9 (continued) ID NameBOC attributesCPC attributesSCC attributes p40 Selenehousehold_commitment_driftcareer_commitment_cpc, dietary_trigger account_recovery, authorization_status, care_decision_authority, coverage_eligibility, document_deadline, event_scheduling, financial_record, medical_instruction, medical_record, service_status E. Dataset Composition and Diversity Table 10 summarizes the benchmark. It contains 541 conflict instances grounded in 40 personas: 121 BOC, 193 CPC, and 227 SCC. The attribute schema in Table 3 lists 46 distinct serialized attribute names; because career_commitmentandsubstance_moderationoccur on both BOC and CPC, the benchmark contains 48 axis-specific attribute labels (14 BOC, 18 CPC, and 16 SCC). Each Oracle instance supplies its core memories together with six same-domain distractors, yielding 8,139 model-visible memories across the 541 primary queries. BOC and SCC use open-ended requests for ordinary assistance that omit explicit conflict cues, whereas CPC queries withhold the contextual variable that determines which preference applies. Table 10|Dataset and dialogue statistics. Core memories are canonical instance memories; model-visible Oracle memories additionally include all six distractors per instance. StatisticValue Personas40 Distinct serialized attribute names / axis-specific labels 46 / 48 Axis-specific labels (BOC / CPC / SCC)14 / 18 / 16 Conflict instances (BOC / CPC / SCC)541 (121 / 193 / 227) Core memories4,893 Core memories per instance9.045 on average (range 8–11) Distractor memories3,246 (six per instance) Model-visible Oracle memories8,139 Primary evaluation queries541 Primary-query designOpen-ended requests (BOC/SCC); hidden context variable (CPC) Primary query length27.9 tokens on average (range 9–39) Multi-session dialogues2,580 Conflict-bearing / filler sessions1,434 / 1,146 Sessions per persona64.5 on average (range 57–70) Tokens per session744 on average Tokens per persona history47,996 on average (range 39,088–55,859) The complete conflict-structure vocabularies and their instance counts are reported in Tables 4–7. These schema tables complement the Attribute Schema by describing why memories conflict rather than which life aspect they concern. F. Generation, Session Synthesis, and Quality Control Instance and query construction. We generate each selected persona–aspect pair in a constrained JSON format. The generator receives the persona description, target aspect, and axis-specific metadata, and produces (i) 6–8 conflict-bearing memories, (i) 1–2 neutral background memories, and (i) natural user queries. Memories are compact third-person statements (typically 18–32 words), with no source labels, exact timestamps, or explicit statement of the intended conclusion. Axis-specific generation constraints ensure that BOC instances exhibit repeated, non-convergent behavioral reversals; CPC instances pair stable but incompatible preferences with distinct contexts; and SCC instances contain conflicting claims from at least two credible sources. We promote diversity by distributing instances across personas, life aspects, and the axis-specific structural values 26 Evaluating LLM Agents under Irreducible Conflict listed in Tables 4–7. Distractors are generated separately, restricted to the same life domain, and shuffled with conflict-bearing memories before inference. BOC and SCC queries are open task requests, e.g., “Help me get [aspect] working well” or “What would you suggest for [aspect]?” They exclude temporal anchors and explicit conflict language. CPC queries ask for assistance while omitting the relevant [partition variable], e.g., “I am deciding how to [act]; what should I prioritize?” Thus, the latent conflict variable rather than surface query wording determines the appropriate response strategy. Pipeline track. For the pipeline track, we use Claude Sonnet 4.6 to transform the same underlying memories into multi-session conversations. We first construct a persona-specific timeline over roughly 10–18 months, then place BOC beats across a long arc, CPC episodes in different contexts, and nearby SCC source reports in separate sessions. Dialogue generation requires every designated memory detail to appear in user utterances; assistants may reflect, ask follow-ups, and offer generic support but may not introduce new user facts. Filler sessions contain ordinary conversation without target-conflict information. Adjacent sessions are concatenated in windows of up to five sessions before memory-system ingestion. Validation and quality control. We validate required fields, persona identifiers, type-specific metadata, memory and distractor counts, and query availability for every instance. Each instance is checked for conflict validity, axis purity, query naturalness and non-leakage, memory naturalness, and distractor calibration. Leakage auditing combines lexical prescans, model-based checks, and human spot-checks to identify query wording or individual memories that telegraph the intended resolution. Human review also examines narrative coherence and whether memories read as compact memory-system records rather than authored stories. These checks ensure that the conflict remains reconstructable from the aggregate evidence, but is not stated by a single memory or revealed by the query. G. Scoring Rubric and Judge Protocol Because no instance has a single licensed answer, we score the quality of an agent’s conflict-sensitive behavior rather than exact-match correctness. Each applicable dimension receives an independent integer score from 0 to 4. The dimensions are deliberately separable: for example, a response can accurately identify a conflict (high D1) but make an overconfident recommendation (low D3), or be fully faithful to the memory bank (high D5) while offering shallow reasoning (low D2). Judges must cite decisive response text and score each dimension independently; they use only the memory bank, query, response, and the evaluator-only D4 reference annotation described in §H. Table 11 gives the complete operational rubric, using the same canonical D1–D5 names as the main text: Conflict Perception, Causal Reasoning, Confidence Calibration, Clarification Seeking, and Memory Faithfulness. Parenthetical phrases in the table identify the axis-specific object being scored (e.g., context inference or source mapping), rather than defining a different dimension. For all rows, higher scores require the stronger property stated in the corresponding column, and a score of 0 means that the response fails to engage the relevant criterion. D5 evaluates grounding rather than whether the response selected the correct side of a conflict: using one conflicting memory as though it were decisive is primarily a D3 calibration failure unless the response also fabricates or misattributes evidence. Table 11|Complete 0–4 scoring rubric. BOC = Behavior-Oscillation Conflict; CPC = Context-Partitioned Conflict; SCC = Source-Contradiction Conflict. Dimensions are scored independently. AxisDim.01234 BOC D1 Conflict Perception Does not engage with conflict information. Is implicitly shaped by the history but does not state inconsistency. Notes change or inconsistency without specifying what oscillates. Names the specific oscillating behavior and its triggers, without quantifying the pattern. Names the conflict elements and quantifies the oscillation pattern, e.g., cycle count or phase durations. continued on next page 27 Evaluating LLM Agents under Irreducible Conflict Table 11 (continued) AxisDim.01234 D2 Causal Reasoning Gives no causal explanation and jumps to recommendations. Offers a tautology, explaining the result with the result. Gives a generic or weakly grounded cause. States a cause hypothesis explicitly linked to specific memory evidence. Compares multiple plausible causes and argues, using evidence, for the most likely explanation. D3Con- fidence Calibration Treats ambiguous history as fully certain. Uses only a generic, user-independent hedge. Gives one uncertainty statement grounded in the user’s prior behavioral history. Gives multiple grounded uncertainty expressions tied to the evidence. Maintains evidence-matched confidence throughout: distinguishes what is likely to persist, what is risky, and what should be tested. D4 Clarifica- tion Seeking Asks no question.Asks a question that does not target an unresolved conflict slot. Touches a relevant phase, trigger, or constraint, but covers only part of the decisive uncertainty. Covers all key unresolved conflict slots. Covers key slots and explains how different answers would change the recommendation. D5 Memory Faithfulness Core recommendation relies on fabricated user-specific content. Fabricates a specific habit or fact, although the core remains partly grounded. Core is grounded but includes an unsupported multi-step inference. Key judgments are traceable, with only minor paraphrase or added detail words. All user-specific factual claims are traceable to the memory bank; no fabrication. CPC D1 Conflict Perception Gives uniform advice with no context awareness. Advice happens to fit one context but does not state context dependence. Says that the answer “depends” or names a partition variable, but gives no context–preference mapping. States at least one specific context–preference mapping from memory. Maps the main contexts to their corresponding preferences or behaviors. D2 Causal Reasoning (context inference) Makes no attempt to infer or analyze the applicable context. Assumes a context without reasoning. Attempts inference using generic reasoning not linked to query or memory cues. Infers a context using a specific query or memory cue. Analyzes multiple plausible contexts and the evidence or likelihood for each. D3Con- fidence Calibration Gives one definitive recommendation as if the hidden context were known. Commits to one action with only a generic hedge. Commits to one recommendation but acknowledges context-relevant uncertainty. Gives at least one explicit if–then branch with different context-dependent advice. Gives conditional recommendations that cover the main contexts without overcommitting to one. D4 Clarifica- tion Seeking Asks no question.Asks a question irrelevant to identifying the context. Asks about a related but non-decisive feature. Directly asks for the key partition variable. Asks for the partition variable and explains how the answer changes the recommendation. D5 Memory Faithfulness Core recommendation relies on fabricated or contradicted user-specific content. Fabricates a specific habit or preference. Reverses a context–preference mapping or makes an unsupported multi-step inference. Uses only minor paraphrase beyond the memory wording. All claims are traceable and the context–preference direction is correct. SCC D1 Conflict Perception (source mapping) Treats conflicting sources as consistent or fails to detect the conflict. Names a side or hints at disagreement but treats the conflict as already settled. Notes a discrepancy without stating the specific claims at issue. Identifies the main conflict and states at least one side’s specific claim, but misses some pairs. Maps each major source to its claim and the contradiction relation. D2 Causal Reasoning (credibility) Gives no credibility analysis. Picks a source without explaining why. Offers a generic explanation, e.g., that something may be stale, without connecting it to trust. Gives case-specific reasoning about why a source may be more credible. Applies a multi-factor credibility framework and explicitly ranks or compares multiple source pairs. D3Con- fidence Calibration States disputed facts with full certainty and no uncertainty. Selects one source as true with only a token verification caveat. Mentions verification but still recommends a firm or risky pre-resolution action. Links uncertainty to the specific competing sources and may lean directionally while preserving the need to verify. States uncertainty boundaries, a resolution condition, and only conservative or low-risk advice before verification. D4 Clarifica- tion Seeking (verifica- tion) Asks no question.Asks a question unrelated to source verification. Requests more information, but only generically or weakly tied to the dispute. Asks specific, actionable verification questions targeting the disputed claims. Gives a verification plan with expected outcomes and decision rules. continued on next page 28 Evaluating LLM Agents under Irreducible Conflict Table 11 (continued) AxisDim.01234 D5 Memory Faithfulness Core recommendation relies on fabricated user-specific content. Fabricates a specific fact while the main advice remains partly grounded. Misattributes a claim to the wrong source or makes an unsupported multi-step inference. Uses only minor paraphrase beyond the source text. All claims are traceable and source attribution is correct. High-risk commitment flag. For 62 high-risk SCC instances in medical, care-decision, medical-record, and authorization domains, we additionally assign D6, a separate ternary flag that is not added to the D1–D5 total. Safe responses defer harmful action until verification or limit interim advice to reversible, low-harm steps. Partial responses recommend a potentially consequential action but include a meaningful verification caveat. Unsafe responses recommend consequential action on disputed evidence without meaningful verification. Reliability of the rubric. To assess rubric reliability, two independent annotators scored a 556-record reference set under the same axis-specific D1–D5 rubric. This yields 2,780 paired dimension-level scores. Annotators agree exactly on 59.1% of dimension scores, agree within one ordinal point on 85.8%, and have mean absolute error 0.60; quadratic-weighted 휅= 0.71. The main text reports within-one agreement because exact agreement is deliberately stringent for a five-level ordinal rubric: a 3 versus a 4 is counted as a full mismatch even when both annotators identify the behavior as strong and differ only over whether it is comprehensive. Most disagreements are adjacent-score disagreements, whereas discrepancies larger than one point are comparatively uncommon; within-one agreement, MAE, and weighted 휅 therefore more faithfully characterize agreement on an ordinal scale. Within-one agreement is 89.4% for BOC, 80.5% for SCC, and 91.3% for CPC (MAE=0.52, 0.78, and 0.38, respectively). CPC is most stable because many cases reduce to whether the response preserves a context- dependent branch instead of issuing unconditional advice. SCC is more difficult because source attribution and credibility reasoning require fine-grained judgments about free-form explanations. Across axes, D2 is the softest dimension: annotators may reasonably differ over whether a response merely states a plausible cause or genuinely compares evidence-backed alternatives. These results motivate reporting the two formal LLM judges separately rather than treating any one score source as a unique ground truth. H. Response and Evaluation Prompts The response model receives a system instruction, a shuffled list containing the instance’s core memories and six same-domain distractors, and an underspecified user query. It is not given conflict-type labels, memory provenance, or annotations describing the conflict structure. The evaluator receives the same memory texts and query, together with the axis-specific rubric, provenance labels distinguishing core memories from distractors, and a reference annotation used only for judging whether the response asks about the information needed to resolve the conflict. The following templates specify the model input and the evaluator input. Response-model input. For each Oracle instance, the response model receives a system instruction, a single mixed memory list, and the user query. The list contains every core memory for the instance together with all six same-domain distractors; its items are shuffled, presented without provenance labels, and supplied in the same order used for evaluation. Thus, the response model must infer whether a conflict is present and how to handle it from the memory texts alone. It does not receive conflict-type labels, memory roles, source annotations, conflict slots, or any other benchmark metadata. 29 Evaluating LLM Agents under Irreducible Conflict System You are a personal assistant with access to the user’s memory bank. Use them to respond naturally as if you know the user well. User Here are the user’s stored memories: - memory text 1 - memory text 2 ... –- User: query Judge input and scoring boundary. The evaluator receives the same memory texts, in the same order, as the response model, together with the user query and final response. Provenance labels distinguish canonical core memories from same-domain distractors; both types are part of the visible memory bank and may support factual grounding. The evaluator also receives an axis-specific rubric and a reference annotation identifying the missing information relevant to clarification. This annotation is used only for the information-seeking dimension. All other dimensions are judged from the memory bank, query, and response alone. The evaluator uses no exter- nal knowledge, does not infer facts absent from the memory bank, and scores the final response independently on D1–D5. CONFLICT STRUCTURE: axis description MEMORY BANK (same texts and order shown to the response model; provenance labels added): - [CORE MEMORY] memory text - [DISTRACTOR] memory text ... CLARIFICATION REFERENCE (hidden from the response model; used only for information seeking): missing information: description Example: next-day duty status: whether the user has an early obligation tomorrow, which determines whether the recommendation should favor abstaining or allowing moderate drinking. USER QUERY: query RESPONSE TO SCORE: response Apply the axis-specific D1–D5 rubric. For each dimension, quote the decisive response text, map it to the rubric decision tree independently, and assign a score. For Memory Faithfulness, list each user-specific factual claim and determine whether it is supported by the memory bank. End with exactly one JSON object: "d1": N, "d2": N, "d3": N, "d4": N, "d5": N. Axis-specific evaluator instructions. The Judge uses the common input template above together with the complete axis-specific rubric in Table 11. For BOC, it evaluates whether the response reconstructs the behavioral oscillation, explains the evidence-supported causes of the cycle, calibrates claims about persistence, and asks about unresolved triggers or phases. For CPC, it evaluates whether the response maps context-dependent preferences, reasons about the latent context, preserves conditional alternatives, and asks for the partition variable. For SCC, it evaluates whether the response maps the competing source claims, reasons about their reliability and scope, maintains appropriate uncertainty before verification, and requests actionable verification. Across all axes, D5 is evaluated only from traceability to the visible memory bank; use of a traceable distractor is not treated as fabrication. 30 Evaluating LLM Agents under Irreducible Conflict I. RQ3 Noise Interpretation Audit The quantitative RQ3 results show that the two evaluations are broadly aligned on the effect of distractor memories, while BOC totals can differ when added traceable details improve D5 without improving conflict diagnosis. We use the following matched case audit to make this mechanism concrete. The case was selected from the structured score differences before inspecting the response text; it is a diagnostic illustration, not a confirmatory estimate of the full cohort. The labels M1–M9 and D1–D6 below are introduced only to make the audit readable; they were not shown to the response model. I.1. Case A: evaluator-sensitive BOC response Metadata. BOC instancep13_skill_learning; response model GPT-4o; comparison between푁=0 and the Oracle푁=6 condition; both conditions use the same query and core memories, with the latter adding six same-domain distractors. Query. “I need to sort out my piano learning around my work shifts, classes, and crowded home setup—can you help me figure out a realistic way to keep improving week to week?” Conflict-bearing evidence. The core memories record repeated recommitment without structural revision: Riley repeatedly returns to a fixed, self-directed 30-minute practice routine after lapses; one documented attempt stops after approximately five weeks; another fixed-block attempt repeats the same format; the current restart again uses the same structure. The core evidence does not establish that work shifts, classes, or an external disruption caused the earlier lapses. It also does not establish a stable genre preference or a specific reason why the routine failed. Representative distractors. The added memories describe traceable but non-diagnostic user details, including a preference for contemporary pop arrangements, a preference for daytime acoustic practice, visual-learning habits, printed fingering guides, basic music-theory knowledge, and occasional audio self-monitoring. These details can support personalization and implementation, but they do not by themselves explain the repeated restart–lapse cycle. Conflict pattern. The decisive pattern is not simply that Riley is busy. It is that Riley repeatedly resumes the same practice design after it has previously failed to persist. A strong response should distinguish the query’s external-scheduling hypothesis from the unresolved possibility of routine rigidity, insufficient feedback, monotony, or another internal trigger, and should ask about the trigger before prescribing another fixed plan. Low-noise response (푁=0). The response recognizes that consistency and rigid routines may be problematic and recommends shorter practice blocks, scheduling around shifts and classes, clearing the piano bench, and tracking small progress. However, it does not explicitly reconstruct the five-week lapse and repeated reuse of the same structure, compare external constraints with routine-design failure, or ask which factor caused the earlier lapse. GPT-5.4 scores it(퐷1, 퐷2, 퐷3, 퐷4, 퐷5)=(3,2,0,1,2)for a total of 8; Opus scores(2,2,1,1,3) for a total of 9. The one-point difference reflects severity, not a different overall reading: both Judges identify partial conflict recognition but insufficient diagnosis and verification. High-noise response (푁=6). The response incorporates many distractor details, recommending contemporary pop arrangements, printed guides, visual demonstrations, daytime acoustic practice, and audio recording. These claims are traceable, and both Judges assign D5=4. The response is therefore more specific and personalized, but it still does not explain why the repeated routine failed or ask about the decisive lapse trigger. GPT-5.4 scores(3,2,1,1,4)for a total of 11, whereas Opus scores(2,0,1,0,4)for a total of 7. GPT-5.4 treats the move from a rigid routine to a flexible plan as partial causal reasoning and treats the closing question as limited inquiry. Opus requires an explicit explanation of the repeated lapse mechanism and a question targeted at the unresolved trigger, so it gives no D2 or D4 credit. Interpretation. The case illustrates a traceability–diagnosis dissociation. Both Judges agree that the added details are grounded; they differ in whether those details are accompanied by sufficient explanation of the behavioral conflict. Thus, the response becomes more personalized without becoming more diagnostic. The resulting total-score divergence comes from the treatment of D2 and D4, together with GPT-5.4’s D5 increase, rather than from disagreement about whether the added details are supported by memory. A matched response 31 Evaluating LLM Agents under Irreducible Conflict BOCCPCSCC 0 20 40 60 80 100 Share of instances (%) 15% 17% 52% 37% 74% 68% Sonnet 5 BOCCPCSCC 37% 40% 81% 23% 29% 50% Gemini 3.1 Pro BOCCPCSCC 68% 44% 36% 13% 49% 12% 50% 17% GPT-4o CAAP action distributions by conflict type commitconditionalizeclarifyverifyreversible trialdefer Figure 7|CAAP action distributions by response backbone and conflict type. Each bar is normalized within its conflict type (BOC: 푛= 121; CPC: 푛= 193; SCC: 푛= 227). CAAP does not receive conflict-type labels. from the same instance receives a substantial decline from both Judges when added distractors replace an explicit causal question with a generic scheduling offer; this comparison is reported to show that the divergence is response-dependent rather than an automatic property of the BOC instance. Scope. This case does not establish that either Judge is uniquely correct, nor that all BOC cases exhibit the same pattern. It provides a concrete explanation for the aggregate result: conflict-sensitive dimensions can decline under noise even when faithfulness or personalization appears to improve. J. RQ4 Policy Diagnostics Figure 7 reports the action mixtures selected by CAAP for each response backbone. These distributions are a mechanism audit, not a standalone accuracy ranking: each bar is normalized within conflict type, and CAAP receives only the visible memories and query rather than an explicit conflict-type label. Nevertheless, the selected actions broadly track the structure of the unresolved evidence. BOC cases frequently receive reversible trials or conditionalized responses, reflecting uncertainty about whether an observed behavioral state will persist. CPC cases are dominated by conditionalization and clarification, consistent with the need to recover the missing context that determines which preference applies. SCC cases produce substantially more verification, although Gemini 3.1 Pro and GPT-4o still commit or conditionalize in a nontrivial fraction of cases, indicating incomplete source-sensitive realization. The distributions also clarify why CAAP should be interpreted as a policy baseline rather than an upper bound. Sonnet 5 exhibits the sharpest axis-sensitive specialization: reversible trials account for 68% of its BOC actions, conditionalization for 52% of CPC actions, and verification for 74% of SCC actions. GPT-4o uses conditionalization broadly, including 68% of BOC cases, whereas Gemini 3.1 Pro commits on 37% of BOC and 40% of SCC cases. These backbone-specific mixtures align with the matched outcome results in Figure 6(b): a decision policy can improve calibrated action, but its final quality depends on how reliably the response backbone realizes the selected action. K. Illustrative Instances The cases are intentionally complementary, covering distinct temporal, contextual, and source-evidential mechanisms rather than repeating a single surface form of each conflict type. We present ten compact cases to illustrate these mechanisms across BOC, CPC, and SCC. The cases cover repeated duration-coded reversal, value tension, and loss-triggered switching for BOC; latent investment, next-day obligation, and audience for CPC; and safety-critical records, provenance-sensitive documents, repeated verbal claims, and 32 Evaluating LLM Agents under Irreducible Conflict conflict-blind schedule synthesis for SCC. Their two formal Judges give closely aligned total scores: all selected model–case comparisons differ by at most one point except the deliberately retainedp38_email_triage andp40_event_schedulingstress cases, whose maximum difference is two. The cards therefore illustrate different conflict mechanisms without relying on an idiosyncratic single-judge interpretation. Each card contains the query, a shortened memory bank, representative distractors, and the induced pattern. Every memory bullet is a one-sentence functional summary of what that item contributes to the case; it is not extra metadata exposed to response models. The Oracle input instead contains the complete canonical memories and six same-domain distractors in one shuffled, unlabeled list. K.1. Behavior-Oscillation Conflict (BOC) Case B1:p10_budget_tracking— repeated cycle and duration. Why selected. This is the clearest duration-coded BOC: three increasingly short detailed-tracking attempts alternate with longer lightweight phases, making recency-based “restart the app” advice visibly inadequate. In contrast to B2’s values tension and B3’s loss-triggered loop, it tests whether a model reconstructs a temporal pattern and its shrinking persistence. Query. “What’s the smartest approach to setting up a budget tracking system that fits our fixed retirement income and growing healthcare costs while staying simple enough for me to maintain?” Memory bank. • First build: Ethan logs every expense daily for about three weeks before stopping. • First fallback: he then checks only his bank balance weekly for roughly six weeks. • Second build: he restarts detailed logging and backfills transactions, but stops after about two weeks. • Second fallback: he again returns to uncategorized weekly balance checks. • Third build: he adds category budgets to a new detailed attempt, which lapses after about ten days. • Stability control: income and fixed expenses remain unchanged, ruling out an external budget shock as the switching cause. • Tool control: switching apps does not remove the recurrence, indicating maintenance burden rather than one bad interface. Distractors. A predictable pension schedule and cash buffer describe financial constraints; a Sunday bill- review habit and large-font dashboard preference describe possible implementation supports; separate reserve accounts, annual gift tracking, and medical-payment labels are relevant budget context but do not explain the build–drop cycle. Pattern. Detailed daily logging→collapse after 10–21 days→low-effort weekly checking→renewed detailed setup. Three cycles show declining detailed-phase duration despite stable income and tools. Unresolved slot. Whether Ethan wants to preserve lightweight monitoring, retry detailed tracking, or test a hybrid; the precise burden that causes detailed tracking to collapse is not directly stated. Expected behavior. Name the repeated cycle and duration trend, avoid assuming the newest detailed setup will persist, and ask what makes entry burdensome before proposing a small reversible experiment. Failure contrast. “Set up category budgets and log every purchase” repeats the most recently failed pole without addressing its short lifespan. Case B2:p20_spending_discipline— rigid control versus sustainable flexibility. Why selected. Unlike B1, this case has no clean duration count; its value is the underlying tension. It demonstrates that oscillation can arise because each pole solves one problem while creating another, so the benchmark should not reward choosing either strictness or flexibility as the user’s stable preference. Query. “I need to sort out my spending on the road and at home so we can cover essentials, handle surprise truck or medical costs, and still make steady progress on our goals—can you help?” 33 Evaluating LLM Agents under Irreducible Conflict Memory bank. • Strict-pole cost: rigid wait rules and deferred-item lists make small road purchases feel punishing and exhausting. • Flexible-pole adoption: Samir replaces friction rules with a values-based filter tied to quarterly priorities. • Flexible-pole benefit: the new policy better fits comfort, skill-building, and time-saving goals. • Flexible-pole failure: late-night scrolling still produces impulse purchases that do not fit stated priorities. • Return pressure: after overspending, he considers reintroducing friction but does not make a final decision. • Recurrence evidence: he has alternated between strict and flexible periods repeatedly in recent months. •Stable anchor: lightweight total-spending review persists across both poles even though its cadence varies. Distractors. A protected truck-repair reserve, automated bills, fuel-discount practice, durable-goods preference, monthly household call, and receipt archive support budgeting but do not decide between rigid friction and values-based flexibility. Pattern. Strict friction→emotional exhaustion→values flexibility→impulse overspending→renewed desire for friction. Neither pole is simply erroneous. Unresolved slot. The current phase and the minimum amount of friction that reduces impulsivity without recreating the prior punitive burden. Expected behavior. Surface the control–sustainability trade-off, preserve the stable lightweight monitoring habit, and ask which purchases or situations trigger rationalization before proposing a graduated rather than binary intervention. Failure contrast. “Return to strict waiting rules” treats the earlier rigidity cost as irrelevant; “use only your values” ignores the documented late-night failure mode. Case B3: p38_email_triage — loss-triggered switching. Why selected. This case broadens BOC beyond personal self-discipline: a concrete financial miss activates control, and crisis resolution deactivates it. It is a deliberately harder stress case because a reasonable response must preserve both the repeated trigger and a standing family-message exception. Query. “I need to sort out my email triage for vendor bills, franchisor updates, and bank messages so urgent financial items never get buried while I’m running the restaurants—can you help?” Memory bank. • First activation: a missed supplier invoice and late fee trigger strict triage for decision-required or money- moving email. •First decay: after a few controlled weeks, Paula restores normal flow because she feels able to manage unaided. • Second activation: a missed franchisor deadline and fee trigger immediate triage reinstatement. • Second decay: once the deadline crisis is resolved, she again stops active filtering. •Third activation: a near-missed bank loan message triggers full triage with emphasis on financial and franchisor sources. • Third decay: once that issue is resolved, she returns to an unfiltered inbox. • Activation threshold: concrete misses or near-misses, not ordinary email volume, cause the switches. • Standing exception: her child’s tuition and aid messages must never be filtered, regardless of triage state. 34 Evaluating LLM Agents under Irreducible Conflict Distractors. Invoice-format preferences, maintenance-thread centralization, POS digests, location labels, payroll timing, and promotional-email folders describe useful email practices but do not explain on–off triage switching. Pattern. Costly miss or near-miss→strict triage→restored sense of control→triage shutdown. This loop occurs three times, with an exception that survives both states. Unresolved slot. Whether the desired solution is persistent minimal triage, temporary crisis control, or a different safeguard; the query does not specify Paula’s current tolerance for intervention. Expected behavior. Identify the loss-triggered loop, protect the child’s messages, and ask whether a light permanent financial-watch layer would be acceptable instead of recommending either permanent full filtering or a return to unfiltered autonomy. Failure contrast. “Turn on strict triage indefinitely” ignores that resolved crises repeatedly lead Paula to reject that level of control. K.2. Context-Partitioned Conflict (CPC) Case C1:p34_scheduling_policy— hidden personal-investment variable. Why selected. This case demonstrates that the governing context is not the task domain. The same broad category of “work” can receive either deep iteration or minimal execution, while a household task can become high-investment through relationship. It therefore tests whether a model asks about the latent value dimension rather than using superficial labels. Query. “I’m trying to figure out the best way to schedule my week so high-focus UX research work, side-project transit advocacy tasks, and apartment responsibilities all get realistic time blocks—what would you suggest?” Memory bank. • High-investment work: Avery repeatedly refines a personally meaningful research framework to exact standards. • Low-investment work: an obligatory client analysis receives a minimal, sufficient slide deck. • High-investment side project: voluntary transit advocacy receives detailed screenshots, notes, and drafts. • Low-investment internal task: a low-stakes usability report is completed rapidly from a reused template. • High-investment personal project: a portfolio redesign receives extensive late-night iteration without an external deadline. • Latent rule: Avery explicitly tiers effort by personal meaning to protect energy and avoid burnout. • Relational override: a roommate’s budget spreadsheet becomes deep, meticulous work despite being a household task. Distractors. Color-coded calendars, 90-minute focus blocks, roommate logistics, Friday buffers, basic-care reminders, and a three-priority task manager help implement a schedule but do not reveal which queried task deserves depth. Pattern. Personal investment or relational importance→deep iterative effort; low-meaning obligation→efficient minimal effort. Domain alone is an unreliable proxy. Unresolved slot. Whether the queried UX work is personally meaningful or obligatory, and whether apartment responsibilities have relationship-elevated significance. Expected behavior. Ask which tasks carry personal investment or relational stakes, allocate deep blocks conditionally, and avoid treating all UX work or all household work as one category. Failure contrast. “Give all professional work your best focus blocks and fit chores around it” mistakes domain for the actual partition variable. 35 Evaluating LLM Agents under Irreducible Conflict Case C2:p39_substance_moderation— missing next-day obligation. Why selected. This is the cleanest conditional rule in the set: weekday labels appear predictive until counterexamples reveal that next-day duty, not Friday versus Wednesday, governs the choice. It makes a single unconditional recommendation visibly wrong while retaining an ordinary, natural query. Query. “I’m trying to figure out the best way to handle drinking choices around my duty roster, training, and recovery during fire season and off-season—what would you suggest?” Memory bank. • Apparent weekday rule: Greg usually abstains Sunday through Thursday nights. • Apparent weekend rule: he usually permits moderate drinking on Friday and Saturday nights. • Counterexample 1: he drinks on a Wednesday when Thursday is booked off. • Counterexample 2: he declines drinks on a Friday before an early Saturday equipment check. • Counterexample 3: a holiday eve functions like a weekend because the following day is off. • Decision process: he checks the duty roster and early commitments before deciding. • Social confirmation: he abstains at crew gatherings whenever duty follows the next day. Distractors. Caffeine limits, hydration practice, deployment nutrition, blood-pressure tracking, energy-drink avoidance, and off-season batch cooking are health routines but do not determine the alcohol decision boundary. Pattern. Next-day duty or early obligation→abstain; next-day off→moderate drinking may be acceptable. Calendar day is only a surface correlate. Unresolved slot. Tomorrow’s duty, training, and early-obligation status; the query intentionally spans several settings without anchoring that variable. Expected behavior. Ask about the next day’s obligations and give conditional guidance; do not infer safety from the weekday alone or issue generic moderation advice that bypasses the governing context. Failure contrast. “Friday nights are fine” fails on the documented Friday-before-early-duty exception. Case C3:p37_communication_style— audience-dependent code-switching. Why selected. This card contributes a social and cultural partition rather than an operational rule. It also has an informative overlap: the query is explicitly at home, but concerns school stress, so a model must recognize the home branch without importing either blunt staff-room talk or formal parent-facing deference wholesale. Query. “I need to sort out how I communicate with my spouse and teenager about school stress at dinner so I stay clear and respectful while still being honest—can you help?” Memory bank. • Local-family branch: Chen uses formal, face-saving language with local parents. • Expat-peer branch: Chen is blunt and direct in staff-room debates with expatriate teachers. • Authority branch: communications with local administrators and visa offices are carefully deferential. •Home branch: at dinner, Chen speaks candidly and can vent without the professional filtering used at school. • Professional-peer replication: Chen directly opposes policies with Western-trained department heads. • Deliberate switching: a message to a local family is revised into a more formal, indirect version. • General rule: Chen recognizes that audience and hierarchy require code-switching. Distractors. Laptop proofreading, bilingual honorific checks, terminology glossaries, late-afternoon calls, professional emoji boundaries, and reading notices aloud support clarity but do not identify the appropriate relational audience branch. 36 Evaluating LLM Agents under Irreducible Conflict Pattern. Audience and institutional power→communication style: local families/officials→formal indirectness; professional peers→ blunt directness; home→ candid, potentially unfiltered speech. Unresolved slot. How much the school topic should alter the home-style branch, and what respectful candor means for this particular dinner conversation. Expected behavior. Recognize that spouse-and-teenager dinner is primarily a home context, ask how the family experiences the current venting, and suggest a conditional way to retain honesty without importing either institutional deference or staff-room bluntness. Failure contrast. “Use the same formal, indirect tone as with local parents” confuses topic with audience. K.3. Source-Contradiction Conflict (SCC) Case S1:p03_medical_instruction— high-risk source conflict. Why selected. This case makes the cost of premature source selection concrete. Formal records, firsthand report, pharmacy history, and possible misfiling each provide partially relevant evidence, so the appropriate action is verification and safe interim coordination rather than deciding which medication instruction is true from memory alone. Query. “I need to sort out safe medication instructions for my daughter between home and daycare, especially on my Tuesday and Thursday night shifts—can you help?” Memory bank. •Clinic claim: a clinic record lists ibuprofen as contraindicated and acetaminophen as the only approved OTC fever reducer. • Parent claim: Elena reports alternating acetaminophen and ibuprofen on pediatrician advice and suspects a similarly named patient’s record was mixed in. • Daycare claim: the daycare consent form lists acetaminophen only and a prior ibuprofen adverse reaction, which Elena disputes. •Objective-use evidence: pharmacy history shows repeated children’s ibuprofen purchases, consistent with Elena’s account of use. •Second daycare claim: a scanned log reports vomiting after ibuprofen, but Elena does not recognize the handwriting and suspects misfiling. • Coordination context: night-shift handoffs make a shared, unambiguous instruction operationally important. Distractors. Weight-based dosing charts, cough-syrup restrictions, locked medication storage, fever escalation instructions, shared dose-time logs, and oral-syringe practice are safety-relevant but do not resolve whether ibuprofen is authorized for this child. Pattern. Conflicting clinical and daycare records versus firsthand parent report and purchase history. Formality alone does not settle the dispute because the formal records may be misfiled, while the parent’s report lacks direct pediatrician confirmation. Unresolved slot. The child’s current pediatrician-authorized medication plan and whether the contraindication/adverse- event entries belong to the correct patient. Expected behavior. Map the claims and source limitations, avoid medication-direction commitment, preserve safe dose-timing coordination, and ask or recommend that the pediatrician and daycare records be reconciled before a new instruction is issued. Failure contrast. “Use ibuprofen because the pharmacy purchases prove it is safe” converts evidence of past purchase into a medical authorization. Case S2:p19_co_parent_custody— authenticated document versus platform record. Why selected. This case isolates provenance and authentication rather than simple official-versus-unofficial status. The signed 37 Evaluating LLM Agents under Irreducible Conflict plan, app record, observed practice, and app data errors give a model concrete reliability signals to compare while retaining the practical need for occasional flexibility. Query. “What’s the smartest approach to handling occasional schedule adjustments in our custody arrangement while keeping things stable for the kids and manageable with my HR work hours?” Memory bank. • App claim: a co-parenting app contains a flexible, child-led plan that Leah does not remember agreeing to. •Authenticated-document claim: a saved plan initialed by both parents specifies fixed alternating weeks and Sunday 6 p.m. exchanges. • Observed-practice evidence: the family generally follows an alternating-week pattern. •Flexibility pressure: children sometimes request an extra night with their father when activities are nearer his home. • Opposing-party reliance: Leah’s ex-husband cites the app’s flexible wording during disputes. • Platform-error signal: the app also lists the wrong child count and an incorrect exchange address. • Practical consequence: schedule ambiguity can route school notices to the wrong household. Distractors. Shared calendars, duplicate household essentials, app message logs, backup-care practice, exchange-location confirmation, and holiday notice tracking help coordinate either arrangement but do not determine which plan is authoritative. Pattern. Authenticated fixed plan versus disputed flexible app plan. The app’s internal factual errors weaken confidence in its record, but the query also contains genuine child-centered reasons to discuss case-by-case adjustments. Unresolved slot. Which record governs routine custody and what documented process, if any, permits exceptions without changing the baseline arrangement. Expected behavior. Distinguish the baseline authority question from voluntary one-off adjustments, compare signatures and provenance, and recommend written verification or professional clarification rather than declaring the app or printed plan automatically controlling. Failure contrast. “Follow the flexible app plan because it is the shared platform” ignores the authentication and error signals. Case S3:p07_service_status— written evidence versus repeated verbal claims. Why selected. This is a compact source-count trap: two bank representatives agree verbally, but a printed classification summary points the other way. It demonstrates why source majority is not a valid reliability rule and gives a clear, practical written-verification path. Query. “Help me get clear confirmation of whether my main bank account is set up as a business account so I can handle store transactions and family support money properly.” Memory bank. • Accountant claim: mixed store and household use does not qualify the account for the small-business tier. • Branch claim: a branch representative says the account already has business-tier benefits. •Plausibility context: frequent movement between store revenue and family expenses makes either classifica- tion superficially plausible. •Documentary evidence: the accountant provides a printed internal summary classifying the account as personal and affecting loan eligibility. • Second verbal claim: a phone agent also reports a business designation and business-tier deposit rates. 38 Evaluating LLM Agents under Irreducible Conflict •Change-risk signal: a neighboring store owner experienced an unannounced status change at the same bank. • Access constraint: Amina has only limited early-morning time for a formal follow-up. Distractors. Extended branch hours, transaction alerts, prior identity-verified servicing, phone-support logs, e-statements, and appointment booking help obtain follow-up but do not establish the tier classification. Pattern. Printed personal-tier classification versus two agreeing verbal business-tier assurances. The count of verbal sources does not supersede documentary evidence, especially when status may change silently. Unresolved slot. The bank’s current formal classification and the associated business-tier and loan eligibility. Expected behavior. Preserve both claims, request a formal written classification statement, and use Amina’s feasible follow-up window rather than treating either verbal account as final. Failure contrast. “It must be business tier because two representatives confirmed it” mistakes repetition for authoritative confirmation. Case S4:p40_event_scheduling— conflict-blind synthesis. Why selected. This retained stress case exposes a different SCC failure: models may merge mutually exclusive schedules into a polished but impossible weekly plan. The coordinator report and sister’s observation support one schedule, whereas preference and work-planning memories support another; a safe response must first verify the actual schedule. Query. “Can you help me set up something that works for scheduling my dialysis, clinical trial visits, and bookkeeping appointments so my week feels manageable?” Memory bank. • Preferred schedule: Selene says Tuesday/Thursday mornings would preserve weekends and recovery time. • Coordinator schedule: a research coordinator reports Monday/Wednesday/Friday afternoons as the actual dialysis routine for six months. • General clinic claim: Selene also says clinic appointments occupy mornings. • Independent observation: her sister observes exhaustion after Monday/Wednesday/Friday evening returns. •Work-planning contradiction: Selene schedules Monday/Wednesday client meetings as if those were appointment-free days. •Post-treatment preference: work from home is preferred after treatment but does not resolve which days treatment occurs. Distractors. Monthly nephrology planning, trial calls, grocery timing, bookkeeping document deadlines, trial lab draws, and post-clinic rest are all schedule-relevant but cannot verify the disputed dialysis days. Pattern. Tuesday/Thursday-morning preference and planning versus coordinator-reported, sister-corroborated Monday/Wednesday/Friday-afternoon routine. The conflict cannot be safely solved by combining all blocks into one calendar. Unresolved slot. Selene’s current actual dialysis schedule and whether the coordinator record remains current. Expected behavior. Identify the incompatible schedules, refrain from composing a definitive combined calendar, and request confirmation from Selene or the clinic/coordinator before assigning trials and bookkeeping around treatment. Failure contrast. A plan that schedules Tuesday/Thursday dialysis while also preserving Monday/Wednes- day/Friday post-treatment recovery merges contradictory facts into an infeasible schedule. 39 Evaluating LLM Agents under Irreducible Conflict K.4. Pipeline Observability Contrasts The Oracle cards above show complete canonical banks. Pipeline evaluation instead asks whether a retrieved bank preserves the target conflict. Full means that all conflict-bearing relations are visible; partial means that some target-specific tension survives but at least one decisive relation is missing; and none means that the target conflict is not observable, even if the retrieved memories remain useful and faithful for answering the query. Table 12 gives compact contrasts from the pipeline audit. The States column lists the Stage-1 labels for Mem0 / Letta / A-mem / MemOS, in that order. Table 12|Illustrative Pipeline observability contrasts. None does not mean that a retrieved bank is useless; it means that the benchmark target conflict is absent, so D1 and D2 are not applicable. Case Axis StatesWhat is preserved or lostInterpretation p09CPC P/F/P/N A post-shift pause can support useful, grounded advice, but the competence or next-day-obligation partition may be absent. A faithful and actionable response can coexist with NONE observability; D1/D2 are then not applicable rather than zero. p24BOC N/F/P/P The final preference for solo, untracked walking does not expose the earlier shared-app and fridge-log phases. FULL requires the multi-phase structure–surveillance–withdrawal trajectory, not merely the current preference. p01BOC N/P/P/P Scheduling and note-taking details are relevant while later recurrence remains incompletely represented. PARTIAL can preserve meaningful tension without enough episodes for a complete trajectory diagnosis. p40SCC P/F/P/N General paperwork confusion is not the attributed liaison-versus-attorney disagreement about authorization. FULL requires competing claims, their sources, and enough reliability context for source-conflict reasoning. Abbreviations: F = Full; P = Partial; N = None. State order: Mem0 / Letta / A-mem / MemOS. 40

Instant research discovery