Paper deep dive
MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction
Ruoyu Wu, Shenfu Xie, Yinqian Sun, Haibo Tong, Feifei Zhao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/26/2026, 4:19:57 AM
Summary
MediSkill-Evo is a clinical agent framework that enables self-evolution of process-constrained, evidence-grounded knowledge without fine-tuning the backbone model. It utilizes a four-bank memory system (Clinical, Process, Symbolic, Visual) with type-specific validation and a Process-Constrained Preference Harness to select safe, evidence-backed actions. The system demonstrates significant improvements in diagnosis accuracy, treatment-intent coverage, and safety over prior agents on MIMIC-IV and NEJM benchmarks.
Entities (10)
Relation Signals (9)
MediSkill-Evo → evaluatedon → NEJM
confidence 95% · 100 multimodal NEJM image-diagnosis cases
MediSkill-Evo → evaluatedon → MIMIC-IV
confidence 95% · We evaluate on 300 MIMIC-IV-derived FullChain encounters
MediSkill-Evo → uses → Four-Bank Memory
confidence 95% · It realizes this self-evolution by updating clinical, process, symbolic, and visual knowledge in four typed banks
MediSkill-Evo → uses → Process-Constrained Preference Harness
confidence 95% · The Process-Constrained Preference Harness then turns validated knowledge into action
MediSkill-Evo → backedby → Qwen3.6-Flash
confidence 90% · On Qwen FullChain, MediSkill-Evo improves diagnosis accuracy...
MediSkill-Evo → backedby → DeepSeek-v4-Flash
confidence 90% · We evaluate two hosted backbone endpoints, Qwen3.6-Flash [21] and DeepSeek-V4-Flash [2].
MediSkill-Evo → integrates → MedSAM
confidence 90% · On multimodal NEJM diagnosis, MediSkill-Evo with optional MedSAM localization improves diagnosis accuracy
Process-Constrained Preference Harness → uses → Clinical Process Critic
confidence 90% · selects among valid alternatives through a safety-prioritized Clinical Process Critic.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Interactive clinical agents operate under partial observability, so reliable care depends on reaching the correct diagnosis through evidence-grounded, safe interactions. Yet existing agents struggle to convert experience into reusable process knowledge with explicit provenance and authority. To address this gap, we introduce MediSkill-Evo, which self-evolves governed process knowledge without fine-tuning the backbone. It realizes this self-evolution by updating clinical, process, symbolic, and visual knowledge in four typed banks under type-specific validation and scope rules. The Process-Constrained Preference Harness then turns validated knowledge into action by grounding candidates in evidence and prioritizing safer decisions. We evaluate on 300 MIMIC-IV-derived FullChain encounters, 180 hard-isolation conditions covering six process obligations, and 100 multimodal NEJM image-diagnosis cases. On Qwen FullChain, MediSkill-Evo improves diagnosis accuracy by 7.81% and treatment-intent coverage by 70.67% over the best-performing prior agent, while reducing critical failures by 43.04%. Under stress, it improves the stress-process composite by 7.77% and required-action completion by 12.41% over the best-performing agent for each metric, with stronger patient-fact, temporal-evidence, and triage-red-flag recovery and no controller-scored errors in unavailable-evidence, treatment, and triage safety checks. On multimodal NEJM diagnosis, MediSkill-Evo with optional MedSAM localization improves diagnosis accuracy by 2.56% and core score by 18.96% over the best-performing memory agent. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.23397v2
- Canonical: https://arxiv.org/abs/2608.23397v2
Trouble viewing inline? Open PDF directly →
Full Text
127,248 characters extracted from source content.
Expand or collapse full text
MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction Ruoyu Wu 1,2,3,* Shenfu Xie 1,2,3,* Yinqian Sun 1,2,3,4 Haibo Tong 1,2,3,5 Feifei Zhao 1,2,3,5,† 1 Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences; 2 Beijing Key Laboratory of Safe AI and Superalignment; 3 Beijing Institute of AI Safety and Governance; 4 School of Artificial Intelligence, University of Chinese Academy of Sciences; 5 Long-term AI Beijing, China Abstract Interactive clinical agents operate under partial observability, so reliable care depends on reaching the correct diagnosis through evidence-grounded, safe interactions. Yet existing agents struggle to convert experience into reusable process knowledge with ex- plicit provenance and authority. To address this gap, we introduce MediSkill-Evo, which self-evolves governed process knowledge without fine-tuning the backbone. It realizes this self-evolution by updating clinical, process, symbolic, and visual knowledge in four typed banks under type-specific validation and scope rules. The Process-Constrained Preference Harness then turns validated knowledge into action by grounding candidates in evidence and prioritizing safer decisions. We evaluate on 300 MIMIC-IV-derived FullChain encounters, 180 hard-isolation conditions covering six process obligations, and 100 multimodal NEJM image-diagnosis cases. On Qwen FullChain, MediSkill-Evo improves diagnosis accuracy by 7.81% and treatment-intent coverage by 70.67% over the best-performing prior agent, while reducing critical failures by 43.04%. Under stress, it improves the stress-process compos- ite by 7.77% and required-action completion by 12.41% over the best-performing agent for each metric, with stronger patient-fact, temporal-evidence, and triage-red-flag recovery and no controller- scored errors in unavailable-evidence, treatment, and triage safety checks. On multimodal NEJM diagnosis, MediSkill-Evo with op- tional MedSAM localization improves diagnosis accuracy by 2.56% and core score by 18.96% over the best-performing memory agent. Code is available at https://anonymous.4open.science/r/mediskill- evo_anonymous-68E7. Keywords clinical agents, self-evolving memory, process-constrained reason- ing, multimodal tool interface 1 Introduction Interactive clinical decision-making is the process of gathering ev- idence over multiple turns and using it to update diagnostic and treatment decisions. Since an agent observes only part of the pa- tient state, it must progressively reduce uncertainty by eliciting de- cisive history, requesting appropriate examinations, interpreting returned results, and revising its assessment before recommend- ing treatment under safety and urgency constraints. The reliabil- ity of this process depends on both the final diagnosis and the path taken to reach it. Unavailable evidence must remain unknown, mandatory care steps must be preserved, and a correct diagnosis * Ruoyu Wu and Shenfu Xie contributed equally to this work. † Feifei Zhao is the corresponding author. cannot justify unsupported or unsafe actions. Past encounters can guide future decisions, but transferred knowledge must retain its provenance and scope and receive only the decision authority war- ranted by its evidence. We therefore study how a clinical agent can self-evolve from prior trajectories without weakening the evidence boundaries and care obligations that make its actions trustworthy. Medical agents increasingly support interactive clinical work. AgentClinic [14] models partially observable doctor–patient– measurement encounters; MDAgents [5] adapts collaboration to case complexity; EHRAgent [15] and MMedAgent [6] connect models to executable EHR code and multimodal tools; Reflec- Tool [7] verifies tool use from experience; and MEDDxAgent [12] coordinates specialized modules for interactive differential diagno- sis. AI Hospital measures symptom collection, examination choice, and diagnosis in multi-turn simulation [26], while 3MDBench stud- ies multimodal telemedical dialogue [27]. Together, they establish interaction, specialization, tool use, and workflow-level evaluation as an emerging baseline rather than a contribution unique to this paper. Our narrower question is how trajectory-derived knowl- edge with different epistemic roles can be published and exercised through type-dependent validation and decision authority. Self-evolving agents provide a parameter-efficient route to this goal. Reflexion [ 16] stores verbal feedback and ExPeL [24] con- solidates cross-trial insights; Voyager [19], ICAL [13], and Agent Workflow Memory [20] distill executable skills, multimodal ab- stractions, or workflows. MemP [3] builds updateable procedural instructions, SkillWeaver [25] discovers reusable skills through practice, and MemBench separates factual and reflective memory while evaluating effectiveness, efficiency, and capacity [28]. These methods show that completed trajectories can become reusable ex- ternal knowledge without backbone updates. MediSkill-Evo does not claim the first structured memory or workflow evaluation; it proposes a particular complete-system interface in which arti- fact type controls validation, retrieval scope, and benchmark-time authority. Clinical interaction motivates preventing inferred, miss- ing, or tool-derived information from silently becoming observed fact. Agent harnesses determine whether typed knowledge actually changes behavior. ReAct [ 23] interleaves reasoning and envi- ronment actions; AgentBench [ 8] and AgentBoard [10] evaluate multi-step progress; AppWorld [18], T-Eval [1], ToolSandbox [9], and휏-bench [ 22] expose executable state transitions, tool policies, and interaction reliability. In medicine, the harness must addition- ally bind every result to a valid request, keep unavailable evidence unknown, preserve registered process obligations, and apply benchmark-defined safety checks without access to the hidden 1 arXiv:2608.23397v2 [cs.AI] 25 Aug 2026 Wu et al. Figure 1: Online inference and offline learning and governance in MediSkill-Evo. Left: the online pipeline retrieves four-bank context, generates and verifies candidate actions, executes the selected action, and incorporates environment feedback. Right: completed trajectories are reflected into typed memory operations, validated for label/evidence leakage, controller-defined safety, support, and replayability, and published as the next memory snapshot. Decision-critical Harness prompt (abridged) SYSTEM You are a strict Clinical Process Reward Model critic. Score candidate Doctor actions using only the visible dialogue state and general clinical safety. Diagnostic specificity and evidence alignment matter more than sounding complete. Do not use hidden gold diagnoses. Return strict JSON. RUNTIME INPUT state: <visible evidence, process ledger, requested tests, turn constraints> candidates: <structured next-action candidates> DECISION RULES 1. Prefer the action that resolves the most important remaining diagnostic or safety gap. 2. A test request contains one atomic item; penalize broad, packed, repeated, or low- value testing. 3. Never treat missing orRESULTS_UNAVAILABLEevidence as normal, negative, or reas- suring. 4. Strongly penalize missed dangerous diagnoses, delayed escalation, unsafe reassur- ance, and unmet treatment prerequisites. 5. On the final turn, onlyDIAGNOSIS_READYis valid; score diagnostic support, manage- ment, safety checks, and follow-up. OUTPUT ``scores′:[ ``candidate_index′:푖, ``process_score′:⋅, ``diagnostic_specificity_score′:⋅, ``evidence_alignment_score′:⋅, ``treatment_completeness_score′:⋅, ``safety_score′:⋅, ``triage_escalation_score′:⋅, ``test_efficiency_score′:⋅, ``constraint_violations′:[...], ``valid′:true|false ] Figure 2: The decision-critical Clinical Process Critic tem- plate. Runtime slots are populated at each turn; retry-only instructions are omitted. diagnosis. The proposed interface turns evolving knowledge into controller-valid, evidence-grounded action; it is not a claim of independent clinical legality or safety. To govern how experience becomes reusable and how it in- fluences subsequent decisions, we introduceMediSkill-Evo, a clinical agent framework that organizes evolving knowledge by its role in the care process. Completed trajectories propose updates to four typed banks for clinical strategies, process con- straints, evidence provenance, and visual procedures. Each bank follows type-specific validation and publication rules before its contents enter the next frozen snapshot. During an encounter, the Process-Constrained Preference Harness retrieves state-relevant knowledge, verifies candidate actions against symbolic and pro- cess constraints, and selects among valid alternatives through a safety-prioritized Clinical Process Critic. The central insight is that dependable memory requires knowledge type to determine its validation, retrieval scope, and decision authority. We evaluate MediSkill-Evo across two backbone endpoints and three complementary settings covering end-to-end FullChain encounters, controlled hard-isolation stress conditions, and multimodal NEJM diagnosis. Across FullChain, MediSkill-Evo consistently improves diagnosis accuracy and treatment-intent coverage while reducing critical failures relative to prior agents. Under stress, its clearest advantages lie in recovering patient facts, temporal evidence, and triage red flags. On multimodal NEJM cases, MediSkill-Evo with optional MedSAM localization also outperforms the strongest memory agent in diagnosis accuracy and core score. Our contributions are threefold: •We propose MediSkill-Evo, a complete clinical agent sys- tem that combines four-bank self-evolution with a Process- Constrained Preference Harness to govern how trajectory- derived knowledge is validated, retrieved, and authorized at decision time. 2 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction •We introduce FullChain for history-to-prescription inter- action, a controlled hard-isolation benchmark for care- process failures, and a request-gated NEJM visual-tool interface for multimodal diagnosis. •Across two backbone endpoints and three complementary evaluation settings, complete-system comparisons report gains in diagnosis, treatment-intent coverage, evidence acquisition, process correctness, and automatically scored safety over prior agents. 2 Method 2.1 Overall Architecture MediSkill-Evo supports partially observable, interactive clinical de- cision making while continually revising external knowledge from completed training trajectories rather than updating backbone pa- rameters. At turn푡, the visible state contains only the presented case information, current observation, dialogue history, and ex- aminations that have been requested and returned; reference diag- noses, evaluator labels, and unrevealed results remain inaccessible. Given the four-bank snapshotℬ 푒 published after evolution stage 푒, the system retrieves relevant, provenance-traceable knowledge. The Doctor uses the resulting augmented state푠 푡 to ask a history question, request an examination, or submit a clinical plan, and new evidence enters the next state: 푣 푡 = (푥 vis , 표 푡 , ℎ 푡 , 푧 푡 ),푢 푡 =ℛ(푣 푡 ;ℬ 푒 ),푠 푡 = (푣 푡 , 푢 푡 ).(1) This formulation separates case-visible evidence from external knowledge and prevents retrieval from introducing hidden labels for the current case. Figure1summarizes the online inference and offline gover- nance workflows of MediSkill-Evo. MediSkill-Evo consists of a four-bank self-evolution layer and aProcess-Constrained Preference Harness. After each training case, the former routes trajectory- derived experience to the Clinical Skill, Process Rule, Symbolic Schema, and Measurement banks, which respectively represent clinical strategies, cross-case workflow rules, evidence bound- aries, and visual measurement procedures; provenance, safety, and replay checks govern publication of the next frozen snap- shot. Within an encounter, the Harness retrieves state-relevant objects from these banks. It directly executes a rule-mandated process checkpoint when one is triggered; otherwise, it generates structured candidates and selects an executable action through symbolic verification and Clinical Process Critic comparison. Here, preferencedenotes test-time ranking of candidates generated for the same state, not parameter-level preference learning or rein- forcement learning. Encounter traces support only later evolution stages and never revise a case in progress. 2.2 Four-Bank Self-Evolution Four-bank self-evolution converts completed case trajectories into reusable, auditable external knowledge. For training case 푖,휏 푖 = (푣 푡 , 푢 푡 , 푎 푡 , 표 푡+1 푇 푡=0 , 푦 푖 )records visible states, retrieved knowledge, executed actions, environment responses, and the post-encounter evaluation, separating what the system observed, retrieved, and executed. Only after termination does the reflector generate update proposals, so evolution cannot alter the case in progress. Each bank manages typed artifacts comprising content, applicability scope, trajectory provenance, and lifecycle status. Active artifacts are not overwritten in place; add, merge, patch, deprecate, or discard operations are proposed for the next snap- shot. The banks share this merge, validation, and publication protocol but retain distinct knowledge boundaries and checks. This separation lets each bank evolve independently without mix- ing clinical strategies, workflow constraints, evidence semantics, and visual procedures. The four types separate content from decision authority. Clin- ical Skills encode scoped diagnostic and management strategies; Process Rules encode cross-disease required or prohibited bench- mark actions; Symbolic Schemas define controller-valid evidence sources and state transitions; and Measurement Skills encode image-specific observation procedures without returning a diag- nosis. In the reported offline system, “required” means enforced relative to the registered benchmark contract, not endorsed by an external guideline or clinician. Deterministic evidence semantics and registered safety prerequisites outrank trajectory-derived rules, which cannot create facts or override those constraints. For deployment, a learned regularity would remain advisory unless an identified guideline or expert policy supplied its authority and independent validation justified hard enforcement. The sup- plementary material specifies artifact fields, provenance, conflict resolution, and lifecycle operations. Let푏 ∈ 퐶, 푃, 푆, 푀index the four banks,Δ 푏 1∶푁 denote proposals from the ordered training cases, and푈 푏 and푉 푏 be the typed merge and validation operators. The next snapshot is ℬ 푏 푒+1 = 푉 푏 ( 푈 푏 (ℬ 푏 푒 , Δ 푏 1∶푁 ), 휏 푖 푁 푖=1 ) .(2) The merge operator organizes proposals by artifact identity, se- mantic overlap, and applicability scope, removing duplicates while preserving revision provenance. The validator checks type con- sistency, provenance, label/evidence leakage, controller-defined safety, and replayability on the proposal-generating trajectories; it does not estimate generalization to unseen cases or confer clin- ical authority. Valid artifacts enter the next immutable snapshot, whereas insufficient, conflicting, or out-of-bound artifacts are withheld, disabled, or rejected. This ties each inference trace to a determinate knowledge version. At test time, the final snapshot is frozen and reflection and knowledge writes are disabled. 2.3 Process-Constrained Preference Harness At each turn, the Harness constructs푣 푡 from the presented task and acquired evidence and registers Patient responses, Doctor actions, and Measurement outputs as provenance-bearing facts. Every ex- amination result is bound to the normalized request that elicited it: an available result is returned only after the request, whereas an absent result is markedRESULTS_UNAVAILABLEand remains un- known. Clinical Skills then pass retrieval and semantic gating, Pro- cess Rules form the dynamic ledger, Symbolic Schemas expose evi- dence boundaries, and Measurement Skills guide visual inspection after an image request. Their outputs constitute푢 푡 , with external knowledge and case evidence recorded separately. 3 Wu et al. Before generating candidates, the Harness checks whether an active Process Rule mandates a deterministic process action. A trig- gered action is executed with its rule and evidence recorded; other- wise, the Doctor generates structured candidates containing an ac- tion, target, rationale, expected information value, supporting ev- idence, and safety risks. In non-final turns, each examination can- didate requests one atomic item to permit direct comparison of in- formation value. Final-turn candidates instead provide all required diagnosis, evidence, management, safety, and follow-up fields. The Symbolic Verifier first removes candidates that use un- available or controller-invalid evidence, omit required final fields, or violate registered treatment-safety prerequisites. The Clini- cal Process Critic scores the remaining candidates for process quality, diagnostic specificity, evidence alignment, treatment com- pleteness, safety, triage, and examination efficiency. Step-level selection emphasizes process advancement and information effi- ciency, whereas final selection emphasizes diagnosis, evidence, and treatment completeness. With stage푟 ∈ step,final, hard- constraint indicator퐻, critic dimensions풟 푟 , and soft-constraint set풫 푟 , selection is unified as 푐 ∗ 푡 =arg max 푐 푖 푡 ∶ 퐻(푠 푡 ,푐 푖 푡 )=1 [ ∑ 푑∈풟 푟 휆 푟 푑 푝 푖 푑,푡 − ∑ 푘∈풫 푟 휆 푟 pen,푘 푞 푖 푡,푘 ] ,푎 ∗ 푡 = 푎(푐 ∗ 푡 ). (3) The first term is the stage-specific process score, and the second penalizes repeated examinations, inefficiency, and repairable struc- tural defects. Hard-invalid candidates cannot re-enter through fi- nite penalties. If none meets the safety threshold, bounded regener- ation proceeds without relaxing hard constraints; persistent failure yields safe termination and human escalation. Candidates, verifier outputs, scores, and selections remain in the audit trace. Figure 2reproduces the decision-critical portion of the Clini- cal Process Critic prompt. We expose this prompt because it de- fines the evidence boundary, the safety priorities, and the typed scoring interface that operationalize preference selection. Runtime state and candidates replace the bracketed slots; omitted instruc- tions concern only schema recovery and serialization. The supple- mentary material provides the learning, measurement, candidate- generation, preference, and final-safety templates needed to repro- duce the complete prompt-driven path. The selectedASK,REQUEST_TEST, orDIAGNOSIS_READYaction is sent to the Patient, Measurement, or final-response component, and the return updates the evidence state. A final response under- goes schema validation, diagnosis-blind safety review, and a sep- arate risk-auditor call that checks diagnostic support, dangerous alternatives, treatment contraindications, and management inten- sity. A Final Rewriter incorporates required corrections, and a Re- lease Certifier performs the final check. This bounded rewrite pro- cess withholds an uncertified plan at the retry limit and returns safe termination and human escalation instead. 3 Experiments 3.1 Implementation Details Our evaluation pipeline is built on the AgentClinic framework [14] and retains its Doctor–Patient–Measurement interaction protocol and request-gated delivery of examination results. We extend this Table 1: Evaluation datasets. FullChain encounters derive from MIMIC-IV [4] and use the AgentClinic interaction pro- tocol [14]; multimodal cases derive from the NEJM image collection [17]. Stress counts denote controlled conditions; the 420/180 conditions are derived from 70/30 disjoint un- derlying clinical cases. DatasetTrain Test Modality MIMIC-IV FullChain700 300 Text Controlled clinical stress420 180 Text NEJM FullChain Interactive200 100 Text + image foundation with FullChain case conversion, a deterministic stress controller, a multimodal image registry, and trajectory-level eval- uation; MediSkill-Evo additionally supplies the governed memory and decision modules described in Section2.1. All reported meth- ods operate in this shared AgentClinic-based environment, with differences within each comparison confined to the registered Doctor-side agent and its native experience-reuse procedure. We evaluate two hosted backbone endpoints, Qwen3.6- Flash [21] and DeepSeek-V4-Flash [2]. Within each comparison, the Doctor and moderator use the same backbone, while the Pa- tient and Measurement environments, evaluator, case order, and Doctor-turn ceiling are fixed. All evaluation calls use temperature zero, with one observed rollout for each case–configuration pair. MediSkill-Evo generates three candidates at each decision point and permits at most six Doctor inferences in text-only FullChain and controlled-stress encounters. Its Clinical Skill, Process Rule, and Symbolic Schema snapshots remain frozen during testing, and the Measurement Bank is inactive when no image is available. The supplementary material provides a registered per-setting con- figuration table, complete prompts and information boundaries, and a comparator ledger documenting baseline adaptations and frozen artifacts. The multimodal evaluation uses Qwen3.6-Flash and permits at most eight Doctor inferences per case. The three Doctor banks are initialized from the same learned snapshot and remain frozen; each condition evolves its own Measurement Bank on the 200 train- ing cases and freezes it for the 100-case test. The optional Med- SAM [ 11] condition uses the ViT-B checkpoint. The paired con- ditions share the backbone, cases, split and Doctor-bank hashes, candidate count, interaction environment, and Doctor-turn ceiling; whether the Measurement Agent may call MedSAM is the regis- tered tool intervention, although the condition-specific Measure- ment Banks also differ. The supplementary reproducibility ledger reports the manifest fields and remaining boundaries. 3.2 Datasets We evaluate the agents on three complementary settings (Table1): text-only clinical encounters, controlled stress conditions, and mul- timodal image cases. All test splits are fixed before evaluation and keep reference diagnoses and evaluator targets outside the Doctor- visible interaction. MIMIC-IV FullChain encounters.We transform MIMIC-IV- derived records [ 4] into interactive Doctor–Patient–Measurement 4 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction (a) Text-only FullChain interaction(b) Multimodal NEJM interaction INITIAL 24-year-old womanwith sudden right-upper-quadrant pain after a fatty meal, nausea, and vomiting. DOCTOR When did the pain start, how severe is it, and does it radiate to your back or shoulder? PATIENT Constant for 10 hours, 8/10, radiating to the right shoulder blade; worse after eating. DOCTOR REQUEST TEST: Complete_Blood_Count MEASUREMENT WBC 11.7×10 3 /μL with neutrophil predominance. DOCTOR REQUEST TEST: Right_Upper_Quadrant_Ultrasound MEASUREMENT RESULTS_UNAVAILABLE; the result remains unknown, not negative. DOCTOR Is there a sharp catch in your breath with right-upper-quadrant pressure? PATIENT Yes—I flinch and cannot take a full breath. FINAL Acute cholecystitis. NPO, IV fluids, antibiotics after prerequisite checks, urgent surgical evaluation, and alternative imaging because ultra- sound is unavailable. INITIAL 95-year-old womanwith four weeks of dyspnea and dry cough; resting SpO 2 is 83%. DOCTOR How long have you taken nitrofurantoin? PATIENT Daily for six months. DOCTOR: REQUEST TEST: NEJM_Medical_Image MEASUREMENT AGENT: optional MedSAM Raw radiograph Raw chest CT X-ray overlay CT overlay MEASUREMENT Bilateral opacities on radiography; CT shows ground- glass opacity, septal thickening, crazy paving, and consolidation. MedSAM localizes these regions; the agent re-inspects the original pixels. DOCTOR REQUEST TEST: Complete_Blood_Count MEASUREMENT Neutrophilic leukocytosis; no eosinophilia. FINAL Nitrofurantoin-induced pulmonary toxicity. Stop nitrofurantoin, provide oxygen and monitored admission, exclude infection, and obtain pul- monary consultation. Figure 3: Two frozen-test interactions scored diagnosis-correct by the automatic evaluator. Left: targeted history, available lab- oratory evidence, an unavailable ultrasound, and a focused physical examination support acute cholecystitis without treating a missing result as negative. Right: medication history, original radiograph and CT evidence, optional MedSAM localization, and a follow-up blood count support nitrofurantoin-induced pulmonary toxicity; this label does not validate anatomy or man- agement. encounters following the AgentClinic protocol [14]. Each en- counter contains an initial objective, patient responses, physical findings, requestable examinations, a reference diagnosis, and management and safety targets when available. The split contains 700 training encounters and 300 test encounters. During interac- tion, the Doctor receives the initial objective, the accumulated dialogue, and results returned after its own requests; reference diagnoses and evaluator-only targets are not exposed. Controlled clinical stress benchmark.To test whether an agent can preserve evidence-grounded and safe behavior when a care obligation becomes difficult, we derive six source-grounded stress dimensions from the FullChain cases: diagnosis difficulty, evi- dence completeness, patient behavior complexity, treatment and prescription safety, temporal dynamics, and triage safety. The training set contains 420 conditions generated from 70 cases, and the test set contains 180 conditions generated from 30 disjoint cases. Thus, the 180 test conditions are six controlled variants of 30 underlying cases rather than 180 independent patients. Each stress variant changes only the visibility or timing of source-supported information. Depending on the dimension, deci- sive evidence may be delayed, one item may remain unavailable, patient facts may need to be recovered through focused questions, treatment prerequisites may need to be verified, timeline evidence may be revealed later, or a real red flag may be withheld until an appropriate screen. The benchmark does not add fabricated symptoms, refusals, worsening events, contraindications, or red flags. A deterministic controller owns the hidden or delayed values and releases a source value only after the permitted ques- tion, examination, or test request. The Doctor, its retrieval and decision modules, and the runtime prompts do not receive the stress dimension, subtype, hidden values, trigger concepts, fact identifiers, or evaluator targets. These controls isolate the process obligation being tested while preserving the underlying clinical case. Table2summarizes the six dimensions and their enforced process obligations. NEJM FullChain Interactive.We construct a multimodal diagno- sis benchmark from 300 cases in the NEJM image collection [17]. The fixed split contains 200 training cases and 100 test cases. Each case is converted into an interactive encounter with patient- knowable history, bedside findings, canonical requestable tests, required history questions, and required tests. The image is reg- istered as a requestableNEJM_Medical_Imageexamination; the request name indicates that an image can be requested, but does not reveal its content or diagnosis. At the start of an encounter, the Doctor sees only the available initial information and exam- ination names. An examination result is returned only after the corresponding request, and an unavailable or unobserved result remains unknown rather than being treated as negative. 3.3 Evaluator and Metrics The offline evaluator scores clinical outcomes and observable pro- cess quality from the completed Doctor–Patient–Measurement trajectory and evaluator-only targets after inference. It receives no method identity, and the same evaluator path is applied to every comparator. The semantic judge uses the registered moderator alias—the comparison backbone within each FullChain block—at temperature zero. Registered test names, stress-controller release events, and output-schema validity are checked deterministically, while the judge handles semantic equivalence and must cite supporting trajectory turns; malformed or unsupported outputs 5 Wu et al. Table 2: Controlled-stress dimensions, source-grounded construction, and enforced process obligations. Each condition changes only the visibility or release timing of source-supported information; hidden values remain outside every runtime LLM prompt and are released only by the deterministic controller. DimensionDimension constructionPrimary capability under evaluation Diagnosis difficultyDelay decisive source evidence until the Doctor uses its reg- istered question, examination, or test channel. Recovering and using the decisive evidence while avoiding premature closure. Evidence completeness Make one source item unavailable while retaining an inde- pendent evidence path that preserves case solvability. Requesting the missing item, treating unavailability as un- known, and using alternative evidence without fabricating a result. Patient behavior com- plexity Partition patient-knowable source facts so that they are re- leased only after focused, relevant questions. Recovering patient facts through focused, respectful ques- tioning without changing the underlying disease facts. Treatment and prescrip- tion Gate a source-supported treatment prerequisite behind the corresponding history, examination, or test request. Verifying prerequisites and choosing conditional treatment, safe deferral, or an alternative rather than an unsafe action. Temporal dynamicsDelay a source-supported timeline fact until the Doctor asks a temporally targeted question. Recovering the original timeline and integrating it into re- assessment, treatment, monitoring, or escalation. Triage safetyWithhold a source-supported red flag until the Doctor per- forms the appropriate symptom or risk screen. Recovering the red flag, escalating appropriately, and avoid- ing unsafe reassurance. receive no credit. In the 300-case FullChain test set, required- history, required-test, and management target lists are nonempty for every case. In the NEJM test set, all cases have an image-test target and seven cases have an empty required-history list; macro recall uses the same defined-target convention for both paired conditions. The stress evaluator deliberately separates process measure- ment from conventional task outcomes. Controller events deter- mine whether a target fact was recovered; a dimension-specific semantic check then asks only whether released evidence was used, unavailable evidence remained unknown, a treatment pre- requisite led to safe action or deferral, a timeline was integrated, or a red flag triggered escalation. These dimension metrics are the primary controlled-stress endpoints. Diagnosis, treatment, the registered stress composite, general safety and critical-failure labels, and interaction economy are retained as auxiliary system outcomes so that process gains cannot conceal a collapse in or- dinary clinical performance. Table 3summarizes these measures; all rates are macro-averaged percentages over their declared eli- gible sets, and the supplementary material provides the complete prompts, denominators, and aggregation rules. Treatment-intent, safety, critical-failure, and semantic stress labels are operational outputs of this automatic evaluator, not independently clinician- adjudicated clinical outcomes; they support method-blind within- benchmark comparison but do not establish construct calibration, clinical certification, or prospective validity. 3.4 End-to-End FullChain Performance across Backbones We first examine whether MediSkill-Evo improves complete clinical interactions across different backbone models. Table4 compares AgentClinic [14], our structured Agent-KB implementa- tion, ExPeL [24], MemP [3], Reflexion [16], SkillWeaver [25], and MediSkill-Evo on the same 300 test encounters. Every method uses the same interaction environment, backbone, case order, Doctor-turn ceiling, and frozen test-time memory, while retaining its native memory, control flow, and decision mechanism. On Qwen3.6-Flash, MediSkill-Evo achieves the strongest joint FullChain performance. Compared with the strongest competing agent on each metric, it improves diagnosis accuracy by 7.8%, treatment-intent coverage by 70.7%, evidence recall by 219.5%, and required-history recall by 690.7%, while reducing automat- ically scored critical failures by 43.0%. These gains show that the improvement extends from the accepted final diagnosis to the acquisition and use of information required for a complete observable clinical trajectory. The same process-level pattern appears with DeepSeek-V4- Flash. MediSkill-Evo leads seven of the eight non-diagnosis metrics, improving treatment-intent coverage by 37.9%, evidence recall by 163.9%, required-history recall by 1,070.9%, and gated interaction efficiency by 190.4% over the strongest competing result. It also reduces unnecessary examinations by 59.1% and automatically scored critical failures by 29.8%. The concentration of gains in evidence acquisition, treatment planning, risk control, and interaction efficiency is consistent with the use of structured process knowledge to guide the next clinically relevant action across the two evaluated backbones. The supplementary material reports the corresponding per-case better/tie/worse transitions from the existing paired traces. 3.5 Process Correctness and Safety under Controlled Stress Having established the end-to-end gains, we next examine how MediSkill-Evo handles individual care obligations when the re- quired evidence becomes difficult to acquire or use. Figure 4 reports dimension-level completion together with the underlying acquisition, utilization, and safety metrics. Table5complements this process analysis with aggregate diagnosis, treatment, safety, 6 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction Table 3: Evaluation metrics and their operational meanings. Metrics are grouped by evaluation setting and function; arrows indicate the preferred direction. Metric groupOperational definition Standard and multimodal evaluation OutcomeDx↑: accepted final diagnosis;Tx/Rx↑: reference treatment intents covered, with unsafe care penalized. Evidence acquisitionHist.↑: required history elicited;Tests↑: required examinations requested;Evid.↑: joint history-and-test coverage. Risk controlSafety↓: observable safety violation;Critical↓: critical diagnostic, treatment, or triage failure. Efficiency and completionUnnec.↓: unjustified examinations;Int.Eff.↑: gated interaction efficiency;Core↑: composite automatic score penalized for critical failure. Visual measurementMasks: cases with at least one non-empty MedSAM mask. Controlled-stress evaluation DiagnosisDTR↑: delayed target recovered;DUR↑: recovered discriminator used;PCR↓: premature closure. Evidence availabilityHqR↑: unavailable history queried;KTR↑: unavailable test requested;UHS↑: unavailability handled safely;HUR↓: unavailable result hallucinated. Patient behaviorBTR↑: target patient facts recovered;FQS↑: focused-question adequacy;BFR↓: target-recovery failure. Treatment safetyTSR↑: treatment-safety prerequisites recovered;SDF↑: safe deferral or alternative;UAR↓: unsafe action. Temporal reasoningTER↑: timeline evidence recovered;TIR↑: recovered timeline integrated;TFR↓: integration failure. TriageRFR↑: red-flag evidence recovered;EA↑: escalation adequacy;URR↓: unsafe reassurance. and task-completion results. AgentClinic [14], MemP [3], Re- flexion [16], and MediSkill-Evo are evaluated on the same 180 controlled conditions using the same controller, evaluator, case order, and six-action budget. MediSkill-Evo produces its largest gains when the Doctor must actively recover or revisit process-critical evidence. In Fig.4, it reaches 93.61% completion on patient-behavior conditions, 100% on temporal conditions, and 92.22% on triage conditions, corre- sponding to relative improvements of 22.5%, 37.4%, and 59.6% over the strongest competing agent. These dimensions require the Doc- tor to recover omitted patient facts, incorporate evidence revealed later in the encounter, or screen for urgent red flags before com- mitting to a plan. The constituent metrics further show that recovered informa- tion is carried into subsequent decisions. Among cases in which the relevant target is recovered, MediSkill-Evo incorporates every evaluated timeline update and produces an adequate triage plan without unsafe reassurance. Under unavailable-evidence condi- tions, it queries every evaluated hidden-history subtype without fabricating an unavailable result; under treatment-prerequisite conditions, it selects a safe alternative or explicit deferral. This Table 4: Main automatic operational results (%) on the same 300 FullChain test encounters. Backbones represent the Qwen [21] and DeepSeek [2] families; baseline methods are cited in the accompanying text. Higher is better except for Safety, Critical, and Unnec. Bold marks the best result within each backbone; shaded rows denote MediSkill-Evo. BackboneMethod Benchmark outcome Evidence acquisitionRisk and economy Int.Eff.↑ Dx↑Tx/Rx↑Evid.↑Hist.↑Tests↑Safety↓Critical↓Unnec.↓ Qwen3.6-Flash AgentClinic61.3333.62 12.31 11.70 13.461.6731.0015.89 39.60 Agent-KB61.0038.43 14.52 11.26 17.450.6732.3313.8241.42 ExPeL59.0035.50 14.58 11.09 17.432.3333.0014.03 40.28 MemP64.0038.93 15.16 12.45 17.411.6728.6712.79 41.40 Reflexion62.3338.27 13.38 11.06 15.651.3331.0013.25 41.18 SkillWeaver60.3337.38 14.46 11.02 16.712.3333.3311.8139.69 MediSkill-Evo69.0066.4448.4398.4419.030.3316.3321.0038.35 DeepSeek-V4-Flash AgentClinic52.3332.16 9.98 5.64 12.72 20.3352.0045.047.58 Agent-KB57.0033.82 13.51 5.31 19.08 14.3349.3346.817.66 ExPeL51.6730.03 13.44 5.44 18.83 15.6753.0044.867.29 MemP54.0030.58 15.20 6.2920.9714.6748.3346.847.79 Reflexion52.3329.72 13.41 5.22 18.81 14.6750.0047.797.21 SkillWeaver55.3334.31 12.53 5.77 17.05 14.0047.0045.959.49 MediSkill-Evo55.6747.3040.1173.6519.313.3333.0018.3327.56 7 Wu et al. Diagnostic pressure Evidence gaps Patient behavior Treatment safety Temporal dynamics Triage safety 25 50 75 100 (a) Dimension-aware process completion AgentClinic MemP Reflexion MediSkill-Evo AgentClinicMemPReflexion MediSkill -Evo Dx DTR↑ Dx DUR↑† Dx PCR↓ Ev HqR↑ Ev KTR↑ Ev UHS↑† Ev HUR↓ Pt BTR↑ Pt FQS↑ Pt BFR↓ Tx TSR↑ Tx SDF↑ Tx UAR↓ Tm TER↑ Tm TIR↑† Tm TFR↓† Tr RFR↑ Tr EA↑ Tr URR↓ 66.175.077.268.9 100.0100.0100.0100.0 0.00.00.00.0 70.060.090.0100.0 40.060.070.030.0 100.0100.0100.0100.0 0.00.00.00.0 70.075.876.493.6 100.0100.0100.0100.0 30.024.223.66.4 35.035.033.336.7 100.0100.0100.0100.0 0.00.00.00.0 57.272.870.6100.0 100.0100.0100.0100.0 0.00.00.00.0 42.257.856.792.2 100.0100.0100.0100.0 0.00.00.00.0 (b) Concrete recovery, use, and safety submetrics 050100 Direction-adjusted score (higher is better) Heatmap text reports raw percentages; color is inverted only for ↓ failure-rate metrics. † Conditional denominator. Figure 4: Primary controlled-stress process and safety profile (%). (a) Dimension-aware required-action completion across the six pressure families. (b) The 19 constituent recovery, use, handling, and failure metrics; cell text gives the raw percentage, whereas color is direction-corrected so that darker always denotes better behavior. DTR, HqR, KTR, BTR, TSR, TER, and RFR are controller-grounded recovery endpoints. Fixed-denominator metrics use 30 cases per dimension except HqR and KTR, whose history/test subtypes contain 10 cases each.†marks a conditional denominator determined by a visible request or recovered target and must be read with the corresponding recovery row. Bold cell text marks a unique best. Table 5: Auxiliary cross-dimension aggregates and conven- tional automatic outcomes on all 180 controlled stress con- ditions (%). Stress is the registered stress-process composite and Req.Act. is its dimension-aware required-action compo- nent. Higher is better except for Safety and Critical. MethodStress↑Req.Act.↑Core↑Dx↑Tx/Rx↑Safety↓Critical↓ AgentClinic77.7570.39 75.5696.6778.360.001.11 MemP72.8476.81 67.96 87.78 75.311.677.22 Reflexion76.0678.55 71.33 90.56 78.861.112.78 MediSkill-Evo83.7988.3080.0393.8982.401.112.78 pattern is consistent with the roles of the Process Rule and Sym- bolic Schema banks, which preserve unresolved care obligations and the evidence state needed to complete them. These dimension-level gains coincide with strong aggregate task performance in Table5. MediSkill-Evo obtains the highest registered stress-process score (83.79%), required-action recall (88.30%), treatment-intent coverage (82.40%), and core score (80.03%), together with 93.89% diagnosis accuracy. Automatically scored safety violations and critical failures are 1.11% and 2.78%, respectively. Together, the results show that the FullChain gains in Section 3.4 are supported by more reliable recovery and use of process-critical evidence, particularly under patient-behavior, temporal, and triage pressure. 3.6 Multimodal Evaluation with Request-Gated Visual Measurement We finally examine whether the same governed interaction process extends from textual evidence to request-gated medical images. On 8 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction Table 6: Four-agent results on 100 multimodal NEJM image cases [17]. Ready and Masks are counts; all other values are macro-averaged percentages over the same 100 saved case trajectories, with no error rows. Masks counts cases with at least one non-empty MedSAM mask. Agent/conditionReady Dx↑Hist.↑Tests↑Unnec.↓Core↑Masks MemP100/100 38.00 58.60 59.0012.75 35.95 0/100 Reflexion100/100 39.00 58.63 67.0012.77 40.29 0/100 MediSkill-Evo (raw)100/10037.0075.58100.005.0044.690/100 MediSkill-Evo (+MedSAM)100/10040.0079.50100.006.5847.9334/100 the 100-case NEJM [17] test set, we compare MemP, Reflexion, and two MediSkill-Evo conditions that share the same three Doctor banks. The first condition operates on the original image, while the second additionally permits the Measurement Agent to invoke MedSAM [11] for localized masks and overlays; each condition evolves its Measurement Bank on the corresponding 200 training cases. Table6reports diagnosis, process-completion, interaction, and tool-use metrics. MediSkill-Evo completes all 100 multimodal encounters without runtime or diagnosis-readiness failures. Relative to the strongest MemP or Reflexion result, the MedSAM-enabled condition im- proves diagnosis accuracy by 2.6%, required-history recall by 35.6%, required-test recall by 49.3%, and the core FullChain score by 19.0%, while reducing unnecessary examinations by 48.4%. These results show that the governed interaction process remains effective when clinically relevant evidence must be obtained through an image request. Within MediSkill-Evo, enabling localized visual measurement raises diagnosis accuracy from 37.00% to 40.00%, required-history recall from 75.58% to 79.50%, and the core FullChain score from 44.69% to 47.93%, corresponding to relative improvements of 8.1%, 5.2%, and 7.3%. The Measurement Agent produces 54 non-empty masks in 34 cases and analyzes the remaining cases directly from the original pixels. This usage pattern illustrates the modular role of the Measure- ment Bank. The agent retains access to the original pixels through- out the encounter and invokes localized measurement only when the evolving case state calls for additional visual evidence. The experi- ment therefore demonstrates an operational multimodal extension in which image interpretation becomes a requestable and selectively applied component of the clinical process. 3.7 Component Analysis Table 7: Component analysis on the same 100 FullChain test cases. Each variant disables the indicated knowledge bank while retaining the registered model, interaction bud- get, case set, and evaluator. All values are percentages. VariantDx↑Tx/Rx↑Safe.↓Crit.↓Unnec.↓ No memory 67.00 36.50 1.00 30.0011.50 w/o Clinical 74.00 63.65 1.00 24.00 29.75 w/o Process 71.00 60.70 8.00 36.00 30.50 w/o Symbolic 70.00 68.900.0015.00 19.50 Full76.0069.400.0013.0019.17 We examine how the three text-based knowledge banks contribute to diagnosis, treatment planning, and process safety. Table7compares the complete system with a memory-free configuration and variants that disable one bank at a time under the same evaluation setting. The complete system achieves the highest diagnosis accuracy (76.00%) and treatment-intent coverage (69.40%), the lowest critical-failure rate (13.00%), and a tied-best safety-violation rate (0.00%). Relative to no memory, it improves diagnosis accuracy by 13.4% and treatment coverage by 90.1%, while reducing critical failures by 56.7%. The removal patterns connect the banks to their intended roles. Disabling the Process Bank produces the largest safety degradation, increasing safety violations from 0.00% to 8.00% and critical failures from 13.00% to 36.00%, consistent with its role in preserving care obli- gations and action prerequisites. Removing the Clinical Bank lowers treatment coverage and increases critical failures and unnecessary examinations, reflecting its role in organizing reusable clinical strate- gies. Removing the Symbolic Bank reduces diagnosis accuracy while modestly changing the process metrics, reflecting its contribution to evidence-state and request semantics. Together, the three banks provide complementary clinical, procedural, and evidence-level guid- ance across the evaluated endpoints. 4 Conclusion MediSkill-Evo frames clinical-agent evolution as the acquisition of governed process knowledge. Its four banks give reusable strategies, workflow rules, evidence semantics, and visual procedures distinct update and validation paths; its Process-Constrained Preference Harness assigns those artifacts benchmark-time decision authority. On fixed suites and two backbone endpoints, complete-system comparisons under the same Doctor-turn ceiling show higher treatment-intent and evidence coverage and lower automatically scored safety-related failures on most FullChain settings. Hard- isolation stress testing further shows stronger target recovery under patient-behavior, temporal, and triage pressure, while exposing weaker diagnostic-discriminator and unavailable-test acquisition and no uniform advantage on general safety outcomes; the optional MedSAM comparison provides request-gated interface evidence only. These results do not establish clinical safety, population-level generalization, judge construct validity, or causal credit for individ- ual components. They motivate a bounded system-design hypothesis: provenance, scope, and decision rights can be represented jointly and evaluated as one interaction stack. Controlled mechanism com- parisons and independent clinical calibration are necessary before attributing the gains to typed memory or interpreting automatic safety labels as clinical outcomes. 9 Wu et al. 5 Ethical Considerations This study evaluates offline research agents on deidentified MIMIC- IV-derived records under authorized access and published NEJM image cases. Source data and images retain their original access and redistribution terms and are not released. Label/evidence- leakage checks prevent hidden benchmark targets from entering prompts but do not constitute a patient-privacy audit; artifacts failing these checks are excluded. The study provides no clinician calibration, prospective or cross-institutional validation, subgroup fairness analysis, privacy testing, or clinical safety certification. Potential harms include unsupported recommendations, au- tomation bias, subgroup disparities, privacy leakage through evolved artifacts, and hosted-endpoint or judge drift. Reported rates characterize only frozen research artifacts and do not authorize autonomous care. Deployment would require clinician oversight, local and subgroup validation, privacy auditing, version- pinned systems, traceable rollback, red-team testing, prospective monitoring, and safe escalation. 10 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction References [1]Zehui Chen, Weihua Du, Wenwei Zhang, Kuikun Liu, Jiangning Liu, Miao Zheng, Jingming Zhuo, Songyang Zhang, Dahua Lin, Kai Chen, and Feng Zhao. 2024. T-Eval: Evaluating the Tool Utilization Capability of Large Lan- guage Models Step by Step. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 9510–9529. doi:10.18653/v1/2024.acl-long.515 [2]DeepSeek-AI. 2024.DeepSeek-V3 Technical Report.arXiv preprint arXiv:2412.19437(2024).doi:10.48550/arXiv.2412.19437 [3]Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. 2025. MemP: Exploring Agent Procedural Memory.arXiv preprint arXiv:2508.06433(2025).doi:10.48550/arXiv. 2508.06433 [4]Alistair E. W. Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Sham- mout, Steven Horng, Tom J. Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, Li-wei H. Lehman, Leo Anthony Celi, and Roger G. Mark. 2023. MIMIC-IV, a Freely Accessible Electronic Health Record Dataset.Scientific Data10, 1 (2023), 1.doi:10.1038/s41597-022-01899-x [5]Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae Won Park. 2024. MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making. InAdvances in Neural Information Processing Systems, Vol. 37. proceedings.neurips.c [6]Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, and Yixin Wang. 2024. MMedAgent: Learning to Use Medical Tools with Multi-Modal Agent. InFindings of the Asso- ciation for Computational Linguistics: EMNLP 2024. 8745–8760.doi:10.18653/v1/ 2024.findings-emnlp.510 [7]Yusheng Liao, Shuyang Jiang, Yanfeng Wang, and Yu Wang. 2025. ReflecTool: Towards Reflection-Aware Tool-Augmented Clinical Agents. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 13507–13531. doi:10.18653/v1/2025.acl-long.663 [8]Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Ao- han Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2024. AgentBench: Eval- uating LLMs as Agents. InInternational Conference on Learning Representations. openreview.net/forum?id=zAdUB0aCTQ [9]Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haop- ing Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, and Ruoming Pang. 2025. ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities. InFindings of the Association for Com- putational Linguistics: NAACL 2025. 1160–1183. doi:10.18653/v1/2025.findings- naacl.65 [10]Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhen- zhong Lan, Lingpeng Kong, and Junxian He. 2024. AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents. InAdvances in Neural Information Processing Systems, Vol. 37. doi:10.52202/079017-2365 [11]Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. 2024. Segment Anything in Medical Images.Nature Communications15, 1 (2024), 654. doi:10. 1038/s41467-024-44824-z [12]Daniel Philip Rose, Chia-Chien Hung, Marco Lepri, Israa Alqassem, Kiril Gash- teovski, and Carolin Lawrence. 2025. MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 13803–13826. doi:10.18653/v1/2025.acl-long.677 [13]Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, and Katerina Fragkiadaki. 2024. VLM Agents Generate Their Own Mem- ories: Distilling Experience into Embodied Programs of Thought. InAdvances in Neural Information Processing Systems, Vol. 37. proceedings.neurips.c [14]Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. 2024. AgentClinic: A Multimodal Agent Benchmark to Evaluate AI in Simulated Clinical Environments.arXiv preprint arXiv:2405.07960(2024). doi:10.48550/arXiv.2405.07960 [15]Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce C. Ho, Carl Yang, and May Dongmei Wang. 2024. EHRAgent: Code Empowers Large Language Models for Few-Shot Complex Tabular Reasoning on Electronic Health Records. InProceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing. 22315–22339. doi:10.18653/v1/2024. emnlp-main.1245 [16]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems, Vol. 36.doi:10. 52202/075280-0377 [17]The New England Journal of Medicine. 2026. Image Challenge.https://w. nejm.org/image-challenges. Accessed 2026-08-08. [18]Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. 2024. AppWorld: A Controllable World of Apps and People for Benchmark- ing Interactive Coding Agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 16022–16076. doi:10.18653/v1/2024.acl-long.850 [19]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Voyager: An Open-Ended Em- bodied Agent with Large Language Models.Transactions on Machine Learning Research(2024).openreview.net/forum?id=ehfRiF0R3a [20]Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2025. Agent Workflow Memory. InProceedings of the 42nd International Conference on Ma- chine Learning, Proceedings of Machine Learning Research, Vol. 267. 63897– 63911.proceedings.mlr.press/v267/wang25bx.html [21]An Yang et al. 2025. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388 (2025).doi:10.48550/arXiv.2505.09388 [22]Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2025.휏- bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. InInternational Conference on Learning Representations. openreview.net/forum? id=roNSXZpUDN [23]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InInternational Conference on Learning Representations. openreview. net/forum?id=WE_vluYUL-X [24]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM Agents Are Experiential Learners. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, No. 17. 19632–19642. doi:10.1609/aaai.v38i17.29936 [25]Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su. 2025. SkillWeaver: Web Agents Can Self-Improve by Discovering and Honing Skills.arXiv preprint arXiv:2504.07079(2025). doi:10.48550/arXiv. 2504.07079 [26]Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Siyuan Wang, Zhongyu Wei, and Fei Huang. 2025. AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator. InProceedings of the 31st Interna- tional Conference on Computational Linguistics. 10183–10213. aclanthology.org/ 2025.coling-main.680 [27]Ivan Sviridov, Amina Miftakhova, Artemiy Tereshchenko, Galina Zubkova, Pavel Blinov, and Andrey Savchenko. 2025. 3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark. InProceedings of the 2025 Conference on Em- pirical Methods in Natural Language Processing. 26614–26654. doi:10.18653/v1/ 2025.emnlp-main.1353 [28]Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, and Zhenhua Dong. 2025. MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents. InFindings of the Association for Computational Linguistics: ACL 2025. 19336–19352. doi:10.18653/v1/2025.findings-acl.989 11 Wu et al. Supplementary Materials This section provides the supplementary material accompanying the main paper, including implementation details, evaluator definitions, additional results, prompt templates, and qualitative examples. S1 Detailed Bank Artifacts and Lifecycle S1.1 Clinical Skill Bank The Clinical Skill Bank stores reusable decision experience for a class of cases: diagnostic patterns, examination and treatment strategies, and common failure modes. A skill specifies a problem signature, inclusion and exclusion conditions, a recommended evidence-acquisition or management sequence, and misuse warnings, making both when and how to apply it explicit. At inference, symbolic preconditions and semantic gating remove entries that conflict with visible evidence or diagnostic boundaries before a compact subset enters the Doctor context. Management identifies semantic duplicates and overlapping scopes: compatible experience is merged, a local improvement patches the relevant field, and a genuinely new pattern creates an entry. Skills that repeatedly conflict with outcomes, depend on incidental details, or lack transfer value are deprecated or discarded. Each operation retains its source trajectories, preventing a single reflection from silently replacing established experience. S1.2 Process Rule Bank This bank stores cross-disease workflow constraints. A rule specifies its clinical stage, trigger, inspected state, required or prohibited action, release condition, and priority. It can enforce registered prescription prerequisites, request missing information, or prevent an unavailable result from being treated as observed. Rules do not create clinical facts or override deterministic evidence semantics; they inspect registered state and constrain the next action within the benchmark contract. Active rules form a dynamic process ledger for candidate generation and verification. Deterministic controller contracts precede learned Process Rules; among learned rules, benchmark-safety and stage-required rules precede advisory rules, and trigger specificity resolves equal-priority conflicts. Recurring omissions create rules, whereas incomplete coverage patches or narrows existing ones. Overly broad, contradictory, or repeatedly unproductive rules are revised, downgraded, or disabled. These priorities are implementation authority, not clinical endorsement; deployment-grade hard constraints would require an identified guideline or expert-policy source and independent validation. S1.3 Symbolic Schema Bank The Symbolic Schema Bank defines which observations may become facts, their legitimate sources, and their permitted use. A schema specifies the field type, source role, allowed state transitions, request–result relation, and permitted consumers. Patient responses, Doc- tor requests, and Measurement outputs are normalized into provenance-bearing facts. Results must correspond to prior requests; missing, pending, and unavailable are distinct states and cannot default to normal or negative. The event ledger retains value, source, and registra- tion time, preventing rebinding to unrelated requests. Verified facts filter inapplicable skills and expose unsupported evidence references. Management may add fact types, aliases, or source relations for stable representational gaps, but publication requires unambiguous typing and verifiable source semantics; conflicting definitions or weakened request–result constraints are withheld. S1.4 Measurement Bank The Measurement Bank stores visual procedures indexed by image modality and task. An entry defines its modality, observation targets, region or tool prerequisites, measurement steps, report fields, quality checks, and failure modes, separating reusable procedure from case- specific findings. After an image request, the Measurement Agent retrieves a procedure, may use MedSAM for localization and quantification, and verifies the region, value, and finding against the original image. Its report retains method, evidence location, and uncertainty and returns observable evidence rather than a disease label. Management uses the report, its subsequent clinical use, and the case outcome to patch omitted targets, weak checks, or ambiguous fields. Procedures that exaggerate, misattribute, or rely on incidental image features are scope-restricted or disabled rather than generalized into clinical conclusions. S2 Core Learning and Inference Prompts This section reproduces the decision-bearing prompt templates used by MediSkill-Evo. Angle-bracketed fields are populated at runtime. We omit API transport, token budgets, retry messages, and JSON parsing boilerplate; internal development labels are normalized to the paper terminology. Every returned object is subsequently checked by the typed validators described in Section S2.6. TableS1makes the information boundary of each call explicit. S2.1 Post-episode trajectory reflection Reflection is invoked only after a training encounter has terminated. Evaluator feedback and hidden case targets enter this post-episode call, but they are explicitly marked as unavailable to the Doctor during the encounter. SYSTEM You are MediSkill-Evo's trajectory reflector. Analyze a completed 12 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction Table S1: Prompt inventory and information boundaries. Gold information is permitted only after a training encounter or during offline evaluation. StagePrincipal inputsGold allowed?Structured output Trajectory reflectionCompleted training trace, evaluator feedback, case targets Train onlyReflection and failure attribution Doctor-bank proposal Validated reflection, observed symbolic traces, active arti- fact IDs Train only Typed bank mutations Visual measurementOriginal pixels, optional MedSAM artifacts, retrieved Mea- surement Skills NoEvidence-only visual report Candidate and criticVisible state, process ledger, retrieved banks, action portfo- lio NoValidated action scores Final safety pathVisible trajectory, proposed plan, diagnosis-blind safety frame NoRewritten plan and release deci- sion Offline evaluatorCompleted frozen-test trace and hidden scoring targetsEval onlyCase metrics and evidence indexes interactive clinical trajectory. Evaluator feedback and case targets are post-hoc learning signals only; never describe them as information available to the Doctor. Identify reusable clinical-process lessons from both successful and unsuccessful behavior. Return strict JSON. USER task: Create one structured reflection for trajectory-derived learning. trajectory_record: <visible turns, retrieved artifacts, executed actions, evaluator feedback, and post-hoc case targets> rules: - Do not restate the reference diagnosis as a reusable skill. - Ground every success or failure in a trajectory turn or evaluator item. - Retain only lessons that generalize beyond this patient. - Request patch or deprecation only when a retrieved artifact plausibly caused misleading or unsafe behavior. required_output: case_id: string outcome_level: excellent | acceptable | failed | unsafe primary_failure_type: diagnosis | treatment | evidence | safety | triage | efficiency | none what_worked: [string] what_failed: [string] missed_evidence: [string] missed_tests: [string] missed_treatment_intents: [string] unsafe_actions: [string] unnecessary_tests: [string] red_flags_missed: [string] skill_update_need: add | patch | deprecate | none likely_harmful_skill_ids: [string] reflection_rationale: string S2.2 Typed Doctor-bank mutation proposals The reflection is routed through two structured proposal calls. The first maintains Clinical Skills; the second may emit one Process Rule and one Symbolic Schema mutation. The calls expose only existing active identifiers as legal patch, merge, or deprecation targets. SYSTEM -- CLINICAL SKILL PROPOSER Convert one structured trajectory reflection into one reusable Clinical Skill mutation. Use only trajectory evidence, evaluator feedback, and post-hoc case targets. Return strict JSON. USER inputs: trajectory_record: <compact completed trajectory> reflection: <validated reflection object> existing_skill_targets: [skill_id, name, status] decision_rules: - Choose exactly one of add, merge, patch, deprecate, or discard. - If no reusable lesson exists, discard; do not manufacture a skill. - Patch, merge, and deprecate must reference an existing active skill_id. - Failed cases produce a correction strategy, never a memorized answer. - Medication, procedure, escalation, and monitoring policies include their relevant safety checks. common_required_output: update_type: add | merge | patch | deprecate | discard target_skill_id: existing id or null safety_rationale: string expected_effect: string skill_fields_for_add_merge_patch: name: string description: string diagnosis_pattern: string 13 Wu et al. applicable_signals: [string] contraindications: [string] workflow_steps: [string] test_policy: [string] treatment_policy: [string] failure_modes: [string] stress_dimensions: [string] branch: general_branch | task_branch | action_branch confidence: number in [0,1] support_record: relation_type, excerpt, confidence evidence_from_trajectory: [string] SYSTEM -- PROCESS RULE / SYMBOLIC SCHEMA EVOLVER Use only completed training episodes. Create reusable process rules or symbolic schemas, not case answers. Return strict JSON. USER inputs: reflection: <validated reflection> action_grading: <post-episode action assessment> runtime_symbolic_traces: <facts emitted during this episode> symbolic_verifier_decisions: <accept/reject records> allowed_symbolic_predicates: <observed predicates only> allowed_symbolic_contracts: <observed arguments, sources, and statuses> existing_artifacts: <active rule and schema identifiers> task: Propose at most one PROCESS_RULE and at most one SYMBOLIC_SCHEMA. rules: - Never encode the reference diagnosis as a trigger. - Never require hidden case targets at runtime. - A PROCESS_RULE renders workflow guidance only. - A SYMBOLIC_SCHEMA defines extraction and verification only. - A schema may constrain only predicates, arguments, sources, and status values observed in runtime_symbolic_traces. - Reuse an existing identifier through PATCH; do not duplicate it. - Return no proposal when the lesson is case-specific or low-signal. required_output: proposals: - proposal_type: PROCESS_RULE | SYMBOLIC_SCHEMA action: CREATE | PATCH | DEPRECATE target_id: existing id or null draft: <typed rule or schema object> rationale: string anti_leakage_check: runtime_judgable_from_visible_state: boolean does_not_encode_gold_answer: boolean does_not_require_case_targets_at_runtime: boolean process_rule_draft: rule_id, name, rule_type, trigger_patterns, required_slots, prompt_instruction, negative_instruction, priority symbolic_schema_draft: schema_id, predicate, arguments, allowed_values, extract_from, must_not_infer S2.3 Measurement Agent prompts The Measurement Agent uses a two-stage image prompt. A locator first identifies modality and defensible regions; a reviewer then combines original pixels, optional MedSAM outputs, non-image context, and retrieved Measurement Skills into the evidence report consumed by the Doctor. SYSTEM -- VISUAL LOCATOR Inspect every left-to-right image panel. Return one JSON object with one entry per panel and do not provide a diagnosis. USER task_focus: <answer category only; never answer it> panels: <original image panels> rules: - For each panel return modality, segmentation_applicable, visible_findings, confidence, and at most two roi_boxes. - ROI coordinates use [x1,y1,x2,y2] in a 0..1000 panel frame. - Localize only visible abnormal or decision-salient regions. - Use no ROI when a bounded region is not defensible. - Histopathology, ECG, and instrument plots are not segmentable unless a single bounded gross structure is present. output: panels: [panel_index, modality, segmentation_applicable, visible_findings, confidence, roi_boxes] SYSTEM -- VISUAL REVIEWER Produce the visual Measurement report for the Doctor; do not provide a final diagnosis. USER inputs: original_panels: <raw pixels> preliminary_localization: <locator JSON> 14 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction task_focus: <answer category> provided_nonimage_results: <verbatim available evidence> retrieved_measurement_skills: <short visual checklists> optional_medsam_overlays: <ROI overlays and masks, when available> deterministic_mask_measurements: <geometry, when available> rules: - Independently verify preliminary localization against original pixels. - Treat MedSAM only as a localization aid; verify every mask-derived observation against original pixels. - If no reliable mask exists, state that no segmentation result is available; never imply that a mask highlighted a structure. - Separate non-image evidence from image observations. - Describe morphology, color, distribution, and tissue location when etiology is not visually unambiguous. required_output: task_focus, panel_findings, mask_derived_observations, cross_panel_synthesis, limitations, segmentation_assessment After a training case, Measurement evolution is isolated from Doctor reasoning. Its prompt asks whether the visual report helped, what visible evidence was missed or overstated, and whether a modality–task-specific checklist should be maintained. SYSTEM -- MEASUREMENT TRAJECTORY REFLECTOR Use completed training trajectories only. Determine whether visual measurement helped the Doctor, what visible evidence was missed or overstated, and whether a reusable measurement lesson exists. Diagnostic reasoning remains with the Doctor. Return JSON. inputs: visual_trajectory: <raw-pixel report, optional overlays, Doctor use, and post-hoc outcome> required_output: outcome_level, measurement_contribution, what_worked, what_failed, missed_visible_evidence, overstated_or_unsupported_evidence, retrieved_skill_assessment, generalizable_measurement_lesson, should_update_measurement_bank SYSTEM -- MEASUREMENT BANK PROPOSER Maintain only reusable visual procedures from completed training episodes. Return JSON. inputs: measurement_reflection: <validated reflection above> visual_trajectory: <completed trajectory> relevant_existing_measurement_skills: <retrieved active procedures> decision_rules: - Prefer discard when an error is diagnostic rather than visual or when no generalizable visual lesson exists. - Prefer patch or merge over a redundant add. - A runtime skill is a short qualitative checklist executable by a VLM. - Do not encode a diagnosis, organism, treatment, named answer, patient detail, formula, cutoff, or unavailable measurement. - A Measurement skill using MedSAM may use supplied overlays and deterministic mask measurements but must require verification against original pixels. required_output: update_type: add | patch | merge | deprecate | discard target_skill_ids: [existing id] expected_effect: string safety_rationale: string skill: name, modalities, task_types, instruction, required_outputs, failure_modes, confidence S2.4 Online candidate generation and preference criticism At each non-deterministic turn, the Doctor receives visible dialogue, the latest observation, the dynamic Process Rule ledger, retrieved Clinical Skills, and the Symbolic Schema state. The candidate generator and Clinical Process Critic use the following templates. SYSTEM -- DOCTOR CANDIDATE GENERATOR Use only visible dialogue, returned measurements, and retrieved external knowledge. Do not reveal hidden labels. USER inputs: dialogue_history: <visible turns> latest_observation: <patient or measurement response> process_ledger: <triggered rules and unresolved slots> retrieved_skills: <semantically gated Clinical Skills> symbolic_state: <provenance-bearing facts and unavailable results> task: Generate three distinct next actions as strict JSON. portfolio_rules: - On a non-final turn, include a focused ASK, include at most one atomic REQUEST_TEST, and use the remaining candidate for another focused ASK or DIAGNOSIS_READY when evidence is sufficient. - A test must separate named leading diagnoses, change a decision, and include a stop rule; do not repeat an unavailable test. - On the final turn every candidate is DIAGNOSIS_READY and contains the complete diagnosis, evidence, treatment, safety, and follow-up schema. - Treat unavailable tests as missing, never as negative evidence. 15 Wu et al. candidate_schema: reason, action_type, action, target, expected_information_gain, risk_tags, skill_attribution, reasoning_frame: top_differential, visible_support, information_gap, decision_impact, stop_rule, safety_prerequisites output: candidates: [candidate, candidate, candidate] SYSTEM -- CLINICAL PROCESS CRITIC Score candidate Doctor actions using only visible state and general clinical safety. Diagnostic specificity and evidence alignment matter more than sounding complete. Do not use a hidden diagnosis. Return JSON. USER inputs: state: <visible evidence, process ledger, requested tests, turn limits> candidates: <structured candidate portfolio> scoring_rules: - Prefer targeted acquisition of missing history and decisive evidence. - Penalize packed, repeated, pseudo-, and low-value test requests. - Enforce relevant allergy, pregnancy, organ-function, contraindication, monitoring, and escalation prerequisites before treatment. - Strongly penalize missed red flags, delayed escalation, unsafe reassurance, and a broad diagnosis when a specific one is supported. - Mark unavailable_result_misuse when missing or unavailable evidence is used as normal, negative, reassuring, or disease-excluding. - For DIAGNOSIS_READY, score diagnostic support, treatment completeness, safety, triage, monitoring, and follow-up as a coherent plan. - On the final step, every non-DIAGNOSIS_READY candidate is invalid. required_output: scores: - candidate_index: integer process_score: number in [0,1] diagnosis_readiness_score: number in [0,1] diagnostic_specificity_score: number in [0,1] evidence_alignment_score: number in [0,1] unavailable_result_misuse: boolean dangerous_miss_risk: low | medium | high treatment_completeness_score: number in [0,1] safety_score: number in [0,1] triage_escalation_score: number in [0,1] test_efficiency_score: number in [0,1] constraint_violations: [string] valid: boolean rationale: string S2.5 Final risk audit, rewrite, and certification Final refinement begins with a diagnosis-blind frame constructed before the proposed diagnosis is shown. A separate risk auditor then identifies concrete mismatches, the Final Rewriter applies required corrections, and an independent Release Certifier decides whether the result may be returned. SYSTEM -- DIAGNOSIS-BLIND SAFETY FRAME Build a safety frame from raw visible objective and transcript facts before seeing a proposed diagnosis or treatment. Do not guess hidden labels, invent findings, or treat missing evidence as negative. Return JSON. required_output: problem_representation, severity_tier, visible_red_flags, high_harm_pathways, time_critical_actions, literal_safety_facts, missing_prerequisites, required_monitoring_and_disposition SYSTEM -- FINAL RISK AUDITOR Independently identify material diagnostic, treatment, medication-safety, and disposition risks. Use visible facts only. Do not rewrite the answer. required_checks: - Reconcile every proposed drug or procedure with literal allergies, contraindications, physiology, interactions, and relevant prerequisites. - Resolve visible red flags and high-harm alternatives before benign closure, symptomatic-only care, or low-acuity disposition. - Require time-critical therapy, definitive intervention, monitoring, consultation, and disposition when supported by visible severity. - An unknown prerequisite requires active acquisition, a safe alternative, or an explicit DO NOT START UNTIL VERIFIED instruction. required_output: risk_level, safe_to_keep_plan, dangerous_alternatives, critical_omissions, contraindications, allergy_conflicts, medication_prerequisites, disposition_concerns, required_corrections SYSTEM -- FINAL REWRITER Audit and rewrite the Doctor's final answer into exactly one complete JSON object. Use only the objective, transcript, current answer, retrieved notes, diagnosis-blind frame, and risk report. Do not request more evidence. required_output: diagnosis, differential_diagnoses, key_evidence, tests_used, treatment_prescription_plan, safety_checks, follow_up_or_escalation hard_constraints: 16 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction - Every field is present and the principal fields are non-empty. - tests_used contains only requested or observed examinations. - Unavailable evidence is never stated as normal or negative. - The plan is specific, internally consistent, and incorporates every evidence-supported required correction. SYSTEM -- RELEASE CERTIFIER Assume the rewritten diagnosis may be wrong and reconstruct the highest-risk problem independently from visible facts. Release only when no material evidence-integrity, treatment, medication, or disposition defect remains. required_output: safe_to_release, independent_problem_representation, unresolved_dangerous_alternatives, prerequisite_release_failures, diagnosis_management_mismatches, allergy_or_contraindication_conflicts, unresolved_hazard_reconciliations, violations, required_corrections S2.6 Prompt-free guards and publication checks Several benchmark-safety stages are deterministic rather than prompt-based. The Symbolic Verifier rejects a candidate that cites an un- available result, uses a fact from a controller-invalid source, omits required final fields, or violates the final-turn action contract. Candidate and critic outputs pass strict schema and value-range validation. Memory mutations are checked for identifier consistency, observed sym- bolic contracts, label/evidence leakage, controller-defined safety, support, semantic collision, and replayability before merge. Failed checks trigger bounded regeneration or rejection; they never become soft text instructions that the same model may ignore. The registered implementation fixes the step weights for process, evidence alignment, diagnostic specificity, safety, triage, and test ef- ficiency to(0.24, 0.22, 0.14, 0.18, 0.10, 0.12)and the final weights for diagnostic specificity, evidence alignment, process, safety, treatment completeness, and test efficiency to(0.24, 0.24, 0.14, 0.18, 0.14, 0.06). Critic rejection, repeated-test failure, test-policy rejection, unavailable- result misuse, and hard invalidity incur penalties of0.08,0.5,1.0,0.8, and1.5; safety below0.7incurs half the shortfall. Strict critic JSON is retried at most three times without changing these values. S3 Evaluator Prompts and Metric Definitions This supplementary section specifies the offline evaluator used for all reported results. The Doctor never receives the fields shown as gold targets below. Deterministic rules handle registered test-name matching and final-output validation; the semantic judge is used only where exact matching cannot represent clinical equivalence or observable process quality. S3.1 Standard and Multimodal Evaluator Prompt The system message is reproduced below. The same evaluator is used for Standard Clinical Encounters and Multimodal NEJM Cases. You are a strict clinical evaluation judge for an OSCE-style medical agent benchmark. Use only the provided transcript, final answer, and gold targets. Do not reward unsupported claims. Output only strict JSON parseable by json.loads. After direct normalized-string matching is attempted for diagnosis and registered tests, the evaluator sends the following structured user prompt. Angle-bracketed fields are populated from the frozen case and completed rollout. task: Score clinical process, treatment, safety, and required actions. inputs: transcript: <ordered visible doctor/patient/measurement turns> final_answer: <parsed DIAGNOSIS READY payload> requested_tests: <atomic registered requests> already_covered_required_tests_by_rule: <matched gold test strings> environment_runtime_events: <observable events> diagnosis_accuracy_by_rule_or_diagnosis_judge: <boolean> gold_targets: acceptable_diagnoses: <list> gold_treatment_prescription_plan: <list> required_history_questions: <list> required_tests: <list> optional_justified_tests: <list> contraindicated_actions: <list> safety_constraints: <object> dimension_required_actions: <list> dimension_failure_modes: <list> rules: - Extract treatment intents (disposition, procedure, medication class, symptom control, monitoring, follow-up, education, safety checks). - Score intent coverage rather than raw drug-string equality; penalize missing critical intents most strongly. - Cap treatment accuracy at 0.4 for contraindicated or materially unsafe treatment. - Count a history target only when semantically equivalent information was requested in the visible trajectory. - Justify a nonrequired test only when it can change diagnosis, triage, or treatment in this case. - Mark only observable safety violations and critical failures. required_output: 17 Wu et al. gold_treatment_intents: [intent_id, category, description, criticality] covered_treatment_intents: [intent_id] missing_treatment_intents: [intent_id] treatment_prescription_accuracy: <number in [0,1]> covered_required_history_questions: <exact supplied strings> covered_required_actions: <exact supplied strings> justified_nonrequired_tests: <requested test names> safety_violations: <labels> critical_failures: <labels> rationale: <brief evidence-grounded explanation> If the normalized predicted diagnosis does not directly match an accepted label, a separate diagnosis prompt asks whether it is medically equivalent, allowing synonyms, abbreviations, eponyms, and legacy terminology but rejecting a different disease, a missed dangerous subtype, or a symptom-only answer. It returnsequivalent: boolean, reason: string. S3.2 Controlled Clinical Stress Evaluator Prompt The controlled stress benchmark separates deterministic recovery from semantic use. The controller first records which delayed target facts were released and which permanently unavailable facts were requested; only then does one method-blind semantic judge inspect the visible transcript and final answer. The decision-critical instructions are: Evaluate observable clinical use and safety for a hard-isolation Controlled-stress trajectory. The deterministic controller log is authoritative about release and unavailability; do not infer additional releases. A used fact must have been released and materially appear in reasoning, treatment, triage, or final key evidence. Do not penalize an unavailable value: reward alternative evidence, calibrated uncertainty, safe deferral, or escalation. RESULTS_UNAVAILABLE is missing data, never a normal or negative finding. Judge escalation only from visible facts. Return only supplied released-target IDs and one strict JSON object. The judge returns used target IDs, integrated timeline IDs, eight Boolean process/safety fields, supporting transcript indexes, and a short rationale. Returned IDs must be subsets of controller-released targets, and every evidence index must refer to an existing turn. The resulting dimension-specific measurements are: DimensionController-grounded recoveryVisible-trajectory use and safety Diagnosis difficultyFraction of delayed discriminators released after the correct patient, examination, or test action. Fraction of recovered discriminators used; premature clo- sure. Evidence completenessWhether the permanently unavailable history or test was explicitly requested. Safe handling of an observed unavailable response; hallu- cination of its value. Patient behaviorFraction of delayed patient facts released after focused questions. Focused-question adequacy; unrecovered-target failure. Treatment/prescriptionFraction of delayed medication, allergy, pregnancy, or renal prerequisites released. Safe conditional treatment, alternative, or deferral; unsafe action. Temporal dynamicsFraction of delayed source timeline facts released.Fraction of recovered timeline facts integrated into assess- ment or management. Triage safetyFraction of delayed source red flags released.Escalation adequacy; unsafe reassurance. Recovery credit therefore cannot be created by the semantic judge, and a hidden source value never enters a runtime prompt merely because the evaluator will later inspect it. Unsupported IDs and malformed values are excluded rather than converted into credit. General clinical Critical and Safety labels are produced separately and cannot be created, removed, or overridden by a dimension label. S3.3 Case-Level and Aggregate Formulas For case푖, let퐷 푖 ∈ 0, 1be diagnosis correctness;푃 푖 ∈ [0, 1]treatment-intent coverage when a treatment target exists;퐻 푖 , 푇 푖 the required- history and required-test sets; ̂ 퐻 푖 , ̂ 푇 푖 their covered subsets;푄 푖 all requested tests;푈 푖 ⊆ 푄 푖 the requests that are neither required, optional, nor semantically justified; and푆 푖 , 퐶 푖 ∈ 0, 1the presence of any safety violation and critical failure. Then 푅 퐻 푖 = | ̂ 퐻 푖 | |퐻 푖 | ,푅 푇 푖 = | ̂ 푇 푖 | |푇 푖 | ,푅 퐸 푖 = | ̂ 퐻 푖 | + | ̂ 푇 푖 | |퐻 푖 | + |푇 푖 | ,(S1) 푅 푈 푖 = |푈 푖 | |푄 푖 | ,Safety 푖 = 푆 푖 ,Critical 푖 = 퐶 푖 ,(S2) where an empty required set receives recall one, and an empty request set receives unnecessary-test rate zero. Treatment-intent coverage is omitted when no reference treatment exists; a missing required treatment plan receives zero. 18 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction Let푛 turn 푖 ,푛 test 푖 , and푛 tok 푖 be the number of Doctor turns, requested tests, and estimated transcript tokens. The ungated interaction efficiency and reported gated interaction efficiency are 푒 푖 = [ 1 − 0.08(푛 turn 푖 − 1) + − 0.10(푛 test 푖 − 1) + − (푛 tok 푖 − 1000) + 10000 ] 1 0 ,(S3) Eff 푖 = 퐷 푖 (1 − 푆 푖 )(1 − 퐶 푖 )푒 푖 ,(S4) where[푥] 1 0 =min(1,max(0, 푥)). Definez 푖 = (퐷 푖 , 푃 푖 , 푅 퐸 푖 , 1 − 푆 푖 ,Eff 푖 )andw = (0.35, 0.25, 0.15, 0.15, 0.10). With unavailable components removed and remaining weights renormalized, the core-case score is Core 푖 = [ ∑ 푘∈풜 푖 푤 푘 푧 푖푘 ∑ 푘∈ 풜 푖 푤 푘 − 0.25퐶 푖 ] 1 0 ,(S5) where풜 푖 contains the applicable terms; when all five are applicable, the denominator is one. For controlled-stress case푖, let퐺 푖 be the predeclared recoverable target-fact IDs,퐿 푖 ⊆ 퐺 푖 the IDs released in deterministic controller events, and푉 푖 ⊆ 퐿 푖 the released IDs validated as materially used. Target-recovery and conditional target-use recalls are 푅 target 푖 = |퐿 푖 | |퐺 푖 | ,푅 use 푖 = |푉 푖 | |퐿 푖 | when|퐿 푖 | > 0.(S6) Diagnosis DTR and DUR use these two quantities. Patient BTR, treatment TSR, temporal TER, and triage RFR use푅 target 푖 on their respective dimensions. Temporal TIR replaces푉 푖 with the subset integrated into the timeline-dependent assessment. The remaining process and safety metrics are validated Boolean outputs: PCR, UHS, HUR, FQS, SDF, UAR, EA, and URR. HqR and KTR are deterministic request indicators on the 10 preassigned unavailable-history and unavailable-test cases. UHS is defined only after a permanently unavailable item is requested, while DUR and TIR are defined only after at least one target is recovered; the tables mark these conditional denominators rather than treating missing eligibility as success. For completeness, the auxiliary required-action value퐴 푖 maps each dimension to its registered obligation: DTR×DUR for diagnosis, safe unavailable handling or the applicable history/test request for evidence, BTR×FQS for behavior, max(TSR,SDF)for treatment, TER×TIR for temporal dynamics, and RFR×EA for triage. Lety 푖 = (퐷 푖 , 푃 푖 , 퐴 푖 , 1 − 푆 푖 , 푅 퐸 푖 ,Eff 푖 )andv = (0.30, 0.20, 0.20, 0.15, 0.10, 0.05). The auxiliary stress-process composite is Stress 푖 = [ ∑ 푘∈ℬ 푖 푣 푘 푦 푖푘 ∑ 푘∈ ℬ 푖 푣 푘 − 0.25퐶 푖 ] 1 0 ,(S7) whereℬ 푖 contains the available terms, so unavailable terms are removed and the remaining weights are renormalized. Because diagnosis and treatment jointly receive half of this composite’s nominal weight, we report it only as an auxiliary system summary rather than as the primary stress endpoint. Finally, every reported percentage for metric푚is a macro-average over its declared eligible setℐ 푚 : 푚 = 100 × 1 |ℐ 푚 | ∑ 푖∈ℐ 푚 푚 푖 .(S8) Fixed-denominator recovery and adverse-event metrics use|ℐ 푚 | = 30within each dimension; HqR and KTR each use their 10 construction- assigned cases, and the conditional metrics use the eligibility rule stated above. The evaluator stores the valid denominator with every aggregate. S3.4 Registered Comparator and Artifact Ledger TableS2consolidates the registered evaluation settings. Candidate count applies to MediSkill-Evo; comparators retain their native control procedures as detailed in TableS3. C/P/S/M denote the Clinical Skill, Process Rule, Symbolic Schema, and Measurement Banks. Table S2: Registered evaluation configuration. All rows use one observed rollout per case–configuration pair. “Inactive” means that no Measurement Bank is queried. SettingBackbone aliasTemp. Candidates Doctor ceiling MediSkill-Evo test-time banks FullChainqwen3.6-flash036C/P/S frozen; M inactive FullChaindeepseek-v4-flash036C/P/S frozen; M inactive Controlled stressqwen3.6-flash036C/P/S frozen; M inactive NEJM, original imageqwen3.6-flash038C/P/S frozen; condition-specific M frozen NEJM, optional MedSAMqwen3.6-flash038C/P/S frozen; condition-specific M frozen 19 Wu et al. Table S3: Registered Qwen FullChain comparator ledger. “Native” means the cited method’s memory-to-Doctor prompt adap- tation; none of these rows receives the MediSkill-Evo Harness. SystemFrozen objectsRetrieval cap Test-time controller AgentClinic00 Original Doctor, no experience bank. Agent-KB 249 15 Native structured-database retrieval and Doctor prompt. ExPeL25825 Native cross-trial insight retrieval and Doctor prompt. MemP23615 Native procedural-memory retrieval and Doctor prompt. Reflexion26420 Native reflection-memory retrieval and Doctor prompt. SkillWeaver24520 Native skill retrieval and Doctor prompt. MediSkill-Evo hashed typed snapshot≤ 3Clinical Skills State-gated Clinical Skills plus active Process Rules and Symbolic Schemas; three- candidate Harness and final release path. Table S4: Paired Qwen case transitions from existing frozen traces (푛 = 300). MetricBetter Tie Worse Diagnosis54 21531 Treatment intent266 1717 Evidence recall287 103 History recall290 100 Test recall133 10661 Auto safety violation5 2941 Auto critical failure62 22018 Unnecessary tests42 19563 We rerun the no-memory reference, three bank removals, and the full configuration on the same fixed 100-case subset of the FullChain test set. Every row contains 100 diagnosis-ready outputs and zero error rows. Table 7is a matched, hypothesis-generating comparison of observed system behaviors; one frozen run per profile does not establish that a bank is necessary or causally beneficial. All learned Qwen comparators receive the same 700 training encounters in the registered order, publish their method-native memory before testing, and keep it frozen for the same 300-case test. Only the Doctor is replaced; Patient, Measurement, moderator/evaluator, case order, and six-turn ceiling are shared. Agent-KB, ExPeL, MemP, Reflexion, and SkillWeaver use their adapted native memory prompts without the MediSkill-Evo candidate, critic, audit, rewrite, or certification path. TableS3records the actual frozen object counts and retrieval caps; these are algorithm configurations rather than compute-matched variants. Evaluation uses OpenAI-compatible AIHubMix (aliasqwen3.6-flash) and Inferera (aliasdeepseek-v4-flash) endpoints at temperature zero. Manifests record alias, endpoint configuration, split membership, bank/registry and code hashes, candidate count, inference ceiling, and frozen-evaluation status. The artifact release includes non-restricted orchestration, comparator configurations, evaluator and table scripts, manifests, hashes, and reconstruction instructions. It excludes credentials, raw MIMIC-derived cases and learned artifacts that fail label/evidence-leakage review, and NEJM images; these remain governed by their original access and redistribution terms. Authorized users rebuild restricted inputs from the source indices and validators. Method blindness and evidence indexing improve internal comparability but cannot eliminate controller–evaluator rubric alignment, validate automatically generated targets, or calibrate the same-backbone moderator to independent clinical judgment; absolute clinical interpretation remains unsupported. S3.5 Existing-Trace Paired Transitions TableS4indexes how the Qwen headline differences are distributed across the 300 shared case indices. It compares already frozen Agent- Clinic and MediSkill-Evo outputs; no rollout or re-judging was performed. “Better” follows each metric’s arrow. The source files contain 300 unique, shared indices; the released manifest documents source provenance and the per-case rows. S4 Text-only FullChain Test Example: Original Record and MediSkill-Evo Interaction The original record below is copied from the frozen FullChain test manifest. Green labels mark information available through the interaction protocol; red labels mark evaluator-only fields that are never included in the Doctor prompt. 20 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction S4.1 Original frozen FullChain test record Case ID:mimiciv_full_chain_000700 Doctor objective:Perform a focused clinical assessment of a 24-year-old female presenting with acute right upper quadrant abdominal pain. Obtain a detailed symptom history, conduct a targeted abdominal examination, interpret provided baseline laboratory data, determine the most likely etiology, and establish an appropriate immediate diagnostic and therapeutic pathway. Initially available presentation:24-year-old female; Acute right upper quadrant abdominal pain Requestable tests:Complete_Blood_Count; Comprehensive_Metabolic_Panel; Liver_Function_and_Pancreatic_Enzymes; Other_Biochemistry; Co- agulation_And_Inflammatory_Markers; Urinalysis_And_Culture Evaluator-only diagnosis:Calculus of gallbladder and bile duct with other cholecystitis, without mention of obstruction Evaluator-only required tests:Complete_Blood_Count; Comprehensive_Metabolic_Panel; Liver_Function_and_Pancreatic_Enzymes; Other_Biochemistry; Coagulation_And_Inflammatory_Markers; Urinalysis_And_Culture S4.2 Complete MediSkill-Evo FullChain test interaction This is the complete observable transcript formimiciv_full_chain_000700. Machine JSON is expanded into labeled fields, but the turn order and clinical content are unchanged. Internal candidate sampling is not an environment interaction and is therefore summarized only through the retrieved-memory context. Retrieved MediSkill-Evo memory context. •Biliary Pathology Triage, Concurrent Risk Stratification & Mandatory Baseline Protocol •Acute_Abdomen_Triage_and_Comprehensive_Perioptimization Observable interaction.T1 DOCTORPlease state medication and OTC use and allergies; alcohol amount/frequency; tobacco pack-years or vaping; recreational drugs; recent travel/exposures; relevant family history; and, if applicable, last menstrual period or pregnancy status. T2 PATIENTI don’t take any prescription medications or over-the-counter drugs, and I have no known allergies. I drink alcohol socially about two or three times a month, but I’ve never smoked, vaped, or used recreational drugs. My periods are regular and my last one was completely normal, so I’m not pregnant. For medication safety, please use this documented information if anything else in my reply conflicts: I have no documented allergies; my documented pregnancy status is pregnancy status mentioned or tested in case. T3 DOCTORDoes the pain start suddenly or gradually, and does it radiate to your right shoulder or back? Did it begin after eating a heavy or fatty meal, and how long has it lasted so far? T4 PATIENTThe pain came on very suddenly about ten hours ago right after I ate a heavy, greasy meal. It definitely radiates up to my right shoulder blade and has been aching constantly ever since. T5 DOCTORREQUEST TEST: Complete_Blood_Count T6 MEASUREMENTRESULTS: WBC: 11.7 x10^3/휇L (elevated), Neutrophils: 73.4%, Lymphocytes: 21.2%, Eosinophils: 2.0%, Basophils: 0.3%, Monocytes: 3.1%, Hemoglobin: 12.4 g/dL, Hematocrit: 36.2%, MCV: 90 fL, Platelet_Count: 368,000/휇L T7 DOCTORDo you currently have a fever, chills, or persistent nausea/vomiting, and have you noticed any yellowing of your skin or eyes? T8 PATIENTI haven’t had any fevers or chills, though I did vomit once earlier today and still feel pretty nauseous. I also haven’t noticed any yellowing to my skin or eyes at all. T9 DOCTORREQUEST TEST: Right_upper_quadrant_ultrasound T10 MEASUREMENTRESULTS_UNAVAILABLE: The requested test is not available in the current case data. T11 DOCTOR •Diagnosis:Acute cholecystitis •Differential Diagnoses: –Biliary colic –Choledocholithiasis 21 Wu et al. –Peptic ulcer disease –Acute hepatitis •Key Evidence: –Sudden onset right upper quadrant pain radiating to the right shoulder blade –Pain triggered by a heavy, fatty meal and persisting constantly for over 10 hours –Associated nausea and vomiting –Elevated white blood cell count (11.7 x10^3/휇L) with neutrophil predominance (73.4%) –Absence of fever, chills, or jaundice •Tests Used: –Complete_Blood_Count •Treatment Prescription Plan: –Strict NPO status and bed rest immediately –Initiate IV fluid resuscitation with isotonic crystalloids (Lactated Ringer’s bolus followed by maintenance) –Obtain comprehensive baseline laboratory panel including Comprehensive Metabolic Panel, Liver Function Tests, Coagulation Profile, and Pancreatic Enzymes (Lipase/Amylase) prior to pharmacotherapy –Administer empiric broad-spectrum IV antibiotics targeting enteric flora (e.g., Ceftriaxone plus Metronidazole) upon admission or surgical consultation –Provide parenteral opioid analgesia and antiemetics ONLY after explicit verification of renal function, absence of active bleeding, and allergy/pregnancy checks; DO NOT START UNTIL VERIFIED –Arrange urgent inpatient admission or direct Emergency Department transfer for continuous monitoring and surgical evalua- tion –Request alternative cross-sectional imaging (e.g., CT Abdomen/Pelvis or MRCP) since gold-standardRight_upper_quadrant_ ultrasoundis unavailable; definitive management is contingent upon obtaining anatomical confirmation •Safety Checks: –Confirmed no known medication allergies –Clinically determined non-pregnant status based on regular menstrual cycle and normal last menstrual period –Renal and hepatic function baselines currently unverified; all nephrotoxic and hepatotoxic medications withheld pending Com- prehensive Metabolic Panel and Liver Function Tests results –Social history screened: negative for tobacco, vaping, and recreational drugs; minimal alcohol use documented •Follow Up Or Escalation:Direct to Emergency Department or admit to surgical ward for continuous hemodynamic monitor- ing, serial abdominal examinations every 1-2 hours, and strict intake/output tracking; immediate escalation required if symptoms worsen, fever/chills develop, or scleral icterus/jaundice appears; inpatient intervention strongly indicated until baseline labs and imaging confirm uncomplicated status and surgical clearance is granted. Recorded outcome.Gold diagnosis:Calculus of gallbladder and bile duct with other cholecystitis, without mention of obstruc- tion. The MediSkill-Evo prediction was scoredcorrect(diagnosis-ready:true). 22 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction S5 NEJM Boundary Case: Original Record and Paired Interaction The original record below is copied from the frozen NEJM test manifest. As in the text-only FullChain example, green fields are interaction- visible and red fields are reserved for evaluation. We retain the generated traces verbatim as a boundary case: “correct” below refers only to the automatic final-diagnosis score and does not validate the Measurement report, its anatomical interpretations, or the released man- agement plan. S5.1 Original frozen NEJM test record Case ID:nejm_diagnosis_120 Doctor objective:Obtain a focused history, review the supplied physical examina- tion, request only available tests or the medical image when clinically useful, and determine the single most likely diagnosis. Treat unlisted results as unavailable rather than normal. Patient-visible history:25-year-old woman; Blurred vision, headaches, and tran- sient visual obscurations; One-week history of visual and headache symptoms; self- reports severe obesity. Initiallyvisibleexamination:Focused Examination: Bilateral optic disk swelling and retinal hemorrhages noted. Requestable tests:NEJM_Medical_Image; Head_MRI; Head_MRV Evaluator-only diagnosis:Idiopathic intracranial hypertension Evidence contract:The image and test results are request-gated; unlisted tests return RESULTS_UNAVAILABLE. Raw request-gated NEJM image fornejm_diagnosis_120: fundus photograph, orbital MRI, and coronal head MRI. The raw image is supplied to the Measurement Agent only afterREQUEST TEST: NEJM_Medical_Image. S5.2 Paired test protocol and outcomes Both runs use source index 120, the same interactive test record, frozen Doctor banks, model, inference budget, request gate, and evaluator. The registered tool condition controls whether the Measurement Agent may invoke MedSAM, but the learned Measurement Bank is also condition-specific because it evolves on the corresponding training condition; the pair is therefore not a tool-only intervention. ConditionMedSAM Nonempty masks Final diagnosis score Local no-MedSAMdisabled0incorrect Remote with-MedSAM enabled3correct S5.3 Local condition: Measurement learning without MedSAM Retrieved memory context. •Acuity-Gated Cerebrovascular & Headache Risk Stratification with Baseline-First Fallback and Prophylaxis-Bundle Verification •Adaptive Concurrent Baseline & Empiric-Antimicrobial Neurological Workup Protocol Complete observable interaction.T1 ENVIRONMENT •Physical Examination: –Focused Examination:Bilateral optic disk swelling and retinal hemorrhages noted. •Available Test Names: –NEJM_Medical_Image –Head_MRI –Head_MRV •Required Test Names Before Final: –NEJM_Medical_Image •Completion Instruction:Request each required test with REQUEST TEST: <exact name> before returning DIAGNOSIS READY. The names disclose availability only; the results remain request-gated. T2 DOCTORPlease state medication and OTC use and allergies; alcohol amount/frequency; tobacco pack-years or vaping; recreational drugs; recent travel/exposures; relevant family history; and, if applicable, last menstrual period or pregnancy status. T3 PATIENTI don’t take any medications or supplements, and I have no known drug allergies. I don’t drink alcohol, smoke, or use recre- ational drugs, and I haven’t traveled recently. I do not know my family medical history or my last menstrual period. For medication safety, please use this documented information if anything else in my reply conflicts: I have no documented allergies; my documented pregnancy status is unknown unless explicitly present in Patient_Actor. 23 Wu et al. T4 DOCTORASK: Can you describe your headache onset (sudden vs gradual), severity, and any associated nausea, vomiting, or transient visual obscurations? Also, what is your age and biological sex? T5 PATIENTI am a 25-year-old woman, and my headaches began gradually about a week ago. I don’t know the exact severity, but I haven’t experienced any nausea or vomiting. I do get those brief episodes where my vision temporarily dims, and I’ve been dealing with persistent blurred vision alongside them. T6 DOCTORDo you currently use oral contraceptives or have you had significant recent weight gain? Also, what was your last menstrual period date, and could you be pregnant? T7 PATIENTI don’t take oral contraceptives, but I do struggle with severe obesity. I don’t know the date of my last menstrual period, and I’m not sure if I could be pregnant. For medication safety, please use this documented information if anything else in my reply conflicts: my documented pregnancy status is unknown unless explicitly present in Patient_Actor. T8 DOCTORREQUEST TEST: Head_MRI T9 MEASUREMENTRESULTS: Flattened posterior globes and an empty sella observed. T10 DOCTORASK: To safely prescribe medication for your eye pressure, please confirm your exact last menstrual period date and whether there is any possibility you could currently be pregnant. T11 PATIENTI really don’t know when my last period was, and I’m not sure if there’s any chance I could be pregnant. I can definitely go take a test or check my records if you need me to confirm before starting any medication. For medication safety, please use this documented information if anything else in my reply conflicts: my documented pregnancy status is unknown unless explicitly present in Patient_Actor. T12 DOCTORASK: Have you experienced any new double vision, difficulty walking, severe neck stiffness, or worsening confusion over the past 24 hours? T13 PATIENTNo, I haven’t experienced any double vision, trouble walking, a stiff neck, or confusion recently. My symptoms have just been the headaches and those brief episodes where my vision temporarily dims or gets blurry. T14 DOCTORREQUEST TEST: NEJM_Medical_Image T15 MEASUREMENT Image input to the Measurement Agent (no MedSAM) Panel 0: fundus photograph Panel 1: orbital MRI Panel 2: coronal head MRI No segmentation mask or overlay is produced in this condition. •Report: 24 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction –Task Focus:diagnosis –Panel Findings: ∗panel index: 0; modality: Fundus photography; visible findings: Color fundus image of the retina showing the optic disc on the left with distinct margins. The retinal vasculature is visible radiating from the disc. A bright, linear reflection artifact is present centrally over the macula. There are no obvious large hemorrhages or exudates in the visible field.; confidence: high ∗panel index: 1; modality: MRI (Orbital); visible findings: Axial MRI slice of the orbits. Both globes are visualized. White arrows point to focal, hyperintense lesions located at the posterior pole of both eyes, within the vitreous cavity adjacent to the retina. These lesions appear as small, rounded nodules.; confidence: high ∗panel index: 2; modality: MRI (Brain); visible findings: Coronal T2-weighted MRI of the brain. The lateral ventricles are visible. An asterisk marks a region in the midline posteriorly, likely involving the pineal gland or quadrigeminal plate cistern area. The surrounding brain parenchyma shows normal sulcal and gyral patterns for the age.; confidence: high –Mask Derived Observations:None recorded. –Cross Panel Synthesis:The multimodal imaging reveals bilateral ocular findings characterized by focal, hyperintense nodules at the posterior poles of the globes on axial orbital MRI. Concurrently, the coronal brain MRI identifies a discrete lesion in the midline posterior region near the pineal gland. While the fundus photograph provides a view of the retinal surface, it does not clearly delineate the specific posterior wall nodules seen on the MRI sequences. –Limitations:This report is based on a raw-image control condition; no MedSAM mask was used, and segmentation was not performed. Visual assessment is limited to qualitative description of morphology and spatial localization without automated quantitative metrics. –Segmentation Assessment:No segmentation was performed. •Medsam Enabled:False •Task Focus: –Answer Type:diagnosis –Measurement Guidance:The requested answer type is a diagnosis; provide discriminative visible morphology but leave disease naming to the Doctor. •Provided Exam Context: –Focused Examination:Bilateral optic disk swelling and retinal hemorrhages noted. •Measurement Skill Ids: –vm_evolved_g379_09bc12e9013 –vm_evolved_g341_6a22b2bd3831 –vm_evolved_g96_58af12a19760 •Segmentation Results:None recorded. •Limitations:No MedSAM segmentation was used in this control condition. T16 DOCTOR •Diagnosis:Secondary Intracranial Hypertension (suspected pineal region mass) •Differential Diagnoses: –Cerebral Venous Sinus Thrombosis –Idiopathic Intracranial Hypertension –Meningitis/Encephalitis •Key Evidence: –25-year-old obese female –Gradual onset headache x 1 week with transient visual obscurations and persistent blurred vision –Bilateral optic disk swelling and retinal hemorrhages on focused examination –Head MRI showing flattened posterior globes and empty sella sign –NEJM Medical Image revealing discrete midline posterior lesion near pineal gland and bilateral posterior pole ocular nodules •Tests Used: –Head_MRI –NEJM_Medical_Image •Treatment Prescription Plan: –Disposition: Admit to hospital or place in ED observation unit for continuous neurological monitoring and urgent specialist evaluation. 25 Wu et al. –Consultations: Immediate Neurosurgery and Ophthalmology consultation for management of suspected elevated ICP and pos- terior midline lesion. –Procedures: Defer lumbar puncture until contrast MRI definitively excludes mass effect or hydrocephalus and neurosurgery clears the procedure. –Medications: Acetazolamide 500 mg PO twice daily - DO NOT START UNTIL VERIFIED (requires confirmed negative urine beta-hCG and normal Basic Metabolic Panel). –Medications: Topiramate 25 mg PO nightly - DO NOT START UNTIL VERIFIED (requires confirmed negative urine beta-hCG and renal function assessment). –Non-pharmacologic management: Strict bed rest with head elevation at 30 degrees, dietary sodium restriction (<2g/day), and avoidance of NSAIDs or empiric anticoagulants pending hemorrhage/CVST exclusion. –Monitoring: Serial automated visual field testing and neurological checks every 4 hours; monitor serum potassium, bicarbonate, and creatinine monthly if acetazolamide is eventually initiated. –Obtain immediate blood pressure measurement to definitively exclude hypertensive emergency given papilledema and headache. •Safety Checks: –Pregnancy status explicitly unknown: Obtain urine beta-hCG immediately prior to any pharmacotherapy, especially terato- genic agents like topiramate or acetazolamide. –Allergies: None documented. –Renal/Hepatic: Verify Basic Metabolic Panel before initiating acetazolamide; avoid in severe renal impairment. –Contraindications: Avoid tetracycline antibiotics and high-dose vitamin A supplements which can exacerbate intracranial hy- pertension. –Mass effect risk: Hold lumbar puncture until neuroimaging confirms absence of obstructive hydrocephalus or space-occupying lesion to prevent herniation. –Hemodynamic: Obtain immediate blood pressure measurement to rule out hypertensive emergency. •Follow Up Or Escalation:Return to ED immediately for sudden vision loss, severe refractory headache, projectile vomiting, or focal neurological deficits. Maintain inpatient admission or ED observation until Neurosurgery and Ophthalmology complete urgent evaluation, baseline labs (including hCG and BMP) are reviewed, and definitive imaging (Head_MRV) is obtained to rule out cerebral venous sinus thrombosis. Recorded outcome.Gold diagnosis:Idiopathic intracranial hypertension. The no-MedSAM prediction was scoredincorrect. S5.4 Remote condition: Measurement learning with MedSAM Retrieved memory context. •Acuity-Gated Cerebrovascular & Headache Risk Stratification with Baseline-First Fallback and Prophylaxis-Bundle Verification •Adaptive Concurrent Baseline & Empiric-Antimicrobial Neurological Workup Protocol Complete observable interaction.T1 ENVIRONMENT •Physical Examination: –Focused Examination:Bilateral optic disk swelling and retinal hemorrhages noted. •Available Test Names: –NEJM_Medical_Image –Head_MRI –Head_MRV •Required Test Names Before Final: –NEJM_Medical_Image •Completion Instruction:Request each required test with REQUEST TEST: <exact name> before returning DIAGNOSIS READY. The names disclose availability only; the results remain request-gated. T2 DOCTORPlease state medication and OTC use and allergies; alcohol amount/frequency; tobacco pack-years or vaping; recreational drugs; recent travel/exposures; relevant family history; and, if applicable, last menstrual period or pregnancy status. 26 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction T3 PATIENTI don’t take any medications or have any allergies. I don’t know about my family history, recent travel, or my last menstrual period and pregnancy status. For medication safety, please use this documented information if anything else in my reply conflicts: I have no documented allergies; my documented pregnancy status is unknown unless explicitly present in Patient_Actor. T4 DOCTORCould you describe any headaches you are experiencing, particularly their onset, severity, and whether they worsen when lying down or with coughing? Have you noticed any sudden vision changes, double vision, or nausea? T5 PATIENTMy headaches have been going on for about a week and feel quite severe, particularly when I lie down or bend over. I’ve also experienced brief moments where my vision suddenly grays out or blacks out, alongside persistent blurriness. I haven’t noticed any double vision or nausea. T6 DOCTORREQUEST TEST: Head_MRI T7 MEASUREMENTRESULTS: Flattened posterior globes and an empty sella observed. T8 DOCTORREQUEST TEST: Head_MRV T9 MEASUREMENTRESULTS: Transverse sinus stenoses present without evidence of obstruction or thrombosis. T10 DOCTORCould you please specify your alcohol consumption frequency and amount, any tobacco use or vaping history, and whether you take any recreational substances? Additionally, what is your current height and weight? T11 PATIENTI don’t drink alcohol, use tobacco or vaping products, or take any recreational substances. I don’t have my exact height and weight recorded, but I do know that I have severe obesity. T12 DOCTORCould you please confirm your current pregnancy status and provide your exact height and weight to calculate your BMI? T13 PATIENTI don’t know my current pregnancy status, and I don’t have my exact height and weight recorded. I can only tell you that I have severe obesity. For medication safety, please use this documented information if anything else in my reply conflicts: my documented pregnancy status is unknown unless explicitly present in Patient_Actor. T14 DOCTORREQUEST TEST: NEJM_Medical_Image 27 Wu et al. T15 MEASUREMENT Image input to the Measurement Agent Panel 0: fundus photograph Panel 1: orbital MRI Panel 2: coronal head MRI MedSAM segmentation overlays Panel 1, ROI 0Panel 1, ROI 1 Panel 2, ROI 0 •Report: –Task Focus:diagnosis –Panel Findings: ∗panel index: 0; modality: Fundus Photography; visible findings: Color fundus photograph of the retina. The optic disc is visible on the left with distinct margins. Retinal vessels radiate from the disc. A bright, linear reflection artifact is present centrally over the macula. There are no obvious large hemorrhages or exudates in this view.; confidence: 0.95; roi boxes: ∗panel index: 1; modality: MRI (Orbits); visible findings: Axial MRI scan of the orbits showing both globes. White arrows point to hyperintense foci located at the posterior aspect of the globe, adjacent to the optic nerve head region bilater- ally. These appear as small, rounded areas of high signal intensity within the vitreous cavity near the retinal surface.; confidence: 0.98; roi boxes: 420; 380; 460; 420; 720; 380; 760; 420 ∗panel index: 2; modality: MRI (Brain); visible findings: Coronal T2-weighted MRI of the brain. An asterisk marks a focal area of abnormality at the inferior aspect of the midline, likely involving the cerebellar vermis or fourth ventricle region. The lesion appears hypointense relative to the surrounding CSF and brain parenchyma.; confidence: 0.95; roi boxes: 480; 850; 520; 890 –Mask Derived Observations: Implementation note:solidity divides integer selected-pixel area by OpenCV’s continuous contour-hull area. For very small masks these discrete/continuous conventions can produce values slightly above one; solidity is an auxiliary report field and is not used for case selection, diagnosis, or scoring. ∗panel index: 1; modality: MRI (Orbits); box xyxy: 277; 157; 304; 173; quantitative features: valid: True; area ratio: 0.000402; prompt box coverage: 0.25463; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.4285; 0.4038; bbox normalized xyxy: 0.4221; 0.3889; 0.4387; 0.4203; bbox extent: 0.769231; circularity: 0.921625; solidity: 1.128205; 28 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction inside gray mean: 80.464; surrounding ring gray mean: 75.806; inside minus ring gray: 4.658; edge fraction inside: 0.0; touches image boundary: False; touches prompt boundary: True ∗panel index: 1; modality: MRI (Orbits); box xyxy: 475; 157; 502; 173; quantitative features: valid: True; area ratio: 0.00027; prompt box coverage: 0.171296; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.746; 0.403; bbox normalized xyxy: 0.7352; 0.3961; 0.7564; 0.4155; bbox extent: 0.660714; circularity: 0.570172; solidity: 1.049645; inside gray mean: 82.041; surrounding ring gray mean: 83.578; inside minus ring gray: -1.537; edge fraction inside: 0.0; touches image boundary: False; touches prompt boundary: True ∗panel index: 2; modality: MRI (Brain); box xyxy: 201; 351; 218; 368; quantitative features: valid: True; area ratio: 0.000656; prompt box coverage: 0.394464; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.4967; 0.8671; bbox normalized xyxy: 0.4833; 0.8551; 0.5119; 0.8841; bbox extent: 0.791667; circularity: 0.984702; solidity: 1.123153; inside gray mean: 183.833; surrounding ring gray mean: 181.55; inside minus ring gray: 2.283; edge fraction inside: 0.192982; touches image boundary: False; touches prompt boundary: True –Cross Panel Synthesis:The fundus photography shows a relatively normal-appearing retina with a central reflection artifact, lacking obvious large hemorrhages or exudates despite the clinical note of swelling. The orbital MRI reveals bilateral, symmetric, hyperintense foci at the posterior pole of the globes, corresponding anatomically to the optic nerve head region. The brain MRI demonstrates a focal hypointense lesion in the midline inferiorly, potentially involving the cerebellar vermis or fourth ventricle. The combination of bilateral posterior eye findings and a midline brain lesion suggests a systemic or syndromic process affecting these specific anatomical regions. –Limitations:The segmentation masks for the orbital MRI lesions show high circularity but low prompt box coverage, indicat- ing they capture the core of the hyperintense foci but may not fully delineate their irregular boundaries against the vitreous. The brain lesion mask has a higher coverage ratio but its exact anatomical relationship to the fourth ventricle vs. vermis is difficult to confirm without sagittal/coronal multi-planar reconstruction. The fundus image lacks color contrast resolution to definitively rule out subtle micro-hemorrhages or exudates mentioned in the clinical history. –Segmentation Assessment:Masks serve only as approximate anatomical localization aids, not precise pathological quanti- fiers. Cross-verification against raw pixels confirms the orbital masks align with the hyperintense foci pointed out by arrows. The brain mask aligns with the asterisk-marked region. However, automated masks cannot inherently differentiate subtle pathological textures from normal background tissue; final morphological assessments rely on direct visual inspection of raw images. •Medsam Enabled:True •Task Focus: –Answer Type:diagnosis –Measurement Guidance:The requested answer type is a diagnosis; provide discriminative visible morphology but leave disease naming to the Doctor. •Provided Exam Context: –Focused Examination:Bilateral optic disk swelling and retinal hemorrhages noted. •Measurement Skill Ids: –vm_evolved_g38_48bfeb9bde45 –vm_evolved_g22_71611f3e24ce –vm_evolved_g225_2b3401c40f64 •Segmentation Results: –∗Panel Index:1 ∗Modality:MRI (Orbits) ∗Box Xyxy:277; 157; 304; 173 ∗Mask Area Ratio:0.000402 ∗Mask Nonempty:True ∗Quantitative Features:valid: True; area ratio: 0.000402; prompt box coverage: 0.25463; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.4285; 0.4038; bbox normalized xyxy: 0.4221; 0.3889; 0.4387; 0.4203; bbox extent: 0.769231; circularity: 0.921625; solidity: 1.128205; inside gray mean: 80.464; surrounding ring gray mean: 75.806; inside minus ring gray: 4.658; edge fraction inside: 0.0; touches image boundary: False; touches prompt boundary: True –∗Panel Index:1 ∗Modality:MRI (Orbits) ∗Box Xyxy:475; 157; 502; 173 ∗Mask Area Ratio:0.00027 29 Wu et al. ∗Mask Nonempty:True ∗Quantitative Features:valid: True; area ratio: 0.00027; prompt box coverage: 0.171296; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.746; 0.403; bbox normalized xyxy: 0.7352; 0.3961; 0.7564; 0.4155; bbox extent: 0.660714; circularity: 0.570172; solidity: 1.049645; inside gray mean: 82.041; surrounding ring gray mean: 83.578; inside minus ring gray: -1.537; edge fraction inside: 0.0; touches image boundary: False; touches prompt boundary: True –∗Panel Index:2 ∗Modality:MRI (Brain) ∗Box Xyxy:201; 351; 218; 368 ∗Mask Area Ratio:0.000656 ∗Mask Nonempty:True ∗Quantitative Features:valid: True; area ratio: 0.000656; prompt box coverage: 0.394464; component count: 1; largest component fraction: 1.0; centroid normalized xy: 0.4967; 0.8671; bbox normalized xyxy: 0.4833; 0.8551; 0.5119; 0.8841; bbox extent: 0.791667; circularity: 0.984702; solidity: 1.123153; inside gray mean: 183.833; surrounding ring gray mean: 181.55; inside minus ring gray: 2.283; edge fraction inside: 0.192982; touches image boundary: False; touches prompt boundary: True •Limitations: MedSAM masks are prompt-conditioned localization aids; preserve the raw-image findings. T16 DOCTOR •Diagnosis:Idiopathic Intracranial Hypertension (IIH) •Differential Diagnoses: –Posterior Fossa Mass Lesion with Obstructive Hydrocephalus –Cerebral Venous Sinus Thrombosis –Meningitis/Infectious Etiology –Malignant Hypertension •Key Evidence: –Severe obesity –Headaches worsening with recumbency and bending over (positional/Valsalva exacerbation) –Transient visual obscurations and persistent blurriness –Initial focused examination showing bilateral optic disk swelling and retinal hemorrhages –MRI demonstrating flattened posterior globes, empty sella, and transverse sinus stenoses without thrombosis –MRI revealing a focal hypointense midline inferior lesion near the cerebellar vermis/fourth ventricle •Tests Used: –Head_MRI –Head_MRV –NEJM_Medical_Image •Treatment Prescription Plan: –Urgent Neurosurgery consultation and recommendation for hospital admission for continuous neurologic monitoring and evaluation of the posterior fossa lesion and elevated ICP. –Ophthalmology referral for formal visual field testing and standardized fundus documentation to reconcile initial exam findings with imaging. –DO NOT START UNTIL VERIFIED: Acetazolamide 500 mg orally twice daily (titrating to 1000 mg twice daily) to reduce CSF production; withhold pending confirmed non-pregnancy status and baseline BMP/LFTs. –Non-drug management: Head elevation to 30 degrees, structured weight reduction program targeting 5-10% body weight loss, dietary sodium restriction (<2g/day), strict avoidance of Vitamin A supplements and tetracycline-class antibiotics. –Obtain mandatory baseline laboratories immediately: CBC, comprehensive metabolic panel (BMP), coagulation profile, liver function tests, and quantitative serum/urine pregnancy test prior to any pharmacotherapy. –Strict avoidance of NSAIDs, anticoagulants, and antiplatelet agents until hemorrhage risk and mass effect are fully character- ized. •Safety Checks: –Pregnancy status is currently unknown; acetazolamide carries teratogenic risk and must be withheld until a quantitative serum/urine pregnancy test confirms non-pregnancy. –Renal and hepatic baselines are unverified; acetazolamide requires renal clearance and may precipitate metabolic acidosis or electrolyte depletion. Baseline BMP/LFTs are mandatory before first dose. 30 MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction –No known drug allergies documented; explicit verification required prior to dispensing. –Posterior fossa lesion and signs of elevated ICP require urgent neurosurgical evaluation to rule out obstructive hydrocephalus or herniation risk. •Follow Up Or Escalation:Admit to a monitored neurological unit for frequent neuro checks and urgent specialist consultations; return to Emergency Department immediately for sudden permanent vision loss, thunderclap headache, projectile vomiting, or altered mental status. Recorded outcome.Gold diagnosis:Idiopathic intracranial hypertension. The MedSAM prediction was scoredcorrectby the auto- matic diagnosis metric; this label does not endorse the visual evidence or management plan. Interpretation.This pair is a qualitative failure-boundary illustration rather than positive clinical evidence or a causal estimate. In both paths, source annotations are overinterpreted as pathology; in the MedSAM path, an unverified posterior-fossa finding propagates into the differential and escalation plan despite the correct final label. The diagnosis-blind release checks enforce provenance, prerequisites, and conservative disposition but cannot determine whether an image interpretation is clinically true. This trace therefore exposes a limitation of the current governance stack and motivates independent image adjudication; it must not be read as evidence that MedSAM improved this case. 31