Paper deep dive
SurgiCLAW: An Interpretable Atomic-Skill Agent for Precision Robotic Micromanipulation
Angen Ye, junjie xie, Yujia Song, Junkai Huang, Dapeng Zhang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent multimodal large language models (MLLMs) and vision--language--action models have provided robots with new capabilities for long-horizon task understanding and planning. In surgical robotics, however, end-to-end prediction of low-level actions remains limited by insufficient precision, opaque decisions, and scarce data. This paper presents SurgiCLAW, a hierarchical skill-based framework for precision surgical manipulation. SurgiCLAW confines an MLLM to intraoperative visual understanding, structured explanation, and atomic surgical-skill selection, while delegating physical execution to verifiable deterministic control programs. For brain--machine-interface implantation, a representative high-precision surgical task, we construct a state-to-skill invocation dataset and expand it through visual and state perturbations. The model first learns structured skill invocation through supervised fine-tuning and is then optimized using three rewards for action correctness, operation prerequisites, and format validity, improving action selection, operational safety, and output parseability. We evaluate subtask success, offline model comparison, real-robot performance, and ablations. On a fixed test set, SurgiCLAW achieves 94.6% skill-selection accuracy and 93.0% macro-averaged F1 (Macro-F1). Across 50 closed-loop real-robot implantation trials, it achieves an 88.0% task success rate.
Tags
Links
- Source: https://arxiv.org/abs/2610.15766v1
- Canonical: https://arxiv.org/abs/2610.15766v1
Trouble viewing inline? Open PDF directly →
Full Text
42,812 characters extracted from source content.
Expand or collapse full text
SurgiCLAW: An Interpretable Atomic-Skill Agent for Precision Robotic Micromanipulation Angen Ye † , Junjie Xie † , Yujia Song, Junkai Huang, Dapeng Zhang * Institute of Automation, Chinese Academy of Sciences University of Chinese Academy of Sciences Beijing, China † These authors contributed equally to this work. * Corresponding author. Abstract—Recent multimodal large language models (MLLMs) and vision–language–action models have provided robots with new capabilities for long-horizon task understanding and plan- ning. In surgical robotics, however, end-to-end prediction of low- level actions remains limited by insufficient precision, opaque decisions, and scarce data. This paper presents SurgiCLAW, a hierarchical skill-based framework for precision surgical manipulation. SurgiCLAW confines an MLLM to intraopera- tive visual understanding, structured explanation, and atomic surgical-skill selection, while delegating physical execution to verifiable deterministic control programs. For brain–machine- interface implantation, a representative high-precision surgical task, we construct a state-to-skill invocation dataset and expand it through visual and state perturbations. The model first learns structured skill invocation through supervised fine-tuning and is then optimized using three rewards for action correctness, opera- tion prerequisites, and format validity, improving action selection, operational safety, and output parseability. We evaluate subtask success, offline model comparison, real-robot performance, and ablations. On a fixed test set, SurgiCLAW achieves 94.6% skill- selection accuracy and 93.0% macro-averaged F1 (Macro-F1). Across 50 closed-loop real-robot implantation trials, it achieves an 88.0% task success rate. Index Terms—surgical robotics, multimodal large language model, atomic skills, precision micromanipulation I. INTRODUCTION The rapid development of multimodal large language mod- els and vision–language–action (VLA) models is changing the design paradigm of robotic systems [1], [2]. Compared with conventional end-to-end imitation-learning or reinforcement- learning policies, VLA models can extract high-level seman- tics from natural-language instructions, visual observations, and task context, showing promise in long-horizon planning, open-vocabulary interaction, and cross-scene generalization. This progress has motivated growing interest in using large models in surgical robots to understand intraoperative scenes, generate procedural plans, and assist with complex surgical tasks [3]. Surgical robotics, however, is fundamentally different from manipulation in ordinary open environments. The challenge is not only to decide what to do next, but also to execute the decision with sufficient precision, safety, and interpretability. Small needle-tip deviations, instrument-pose errors, or abnor- mal motion paths may cause tissue damage, bleeding, or other medical risks. Directly transferring the general VLA paradigm to surgical robots therefore has three limitations. First, end-to- end VLA models commonly predict continuous actions or low- level commands, and their local precision is difficult to recon- cile with surgical error tolerances. Second, large-model outputs are often opaque: clinicians cannot easily determine why an action was selected, which visual and robot states support it, or whether current safety constraints are satisfied. Third, surgical data are inherently scarce. Acquiring precise manipulation tra- jectories requires domain experts, surgical environments, and high-accuracy synchronized recording, making large-scale real interaction data difficult to obtain. We propose SurgiCLAW, an interpretable atomic-skill agent for precision surgical robotics. Its central principle is to decouple complex semantic decisions from high-precision control. An MLLM interprets images, robot states, and task instructions, produces a reviewable structured rationale, and selects an atomic surgical program. Execution is performed by predefined atomic-skill controllers. Each skill has explicit inputs, prerequisites, motion boundaries, and safety constraints, and can use geometry, optimization, visual servoing, or trajectory planning for precise execution. SurgiCLAW consequently does not ask a large model to control an instrument directly; it trains the model to perform an auditable state-to-skill mapping while verifiable controllers retain responsibility for precision motion. This design also changes how training data are constructed. Instead of collecting complete continuous surgical trajecto- ries, the task is formulated as supervised mapping from an image, robot state, and query to an atomic-skill invocation. Determining the appropriate atomic action and its essential parameters is generally easier, more consistent, and easier to audit than demonstrating a complete high-precision trajectory. For brain–machine-interface implantation, we propose a multi- MLLM collaborative data-generation method that expands 5,000 manually collected state–skill samples to approximately 30,000 structured training samples. A structured intermediate decision basis is also generated for every sample, so the output contains not only the function call but also the state assessment, visual evidence, task stage, and safety check. The contributions are threefold: 1) We introduce SurgiCLAW, a hierarchical atomic-skill arXiv:2610.15766v1 [cs.RO] 11 Oct 2026 TABLE I ATOMIC ACTION INTERFACE FunctionMeaningPrimary prerequisite ADJUST_EXPOSUREAdjust microscopic exposureCamera online; exposure outside the calibrated range ADJUST_FOCUSAdjust microscopic focusCamera online; image out of focus E_STOPTrigger a safety stopSingular configuration, system fault, or other high-risk anomaly SPATIAL_REGISTRATIONRegister image and robot coordi- nates Usable image; system self-check passed RESTART_CAMERARestart a failed or disconnected camera Abnormal camera connection; robot held in a safe state ALIGN_NEEDLEDeterministic visual-servo needle alignment Registration complete; image sharp; no singular configuration INSERTExecute flexible-electrode insertionRegistration and alignment complete; camera online; all safety checks passed agent for high-precision surgery. It confines the MLLM to intraoperative visual understanding, structured deci- sion explanation, and atomic-skill selection, and con- nects it through a safety parser to deterministic con- trollers, separating high-level semantic decisions from low-level precision execution. 2) Weproposeamulti-MLLMcollaborativedata- generation method for learning atomic surgical skills. Visual and state perturbations, candidate generation by multiple MLLMs, rule-based pre-validation, and cross-model verification allow high-quality state–skill training samples to be generated at scale from a limited set of robot images and state records. 3) We design three rewards—action correctness, operation prerequisites, and format validity—and optimize the pol- icy with group reward-decoupled normalization policy optimization (GDPO) [4], systematically evaluating the method offline, on atomic subtasks, in a real-robot closed loop, and through ablation. SurgiCLAW achieves 94.6% accuracy and 93.0% Macro-F1 on the fixed test set and an 88.0% task success rate in real-robot trials. I. RELATED WORK A. Vision-Language-Action Policies for Robot Control Vision-language-action models couple web-scale semantic understanding with robot control: SayCan grounds language plans in executable skill affordances [5], while RT-2 [6] and OpenVLA [7] predict actions directly from a unified vision- language model. These policies generalize across instructions and scenes, but emit continuous or token-level actions and rely on large amounts of interaction data. In a safety-critical, high- precision setting this raises two concerns that the present work targets: the predicted low-level motion cannot be guaranteed to meet a tight error bound, and the decision is opaque, so a clinician cannot inspect why an action was chosen. Surgi- CLAW therefore keeps the semantic reasoning of a multimodal model but constrains its output to a small set of bounded, verifiable skills, so the model decides what to do while verified controllers guarantee how precisely it is done. B. Scene Understanding in Medical Robotics Autonomous surgical robotics spans supervised autonomy (STAR [8]), imitation-based control (SRT [9], SRT-H [10]), and language-guided assistance (SuFIA [11]); a parallel line builds medical vision-language models such as SurgVLM [12] and SurgVidLM [13] for operative scene understanding and query answering. These lines advance either task autonomy or scene semantic interpretation, but neither provides a safety- checked interface that converts a model’s high-level decision into a precise, auditable motion primitive. Geometric-vision controllers for registration, alignment, and insertion [14]– [16] already supply such verified primitives for implantation; SurgiCLAW adds an adaptive, safety-gated decision layer that lets a large model schedule these primitives without directly controlling the end-effector. I. METHOD A. Hierarchical Architecture and Action Interface At decision step t, the system input is x t =I t ,s t ,q t ,(1) where I t is the microscopic image, s t is the robot state, and q t is the current operation query. The state s t encodes the current joint values together with the non-visual safety conditions measured by the system—spatial-registration com- pletion, camera connectivity, and robot singularity. The visual conditions that determine the current sub-skill, such as image- quality anomalies (blur, exposure), needle alignment, and scene usability, are perceived by the model directly from I t . Each step is inferred independently without relying on an implicit dialogue history. The model output is y t =r t ,c t ,f t ,(2) andisencodedwiththefixedfields <think>r t </think>, <answer>c t </answer>, and <function>f t </function>. Here, r t describes observable visual evidence, the state assessment, and action prerequisites; c t is a concise operator-facing conclusion; and f t is the single skill to invoke. This explicit text is the model’s chain-of-thought reasoning; exposing it to the 1 2 3 4 5 Raw Samples Image Surgical Skill Label Instruction Image & Text Augmentation 1 Image Augmentation (Examples) Noise OcclusionExposure Blur • Text Augmentation (Paraphrasing) Instruction MLLM-1 • • • Multi-MLLM CoT Generation Input Combination (Examples) • ＋ ＋ ＋ Qwen3-VL DeepSeek-VL2 ChatGPT-5.5 CoT CoT CoT <think> <answer <function> <think> <answer <function> <think> <answer <function> • • • Multi-MLLM Cross-Validation Logical Consistency Skill Correctness Visual Consistency Qwen3-VL DeepSeek-VL2 ChatGPT-5.5 Manual Inspection Pass Rate >98% Filtered Out Final CoT Dataset CoT <think> <answer <function> Robot state 1 Fig. 1. Multi-MLLM collaborative data-generation pipeline. Starting from raw image, robot-state, instruction, and skill-label samples, the pipeline expands the data through image/text augmentation and constructs structured rationales using three MLLMs. Candidate outputs are filtered by rule-based checks, leave- one-generator-out cross-validation, and human spot inspection before being retained in the final structured dataset. operator lets a clinician inspect and verify why the particular skill was selected, which to some extent improves the interpretability of the decision. We caution that this rationale is a model-generated summary and is not verified to faithfully reflect the model’s internal reasoning; we therefore present it as an auditable decision basis rather than as evidence of the model’s actual decision process. The model is restricted to the seven actions in Table I. Target localization, pose correction, and electrode pickup are deterministic internal stages of SPATIAL_REGISTRATION, ALIGN_NEEDLE, or INSERT, rather than additional labels. Manual confirmation is a parser fallback and is not part of the model action set. Each skill is represented as A k =⟨n k ,P k ,C k ,T k ⟩,(3) where n k is its name, P k its prerequisite, C k its execution program, and T k its completion or termination criterion. The model output first enters a safety parser. The parser verifies that each field appears exactly once, only one legal function is present, the function belongs to the seven-class set, and P k (s t ,I t ) holds. A malformed output, illegal or multiple function, or motion request in a high-risk state is never dispatched. The system observes again and requests a new inference, entering manual confirmation when necessary. Images and states are reacquired after every skill, forming an observe–reason–verify–execute–observe loop. B. Structured Data Generation and Partitioning Each sample is d i =I i ,s i ,q i ,r i ,c i ,f i .(4) Original samples comprise robot-platform images together with synchronized robot-state (joint-value) records, covering scenarios such as normal operation, image-quality anomalies, incomplete registration, camera failures, singular configura- tions, and needle misalignment. Visual perturbations include blur, exposure, noise, local occlusion, and target-region vari- ation; state perturbations perturb the joint values themselves. Image and state perturbations are applied within consistent templates so that the visual evidence and the measured robot state do not contradict each other. Before any augmentation, a grouped split is performed on the original sample IDs. With random seed 7, the 5,000 original samples are divided into 4,000 training sources, 500 validation sources, and 500 test sources. Each original sam- ple and every augmented descendant generated from it are assigned exclusively to the same split. Sixfold augmentation is then applied independently in each split, so that every original sample yields six samples including the original (i.e., five augmented descendants), producing 24,000, 3,000, and 3,000 samples, respectively, for 30,000 in total. The validation and test sets use the same perturbation types as training, but no original sample or augmented descendant is shared across splits. Thus, the final 8:1:1 ratio applies to both original source groups and augmented samples. As shown in Fig. 1, to construct structured rationales, we use the same prompt template to query DeepSeek-VL2, Qwen3-VL-Flash, and ChatGPT-5.5 through API calls, with each model independently generating one structured rationale for each input. This multi-MLLM strategy leverages comple- mentary model behaviors and reduces model-specific pattern bias in the generated dataset. All generators use the same seven-function vocabulary and output format, with temperature 0.7, top-p 0.9, and a maximum output length of 512 tokens. Rule-based validation first checks tag completeness, function uniqueness, state consistency, and explicit safety rules. Candidates that pass rule-based validation are cross-verified by the same three MLLMs in a leave-one-generator-out man- Opreration Prerequisite Reward MLLM Stage: 2 GDPO Align the insertion needle with the eletrode. <think>......</think> <answer>......</answer> <function>......</function> Action Correctness Reward Format Reward: R format = 1, 0, if the reply conforms to the format otherwise � Stage: 1 SFT � Image 2D Trajectory / Affordance Box Reasioning Process Multimodal Large Language Model Robot state Instruction Surgical Skill Library (Selected Examples) S a m p l e n Reference MLLM ️ Update KL-Divergence ❄ AnswerFunction fwuu 插入针对齐 ALIGN_NEEDLE INSERT 执行植入 <think>......</think> <output>......</output> ALIGN_ NEEDLE E_STOP RESTART_ CAMERA ... Function： INSERT The image is clear, spatial registration is complete, and the insertion needle is aligned with the target electrode. Function： INSERT Sample：A The image is clear, spatial registration is complete, and the system is operating normally. Sample：B R act =1 R proc =1 R proc =0 R act =0 SPATIAL_ REGISTRATION ADJUST_EXPOSURE R act =0 Can insertion be performed safely? Fig. 2. SFT and GDPO training pipeline. The MLLM first learns structured reasoning, answers, and function calls through SFT. GDPO then optimizes action correctness, operation prerequisites, and output format under a reference-model constraint. ner. Each model scores only the candidates generated by the other two models, rather than its own outputs; for exam- ple, Qwen3-VL-Flash scores only candidates generated by DeepSeek-VL2 and ChatGPT-5.5. This design reduces self- preference bias during verification. Each verifier assigns scores from 0 to 2 for visual consistency, state consistency, rationale quality, and function correctness, weighted by 0.30, 0.25, 0.20, and 0.25. A candidate is retained if the mean score is at least 1.6 and the verifier-score difference is no greater than 0.4. The final generated dataset is further audited through a random human spot check by two reviewers. A generated batch is considered valid only when the human acceptance rate exceeds 98%; in our audit, 299 of 300 sampled instances were accepted (99.7%). C. Supervised Fine-Tuning and GDPO 1) Supervised Fine-Tuning: The base model is Qwen3-VL- 4B-Instruct [17]. The autoregressive objective is L SFT =− T X k=1 logπ θ (y k | x,y <k ).(5) LoRA [18] (rank 8, α = 16) is applied to all linear layers, with global batch 16, per-GPU micro-batch 1, maximum sequence length 8192, and learning rate 1×10 −5 (cosine schedule, 0.03 warmup, five epochs), using bfloat16, FSDP2 [19], gradient checkpointing, and eight NVIDIA A800-SXM4-80GB GPUs, with validation every 250 steps. 2) Three GDPO Rewards: For input x, the policy samples G = 4 candidates, ˆy j G j=1 ∼ π θ (· | x). Each candidate is scored by three binary rewards: action correctness, opera- tion prerequisites, and format validity. Because these rewards encode distinct and partially competing objectives, they are combined with group reward-decoupled normalization policy optimization (GDPO) [4], which normalizes each reward inde- pendently within the sampled group before aggregating them into the combined advantage. The action-correctness reward evaluates whether the pre- dicted function matches the target atomic skill for the current image, robot state, and operation query: R act (ˆy j | x) = 1[ ˆ f j ∈F + (x)],(6) whereF + (x) is the singleton set containing the single correct function call for the current input. For example, when needle alignment is the correct next step, only ALIGN_NEEDLE receives this reward. The operation-prerequisite reward checks whether the cur- rent state satisfies the prerequisites of the predicted function; it therefore acts as a safety/precondition constraint rather than as a correctness signal: R proc (ˆy j | x) = ( 1, P ˆ f j (I,s) = 1, 0, P ˆ f j (I,s) = 0. (7) For example, INSERT receives this reward only when the image is usable, registration is complete, the camera is online, the robot is away from singularity, and the needle is aligned. These prerequisites are assessed by the system from the state s t (registration, camera, singularity) and visual detection from I t (image usability, needle alignment), all of which are available to the model as its input. The format-validity reward is 1 when the output contains complete think, answer, and function fields with ex- actly one legal function call, and 0 otherwise. Let R (j) k ∈ 0, 1 denote the value of reward k ∈ act, proc, fmt for candidate j. GDPO first normalizes each reward independently over the group, decoupling the normal- ization of the three objectives as ̃ A (j) k = R (j) k − meanR (1) k ,...,R (G) k stdR (1) k ,...,R (G) k ,(8) where the mean and standard deviation are computed over the G candidates of the group and, when a reward is constant within the group (zero variance), its normalized advantage is set to zero to avoid an undefined division. The decoupled advantages are then combined with the reward weights, which are applied to the normalized values rather than to the raw reward scores: ̃ A (j) = 0.55 ̃ A (j) act + 0.35 ̃ A (j) proc + 0.10 ̃ A (j) fmt .(9) To keep the advantage magnitude stable as the number of rewards grows, we apply a final batch-wise normalization over the minibatch D B : ˆ A (j) = ̃ A (j) − mean ̃ A (i) i∈D B std ̃ A (i) i∈D B + ε .(10) The policy is updated with a clipped surrogate objective on the normalized advantages, with an importance ratio s t (θ) = π θ (y t | x,y <t )/π θ old (y t | x,y <t ) and a KL-toward-reference regularizer β KL[π θ ∥π ref ]: J GDPO (θ) =E h 1 G G X j=1 1 |y j | |y j | X t=1 min s t ˆ A (j) , clip s t , 1− ε c , 1 + ε c ˆ A (j) − β KL π θ ∥π ref i , (11) where π ref is the SFT reference model, and ε c = 0.1 and β = 0.01 are the clipping threshold and KL coefficient, re- spectively. The weights emphasize correct skill selection while retaining explicit penalty for unsafe prerequisite violations and parser-invalid outputs. We adopt GDPO as defined in [4] without modification; the reward weights (0.55/0.35/0.10) are the only task-specific configuration. IV. EXPERIMENTAL DESIGN A. Task and Data Split Experiments use closed-loop flexible-electrode implanta- tion as the representative task. The model selects the next action from seven atomic skills; target localization, pose correction, and electrode pickup are performed internally by controllers. Before augmentation, the 5,000 original samples are grouped by original sample ID and split using seed 7 into 4,000/500/500 sources. All descendants of an original sample remain in the same split, after which independent within- split augmentation produces 24,000/3,000/3,000 samples. All offline models use the same fixed test set, which is excluded from training, hyperparameter selection, and reward design. B. Metrics and Statistical Protocol Offline skill selection is treated as seven-class classifica- tion. Accuracy is the proportion of samples with the correct function name. Precision, recall, and F1 are computed per class and macro-averaged so that frequent actions such as ADJUST_EXPOSURE do not obscure minority safety actions. Illegal, missing, or multiple functions are counted as incor- rect predictions. Safety violation is evaluated over 2,529 test samples with explicit safety prerequisites and counts a motion command or INSERT proposed when the image is unusable, registration is incomplete, the needle is misaligned, the camera is offline, self-check fails, or the robot approaches singularity. Parser-blocked violations are never sent to the robot. Physical subtasks report success, positional error, orien- tation error, and time. Spatial registration uses target reg- istration error (TRE), the post-registration distance between corresponding targets in image and robot coordinates. Align- ment and insertion report orientation and positional errors separately; positional error is not added to TRE. Real-robot task success requires the system to execute all prescribed atomic skills and reach the terminal state. Mean stage count uses all trials. Offline metrics are reported as descriptive point estimates over the fixed augmented test set because samples derived from the same original ID are not independent observations. Binomial success rates in physical experiments use Wilson 95% confidence intervals [20], and paired real- robot methods use two-sided McNemar tests [21]. C. Robot Platform and Material Scope The real-robot platform follows the system in [15]: a UR5 arm performs large-range motion while the implantation head handles precise local motion, with two 2× microscope cameras (MD65-2T26X lens, 2048 × 2048), a voice-coil- actuated insertion needle, a KOHZU XA04A-R102 stage (0.5-μm resolution, ±0.3 μm repeatability), and a KOHZU SA07A-RM02 electrode stage (0.0012 ◦ /0.0015 ◦ about x/y). A measured 2.4-m field of view gives an image-plane scale of 2.4/2048 = 0.001171875 m/px, used only for local image residuals. Success requires TRE < 0.50 m for spatial registra- tion and positional error < 0.10 m for the controller- internal target-localization stage. ALIGN_NEEDLE requires orientation error < 2 ◦ and image-plane error < 0.005 m; INSERT requires orientation error < 5 ◦ and three-dimensional landing error < 0.030 m. These thresholds evaluate the corresponding controller-level subtasks and are not used to define full-task completion. Spatial registration, target local- ization, image adjustment, and safety stopping use phantoms, calibration targets, and the robot platform without animal material. Only the physical INSERT subtask and complete closed-loop implantation trials used legally sourced ex-vivo rat brain tissue obtained under institutional ethical approval and reported in accordance with the ARRIVE guidelines [22]. V. RESULTS AND ANALYSIS A. Offline Model Comparison All models are compared under one offline protocol. Gen- eral VLMs without SFT estimate the zero-shot ceiling of generic multimodal ability; general VLMs with SFT quantify the benefit of state–skill labels; surgical VLMs test whether domain pretraining can replace task adaptation; and Surgi- CLAW uses SFT followed by GDPO. Every model receives identical images, states, and queries and must use the same structured interface, which imposes a strict output format; prompt-only results therefore partly reflect adaptation to this interface rather than task ability alone. On the fixed augmented test set, SurgiCLAW reaches 94.6% accuracy and 93.0% Macro-F1, improving over the strongest SFT general model by 5.5/6.5 points and over Qwen3-VL-4B- Instruct + SFT by 11.6/11.9 points, while its safety-violation rate drops to 0.5% (13/2,529) versus 3.2% and 1.9% for the 4B and 32B SFT models. Prompt-only general VLMs reach only 14.2–38.4% accuracy (Qwen3-VL-235B strongest at 31.2% Macro-F1), indicating that scale alone does not solve the state-to-skill mapping. After SFT the 4B model rises to 83.0%/81.1%, the 32B to 89.1%/86.5%, and surgical-specific VLMs remain below SFT, showing that domain semantics do not replace interface adaptation. Because augmented samples derived from the same original ID are correlated, these of- fline results are descriptive point estimates without confidence intervals or significance tests. B. Real-Robot Comparison Real-robot experiments evaluate the complete loop of high-level selection, deterministic control, and state update. SurgVLM-7B, SurgVidLM, Qwen3-VL-4B-Instruct + SFT, Qwen3-VL-32B-Instruct + SFT, and SurgiCLAW receive the same microscopic images, robot states, and task queries and share the same parser, action vocabulary, and low-level controllers. Each method is evaluated on 50 paired initial conditions, with randomized method order to reduce platform drift. Fig. 3 illustrates a representative closed-loop decision sequence, showing how SurgiCLAW maps successive image and robot-state observations to atomic-skill calls. Table I reports closed-loop results. The safety-violation denominator is the number of safety-critical decision opportu- nities across trials, which differs by method because failed or repeated loops expose different numbers of such states; every violation is blocked by the parser. SurgiCLAW completes 44/50 trials (Wilson 95% interval 76.2–94.4%), improving over Qwen3-VL-4B-Instruct + SFT by 22 points (33/50 to 44/50) with 19.8% fewer stages and takeover falling from 20.0% to 4.0%; a paired McNemar test over this pair is statistically significant (p < 0.05). Relative to Qwen3-VL- 32B-Instruct + SFT, it improves by 12 points (38/50 to 44/50), but this difference is not statistically significant (two-sided McNemar p = 0.070), so we treat the gain over the 32B model as a trend rather than a confirmed improvement at 50 trials. Because all methods share the same low-level controllers, differences arise from the high-level policy triggering registra- tion, alignment, insertion, recovery, and stopping at the right state: the surgical VLMs delay registration, repeat alignment, or request insertion before prerequisites hold, while the 32B SFT model is more robust than the 4B model, which is more sensitive to ambiguous visual states and recovery decisions. Failure categories are high-level selection, repeated align- ment without convergence, visual anomaly/occlusion, safety- parser refusal, and insertion-control failure; total failures are 27/25/17/12/6, and SurgiCLAW’s six failures break down as 1/2/1/1/1, so repeated alignment under occlusion and physical insertion remain the principal bottlenecks. C. Ablation Study Ablations remove only model-side training or output com- ponents, while keeping low-level controllers, data splits, and metrics unchanged. All variants are evaluated on the same fixed test set with accuracy, macro-precision, macro-recall, Macro-F1, and safety violation. Table IV shows the ablation results. Removing the struc- tured rationale lowers accuracy/Macro-F1 by 6.4/6.6 points, and the SFT-only variant (w/o GDPO, equal to Qwen3-VL- 4B-Instruct + SFT) remains far above prompt-only baselines but below the full model, indicating that supervised skill labels alone do not fully cover boundary and safety-critical states: adding GDPO raises accuracy from 83.0% to 94.6%, Macro- F1 from 81.1% to 93.0%, and cuts safety violation from 3.2% to 0.5%. Removing the action-correctness reward mainly degrades class-level selection, removing the prerequisite re- ward gives the highest safety-violation rate (confirming its role in constraining state/function conflicts), and removing the format reward mainly lowers recall. GDPO therefore improves accuracy, parseability, and safety-aware selection. D. Controller-Level Subtask Verification Each deterministic controller was tested 40 times under fixed perturbations, independently of the large-model deci- sions, following the frameworks of [14]–[16]. Table V reports controller-level success and accuracy. All controllers succeed on 90.0%–100.0% of trials; the spatial- registration TRE of 0.347±0.126 m, needle-alignment error of 0.91 ± 0.42 ◦ , and insertion error of 0.015 ± 0.006 m fall within the corresponding thresholds. Errors and times are means and standard deviations over successful trials. VI. CONCLUSION This paper presented SurgiCLAW, a hierarchical atomic- skill interface for precision robotic micromanipulation. Its central idea is to keep the semantic decision making of a multimodal language model within a small, bounded, and au- ditable skill space, and to gate every decision through a safety parser before delegating execution to verified deterministic controllers; this separates what to do next from how precisely to do it. It also makes the interface learnable from limited TABLE I ATOMIC-SKILL SELECTION ON THE FIXED TEST SET MethodAccuracy (%)↑Macro-P (%)↑Macro-R (%)↑Macro-F1 (%)↑Safety violation (%)↓ General VLMs (non-SFT) MiniCPM-o-2.6 [23]14.22.014.33.618.6 DeepSeek-VL2 [24]20.622.420.914.814.2 Llama-4-Maverick-17B-128E-Instruct [25]24.728.925.117.211.6 InternVL3.5-241B-A28B [26]31.637.831.224.47.4 Qwen2.5-VL-3B-Instruct [27]15.427.815.55.915.8 Qwen2.5-VL-7B-Instruct [27]28.029.728.218.210.9 Qwen3-VL-4B-Instruct [17]30.040.830.323.87.8 Qwen3-VL-30B-A3B-Instruct [17]35.044.135.227.65.9 Qwen3-VL-235B-A22B-Thinking [17]38.443.638.031.24.8 Surgical-specific VLMs SurgVLM-7B [12]41.847.239.738.65.3 SurgVidLM [13]44.649.143.842.74.9 General VLMs + SFT Qwen3-VL-4B-Instruct + SFT83.081.481.081.13.2 Qwen3-VL-32B-Instruct + SFT89.187.685.686.51.9 DeepSeek-VL2-Small + SFT81.778.778.078.04.2 SFT + GDPO SurgiCLAW94.694.092.093.00.5 Please determine whether the system can proceed with the implantation task based on the current image and system status. <think> The current registration result is not avail able because the system self-check has not been completed. The image and robot coordinate systems have not yet been spatially registered. The camera is working normally and no singular configuration is detected, but the insertion needle is not aligned with the target electrode. The system should complete spatial registration before any alignment or insertion operation. </think> <answer> Spatial registration is incomplete. The system should complete registration before proceeding. </answer> <function> SPATIAL_REGISTRATION </function> Based on the image and system state, determine whether the operation can continue normally. Based on the current image and system state, determine the needle alignment status and provide the next operation decision. Please combine the current image feedback and system parameters to determine whether the implantation operation can proceed. <think> The current image is underexposed and appears noticeably dark. The electrode-hole region is difficult to distinguish and the image quality is insufficient for reliable visual alignment. Spatial registration has been completed, the camera is online, and no singular configuration is detected. The insertion needle is not yet aligned, but alignment should be performed only after the image exposure is corrected. </think> <answer> The image exposure is insufficient. Adjust the exposure before continuing. </answer> <function> ADJUST_EXPOSURE </function> <think> The image exposure and focus are within the normal range, and the visual details are sufficiently clear. Spatial registration has been completed, the camera is available, and no singular configuration is detected. However, the insertion needle is still offset from the target electrode, and its orientation error exceeds the alignment threshold. The needle should be aligned before insertion. </think> <answer> The insertion needle is not aligned with the target electrode. Perform needle alignment first. </answer> <function> ALIGN_NEEDLE </function> <think> The image is clear and the exposure and focus are appropriate. Spatial registration has been completed, the camera is working normally, and no singular configuration is detected. The insertion needle is aligned with the target electrode, and the orientation and positional errors satisfy the insertion prerequisites. All safety conditions for physical insertion are satisfied. </think> <answer> All prerequisites are satisfied. The system can proceed with electrode insertion. </answer> <function> INSERT </function> Fig. 3. Closed-loop decision example for flexible-electrode implantation. At each step, SurgiCLAW receives the current microscopic image, robot state, and operation query, then outputs a structured rationale, an operator-facing answer, and a single atomic-skill call. The example illustrates a typical decision sequence from spatial registration and image-quality correction to needle alignment and final insertion. surgical data: a multi-MLLM data-generation pipeline expands 5,000 state–skill samples into 30,000 structured ones, and a three-reward group reward-decoupled normalization policy optimization objective improves skill selection, prerequisite safety, and output parseability. On the fixed test set Surgi- CLAW reaches 94.6% accuracy and 93.0% Macro-F1, and it completes 88.0% of 50 closed-loop implantation trials. We interpret these results as a feasibility demonstration rather than evidence of clinical efficacy: the evaluation covers a single high-precision task using phantom and ex-vivo material, and the closed-loop comparisons are constrained by the modest number of paired trials. Whether the safety and interpretability benefits generalize to other tasks, to more stringent clinical conditions, or to long-horizon unattended operation remains to be established. REFERENCES [1] K. Kawaharazuka, J. Oh, J. Yamada, I. Posner, and Y. Zhu, “Vision- language-action models for robotics: A review towards real-world ap- plications,” IEEE Access, 2025. [2] P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai et al., “pi05: a vision- language-action model with open-world generalization,” arXiv preprint arXiv:2504.16054, 2025. [3] M. Xu, Z. Huang, J. Zhang, X. Zhang, and Q. Dou, “Surgical action planning with large language models,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2025, p. 563–572. [4] S.-Y. Liu, X. Dong, X. Lu, S. Diao, P. Belcak, M. Liu, M.-H. Chen, H. Yin, Y.-C. F. Wang, K.-T. Cheng et al., “Gdpo: Group reward- decoupled normalization policy optimization for multi-reward rl opti- mization,” arXiv preprint arXiv:2601.05242, 2026. [5] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al., “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022. [6] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choroman- ski, T. Ding, D. Driess, A. Dubey, C. Finn et al., “Rt-2: Vision-language- action models transfer web knowledge to robotic control,” arXiv preprint arXiv:2307.15818, 2023. [7] M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi et al., “Openvla: An open- source vision-language-action model,” arXiv preprint arXiv:2406.09246, 2024. [8] A. Shademan, R. S. Decker, J. D. Opfermann, S. Leonard, A. Krieger, TABLE I CLOSED-LOOP REAL-ROBOT IMPLANTATION RESULTS MethodSuccess (%)↑Stages↓Takeover (%)↓ Unsafe proposal (%)↓ Surgical-specific VLMs SurgVLM-7B46.0 (23/50) 10.4± 2.638.0 (19/50)5.4 (11/204) SurgVidLM50.0 (25/50) 9.8± 2.434.0 (17/50)4.9 (10/205) General VLM + SFT Qwen3-VL-4B-Instruct + SFT66.0 (33/50) 8.6± 1.920.0 (10/50)3.4 (7/207) Qwen3-VL-32B-Instruct + SFT76.0 (38/50) 7.7± 1.610.0 (5/50)1.9 (4/206) SFT + GDPO SurgiCLAW88.0 (44/50) 6.9± 1.54.0 (2/50)0.5 (1/194) TABLE IV ABLATION OF MODEL-SIDE COMPONENTS SettingAccuracy (%)↑Macro-P (%)↑Macro-R (%)↑Macro-F1 (%)↑Safety violation (%)↓ Full model94.694.092.093.00.5 w/o structured rationale88.287.485.986.42.3 w/o GDPO83.081.481.081.13.2 w/o action-correctness reward90.488.188.788.21.6 w/o prerequisite reward91.290.888.989.73.2 w/o format reward92.092.888.690.41.1 TABLE V ATOMIC-SUBTASK SUCCESS AND EXECUTION ACCURACY SubtaskSuccess criterionSuccess (%)↑Mean error↓Time (s)↓ Spatial registrationTRE< 0.50 m and coordinate transform completed95.0 (38/40) 0.347± 0.126 m (TRE) 3.18± 0.37 Target localization (internal)Stable target detection available for alignment97.5 (39/40) 0.051± 0.028 m (position) 0.82± 0.26 Needle alignmentOrientation and image-plane position within thresholds95.0 (38/40) 0.91± 0.42 ◦ / 0.0018± 0.0010 m 19.4± 4.8 Insertion executionDirection and 3-D landing position within thresholds90.0 (36/40) 2.57± 1.16 ◦ / 0.015± 0.006 m 45.73± 13.49 Image-quality adjustmentImage restored to a parseable quality range97.5 (39/40)– 1.63± 0.61 Safety stopTimely refusal in a high-risk state100.0 (40/40)– 0.12± 0.02 and P. C. Kim, “Supervised autonomous robotic soft tissue surgery,” Science translational medicine, vol. 8, no. 337, p. 337ra64–337ra64, 2016. [9] J. W. Kim, T. Z. Zhao, S. Schmidgall, A. Deguet, M. Kobilarov, C. Finn, and A. Krieger, “Surgical robot transformer (srt): Imitation learning for surgical tasks,” arXiv preprint arXiv:2407.12998, 2024. [10] J. W. Kim, J.-T. Chen, P. Hansen, L. X. Shi, A. Goldenberg, S. Schmidgall, P. M. Scheikl, A. Deguet, B. M. White, D. R. Tsai et al., “Srt-h: A hierarchical framework for autonomous surgery via language- conditioned imitation learning,” Science robotics, vol. 10, no. 104, p. eadt5254, 2025. [11] M. Moghani, L. Doorenbos, W. C.-H. Panitch, S. Huver, M. Azizian, K. Goldberg, and A. Garg, “Sufia: language-guided augmented dex- terity for robotic surgical assistants,” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, p. 6969–6976. [12] Z. Zeng, Z. Zhuo, X. Jia, E. Zhang, J. Wu, J. Zhang, Y. Wang, C. H. Low, J. Jiang, Z. Zheng et al., “Surgvlm: A large vision-language model and systematic evaluation benchmark for surgical intelligence,” arXiv preprint arXiv:2506.02555, 2025. [13] G. Wang, J. Wang, W. Mo, L. Bai, K. Yuan, M. Hu, J. Wu, J. He, Y. Huang, N. Padoy et al., “Surgvidlm: Towards multi-grained sur- gical video understanding with large language model,” arXiv preprint arXiv:2506.17873, 2025. [14] A. Ye, Y. Song, J. Su, and D. Zhang, “Non-invasive spatial registration using customized dental bracket and improved genetic algorithms,” in 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE). IEEE, 2024, p. 3250–3255. [15] A. Ye, X. Wang, Y. Song, Y. Li, Z. Zhu, and D. Zhang, “Fully automated insertion algorithm of flexible electrodes for invasive brain-machine interface,” Pattern Recognition Letters, 2026. [16] A. Ye, X. Wang, Y. Song, Y. Li, S. Yu, and D. Zhang, “Holenet: Neural network for 5-dof skull-hole-center localization for minimally invasive brain-machine interfaces surgery,” IEEE Transactions on Med- ical Robotics and Bionics, 2026. [17] S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge et al., “Qwen3-vl technical report,” arXiv preprint arXiv:2511.21631, 2025. [18] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” Iclr, vol. 1, no. 2, p. 3, 2022. [19] Y. Zhao, A. Gu, R. Varma et al., “PyTorch FSDP: Experiences on scaling fully sharded data parallel,” Proceedings of the VLDB Endowment, vol. 16, no. 12, p. 3848–3860, 2023. [20] E. B. Wilson, “Probable inference, the law of succession, and statistical inference,” Journal of the American Statistical Association, vol. 22, no. 158, p. 209–212, 1927. [21] Q. McNemar, “Note on the sampling error of the difference between correlated proportions or percentages,” Psychometrika, vol. 12, no. 2, p. 153–157, 1947. [22] N. Percie du Sert, V. Hurst, A. Ahluwalia et al., “The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research,” PLOS Biology, vol. 18, no. 7, p. e3000410, 2020. [23] Y. Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He et al., “Minicpm-v: A gpt-4v level mllm on your phone,” arXiv preprint arXiv:2408.01800, 2024. [24] Z. Wu, X. Chen, Z. Pan, X. Liu, W. Liu, D. Dai, H. Gao, Y. Ma, C. Wu, B. Wang et al., “Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding,” arXiv preprint arXiv:2412.10302, 2024. [25] MetaAI,“TheLlama4herd:Thebeginningofanew eraofnativelymultimodalAIinnovation,”Online,2025, accessed: 2026-08-07. [Online]. Available: https://ai.meta.com/blog/ llama-4-multimodal-intelligence/ [26] W. Wang et al., “InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency,” arXiv preprint arXiv:2508.18265, 2025. [27] QwenTeam,“Qwen2.5-VLtechnicalreport,”arXivpreprint arXiv:2502.13923, 2025.