Paper deep dive

Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation

Xuesong Li, Shuai Chen, Feng Li, Zhongliang Jiang, Nassir Navab, Yuan Bi

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 47

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Ultrasound (US) acquisition depends on the operator's ability to interpret anatomy and anticipate how the view will change with probe motion. Many robotic US navigation methods select actions without explicitly predicting these anatomical changes. We propose SonoGraph-WM, an action- and goal-conditioned world model for anticipatory probe navigation. The model represents anatomy as scene graphs (SGs), capturing visible structures, their geometry, and spatial relationships without synthesizing US images. Given a history of SGs and probe poses, a unified Transformer jointly predicts future SGs and poses. A receding-horizon planner recursively imagines candidate trajectories, selects the shortest predicted path reaching a goal graph, and follows it over a short execution horizon before replanning from new observations. To reduce reliance on tracked and anatomically annotated US sequences, we generate aligned SG--pose training data from computed tomography (CT) label maps along surface-constrained probe trajectories. On four held-out CT cases, spatial relation F1 remains above 93% over 20 prediction steps, and closed-loop navigation achieves 77.50% and 75.00% success for the gallbladder and pancreas, respectively, using annotation-derived SGs. In robot--phantom navigation experiments with label-map-derived SGs, the planner reached the target view in 73.7% of trials. These findings support CT-supervised anatomical world modeling for probe planning and highlight the importance of frequent observation updates for reliable navigation. Project Page: this https URL

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

44,172 characters extracted from source content.

Expand or collapse full text

Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation Xuesong Li a,b , Shuai Chen a , Feng Li a,b , Zhongliang Jiang c , Nassir Navab a,b , and Yuan Bi a,b Abstract— Ultrasound (US) acquisition depends on the op- erator’s ability to interpret anatomy and anticipate how the view will change with probe motion. Many robotic US nav- igation methods select actions without explicitly predicting these anatomical changes. We propose SonoGraph-WM, an action- and goal-conditioned world model for anticipatory probe navigation. The model represents anatomy as scene graphs (SGs), capturing visible structures, their geometry, and spatial relationships without synthesizing US images. Given a history of SGs and probe poses, a unified Transformer jointly predicts future SGs and poses. A receding-horizon planner recursively imagines candidate trajectories, selects the shortest predicted path reaching a goal graph, and follows it over a short execution horizon before replanning from new observations. To reduce reliance on tracked and anatomically annotated US sequences, we generate aligned SG–pose training data from computed tomography (CT) label maps along surface- constrained probe trajectories. On four held-out CT cases, spatial relation F1 remains above 93% over 20 prediction steps, and closed-loop navigation achieves 77.50% and 75.00% success for the gallbladder and pancreas, respectively, using annotation-derived SGs. In robot–phantom navigation exper- iments with label-map-derived SGs, the planner reached the target view in 73.7% of trials. These findings support CT- supervised anatomical world modeling for probe planning and highlight the importance of frequent observation updates for reliable navigation. Project Page: https://noseefood. github.io/us-sonograph-wm/. I. INTRODUCTION Medical ultrasound (US) is widely used for its portabil- ity and real-time imaging without ionizing radiation [1]. Obtaining the desired view requires skilled adjustment of probe position and orientation, which can limit acquisition reproducibility [1]. Robotic US systems (RUSS) aim to improve repeatability and reduce the physical workload of scanning [2]. During autonomous scanning, the robot must determine how to move the probe based on the anatomy visible in the current US image. Existing RUSS navigation methods select probe motions in several ways. Registration-based approaches use rib-cartilage geometry to align US with computed tomography (CT) and transfer template scanning paths to individual patients [3], [4]. Rule-based and visual-servoing methods [5] adjust the probe using detected anatomical features [6]. Model-free reinforcement learning (RL) and imitation learning (IL) learn motions from interaction or expert demonstrations [7]–[9]. Rule-based controllers and model-free policies typically act without explicitly predicting which structures will become a Computer Aided Medical Procedures (CAMP), Technical University of Munich, Munich, Germany; b Munich Center for Machine Learning (MCML), Munich, Germany; c Department of Mechanical Engineering, The University of Hong Kong, Hong Kong SAR, China. Fig. 1.Concept overview of SonoGraph-WM. For each candidate probe motion, the model predicts anatomical scene graphs (SGs) and probe poses. The planner selects the shortest predicted path among those whose SGs match a goal graph. B-mode examples illustrate the anatomical views represented by the graphs. visible. Such predictions would allow candidate motions to be compared before execution, reflecting how sonographers use anatomical knowledge to anticipate changes in the view. Predicting these changes requires anatomical information beyond the current image. B-mode US shows a cross-section rather than the complete three-dimensional arrangement of organs. The target may lie outside the imaging plane, and probe movement changes which structures are visible [10]. Sonographers use anatomical knowledge to interpret visible structures, guide probe adjustments [1], and anticipate how neighboring structures and their spatial arrangement may appear. We aim to give robots this ability to anticipate anatomical changes before choosing their next motion. A learned world model predicts what a robot would observe after an action [11], [12]. Recursive predictions enable comparison of paths before execution, as explored in navigation and manipulation [13], [14]. In US, DreamReg uses predicted observations to refine registration [15], while Fan et al. [16] use rewards from predicted US images to train a predictor of probe motions toward a target view. However, predicting future US images requires modeling arXiv:2609.32837v1 [cs.RO] 26 Sep 2026 both anatomical changes and appearance variations caused by speckle [17], low contrast, and acquisition settings [10]. Anatomical scene graphs (SGs) describe structures without reproducing image texture. Li et al. [10] use SGs to explain US images and guide scanning toward missing structures. Nodes encode anatomical structures, positions, and sizes, while edges describe spatial relationships. This represen- tation predicts which structures candidate motions would reveal and where they would appear. However, the guidance remains coarse and does not directly specify executable robot motions. Extending it to motion planning requires learning how SGs evolve with probe motion from aligned SG–pose sequences, which remain scarce in US. Anatomically labeled CT volumes offer an alternative source of these sequences. We sample simulated US imaging planes along generated probe paths using CADS organ label maps [18]. At each plane, intersecting structures, their geom- etry, and spatial relationships form an SG, which is paired with the corresponding probe pose. This produces multiple SG–pose sequences from each volume without acquiring and annotating US scans for every path. We use these sequences to learn anatomical changes under probe motion and evaluate prediction and navigation using annotation- derived SG observations. We train SonoGraph-WM on these sequences and use its predictions in a receding-horizon planner (Fig. 1). A unified Transformer takes the SG–pose history, a candidate action for the first predicted step, and goal graphs specifying the desired views. It learns subsequent graph and pose changes from generated reference trajectories. The planner recursively extends candidate paths and checks for predicted goal-graph matches. It selects the shortest predicted goal- reaching path and follows it over a short execution horizon, then repeats prediction and planning using newly observed SGs and probe poses. The main contributions are: • We propose SonoGraph-WM, an action- and goal- conditioned world model that jointly predicts anatomi- cal SGs and probe poses for interpretable, anticipatory probe navigation. A receding-horizon planner selects the shortest predicted path reaching a goal graph and follows it over a short horizon before replanning using new observations. • We introduce a CT-derived training-data pipeline that generates aligned SG–pose sequences from anatomical label maps, reducing the need for tracked and anatom- ically annotated US training sequences. • We evaluate prediction and closed-loop navigation on held-out CT cases and demonstrate the feasibility of real-world robotic US navigation through phantom ex- periments. I. RELATED WORK A. Autonomous Probe Navigation in Robotic Ultrasound Autonomous robotic US scanning combines control of probe contact and orientation with navigation toward a target view [2]. Force- and confidence-based controllers address the contact and alignment requirements [19], [20]. Building on these controls, navigation methods use anatomical geometry, image feedback, or learned policies to determine how the probe should move. Geometry-based methods transfer template scanning paths through rib-cartilage US–CT registration [3], [4] or use seg- mented vessel geometry to maintain centering and perpen- dicular imaging during scanning [6]. View-quality methods guide target search through Bayesian optimization with an expert-derived prior [21] or reward functions learned from ranked image pairs [22]. Learned policies map observations to probe motions. Deep RL has been applied to target-plane navigation [7], [23], including transfer from simulated vessel masks to real US segmentations [8]. SonoGym supports policy training and evaluation in simulation [24], while UltraBot learns probe adjustments through imitation of expert demonstrations [9]. While these approaches can guide probe motion based on the US observations, they generally do not explicitly predict how the anatomical view will evolve along candidate paths. A world model that predicts future anatomical views would enable the robot to compare candidate paths before execution and select those expected to reach the target view efficiently, providing an explicit, interpretable basis for navigation. B. World Models for Ultrasound A world model learns how actions change subsequent observations or representations of the scene. The quantities it predicts depend on how they will be used: PlaNet plans through learned latent states [12], while MuZero predicts rewards and values needed for planning [25]. Contrastive structured world models learn how object-level states change under actions [26]. For US, this raises two related questions: what information about a future view should be predicted, and how should that prediction guide probe movement? One use of world modeling is to learn anatomical and mo- tion information for a probe-guidance network. EchoWorld predicts masked and motion-conditioned image features dur- ing pretraining, then fine-tunes a guidance network that uses previous images and motions [27]. Fan et al. [16] train an action-conditioned diffusion model to predict future US images. They then use rewards from the frozen world model to fine-tune a goal-conditioned action predictor. In these approaches, learning how images change with probe motion supports the training of a network that produces probe guidance. World-model predictions can also be used during infer- ence, as illustrated by DreamReg for 2D–3D US registra- tion [15]. DreamReg summarizes previous observations and poses in a latent estimate of the alignment, then simulates candidate probe motions and their predicted observations to refine the registration transform. Anticipating the consequences of probe motion does not necessarily require predicting every detail of a future US image. Sonographers can instead draw on anatomical knowl- edge to reason about which structures may appear and how Fig. 2.System overview of SonoGraph-WM. CT label maps supply SG–pose sequences for joint supervision. During navigation, the frozen model recursively predicts independent branches, each with an upper limit of 300 states. Branches stop at their first predicted goal match or an invalid state. Selection minimizes predicted probe travel among goal-matching branches; one action is executed before the observed SG is checked and history updated. their spatial arrangement may change. Given the noisy nature of US images, high-fidelity image prediction introduces the additional burden of modeling appearance variations that may be irrelevant to navigation. Scene graphs offer a practical alternative by explicitly representing anatomical structures, their geometry, and spatial relationships, allowing a world model to focus on navigation-relevant anatomical changes. C. Anatomical Scene Graphs and CT-Derived Training Data Scene graphs explicitly represent objects and their rela- tionships, allowing image content to be described through the objects present and how they are arranged [28]. In robotics, 3D dynamic scene graphs organize objects, places, and their spatial relationships [29]; SayPlan uses these graphs to identify objects and locations relevant to a task and plan robot actions [30]. For US, Li et al. [10] estimate anatomical structures and their relations from neck images, then use the graphs for image explanation and scanning guidance toward missing structures. Their work shows how the structures visible in a US image can provide information for deciding where to scan next. Predicting SGs along a probe path adds a temporal require- ment to image-based SG estimation. Li et al. [10] annotate structures and relations in individual neck US images and identify annotation effort and limited anatomical coverage as constraints. World-model training additionally requires these labels across successive scanning steps, paired with the probe’s position and orientation. Image-level graph an- notations alone do not provide these SG–pose sequences. CT segmentation resources offer a source of anatomical labels from which such sequences can be constructed. To- talSegmentator provides CT data and segmentation mod- els covering multiple organs and other anatomical struc- tures [31]. CADS combines and standardizes CT datasets to provide labels with broad anatomical coverage [18]. These volume labels allow structures and their relationships to be extracted at successive imaging planes along a probe path. We use CADS label maps to construct SGs at sampled probe poses and assemble them into training sequences. The resulting SG–pose pairs let the world model learn how structures in the imaging plane change as the probe moves. During inference, SGs can be extracted from acquired US images using specialized anatomical detection models. I. METHODS A. Problem Formulation SonoGraph-WM predicts anatomical views and probe poses to compare candidate motions before execution (Fig. 2). At time t, H t contains the latest H observed pairs (S i , P b i ), including the current state. Here, S i denotes the observed SG. The pose P b i ∈ SE(3) maps the probe frame to the anatomy-specific body frame. Consecutive poses define the local motion A t = (P b t ) −1 P b t+1 ,(1) so P b t+1 = P b t A t . This gives action labels during training; at inference, candidate actions are proposed before execution. Their six-dimensional encoding a t comprises local transla- tion (m) and a rotation vector (rad). The goal is an unordered set of acceptable views, G goal = S goal m M m=1 , supplied for the requested target. Goal poses are not model inputs; goal-set construction and availability are specified in Sec. IV-A. The world model predicts b T t = F θ (H t , a t ,G goal ),(2) where b T t contains K predicted pairs ( b S t+k , b P b t+k ) for k = 1,...,K. Each graph describes the anatomy expected at its paired pose. The candidate action conditions the first transition. Later steps form a learned goal-conditioned con- tinuation. Fig. 3.Ultrasound scene graph representation with per-node attributes and relations. B. Ultrasound Scene-Graph World Model Architecture The world model jointly predicts future anatomical SGs and probe poses, conditioned on the observed SG–pose history, a candidate action, and a set of goal SGs. Each SG describes the anatomical structures intersecting the imaging plane. Nodes have fixed anatomical identities and eight nu- merical attributes: component count, normalized bounding- box coordinates (x,y,w,h), normalized centroid coordinates (x,y), and visible-area ratio. Spatial relationships between structures are represented by three relation types: adjacent to, left of, and superficial to. One example is shown in Fig. 3. A shared SG encoder combines node features and pairwise features, including spatial relations, into a d-dimensional em- bedding. Probe poses are encoded from a 9-D representation comprising 3-D translation and continuous 6-D rotation. A multilayer perceptron fuses each pair of SG and pose em- beddings into a state token. The Transformer encoder takes H history state tokens, one candidate action token, and M goal tokens obtained using the shared SG encoder. Temporal embeddings encode the order of the history states, while type embeddings distinguish the action and goal tokens. Goal tokens receive no order-dependent positional encoding, allowing the goals to be represented as an unordered set. The Transformer decoder uses K learned queries to predict the next K SG–pose pairs in parallel. Each decoded token is passed to an SG head and a pose head. The SG head predicts node presence, numerical attributes, pair features, and spatial relations under the fixed graph schema. The pose head predicts relative 3-D translation and continuous 6-D rotation, which are converted into a rigid transformation d ∆P t→t+k ∈ SE(3). The future probe pose is reconstructed as b P b t+k = P b t d ∆P t→t+k . All predicted poses within a chunk are expressed relative to the same current pose P b t . Accordingly, the supervision target for step k is ∆P t→t+k = (P b t ) −1 P b t+k . C. CT-Derived Supervision and Training We train on SG–pose sequences generated from CADS label maps [18]; trajectory construction and dataset settings are given in Sec. IV-A. Predicting anatomical structure rather than image texture lets CT labels supply this supervision without paired annotated US scans. At each waypoint, the imaging plane intersects the label map to obtain the anatomical structures and their spatial relationships. The same procedure generates history, future, and goal SGs. The reference trajectories are geometrically generated, not recorded sonographer demonstrations. Each predicted SG–pose pair is supervised by the corre- sponding future waypoint in the CT-derived trajectory. The training objective combines graph and pose losses: L =L SG +L pose , L SG =L pres +L attr +L pair +L rel , L pose =L trans +L rot . (3) L pres applies binary cross-entropy to each anatomical node’s presence label. L rel applies binary cross-entropy indepen- dently to each relation label, allowing multiple relations per node pair.L attr andL pair use SmoothL1 for numerical node attributes and pair features, respectively; component count is regressed. Pose supervision uses the relative transforms ∆P t→t+k . Let R k and b R k denote their reference and pre- dicted rotation matrices, respectively. The per-step rotation loss is ℓ rot,k = arccos tr( b R ⊤ k R k )−1 2 . L rot aggregates these angles over valid future steps, while L trans penalizes relative translation error. D. Planning by Recursive Imagination The planner uses the world model to compare candidate probe trajectories before execution. At each planning iter- ation, it generates candidate motions, recursively predicts their anatomical outcomes, and selects the shortest predicted path that reaches a goal view. The robot follows the selected trajectory over a short execution horizon before replanning from new observations. At time t, we sample C candidate probe motions by randomly perturbing the current probe velocity. Each candi- date initializes an independent imagined branch, conditioned on the same observed history H t and goal set. The world model extends each branch in K-step chunks. An observed or predicted SG matches the goal set if its distance to at least one goal graph falls below a threshold calibrated on training cases. The graph distance compares node presence, numerical attributes, pair features, and spatial relations. For branch j, a chunk starting at index s predicts SGs and body-frame probe poses at s + 1,...,s +K, where s = t for the first chunk. To continue the rollout, we derive the next conditioning motion from the final two predicted poses: b A (j) next = ( b P b,(j) s+K−1 ) −1 b P b,(j) s+K .(4) Its 6-D encoding serves as the first-action input for the next chunk, starting at s + K. The branch history is updated using the latest H SG–pose pairs, including predicted pairs, while the goal set remains fixed. This process extends the imagined trajectory without acquiring new observations. Predicted states are checked in temporal order, and each branch terminates at its first goal match, immediately before an invalid state, or upon reaching the imagination limit n max . Let B goal denote the branches that reach a predicted goal within the imagination limit, and let n j denote the first goal-matching step of branch j. The cumulative translational distance along this branch to the predicted goal is D j = n j X k=1 b p b,(j) t+k − b p b,(j) t+k−1 2 ,(5) where b p b,(j) t+k is the translational component of the predicted probe pose, and b p b,(j) t = p b t is the current observed posi- tion. The planner selects the goal-reaching branch with the shortest predicted travel distance: j ∗ = arg min j∈B goal D j .(6) Thus, selection minimizes predicted translational travel to a goal, measured in millimeters. Probe orientation is included in the predicted poses but does not contribute to the path- length criterion. If no branch reaches a predicted goal, the planner returns NO GOALREACHED without selecting a trajectory. The robot follows the selected trajectory under the surface constraint over a short execution horizon. Newly acquired SGs and measured probe poses then update the observed history for the next planning iteration. Navigation succeeds only when an observed SG satisfies d goal (S)≤ τ goal ; a pre- dicted goal match alone does not establish task completion. The prediction, selection, and execution cycle repeats until observed success, no eligible branch, or the execution limit is reached. IV. EXPERIMENTAL EVALUATION A. Data and Implementation Details 1) Dataset and Reference Goals: We use CT volumes and corresponding anatomical label maps from CADS [18]. From 24 patient cases, we generate 1,839 probe trajectories containing 63,164 waypoints. For each patient CT and target organ, we predefine probe positions and orientations that yield acceptable views of that organ, termed target-view poses. We extract a reference SG from the CT label map at each of these poses. By default, each target organ has M = 3 reference SGs representing alternative acceptable views. The complete set of reference SGs is supplied asG goal at inference, including for held-out patients. Training, future-state prediction, and CT-based navigation use annotation-derived SGs. The robot–phantom protocol generates SG observations from a registered phantom label map and measured probe poses. These settings test prediction and planning separately from B-mode-to-SG estimation. The graph schema uses at most seven anatomical nodes, 21 node pairs, and 51 relation triplets. Pair geometry comprises horizontal displacement, relative depth, normalized distance, and an adjacency score. 2) Geometric Reference Trajectory Generation: For each patient, we first select and manually validate a set of clin- ically meaningful target views for each target organ. We then sample diverse initial probe poses across the acces- sible surface of the lower abdomen and manually exclude infeasible or highly redundant configurations. Each retained start pose is paired with every verified target view for the corresponding organ, and a smooth reference trajectory constrained to the body surface is generated. Along each trajectory, the probe progressively adjusts its position and orientation toward the target view. All trajectories undergo quality control for contact validity, geometric feasibility, motion continuity, and terminal consistency, with manual review and correction when necessary. At each sampled waypoint, the corresponding scene graph is constructed based on CT label maps, yielding synchronized SG–pose pairs (Sec. I-C). The transformation between consecutive poses is represented as a relative action in the probe-local coordinate frame (Eq. (1)). Each expert trajectory therefore provides a temporally aligned sequence of SG observations, probe poses, and local actions for subsequent world model training. 3) Temporal Samples and Case Split: Each sample con- tains H = 10 history SG–pose pairs, the current trajectory action, the target-specific goal set, and K = 10 future SG– pose targets. The patient-disjoint split comprises 20 training cases (53,344 samples) and four test cases (7,981 samples). Nor- malization statistics and the goal threshold use training cases only. 4) Model and Planner Configuration: The Transformer uses embedding dimension d = 384, four encoder layers, four decoder layers, eight attention heads, a feed-forward dimension of 1,536, and dropout of 0.1. Action vectors and pose translations are normalized using training-set statistics before network encoding. This prepro- cessing is omitted from the model notation; execution uses physical units. At each planning iteration, the planner samples C = 8 candidate motions by randomly perturbing the current probe velocity. Each branch can extend to at most n max = 300 predicted states (30 chunks of K = 10). Goal matching or an invalid state terminates a branch earlier. Selection uses the predicted path length up to each eligible branch’s first goal match; each planning iteration executes one physical step. Because H = K in this configuration, each recursive call uses the previous chunk’s predicted SG–pose pairs as its history. 5) Robotic Ultrasound Setup: The robotic US platform comprises an LBR iiwa 7 R800 robot (KUKA, Germany) and a bk3000 US system (BK Medical, Herlev, Denmark) equipped with a 5C1e curved-array transducer. The probe is mounted on the robot end effector, and the B-mode imaging depth is set to 16 cm. Robot experiments use a US-22 CT/US abdominal phantom (Kyoto Kagaku, Japan). B. Evaluation Questions and Common Protocol The evaluation addresses three questions: (1) How accu- rately do predicted SGs represent the anatomy at their cor- responding predicted probe poses, including over recursively extended rollouts? (2) How reliably does the planner reach an acceptable target view from different initial distances? (3) Can the system guide a physical robot to the target view in (a) Start state(b) Navigation and replanning(c) Stop state(d) Target SG Gallbladder Spine Spleen Pancreas Vessels Liver Vessels Liver Gallbladder Start SG + poseFinal SG + pose Vessels Liver Gallbladder Target SG Local planningReplan Goal Start Stop Pancreas Vessels Liver Kidneys Spine Pancreas Gallbladder Liver Vessels Kidneys Spine Start SG + poseFinal SG + pose Pancreas Gallbladder Liver Vessels Kidneys Spine Target SG Local planningReplan Goal Start Stop StartExecuted path8 candidate trajectoriesPreset goalGoal regionModel stop Fig. 4.Closed-loop navigation examples for the gallbladder (top) and pancreas (bottom). (a) Initial SG and probe pose. (b) SonoGraph-WM evaluates eight candidate future trajectories, executes the first action of the selected trajectory, and replans from the updated observation. Purple dashed curves show predicted candidates and red curves show the executed path. (c) Final SG and probe pose at the model-selected stopping state. (d) Goal SG used for navigation conditioning. TABLE I SCENE-GRAPH PREDICTION ACCURACY ON HELD-OUT CASES WITH TRAJECTORY-ACTION INPUTS. GallbladderPancreas Prediction stage Local step k Total steps n Node visibility F1(%) ↑ Spatial relation F1(%) ↑ BBox mIoU (%) ↑ Node visibility F1(%) ↑ Spatial relation F1(%) ↑ BBox mIoU (%) ↑ Initial11 88.74± 0.21 94.45± 0.08 39.92± 0.32 93.13± 0.19 94.89± 0.10 42.44± 0.37 33 87.69± 0.23 94.35± 0.08 38.95± 0.34 92.33± 0.22 94.77± 0.11 41.13± 0.40 55 86.95± 0.24 94.20± 0.09 38.34± 0.34 91.42± 0.26 94.67± 0.11 39.79± 0.42 1010 84.84± 0.30 93.80± 0.13 36.09± 0.42 89.97± 0.33 94.69± 0.11 38.87± 0.49 Recursive111 84.21± 0.31 94.14± 0.09 34.39± 0.37 89.21± 0.33 94.26± 0.15 37.00± 0.50 313 84.10± 0.34 94.16± 0.10 33.57± 0.43 89.07± 0.38 94.19± 0.15 35.60± 0.56 515 83.94± 0.37 94.12± 0.12 32.92± 0.46 88.57± 0.42 94.28± 0.16 34.60± 0.60 1020 82.70± 0.51 93.59± 0.18 31.58± 0.57 87.50± 0.54 94.19± 0.19 34.09± 0.70 phantom experiments? We evaluate navigation toward two target organs, gallbladder and pancreas. For each target q, G (q) goal denotes the set of all acceptable goal graphs. Navigation trials use a frozen world model, fixed planner settings, and a fixed execution budget. Initial poses are divided into near (d 0 ≤ d split ) and far (d 0 > d split ) groups according to their reference surface-path distance d 0 to the nearest hidden goal pose. The threshold d split is set before evaluation on held-out cases. The goal poses are used only for trial sampling and evaluation and are not available to the planner. Results are reported for each target and distance group, together with an overall summary. C. Scene-Graph Prediction Accuracy We evaluate the world model’s ability to predict anatom- ical SGs over an initial prediction chunk and its first re- cursive extension on four held-out CT cases. Rollouts are conditioned on either a gallbladder or pancreas goal set, with both groups evaluated over the full anatomical graph. Each rollout begins with a ten-frame reference history and the cor- responding trajectory action. The recursive extension uses the model’s predicted SG–pose history and the motion derived from Eq. (4), without additional reference observations. We evaluate local steps k ∈1, 3, 5, 10 in each chunk, covering cumulative prediction steps up to n = 20. For each predicted SG–pose pair, we extract a reference SG from the held-out CT label map at the predicted pose. This comparison assesses whether the imagined anatomy agrees with the predicted probe location. We report three complementary metrics: node visibility F1 measures which anatomical structures are present, spatial relation F1 mea- sures their pairwise relationships, and BBox mIoU measures the overlap between predicted and reference anatomical bounding boxes. Table I reports the mean and standard deviation of each metric at each evaluated step. Spatial relation prediction remains stable throughout the evaluated horizon. Between steps 1 and 20, relation F1 decreases only from 94.45% to 93.59% for gallbladder- conditioned rollouts and from 94.89% to 94.19% for pancreas-conditioned rollouts. In contrast, node visibility F1 decreases from 88.74% to 82.70% and from 93.13% to 87.50%, respectively. Geometric localization also deterio- rates: BBox mIoU decreases from 39.92% to 31.58% for the gallbladder group and from 42.44% to 34.09% for the pancreas group. Both visibility and bounding-box overlap continue to decline during recursive extension. These results suggest that the model retains the broad spatial relationships between anatomical structures over ex- tended imagined rollouts, while predicting their visibility and precise geometry becomes more difficult. The gap between high relation F1 and limited bounding-box overlap identifies geometric precision as a remaining challenge for long- horizon prediction. Whether these predictions support target- TABLE I CLOSED-LOOP TARGET-VIEW ACQUISITION ON HELD-OUT CT CASES WITH EXECUTION HORIZONS OF 1, 5, AND 10 STEPS. Execute 1Execute 5Execute 10 Initial distance Successes / trials n/N Success rate (%) ↑ Actions to goal (steps) ↓ Final position error (m) ↓ Successes / trials n/N Success rate (%) ↑ Actions to goal (steps) ↓ Final position error (m) ↓ Successes / trials n/N Success rate (%) ↑ Actions to goal (steps) ↓ Final position error (m) ↓ Gallbladder Near52/6481.25 19.37± 11.76 12.43± 11.8622/6434.38 19.64± 15.84 15.32± 13.2925/6439.06 27.64± 28.30 11.69± 9.60 Far41/5673.21 37.51± 15.67 16.25± 11.7727/5648.21 38.59± 16.32 12.14± 9.0821/5637.50 42.76± 25.40 13.38± 11.04 Overall93/12077.50 27.37± 16.29 14.11± 11.9149/12040.83 30.08± 18.57 13.56± 11.1646/12038.33 34.54± 27.78 12.46± 10.20 Pancreas Near52/6580.00 18.33± 10.57 15.15± 8.1119/6529.23 13.95± 22.02 15.80± 9.5821/6532.31 17.57± 21.42 12.38± 7.97 Far38/5569.09 33.18± 13.56 13.10± 7.089/5516.36 46.89± 34.60 19.77± 3.526/5510.91 58.00± 35.01 16.80± 2.55 Overall90/12075.00 24.60± 13.96 14.28± 7.7228/12023.33 24.54± 30.39 17.08± 8.2727/12022.50 26.56± 29.70 13.36± 7.33 view acquisition is assessed separately through closed-loop navigation experiments. D. Closed-Loop Navigation Performance We evaluate closed-loop target-view acquisition in held- out CT anatomical environments for the gallbladder and pancreas. To examine the effect of replanning frequency, we compare execution horizons of 1, 5, and 10 steps, denoted Execute 1, Execute 5, and Execute 10. At each planning iteration, the robot follows the selected predicted trajectory for the specified horizon before updating its SG–pose history from new observations and replanning. A trial succeeds when an observed SG satisfies the goal criterion within the execution budget. Table I reports results for 120 trials per target under each execution setting, grouped by near and far initial distances. Success rate is the proportion of all attempted trials that reach an acceptable target view. Actions to goal count the executed probe-motion steps until the first successful obser- vation. Final position error measures the minimum Euclidean distance from the terminal probe position to the region defined by the three predefined target-view positions. Action counts and position errors are reported as mean ± standard deviation over successful trials only. Overall action counts and position errors are calculated across all successful trials from both distance groups. Navigation success is determined by whether the observed SG matches an acceptable goal graph; position error separately measures the distance to the nearest reference target-view position. Replanning after every step achieves the highest success rates for both targets and in both distance groups. With Execute 1, the planner succeeds in 93/120 gallbladder trials (77.50%) and 90/120 pancreas trials (75.00%). Increasing the execution horizon to five steps reduces overall success to 40.83% and 23.33%, respectively. With ten-step execution, success decreases further to 38.33% and 22.50%. These results demonstrate the importance of frequent observation updates for navigation with the current world model. The degradation with longer execution horizons is consistent with the reduced prediction accuracy observed over extended rollouts. Under Execute 1, distant starts are associated with lower success rates and longer successful trajectories. Gallbladder success decreases from 81.25% for near starts to 73.21% for far starts, while the mean action count increases from 19.37 to 37.51. For the pancreas, success decreases from 80.00% to 69.09%, and the mean action count increases from 18.33 to 33.18. Longer execution horizons particularly affect pancreas navigation from far starts, where success falls to 16.36% with Execute 5 and 10.91% with Execute 10. For successful trials under Execute 1, the mean final position error is 14.11 m for the gallbladder and 14.28 m for the pancreas. Thus, reaching an acceptable anatomical view does not require exact recovery of a predefined refer- ence position. Although some longer-horizon settings yield lower mean position errors, these values describe only the successful subset and should be interpreted alongside their substantially lower success rates. Overall, the results support the use of predicted anatomical trajectories for target-view acquisition, while showing that re- liable performance depends strongly on frequent replanning from observed anatomy. E. Robot–Phantom Navigation Performance We further evaluate SonoGraph-WM on the US-22 phan- tom using the robotic US platform described in Sec. IV-A.5. A compliant position–force controller maintains acoustic contact and workspace limits. The phantom, robot-base, and probe frames are registered before each session. A registered phantom label map and the measured probe pose generate the current SG, matching the similar SG used for training. The acquired B-mode stream is recorded for target-view and contact assessment but is not an input to the planner. This experiment therefore tests physical execution and repeated replanning, not B-mode-to-SG perception. Trials start from free poses across phantom surface. Fig. 5.Experimental setup for scene graph-guided robotic ultrasound navigation on an abdominal phantom. The planner reached the target view in 8/10 gallbladder trials (80.0%) and 6/9 pancreas trials (66.7%), yielding an overall success rate of 73.7% (14/19). Among successful trials, actions to goal were 22.0 ± 12.81 steps for the gallbladder and 23.5 ± 12.44 steps for the pancreas, with 22.64 ± 12.18 steps overall (mean ± standard deviation). Across all trials, final position errors were 37.70± 19.08 m for the gallbladder and 23.07± 10.45 m for the pancreas (mean ± standard deviation), with an overall mean of 30.77 m. V. CONCLUSION AND LIMITATIONS SonoGraph-WM learns anatomical transitions from CT- derived SG–pose sequences and selects the shortest predicted path among branches that match a goal view. On four held- out cases, spatial relation F1 remained above 93% through 20 predicted steps, while bounding-box overlap declined with the prediction horizon. Closed-loop acquisition succeeded in 77.50% of gallbladder trials and 75.00% of pancreas trials. Robot–phantom trials reached the target view in 14/19 cases (73.7%), showing that the predicted trajectories can be executed on a physical system. These results support anatomical prediction and navigation with annotation-derived SG observations and case-specific goal graphs. Stable relation prediction coexisted with limited geometric precision. Far-start trials had lower success rates and required more actions among successful trials. Prediction accuracy beyond 20 steps remains uncharacterized despite a maximum planning horizon of 300 states. Predicted goal matches guide selection, while observed SG feedback verifies actual completion. Evaluation covers four test anatomies and assesses the complete system without isolating individual design con- tributions. Extending the approach to B-mode input re- quires a schema-compatible SG estimator and evaluation of perception errors during navigation. This would test whether CT-trained structural predictions remain useful with image-derived observations. Cross-subject goal representa- tions would further reduce reliance on case-specific reference graphs. REFERENCES [1] Z. Jiang, S. E. Salcudean, and N. Navab, “Robotic ultrasound imaging: State-of-the-art and future perspectives,” Medical Image Analysis, vol. 89, p. 102878, 2023. [2] Y. Bi, Z. Jiang, F. Duelmer, D. Huang, and N. Navab, “Machine learning in robotic ultrasound imaging: Challenges and perspectives,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 7, p. 335–357, 2024. [3] Z. Jiang, X. Li, C. Zhang, Y. Bi, W. Stechele, and N. Navab, “Skeleton graph-based ultrasound-CT non-rigid registration,” IEEE Robotics and Automation Letters, vol. 8, no. 8, p. 4394–4401, 2023. [4] Z. Jiang, Y. Kang, Y. Bi, X. Li, C. Li, and N. Navab, “Class-aware cartilage segmentation for autonomous US-CT registration in robotic intercostal ultrasound imaging,” IEEE Transactions on Automation Science and Engineering, vol. 22, p. 4818–4830, 2025. [5] Z. Jiang et al., “Needle segmentation using GAN: Restoring thin instrument visibility in robotic ultrasound,” IEEE Transactions on Instrumentation and Measurement, vol. 73, p. 1–11, 2024. [6] Z. Jiang et al., “Autonomous robotic screening of tubular structures based only on real-time ultrasound imaging feedback,” IEEE Transac- tions on Industrial Electronics, vol. 69, no. 7, p. 7064–7075, 2022. [7] H. Hase et al., “Ultrasound-guided robotic navigation with deep reinforcement learning,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, p. 5534–5541. [8] Y. Bi, Z. Jiang, Y. Gao, T. Wendler, A. Karlas, and N. Navab, “VesNet- RL: Simulation-based reinforcement learning for real-world US probe navigation,” IEEE Robotics and Automation Letters, 2022. [9] H. Jiang et al., “Towards expert-level autonomous carotid ultra- sonography with large-scale learning-based robotic system,” Nature Communications, vol. 16, no. 1, p. 7893, 2025. [10] X. Li, D. Huang, Y. Zhang, N. Navab, and Z. Jiang, “Semantic scene graph for ultrasound image explanation and scanning guidance,” in In- ternational Conference on Medical Image Computing and Computer- Assisted Intervention (MICCAI), 2025. [11] D. Ha and J. Schmidhuber, “Recurrent world models facilitate policy evolution,” in Advances in Neural Information Processing Systems (NeurIPS), 2018. [12] D. Hafner et al., “Learning latent dynamics for planning from pixels,” in International Conference on Machine Learning (ICML), 2019. [13] A. Bar, G. Zhou, D. Tran, T. Darrell, and Y. LeCun, “Navigation world models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. [14] Y. Du et al., “Learning universal policies via text-guided video generation,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [15] L. Kang, Y. Zhang, J. Shan, H. Gong, Q. Ding, and S. S. Cheng, “DreamReg: Belief-driven world model for 2D-3D ultrasound regis- tration,” arXiv preprint arXiv:2606.18825, 2026. [16] S. Fan et al., “Action-conditioned world model for goal plane probe guidance in robotic ultrasound,” arXiv preprint arXiv:2607.21918, 2026. [17] X. Li, N. Navab, and Z. Jiang, “Speckle2self: Self-supervised ultra- sound speckle reduction without clean data,” Medical Image Analysis, 2025. [18] M. Xu, T. Amiranashvili et al., “CADS: A comprehensive anatomical dataset and segmentation for whole-body anatomy in computed tomography,” arXiv preprint arXiv:2507.22953, 2025. [Online]. Available: https://arxiv.org/abs/2507.22953 [19] Z. Jiang, M. Grimm, M. Zhou, Y. Hu, J. Esteban, and N. Navab, “Au- tomatic force-based probe positioning for precise robotic ultrasound acquisition,” IEEE Transactions on Industrial Electronics, vol. 68, no. 11, p. 11 200–11 211, 2021. [20] Z. Jiang et al., “Automatic normal positioning of robotic ultrasound probe based only on confidence map optimization and force measure- ment,” IEEE Robotics and Automation Letters, vol. 5, p. 1342–1349, 2020. [21] D. Raina, S. H. Chandrashekhara, R. Voyles, J. Wachs, and S. K. Saha, “Robotic sonographer: Autonomous robotic ultrasound using domain expertise in Bayesian optimization,” in IEEE International Conference on Robotics and Automation (ICRA), 2023. [22] Z. Jiang, Y. Bi, M. Zhou, Y. Hu, M. Burke, and N. Navab, “Intelligent robotic sonographer: Mutual information-based disentangled reward learning from few demonstrations,” The International Journal of Robotics Research, 2024. [23] K. Li et al., “Autonomous navigation of an ultrasound probe towards standard scan planes with deep reinforcement learning,” in IEEE International Conference on Robotics and Automation (ICRA), 2021, p. 8302–8308. [24] Y. Ao et al., “SonoGym: High performance simulation for chal- lenging surgical tasks with robotic ultrasound,” arXiv preprint arXiv:2507.01152, 2025. [25] J. Schrittwieser et al., “Mastering Atari, Go, chess and shogi by planning with a learned model,” Nature, vol. 588, no. 7839, p. 604– 609, 2020. [26] T. Kipf, E. van der Pol, and M. Welling, “Contrastive learning of structured world models,” in International Conference on Learning Representations (ICLR), 2020. [27] Y. Yue, Y. Wang, H. Jiang, P. Liu, S. Song, and G. Huang, “EchoWorld: Learning motion-aware world models for echocardiography probe guidance,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, p. 25 993–26 003. [28] J. Johnson et al., “Image retrieval using scene graphs,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. [29] A. Rosinol, A. Gupta, M. Abate, J. Shi, and L. Carlone, “3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans,” in Robotics: Science and Systems (RSS), 2020. [30] K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. S ̈ underhauf, “SayPlan: Grounding large language models using 3D scene graphs for scalable robot task planning,” in Conference on Robot Learning (CoRL), 2023. [31] J. Wasserthal et al., “TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images,” Radiology: Artificial Intelligence, vol. 5, no. 5, p. e230024, 2023.

Instant research discovery