Paper deep dive
SAGE: A Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation
Lan Hu, Minghui Liwang, Wenbo Zhu, Xinlei Yi, Yiguang Hong, Xianbin Wang, Zhenzhen Jiao, Seyyedali Hosseinalipour
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/24/2026, 1:44:22 AM
Summary
The paper introduces SAGE, a socially-aware generative engine for heterogeneous multi-agent navigation. SAGE addresses the challenge of safe and socially compliant navigation in open human-robot environments by modeling robots and surrounding entities (pedestrians, cyclists, etc.) as a directed heterogeneous graph. It employs a Heterogeneous Graph Transformer (HGT) to encode type-specific asymmetric interactions and uses a diffusion-based generative model to jointly predict future entity trajectories and plan robot trajectories. A key innovation is a training-free safety-social energy guidance mechanism that refines sampled robot trajectories during inference using differentiable collision, kinematic, task-progress, and social-compliance terms, without requiring retraining.
Entities (10)
Relation Signals (8)
SAGE → evaluatedon → ETH/UCY
confidence 95% · Extensive experiments on real-world (ETH/UCY and SDD) and synthetic datasets verify the effectiveness of SAGE
SAGE → evaluatedon → SDD
confidence 95% · Extensive experiments on real-world (ETH/UCY and SDD) and synthetic datasets verify the effectiveness of SAGE
SAGE → uses → Diffusion-based generative model
confidence 95% · Conditioned on the resulting context, a diffusion-based generative module jointly models future entity trajectories and robot trajectory plans.
SAGE → uses → Heterogeneous Graph Transformer
confidence 95% · SAGE represents robots and surrounding entities as a directed heterogeneous graph and employs a Heterogeneous Graph Transformer (HGT) to encode type-specific asymmetric interactions.
SAGE → employs → Safety-social energy guidance
confidence 92% · During inference, a training-free safety-social energy guidance mechanism refines sampled robot trajectories
Heterogeneous Graph Transformer → models → Asymmetric Interactions
confidence 92% · employs a Heterogeneous Graph Transformer (HGT) to encode type-specific asymmetric interactions.
Safety-social energy guidance → optimizes → Social Compliance
confidence 90% · refines sampled robot trajectories using differentiable collision, kinematic, task-progress, and role-conditioned social-compliance terms.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safe and socially compliant navigation in open human-robot environments requires robots to reason about heterogeneous participants with different dynamics, autonomy levels, and social roles. Existing trajectory prediction and planning methods often rely on homogeneous interaction assumptions or enforce only geometric collision constraints, making it difficult to jointly model asymmetric interactions, coupled prediction-planning, and soft social norms. This paper proposes SAGE, a socially-aware generative engine for heterogeneous multi-agent navigation. SAGE represents robots and surrounding entities as a directed heterogeneous graph and employs a Heterogeneous Graph Transformer (HGT) to encode type-specific asymmetric interactions. Conditioned on the resulting context, a diffusion-based generative module jointly models future entity trajectories and robot trajectory plans. During inference, a training-free safety-social energy guidance mechanism refines sampled robot trajectories using differentiable collision, kinematic, task-progress, and role-conditioned social-compliance terms. Extensive experiments on real-world (ETH/UCY and SDD) and synthetic datasets verify the effectiveness of SAGE in improving safety and social compliance while maintaining task performance. The proposed guidance mechanism consistently reduces collision and social-violation rates, scales to teams of up to 20 robots, and enables explicit control of the safety-accuracy-task trade-off without retraining. These findings demonstrate the potential of SAGE as a scalable framework for socially-aware multi-agent navigation in complex environments.
Tags
Links
- Source: https://arxiv.org/abs/2607.16619v2
- Canonical: https://arxiv.org/abs/2607.16619v2
Trouble viewing inline? Open PDF directly →
Full Text
120,483 characters extracted from source content.
Expand or collapse full text
/ SAGE: A Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation Lan Hu, Minghui Liwang, Senior Member, IEEE, Wenbo Zhu, Xinlei Yi, Senior Member, IEEE, Yiguang Hong, Fellow, IEEE, Xianbin Wang, Fellow, IEEE, Zhenzhen Jiao, Seyyedali Hosseinalipour L. Hu (2453781@tongji.edu.cn) is with the Guohao School, Tongji University, Shanghai, China. M. Liwang (minghuiliwang@tongji.edu.cn), W. Zhu (wbzhu@tongji.edu.cn), X. Yi (xinleiyi@tongji.edu.cn) and Y. Hong (yghong@tongji.edu.cn) are with the Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Autonomous Intelligent Unmanned Systems, Department of Control Science and Engineering, Tongji University, Shanghai, China. X. Wang (xianbin.wang@uwo.ca) is with the Department of Electrical and Computer Engineering, Western University, Ontario, Canada. Z. Jiao (jiaozhenzhen@gmail.com) is with Aeromind Technology Co., Ltd., Shenzhen, China. S. Hosseinalipour (alipour@buffalo.edu) is with the Department of Electrical Engineering, University at Buffalo-SUNY, USA. Corresponding author: M. Liwang. Abstract Safe and socially compliant navigation in open human-robot environments requires robots to predict the motions of heterogeneous agents, such as pedestrians, cyclists, and other robots, while planning trajectories that satisfy physical safety requirements and social interaction norms. Nevertheless, existing navigation methods typically assume homogeneous interactions across agents/entities or enforce only geometric collision constraints, making them inadequate for modeling the interaction patterns among heterogeneous agents and social-compliance constraints (i.e., maintaining appropriate interpersonal distances). Motivated by this, we propose SAGE, a socially-aware generative engine for heterogeneous multi-agent navigation. SAGE represents robots and surrounding agents as a directed heterogeneous graph and employs a heterogeneous graph transformer (HGT) to learn interaction representations that distinguish different agent types and interaction patterns. Conditioned on this representation, we develop a diffusion-based generative model that jointly generates surrounding-agent trajectory predictions and robot trajectory plans within a unified probabilistic framework. During inference, we introduce a training-free safety-social energy guidance mechanism that injects differentiable collision-avoidance, kinematic-feasibility, task-progress, and role-dependent social-compliance objectives into the diffusion sampling process, thereby refining the generated robot trajectories without retraining. Through experiments on real-world (ETH/UCY and SDD) and synthetic datasets, we demonstrate that SAGE consistently improves safety and social compliance while maintaining task performance; further, its training-free guidance mechanism reduces collision and social-violation rates, scales to teams of up to 20 robots, and enables control of the trade-off among safety, task performance, and trajectory accuracy. Index Terms: Socially-aware navigation, heterogeneous multi-agent systems, diffusion models, trajectory generation. I Introduction Driven by recent advances in large-scale robotic systems [19], autonomous agents are rapidly transitioning from structured and controlled environments to unstructured, open environments characterized by dense human-robot coexistence and interaction. In such mixed-traffic scenarios, achieving safe, efficient, and socially compliant autonomous navigation remains a major challenge [21, 7]. This is because unlike structured environments, these scenarios feature heterogeneous agents with diverse behavioral patterns and interactions. Specifically, these agents can be classified into four categories based on their levels of autonomy and control authority: (i) Purely human agents (PHAs, e.g., pedestrians and runners), who are driven entirely by human will, exhibit highly random behavior and adhere to a wide range of social interaction norms [14]; (i) Human-controlled non-autonomous agents (HNAAs, e.g., skateboarders and cyclists), who rely on manual power or simple mechanical propulsion without autonomous sensing, making their trajectories highly dependent on users’ real-time reflexes; (i) Shared-control semi-autonomous agents (SSAAs, e.g., human-driven vehicles with advanced driver-assistance systems), where human decision-making operates in tandem with intelligent assistance, retaining inherent behavioral unpredictability despite partial rule compliance [35]; and (iv) Execution-autonomous agents (EAAs, e.g., automated robots), which are data-driven, with their behavior being relatively predictable under limited compatibility with human social norms [25]. Among these four categories, PHAs, HNAAs, and SSAAs are independently behaving entities whose future motions cannot be directly controlled, requiring robots to continuously anticipate their future behaviors and interactions [12]. In contrast, EAAs correspond to the controllable robots whose trajectories must be planned according to these predicted behaviors while simultaneously satisfying physical safety and social interaction requirements. This tight coupling between environment prediction and robot trajectory planning is a defining characteristic of socially-aware navigation in heterogeneous environments. However, jointly solving these two tasks (i.e., predicting the future behaviors of surrounding heterogeneous agents and planning safe and socially compliant trajectories for autonomous robots) remains challenging because robots must reason about human-driven and data-driven agents that exhibit diverse motion dynamics, autonomy levels, and social interaction behaviors. Addressing these diverse interactions requires navigation frameworks capable of reasoning about both heterogeneous agent behaviors and their influence on robot decision-making. Nevertheless, existing trajectory prediction and planning methods typically rely on homogeneous interaction patterns [2, 27] and/or solely enforce geometric collision avoidance [39, 48], making them incapable of capturing the heterogeneous agent interactions and the associated social behaviors. Consequently, achieving safe and socially compliant navigation in heterogeneous multi-agent environments remains an open research area [30, 25, 23, 45]. I-A Core Motivation To address the aforementioned challenges, we investigate the following key research questions (RQs): ∙ RQ 1: How can we develop principled models that capture structural and behavioral asymmetries in heterogeneous multi-agent interactions, where agents differ in dynamics, perception, and influence over the shared environment? Most mainstream trajectory generation models are developed under the homogeneous-agent assumption, employing shared parameters to uniformly model diverse agent types [2, 27]. However, in real-world environments, interactions vary significantly across agent categories. For example, a robot may adopt proactive yielding when encountering pedestrians, yet switch to predictive trajectory tracking when navigating around vehicles. Ignoring these role-dependent interactions limits the ability of existing models to accurately predict agent trajectories and plan robot trajectories. Since these interaction asymmetries naturally arise from the semantic roles of different agents and their relationships, they can be represented as a heterogeneous interaction graph, where nodes correspond to different agent categories and edges encode their interaction relationships [20]. Based on this representation, we develop a heterogeneous graph transformer (HGT), inspired by [16], that learns distinct interaction patterns across different agent categories. By explicitly distinguishing semantic types of nodes and edges, HGT learns independent attention projection matrices for each interaction relationship (e.g., “robot→ →pedestrian”, “pedestrian→ →robot”, and “robot→ →robot”), thereby capturing asymmetric interactions and producing socially-aware scene representations. ∙ RQ 2: How can physical feasibility constraints and culturally induced behavioral norms be jointly modeled within a unified framework that reconciles hard kinematic limitations with context-dependent social regularities? Most existing generative models primarily enforce hard physical constraints, such as collision avoidance and kinematic feasibility, while largely overlooking soft social constraints. Nevertheless, in human-robot coexistence scenarios, navigation should ensure physical safety, while complying with social norms and human comfort expectations [13]. For instance, even if a trajectory is collision-free, it may still be socially unacceptable when it frequently intrudes into pedestrians’ personal space or disrupts ongoing social interactions [13]. To address this, we employ a conditional diffusion model [15, 11] as the generative backbone and propose a safety-social energy guidance mechanism based on a differentiable energy function [37] during the inference stage. Specifically, social norms, such as maintaining comfortable interpersonal distances and exhibiting polite avoidance behaviors, are formulated as heterogeneous anisotropic potential fields, whose gradients are incorporated into each diffusion denoising step to iteratively steer the generated trajectories toward lower-energy solutions. Unlike methods based on conditional variational autoencoders (CVAEs) and generative adversarial networks (GANs) [34, 12], where constraints are implicitly learned during training and are difficult to modify afterward, diffusion models enable differentiable guidance to be injected directly into the sampling process. Consequently, new physical or social constraints can be incorporated or removed at inference time without retraining the generative model. Through this safety-social energy guidance, generated trajectories are encouraged toward kinematic feasibility [5] and social compliance in a training-free manner. I-B Novelty and Contribution To address the aforementioned research questions, we develop a socially-aware generative engine, called SAGE, that integrates heterogeneous interaction modeling, diffusion-based trajectory generation, and safety-social guidance. The key contributions of SAGE are summarized as follows: ∙ A novel problem formulation for safe and socially compliant navigation in heterogeneous multi-agent environments. We formulate a new navigation problem in which autonomous robots operate alongside heterogeneous entities (e.g., pedestrians, cyclists, and semi-autonomous vehicles) exhibiting distinct dynamics, autonomy levels, and social behaviors. Unlike existing formulations, our problem explicitly captures the tight coupling among heterogeneous agent interactions, surrounding agent trajectory prediction, robot trajectory planning, and the joint consideration of hard physical safety constraints and soft social interaction norms. This unified formulation provides the foundation for developing socially-aware navigation algorithms in heterogeneous environments. ∙ A unified generative framework for heterogeneous interaction modeling, trajectory prediction, and robot trajectory planning. We develop SAGE, a socially-aware generative framework that unifies heterogeneous interaction modeling, surrounding agent trajectory prediction, and robot trajectory planning within a single probabilistic framework. Specifically, SAGE represents heterogeneous agents and their interactions using a directed heterogeneous graph and develops an HGT to explicitly learn role-dependent and asymmetric interaction patterns across different agent categories. Building upon the resulting socially-aware scene representations, SAGE further introduces a diffusion-based generative model that simultaneously predicts the future trajectories of surrounding agents and generates robot trajectory plans, thereby explicitly capturing the coupling between environment prediction and robot motion planning. ∙ A training-free safety-social guidance framework for controllable robot trajectory generation. We integrate a novel safety-social energy guidance mechanism in SAGE that enables explicit constraint-aware correction during diffusion-based trajectory generation. Specifically, we formulate collision avoidance, kinematic feasibility, task progress, and role-dependent social norms as differentiable energy functions, where heterogeneous anisotropic potential fields capture category-specific social behaviors. By injecting the resulting energy gradients into each diffusion denoising step, the proposed guidance iteratively reduces physical-safety and social-compliance violations at inference time, enabling controllable correction without modifying or retraining the underlying generative model. ∙ Empirical validation. We validate the effectiveness of SAGE through experiments on real-world (ETH/UCY and SDD) and synthetic datasets, demonstrating its notable performance on reducing personal space intrusion and collision rates while maintaining trajectory diversity and kinematic feasibility. I Literature Review In the following, we review the related literature while highlighting differences between prior studies and this work. I-A Trajectory Prediction for Socially-Aware Navigation Safe robot navigation in mixed human-robot environments relies on accurate prediction of surrounding entities’ future trajectories for collision avoidance and motion planning. Early approaches predominantly used recurrent neural networks (RNNs), with Social-LSTM [2] introducing social pooling (i.e., aggregating hidden states across neighboring agents) to capture inter-agent interactions and Social-GAN [12] incorporating adversarial learning to model multimodal human motion. Subsequent works advanced these early studies by adopting attention and graph-based architectures to capture richer interactions. For instance, SoPhie [33] and Social-BiGAT [20] combined physical scene context with social attention, while Social-STGCNN [27] modeled interactions through spatio-temporal graphs. Trajectron++ [34] introduced dynamic graphs to accommodate varying agent populations, and AgentFormer [45] employed self-attention to jointly capture agent-agent and agent-environment dependencies. Despite these advances, existing methods generally assume similar interaction mechanisms across agent categories or use lightweight semantic embeddings that cannot fully capture distinct behavioral and social interaction patterns. Moreover, they focus on trajectory prediction rather than jointly predicting surrounding-agent trajectories and planning robot motion, limiting their applicability to socially-aware navigation in heterogeneous environments. These limitations motivate heterogeneous graph learning, which explicitly distinguishes agent categories and their interaction relationships. For example, heterogeneous graph neural networks (GNNs) [46] and HGT [16] learn type-specific representations for different nodes and edges, yet their application to socially-aware robot navigation remains largely unexplored. While recent works have considered task-motion planning and heterogeneous multi-robot coordination in dynamic environments [26, 9], heterogeneous graph learning has rarely been used to jointly reason about surrounding-agent trajectory prediction and robot trajectory planning. To bridge this gap, we develop an HGT-based framework that learns semantic, social, and kinematic representations of heterogeneous agents, providing a socially-aware scene representation that supports both surrounding-agent trajectory prediction and downstream robot trajectory planning. I-B Diffusion-Based Trajectory Generation for Robot Navigation Diffusion probabilistic models [36, 15] have emerged as a powerful generative paradigm for flexible and controllable trajectory synthesis through iterative denoising. In robotics, Diffuser [17] and Decision Diffuser [1] formulated trajectory planning as conditional diffusion for long-horizon decision making. For trajectory generation, MID [11] modeled human-motion uncertainty, while MotionDiffuser [18] extended diffusion to controllable multi-agent prediction. Subsequent works incorporated scene constraints for collision-free generation [44], controllable traffic simulation [47], and interactive robot navigation [28]. Recently, joint prediction–planning diffusion (JPPD) [43] jointly sampled single-robot and surrounding-participant trajectories with an occupancy-based differentiable safety potential during conditional flow-matching inference. However, it does not explicitly consider multiple controllable robots, robot-entity and robot-robot interactions, or role-conditioned social distances. Collectively, these studies establish diffusion models as a flexible framework for controllable trajectory generation, which remains an active topic in cybernetic systems [24]. Advancing this literature, we develop a differentiable safety-social energy guidance framework that injects collision avoidance, kinematic feasibility, task progress, and role-dependent social norms directly into diffusion sampling. By formulating these objectives as differentiable energy functions, their gradients iteratively guide denoising toward physically feasible and socially compliant robot trajectories; moreover, additional physical or social objectives can be incorporated as new energy terms at inference time without retraining the underlying diffusion model. I-C Social Norm Modeling and Human-Aware Robot Navigation Socially-aware navigation enables robots to navigate while respecting implicit human social norms, such as maintaining appropriate interpersonal distances [22, 25]. Early approaches relied on hand-crafted models: the Social Force Model [14] modeled interactions through attractive and repulsive forces, while ORCA [39] used reciprocal velocity optimization for efficient collision avoidance, often resulting in conservative and mechanically rigid behaviors. More recent data-driven approaches include deep reinforcement learning (DRL)-based methods such as SARL [6] and interactive model predictive control (MPC) for dense-crowd navigation [7]. Recent studies have further incorporated richer social cues, including group behavior and proxemics [13], while Transformer-based methods have advanced promptable human trajectory prediction [32]. Concurrently, evaluation frameworks increasingly assess socially-aware navigation beyond trajectory accuracy by considering safety, comfort, and social compliance [10, 4, 31]. Despite these advances, the above methods primarily learn socially plausible behaviors through policy optimization or trajectory prediction, leaving social norms implicitly encoded in model parameters and difficult to interpret, modify, or adapt. Moreover, most approaches focus on surrounding-agent trajectory prediction with limited integration of robot trajectory planning within a unified generative framework. To address these limitations, we formulate physical safety and social interaction principles, including interpersonal comfort distances and category-dependent social margins, as differentiable energy functions that directly guide diffusion-based trajectory generation. By injecting their gradients during diffusion sampling, SAGE promotes physical feasibility and social compliance in a training-free and controllable manner, avoiding opaque black-box supervision and rigid rule-based engineering. Beyond human-robot interactions, multi-robot coordination has been studied through switching-topology coordination [41], distributed collision and deadlock avoidance [48], swarm formation [42], task allocation [38], and learning-based flocking [3]. However, these studies typically treat robot-robot and human-robot coordination separately. In contrast, SAGE models both robot-robot and robot-entity interactions within a unified heterogeneous graph and regulates them through a safety-social energy guidance mechanism, enabling socially-aware navigation in heterogeneous multi-agent systems. I Core Models and Problem Formulation In heterogeneous urban environments characterized by evolving human-machine interactions, robot navigation necessitates the tight coupling of heterogeneous-entity trajectory prediction and socially-aware robot trajectory planning. We formalize this problem under dynamic multi-agent interactions, where each robot, i.e., EAA, is assigned a navigation task such as moving from its current/start state toward an assigned waypoint while satisfying both physical feasibility and social compliance. Meanwhile, surrounding heterogeneous entities, including PHAs, HNAAs, and SSAAs, exhibit evolving and uncontrollable behaviors, requiring the robot to continuously anticipate their future motions and interaction patterns for reliable navigation111In this work, we use navigation to denote the overall task, prediction to describe the future trajectories of entities, trajectory planning to describe the trajectory-planning problem for the main robots, and trajectory generation to describe the diffusion-based solution process.. For clarity, we use agents as a general term for all participants, robots to denote controllable agents, entities to describe heterogeneous non-robot participants, and neighbors to represent entities within the perception neighborhood of a specific robot. As illustrated in Fig. 1, SAGE, which is a multi-agent social perception and navigation framework, consists of the following three synergistic phases: Figure 1: Overview of the three phases of SAGE. Phase 1 encodes scene histories, agent attributes, and robot waypoints into context tokens using a directed heterogeneous graph and HGT. At each reverse diffusion step, Phase 2 produces a clean estimate of the joint robot–entity trajectories, and Phase 3 applies bounded robot-only energy guidance. The corrected estimate is then re-noised for the next reverse step, and this cycle repeats until yielding the final guided robot plans and entity trajectory predictions. ∙ Phase 1: Heterogeneous scene encoding and interaction modeling. Unlike traditional trajectory planning methods that treat agents homogeneously, this phase integrates neighbors’ historical trajectories with semantic and kinematic attributes (such as pedestrian vulnerability and vehicle motion characteristics). Through a directed heterogeneous interaction graph, the framework captures asymmetric interaction dynamics and implicit social interactions, enabling socially-aware navigation behaviors, including pedestrian yielding and predictive vehicle interaction. The resulting scene representation, which utilizes HGT, provides structured social semantics for downstream robot trajectory generation. ∙ Phase 2: Joint trajectory generation via conditional diffusion. Given the inherent uncertainty in future entity behaviors, this phase does not produce a single deterministic trajectory. Instead, we leverage a conditional diffusion model to model a multimodal joint trajectory distribution over both the predicted future behaviors of heterogeneous entities and the task-directed trajectory plans of robots. During inference, the reverse diffusion process iteratively denoises a joint robot–entity trajectory sample, producing at each denoising step a clean estimate of the joint trajectories for subsequent safety-social guidance. ∙ Phase 3: Safety-social guided reverse diffusion. At each reverse diffusion step, the clean joint trajectory estimate obtained in Phase 2 may violate safety, social, kinematic, or task-progress requirements. Phase 3 encodes these requirements as differentiable energies and applies a bounded correction to the robot trajectory components while leaving the predicted entity trajectories unchanged. The corrected joint trajectory estimate is then re-noised to the next diffusion level and returned to Phase 2 for further denoising. This iterative denoising-guidance-re-noising cycle continues until the final diffusion step, yielding safety- and social-aware robot trajectory plans together with the corresponding entity trajectory predictions. In the following, we define agents and trajectories (Secs. I-A–I-B), introduce perception neighborhoods and kinematic quantities (Sec. I-C), and formulate the joint prediction–planning objective and safety-social constraints (Sec. I-D). I-A Agent Modeling in Heterogeneous Environments We consider two key agent types: (i) environmental entities such as heterogeneous PHAs, HNAAs, and SSAAs, which are dynamic and uncontrollable, gathered by the set =o1,…,oi,…,o||O=\o_1,…,o_i,…,o_|O|\; and (i) robots that fall in the category of EAAs, collected by the set ℛ=r1,…,rj,…,r|ℛ|R=\r_1,…,r_j,…,r_|R|\. For each entity oio_i and robot rjr_j, we collect their semantic and geometric attributes in the following tuples: i()=(i(,),i(,)),j()=(j(,),j(,)), -5.69054pt a_i^( o)= (C_i^( o,role),C_i^( o,geom) ),~ a_j^( r)= (C_j^( r,role),C_j^( r,geom) ), -5.69054pt (1) where i(,)∈()∖C_i^( o,role) ^( role) \ robot\ represents the semantic category (e.g., ()=,,,…,C^( role)=\ pedestrian,cyclist,vehicle,…,robot\), describing its behavioral patterns and social roles; and i(,),j(,)∈ℝ2C_i^( o,geom),C_j^( r,geom) ^2 represents the geometric properties (i.e., length and width) that define physical dimensions and collision boundaries. All robots share fixed geometric dimensions j(,)=(,)C_j^( r,geom)=C^( r,geom), ∀j∀ j, same role j(,)=(,)=C_j^( r,role)=C^( r,role)= robot, ∀j∀ j, and kinematic constraints, while each robot rjr_j is assigned a waypoint j=(xj(),yj())g_j=(x_j^( g),y_j^( g)) as its navigation objective within the current prediction horizon T()T^( prd). I-B Trajectory Modeling At each planning instant, SAGE processes a single observation window of length T()T^( obs) and generates joint trajectories over a prediction horizon of length T()T^( prd). Using a relative time index, t=0t=0 denotes the current planning time, the observed time steps are t∈−T()+1,…,0t∈\-T^( obs)+1,…,0\, and the future time steps are t∈1,…,T()t∈\1,…,T^( prd)\. The same procedure can be repeatedly invoked as new observations become available, while each planning instance is formulated independently. For each entity oi∈o_i , we use the observed planar position i(,t)=(xi(,t),yi(,t))p_i^( o,t)=(x_i^( o,t),y_i^( o,t)) at time t to construct the position history i()=i(,t)∣t∈−T()+1,…,0 s_i^( past)=\p_i^( o,t) t∈\-T^( obs)+1,…,0\\. Velocity components can be obtained by finite differences when required. For each robot rj∈ℛr_j , its planar position and its Cartesian state at time t are j(,t)=(xj(,t),yj(,t))p_j^( r,t)=(x_j^( r,t),y_j^( r,t)) and j(,t)=(xj(,t),yj(,t),vj(,t),vj(,t)) s_j^( r,t)=(x_j^( r,t),y_j^( r,t),v_j^( rx,t),v_j^( ry,t)), respectively, while its current Cartesian state is given by j(,0)=(xj(,0),yj(,0),vj(,0),vj(,0)) s_j^( r,0)=(x_j^( r,0),y_j^( r,0),v_j^( rx,0),v_j^( ry,0)), where xj(,0)x_j^( r,0) and yj(,0)y_j^( r,0) denote its current planar position coordinates, and vj(,0)v_j^( rx,0) and vj(,0)v_j^( ry,0) denote the velocity components along the x- and y-axes, estimated from the last two observed positions. Each robot rjr_j aims to generate a trajectory from this current state toward its assigned waypoint jg_j over T()T^( prd), subject to safety and social-interaction requirements. I-C Perception Neighborhoods and Kinematic Quantities To facilitate constraint modeling, we next formalize the robot’s local perception neighborhood (Sec. I-C1) and relevant kinematic quantities (Sec. I-C2). I-C1 Heterogeneity-aware neighborhood In populated environments, exhaustively modeling all entities in O incurs substantial computational overhead and may introduce redundant or irrelevant interaction dependencies that degrade representation quality. Thus, for each robot rj∈ℛr_j at time t, we define its effective perception neighborhood as the set of nearby entities within a pair-dependent interaction radius Rj,i()R_j,i^( val) as follows: jt=oi|oi∈,‖j(,t)−i(,t)‖2≤Rj,i(),N_j^t= \o_i\; |\;o_i ,\; \|p_j^( r,t)-p_i^( o,t) \|_2≤ R_j,i^( val) \, (2) where Rj,i()R_j,i^( val) is constrained by two factors: (i) the sensing range of robot rjr_j, which is bounded by its maximum observable range, and (i) the category-dependent social influence range of neighbor oio_i, which depends on its semantic role, motion state, and geometric properties. For instance, for pedestrians, we use Hall’s proxemics theory [13] as a reference (e.g., the boundary between the social and public zones is approximately 3.6m3.6\,m). For other entity categories, such as cyclists or vehicles, the interaction radius can be adjusted according to their speed, size, and safety-relevant motion characteristics222Note that, jtN_j^t contains only heterogeneous non-robot neighbors observed by robot rjr_j at time t, while nearby robots are later modeled separately through robot-to-robot (R2R) interactions.. I-C2 Kinematic quantities To characterize the nonholonomic constraints of a robot, we define three derived quantities from its state vector j(,t) s_j^( r,t) in the following. Let jt=(vj(,t),vj(,t))v_j^t=(v_j^( rx,t),v_j^( ry,t)) denote the planar translational velocity of robot rjr_j at time t. First, the robot’s heading angle at time t is given by: θjt=atan2(vj(,t),vj(,t)+ε), _j^t=atan2\! (v_j^( ry,t),v_j^( rx,t)+ ), (3) where 0<ε≪10< 1 is a numerical stabilizer to avoid ambiguity in the case of vj(,t)=vj(,t)=0v_j^( ry,t)=v_j^( rx,t)=0. Subsequently, the lateral velocity (i.e., the component of the current velocity perpendicular to the preceding heading) is given by: vj(,t)=−vj(,t)sin(θjt−1)+vj(,t)cos(θjt−1).v_j^( lat,t)=-v_j^( rx,t) ( _j^t-1)+v_j^( ry,t) ( _j^t-1). (4) Finally, the angular velocity is computed as follows: ωjt=0,t=1,atan2(sinΔθjt,cosΔθjt)Δt,t≥2, _j^t= cases0,&t=1,\\ atan2( _j^t, _j^t) t,&t≥ 2, cases (5) where Δθjt=θjt−θjt−1 _j^t= _j^t- _j^t-1, and Δt t is the sampling interval (set to 0.4s0.4\,s in our experiments). For t=1t=1, θj0 _j^0 estimated from the last two observations is used as the reference for vj(,1)v_j^( lat,1). I-D Optimization Objective and Safety-Social Constraints Next, we formulate the navigation problem of our interest with two coupled components. First, we define a joint generative objective that maps current robot states, assigned waypoints, and historical observations of neighbor entities to a distribution over predicted entity trajectories and task-directed robot trajectory plans (Sec. I-D1). Second, we impose task and safety-social feasibility constraints on samples from this distribution, including task progress, collision avoidance, kinematic feasibility, and social compliance (Sec. I-D2). I-D1 Joint generative objective Given the robot states at the current planning time (i.e., j(,0)rj∈ℛ\ s_j^( r,0)\_r_j ), their waypoints (i.e., jrj∈ℛ\g_j\_r_j ), and the historical observations of the currently perceived heterogeneous entities (i.e., i()oi∈0\ s_i^( past)\_o_i _N^0), we define the scene-level conditioning input as follows: S():=⋃rj∈ℛj(,0),⋃rj∈ℛj,⋃oi∈0i().S^( in):= \ _r_j s_j^( r,0),\; _r_j g_j,\; _o_i _N^0 s_i^( past) \. (6) where 0=⋃rj∈ℛj0O_N^0= _r_j N_j^0 denotes the set of neighbor entities currently perceived by the robot team. Given S()S^( in), our objective is to jointly generate (i) the future trajectories of the perceived heterogeneous entities and (i) task-directed trajectories for the robots. Accordingly, we define the system output as the joint future velocity set S()=S(,),S(,),S^( out)= \S^( r,out),S^( o,out) \, where S(,)S^( r,out) contains the planned robot velocities and S(,)S^( o,out) contains the predicted entity velocities over the prediction horizon T()T^( prd). Since S()S^( out) consists of velocity sequences, the future robot and entity positions can be obtained by integrating their velocities as j(,t)=j(,0)+Δt∑τ=1tjτ,i(,t)=i(,0)+Δt∑τ=1tiτ.p_j^( r,t)=p_j^( r,0)+ t _τ=1^tv_j^τ,~~p_i^( o,t)=p_i^( o,0)+ t _τ=1^tv_i^τ. Since entity trajectory prediction and robot trajectory planning are inherently coupled, we model their joint output through the conditional distribution pθ(S()∣S()),p_θ\! (S^( out) S^( in) ), where θ denotes learnable parameters. This distribution captures the multimodal future behaviors of heterogeneous entities together with the task-directed robot trajectory plans. In Sec. IV-B, we realize this conditional distribution using a diffusion-based generative model, whose parameters (i.e., θ) are learned by minimizing the diffusion reconstruction objective ℒL_ diff. I-D2 Safety-social constrained trajectory planning Although samples from the learned conditional joint trajectory distribution pθ(S()∣S())p_θ(S^( out) S^( in)) are statistically plausible, they may fail to make task progress or violate safety, kinematic, or social requirements, especially in dense/long-tail interaction scenarios. Hence, we seek robot trajectories that are both likely under this distribution and feasible with respect to task, physical, and social constraints, leading to our problem formulation P: (): ( P): S()∼pθ(⋅∣S())s.t.(C1)–(C5). S^( out) p_θ (· S^( in) ) s.t. (C1)--(C5). (7) Specifically, constraints (C1)-(C5) are detailed below. • (C1) Robot-to-neighbor collision avoidance. Each robot should maintain its pairwise safety margin from neighbors: ‖j(,t)−i(,t)‖2 \|p_j^( r,t)-p_i^( o,t) \|_2 ≥mj,i, ≥ m_j,i, (8) ∀t∈1,…,T(), ∀ t∈\1,…,T^( prd)\, ∀rj∈ℛ,∀oi∈jt, ∀ r_j ,\ ∀ o_i _j^t, where mj,im_j,i denotes the safety margin between robot rjr_j and entity neighbor oio_i. • (C2) Robot-to-robot (R2R) collision avoidance. Any pair of robots should preserve their pairwise safety margin: ‖j(,t)−j′(,t)‖2 \|p_j^( r,t)-p_j ^( r,t) \|_2 ≥mj,j′, ≥ m_j,j , (9) ∀t∈1,…,T(), ∀ t∈\1,…,T^( prd)\, ∀rj,rj′∈ℛ,j′≠j, ∀ r_j,r_j ,\ j ≠ j, where mj,j′m_j,j is the safety margin between robots rjr_j and rj′r_j . In the above constraints (C1) and (C2), the safety margins mj,im_j,i and mj,j′m_j,j account for both a prescribed minimum clearance d()d^( safe) and the physical dimensions of the interacting agents. Specifically, approximating each robot and entity by its circumscribed circle, the safety margins are given by mj,i=d()+12‖j(,)‖2+12‖i(,)‖2,m_j,i=d^( safe)+ 12\|C_j^( r,geom)\|_2+ 12\|C_i^( o,geom)\|_2, and mj,j′=d()+12‖j(,)‖2+12‖j′(,)‖2m_j,j =d^( safe)+ 12\|C_j^( r,geom)\|_2+ 12\|C_j ^( r,geom)\|_2. • (C3) Kinematic feasibility. The generated robot trajectories should satisfy the prescribed kinematic limits: ‖jt‖2≤vmax,|vj(,t)|≤vmax,|ωjt|≤ωmax,∀t∈1,…,T(),∀rj∈ℛ. gathered\|v_j^t\|_2≤ v_ , |v_j^( lat,t)|≤ v_ lat, | _j^t|≤ _ ,\\[-2.84526pt] ∀ t∈\1,…,T^( prd)\,\ ∀ r_j . gathered (10) • (C4) Social compliance. Beyond physical collision avoidance, robot trajectories should remain outside the category-conditioned social compliance region induced by neighbors in the robot’s forward half-plane: (di,j(,t)σ(i(,)))2+(di,j(,t)σ(i(,)))2≥1, ( d_i,j^( lon,t)σ lon\! (C_i^( o,role) ) )^2+ ( d_i,j^( lat,t)σ lat\! (C_i^( o,role) ) )^2≥ 1, (11) ∀t∈1,…,T(),∀rj∈ℛ,∀oi∈jt,di,j(,t)≥0. ∀ t∈\1,…,T^( prd)\,\ ∀ r_j ,\ ∀ o_i _j^t,d_i,j^( lon,t)≥ 0. In (11), di,j(,t)d_i,j^( lon,t) and di,j(,t)d_i,j^( lat,t) denote the longitudinal and lateral components, respectively, of the position of neighbor oio_i with respect to robot rjr_j, expressed in the robot-centric coordinate: [di,j(,t)di,j(,t)]=[cos(θjt)sin(θjt)−sin(θjt)cos(θjt)][xi(,t)−xj(,t)yi(,t)−yj(,t)]. bmatrixd_i,j^( lon,t)\\ d_i,j^( lat,t) bmatrix= bmatrix ( _j^t)& ( _j^t)\\ - ( _j^t)& ( _j^t) bmatrix bmatrixx_i^( o,t)-x_j^( r,t)\\ y_i^( o,t)-y_j^( r,t) bmatrix. (12) Also, σ(⋅)σ lon(·) and σ(⋅)σ lat(·) define the longitudinal and lateral extents of the elliptical comfort region, respectively, based on the neighbor’s semantic role (e.g., i(,)∈,,C_i^( o,role)∈ \pedestrian,cyclist,vehicle\). In words, for each forward neighbor (imposed by di,j(,t)≥0d_i,j^( lon,t)≥ 0), (11) requires the robot to remain on or outside this ellipse. The right-hand side of (11) is set to one because the longitudinal and lateral separations are normalized by their corresponding role-dependent comfort-region extents. Thus, values below one indicate intrusion into the comfort region, whereas values greater than or equal to one satisfy the social-compliance constraint. These role-dependent margins provide an interpretable social prior, with larger margins for pedestrians reflecting interpersonal comfort and smaller margins for vehicles capturing safety considerations. • (C5) Task progress. The generated robot trajectory should reach the assigned waypoint jg_j or terminate sufficiently close to it within the planning horizon, which results in ‖j(,T())−j‖2≤d(),∀rj∈ℛ, \|p_j^( r,T^( prd))-g_j \|_2≤ d^( goal), ∀ r_j , (13) where d()d^( goal) is the tolerance of vicinity to the waypoint. The formulation P in (7) specifies both the generative target and the task-feasibility requirements of socially-aware robot navigation. Nevertheless, directly solving P is fundamentally challenging due to the intricate coupling among multimodal entity trajectory prediction and robot trajectory planning, heterogeneous interaction patterns of agents, collision avoidance, kinematic feasibility, and socially compliant behavior modeling. These are inherently interdependent and often exhibit competing constraints across spatial, temporal, and social dimensions. To address this complexity, we next develop SAGE. IV SAGE: A Socially-Aware Generative Engine Building on Sec. I-D, this section presents how SAGE parameterizes and subsequently samples from the task-aware generative prior pθ(S()∣S())p_θ(S^( out) S^( in)). During training, SAGE learns this prior to capture the multimodal joint future distribution of controllable robots and heterogeneous entities within their perception neighborhoods. During inference, SAGE samples from this prior through an iterative reverse diffusion process, during which safety-social energy guidance is applied to promote task progress, collision avoidance, kinematic feasibility, and social compliance while remaining likely under the learned distribution. Specifically, SAGE consists of three synergistic phases. First, an HGT encodes asymmetric robot-entity interactions into a contextual representation C()C^( ctx) that captures social semantics and interaction patterns (Phase 1). Second, conditioned on C()C^( ctx), a diffusion model iteratively denoises a joint robot–entity trajectory sample, enabling coherent reasoning over coupled multi-agent dynamics (Phase 2). Third, at each reverse diffusion step, differentiable safety-social energy guidance corrects the denoised robot trajectories before proceeding to the next diffusion step, reducing physical-safety and social-compliance energy penalties (Phase 3). The overall procedure of SAGE is summarized in Alg. 1 and its three phases are elaborated in the following. IV-A Phase 1: Scene Encoding and Interaction Modeling This phase transforms/encodes the heterogeneous scene described in Sec. I-A into a context-token sequence C()C^( ctx) that conditions the diffusion model on agent-specific attributes and interaction patterns. To capture the role-dependent and asymmetric nature of agent interactions, we represent the observed scene as a directed heterogeneous graph: =(,ℰ),G=(V,E), (14) where =ℛ∪0V=R _N^0 comprises the robots and currently perceived entities defined in Sec. I-D. The edge set ℰE captures directed robot-to-neighbor (R2N), neighbor-to-robot (N2R), and robot-to-robot (R2R) interactions, with R2R edges defined over all ordered robot pairs, including self-loops; the direction of each edge allows opposite interaction directions (e.g., R2N vs. N2R) to be modeled explicitly. We next construct the context-token sequence C()C^( ctx) in three steps, where a tailored HGT performs relation-specific information exchange over G to produce interaction-aware node embeddings. Input: Scene-level conditioning input S()S^( in), robot and entity attributes, diffusion steps K, sampling stride s Output: Robot trajectory plans S(,)S^( r,out) and entity trajectory predictions S(,)S^( o,out) 1 Construct the heterogeneous graph G with R2N, N2R, and R2R edges following (14); 2 Encode node features and obtain context tokens C()C^( ctx) following (15)–(23); 3 Initialize SK()∼(0,I)S_K^( out) (0,I) as the large-K limit of (25); 4 for k=K,K−s,…k=K,K-s,… while k>0k>0 do 5 Predict the noise ϵθ(Sk(),βk,C()) _θ(S_k^( out), _k,C^( ctx)) using the denoiser trained via (26); 6 Estimate the clean velocity sequence S^k() S_k^( out) following (27); 7 Compute the four energy terms following (28)–(32); 8 Apply the bounded robot-only correction following (33); 9 Set k′=max(k−s,0)k = (k-s,0) and sample Sk′()S_k ^( out) following (33); 10 return S(,)S^( r,out) as robot trajectory plans and S(,)S^( o,out) as entity trajectory predictions Algorithm 1 Workflow of SAGE ∙ Step 1: Node feature embedding. Before performing information exchange over G, we construct a layer-0 embedding for each entity and robot node. Since entity motion is inferred from historical observations, whereas robot planning depends on the current state and assigned waypoint, we use a long short-term memory (LSTM) network to encode entity trajectories and a multilayer perceptron (MLP) to encode robot states. These motion representations are then combined with semantic role, control status, geometry, and waypoint information to obtain node embeddings in a common latent space. Specifically, we concatenate (⊕ ) these features and pass them through a fusion projection Φ _ fuse, which consists of layer normalization followed by an MLP. Accordingly, for each entity oi∈0o_i _N^0, the layer-0 embedding is given by: i(,0)=Φ(LSTM(i())⊕Emb(i(,)) _i^( o,0)= _ fuse (LSTM( s_i^( past)) _ role(C_i^( o,role)) (15) ⊕Emb(0)⊕MLP(i(,))⊕MLP()), _ ctrl(0) _ geom(C_i^( o,geom)) _ goal(0) ), where the LSTM encodes the observed position history i() s_i^( past) into its final hidden state, while Emb(⋅)Emb_ role(·) denotes a learnable embedding that maps the entity’s semantic role (e.g., pedestrian, cyclist, or vehicle) to a latent representation, and MLP(⋅)MLP_ geom(·) encodes its physical dimensions. Moreover, Emb(0)Emb_ ctrl(0) identifies the entity as uncontrollable, while the zero waypoint input (i.e., MLP()MLP_ goal(0)) indicates that no task objective is assigned to entity nodes. Similarly, for each robot rj∈ℛr_j , the layer-0 embedding is given by: j(,0)=Φ(MLP(j(,0))⊕Emb(1)⊕Emb((,))⊕MLP((,))⊕MLP(j−j(,0))), aligned &h_j^( r,0)= _ fuse (MLP_ state( s_j^( r,0)) _ ctrl(1) _ role(C^( r,role))\\[-2.84526pt] & _ geom(C^( r,geom)) _ goal (g_j-p_j^( r,0) ) ), aligned (16) where MLP(⋅)MLP_ state(·) encodes the robot’s current position and velocity, Emb(1)Emb_ ctrl(1) identifies the robot as controllable, and MLP(⋅)MLP_ goal(·) encodes the displacement from its current position to the assigned waypoint. In (16), the role and geometry features are encoded analogously to those of the entity nodes. The resulting layer-0 embeddings provide a common representation of motion, semantic role, control status, physical geometry, and task information, and are subsequently processed by the relation-specific HGT in Step 2. All components of the scene encoder, including the LSTM, embeddings, feature MLPs, fusion projection, and HGT, are jointly trained with the diffusion denoiser through ℒL_ diff (later detailed in Sec. IV-B). ∙ Step 2: Meta-relation-based attention mechanism. For notational convenience, let ua,ub∈u_a,u_b denote arbitrary nodes in G defined in (14), where each node represents either a robot in ℛR or a perceived entity in 0O_N^0. We use a(0)h_a^(0) to denote the layer-0 embedding of node uau_a, obtained from Step 1. Specifically, a(0)=i(,0),ifua=oi∈0,j(,0),ifua=rj∈ℛ.h_a^(0)= casesh_i^( o,0),&if~u_a=o_i _N^0,\\ h_j^( r,0),&if~u_a=r_j . cases (17) These layer-0 embeddings serve as the initial node representations processed by the HGT. As discussed in the following, the subsequent HGT layers then produce updated representations a(l)h_a^(l), l=1,…,Ll=1,…,L, by exchanging information over the directed edges of G. To this end, to capture asymmetric, role-dependent interactions among robots and heterogeneous entities, we assign a distinct relation type to each directed edge in G based on the semantic roles of its source and target nodes.333A Graph Attention Network (GAT) [40] typically applies shared attention parameters across edges and therefore does not explicitly distinguish heterogeneous, direction-dependent interaction types. In contrast, our setting requires different parameterizations for ordered role pairs (e.g., robot-to-pedestrian versus pedestrian-to-robot), motivating the relation-specific HGT formulation. Specifically, for a directed edge ub→uau_b→ u_a, with ubu_b and uau_a denoting the source and target nodes, respectively, we define the relation ID as: ψ(ub→ua)=ι(ub)N+ι(ua),ψ(u_b\!→ u_a)= (u_b)N_ role+ (u_a), (18) where ι(u)∈0,…,N−1 (u)∈\0,…,N_ role-1\ denotes the semantic-role ID of node u, and N_ role is the total number of semantic roles. Since the source and target roles are ordered, reversing an edge generally results in a different relation ID. This allows SAGE to distinguish asymmetric interactions, such as the influence of a pedestrian on a robot from that of a robot on a pedestrian. Based on these relation types, the HGT iteratively updates each node by attending to and aggregating information from its incoming neighbors. Specifically, for a target node uau_a, let ℳa=ub∈∣(ub→ua)∈ℰM_a=\u_b (u_b→ u_a) \ (19) denote its set of incoming source nodes. Each incoming edge ub→uau_b→ u_a is associated with a relation ID ψ(a,b):=ψ(ub→ua)ψ(a,b):=ψ(u_b→ u_a). To capture different aspects of an interaction, the HGT employs multi-head attention with H parallel attention heads, indexed by h∈1,…,Hh∈\1,…,H\. Each attention head independently evaluates the relevance of incoming source nodes and extracts the information they contribute to the target node. To further allow various interaction types to be processed differently, each relation type ψ(a,b)ψ(a,b) and attention head h are associated with learnable/trainable query, key, and value projection matrices (Wψ(a,b),h,Wψ(a,b),h,Wψ(a,b),h)(W_ψ(a,b),h Q,W_ψ(a,b),h K,W_ψ(a,b),h V), along with an attention-logit bias bψ(a,b),hb_ψ(a,b),h. The query and key projections transform the target and source node representations into a common space in which their interaction relevance is measured, while the value projection transforms the source-node representation into the information to be passed to the target node. Thus, within each attention head, the relation ID ψ(a,b)ψ(a,b) selects the set of learnable parameters used to determine both how relevant source node ubu_b is to target node uau_a and what information ubu_b contributes to uau_a. Subsequently, at HGT layer l∈1,…,Ll∈\1,…,L\, the attention score between source node ubu_b and target node uau_a, and its normalized attention weight, are given by: ea,b,h(l)=(a(l−1)Wψ(a,b),h)(b(l−1)Wψ(a,b),h)⊤dh+bψ(a,b),h,αa,b,h(l)=Softmaxub∈ℳa(ea,b,h(l)), aligned e_a,b,h^(l)&= (h_a^(l-1)W_ψ(a,b),h Q)(h_b^(l-1)W_ψ(a,b),h K) d_h+b_ψ(a,b),h~ -2.27621pt,\\ _a,b,h^(l)&=Softmax_u_b _a (e_a,b,h^(l) ), aligned (20) where dhd_h denotes the per-head projection dimension. In essence, the attention weight αa,b,h(l) _a,b,h^(l) determines the relative importance of source node ubu_b to target node uau_a at layer l. Using these attention weights, each attention head aggregates the value-projected representations of all incoming source nodes. We refer to this weighted aggregate as the head-wise message a,h(l)m_a,h^(l) received by target node uau_a, which is given by: a,h(l)=∑ub∈ℳaαa,b,h(l)b(l−1)Wψ(a,b),h.m_a,h^(l)= _u_b _a _a,b,h^(l)h_b^(l-1)W_ψ(a,b),h V. (21) The H head-wise messages are then concatenated and modulated by a relation-dependent gate before being used to update the representation of target node uau_a as: a(l)=ρ(a(l−1),W[a(l)⊙Concath=1H(a,h(l))]), -5.69054pt aligned h_a^(l)= _ HGT\! (h_a^(l-1),W O\! [ η_a^(l) _h=1^H (m_a,h^(l) ) ] ), aligned -5.69054pt (22) where a(l)=sigmoid(1max(1,|Ψa|)∑ψ∈Ψaψ) η_a^(l)=sigmoid\! ( 1 (1,| _a|) _ψ∈ _a γ_ψ ) is the relation-dependent gate that modulates the aggregated messages, with Ψa=ψ(ub→ua)∣ub∈ℳa _a=\ψ(u_b→ u_a) u_b _a\ denoting the set of relation types associated with the incoming edges of uau_a and ψ γ_ψ denoting the learnable gate associated with relation ψ; Concath=1H(⋅)Concat_h=1^H(·) concatenates the H head-wise messages; ⊙ denotes element-wise multiplication; W O is a learnable output projection that maps the concatenated multi-head representation back to the node-embedding space; and ρ _ HGT applies the residual connection, layer normalization, and GELU-based feed-forward update. Repeating this process over L HGT layers enables each node to progressively incorporate role-dependent information from its multi-hop interaction neighborhood. The final representation a(L)h_a^(L) therefore constitutes the learned interaction-aware embedding of node uau_a produced by the HGT. ∙ Step 3: Scene context construction. After L HGT layers, the final interaction-aware node representations a(L)h_a^(L) are retained as individual node tokens. To additionally capture global scene-level information, we aggregate the final representations of all robot and entity nodes through mean pooling (i.e., element-wise averaging) to obtain a graph-level token ()=Norm(Φ(1||∑a=1||a(L)))c^( G)=Norm\! ( _ ctx\! ( 1|V| _a=1^|V|h_a^(L) ) ), where Φ _ ctx is a learnable projection that maps the pooled graph representation into the node-token dimension and Norm(⋅)Norm(·) denotes ℓ2 _2 normalization. Finally, the context-token sequence is constructed as follows: C()=[1(L),…,||(L),()].C^( ctx)= [h_1^(L),…,h_|V|^(L),c^( G) ]. (23) Consequently, C()C^( ctx) provides the diffusion model in Phase 2 with both agent-specific interaction representations and a global representation of the overall scene. IV-B Phase 2: Conditional Diffusion Trajectory Generation Conditioned on the context tokens C()C^( ctx) obtained in Phase 1, Phase 2 employs a denoising diffusion probabilistic model (DDPM) [15] to model the conditional joint trajectory distribution pθ(S()∣C())p_θ\! (S^( out) C^( ctx) ). By sampling from this distribution, SAGE jointly generates robot plan velocities and entity velocity predictions, thereby capturing their coupled and multimodal future dynamics. We next describe the forward diffusion process, context-conditioned denoiser and training objective, and reverse sampling process underlying our DDPM. ∙ Forward diffusion process. During training, each data sample consists of a scene-level conditioning input S()S^( in) (defined in Sec. I-D1) and its corresponding ground-truth joint future velocity sequence S()S^( out). Phase 1 maps S()S^( in) to the context tokens C()C^( ctx), whereas the forward diffusion process progressively adds Gaussian noise to S()S^( out) over K diffusion steps. Specifically, at diffusion step k∈1,…,Kk∈\1,…,K\, the transition from Sk−1()S_k-1^( out) to Sk()S_k^( out) is defined as follows: q(Sk()∣Sk−1())=(αkSk−1(),βkI),q\! (S_k^( out) S_k-1^( out) )=N\! ( _kS_k-1^( out), _kI ), (24) where S0()=S()S_0^( out)=S^( out) is the clean ground-truth joint future velocity sequence, βk _k is the prescribed noise variance at diffusion step k, and αk=1−βk _k=1- _k. Thus, at each forward diffusion step, Sk()S_k^( out) is sampled from a Gaussian distribution with mean αkSk−1() _kS_k-1^( out) and covariance βkI _kI, thereby progressively perturbing the joint future velocity sequence of robots and entities with Gaussian noise. Rather than sequentially applying all preceding diffusion steps, a noisy sequence at any diffusion step k can be sampled directly from the clean sequence as: Sk()=α¯kS()+1−α¯kϵ,S_k^( out)= α_kS^( out)+ 1- α_k\,ε, (25) where α¯k=∏κ=1kακ α_k= _κ=1^k _κ and ϵ∼(0,I)ε (0,I) is a random noise with the same dimensions as S()S^( out), whose entries are independently drawn from a standard normal distribution with zero mean and unit variance. As k increases, the contribution of the clean sequence decreases while that of the Gaussian noise increases; for large k, Sk()S_k^( out) approaches isotropic Gaussian noise. ∙ Context-conditioned denoiser and training objective. Given a noisy joint velocity sequence Sk()S_k^( out) generated by the forward diffusion process, the denoiser ϵθ _θ is trained to predict the Gaussian noise added to the original sequence S()S^( out). Specifically, the denoiser takes as input Sk()S_k^( out), the corresponding diffusion level represented by βk _k, and the context tokens C()C^( ctx) obtained from Phase 1. We implement ϵθ _θ using a Transformer [45] that employs temporal self-attention to capture motion dependencies within each trajectory, agent-wise attention to capture interactions among robot and entity trajectories, and cross-attention to incorporate the scene and waypoint information encoded in C()C^( ctx). To train this denoiser across different noise levels, for each training sample, we randomly select a diffusion step k and sample Gaussian noise ϵ∼(0,I)ε (0,I). We then construct the corresponding noisy sequence Sk()S_k^( out) using (24) and train ϵθ _θ to predict the sampled noise ϵε by minimizing the following loss function: ℒ=[‖ϵ−ϵθ(Sk(),βk,C())‖22].L_ diff=E [ \|ε- _θ (S_k^( out), _k,C^( ctx) ) \|_2^2 ]. (26) The expectation in (26) is taken over the training samples, randomly selected diffusion steps k, and sampled noise ϵε. During training, this loss is minimized using gradient-based optimization, which updates the learnable parameters of both the denoiser ϵθ _θ and the learnable parameters of Phase 1 scene encoder that produces C()C^( ctx). Consequently, the model jointly learns to encode the conditioning scene and denoise the coupled robot–entity future velocity sequences. ∙ Reverse sampling process. At inference, the trained denoiser ϵθ _θ generates NsampN_samp stochastic samples from the learned conditional joint trajectory distribution. For each sample, the reverse process starts from SK()∼(0,I)S_K^( out) (0,I) and proceeds from k=Kk=K to k=0k=0. At each diffusion level k, the denoiser predicts the noise contained in the current sequence Sk()S_k^( out), which is then removed to obtain the corresponding clean joint velocity estimate S^k() S_k^( out). With a sampling stride s and next diffusion level k′=max(k−s,0)k = (k-s,0), the reverse update is given by: S^k() S_k^( out) =Sk()−1−α¯kϵθ(Sk(),βk,C())α¯k, = S_k^( out)- 1- α_k\, _θ(S_k^( out), _k,C^( ctx)) α_k, (27) Sk′() S_k ^( out) =α¯k′S^k()+1−α¯k′z,k′>0,S^k(),k′=0, = where z∼(0,I)z (0,I) is newly sampled Gaussian noise. Here, S^k() S_k^( out) represents the clean joint robot–entity velocity sequence estimated from the current noisy sequence Sk()S_k^( out). If k′>0k >0, a controlled amount of Gaussian noise, determined by α¯k′ α_k , is added to this clean estimate to construct Sk′()S_k ^( out). We refer to this operation as re-noising. Since k′<k <k, Sk′()S_k ^( out) corresponds to a lower diffusion level and is therefore less noisy than Sk()S_k^( out); it then serves as the input to the next reverse step. This iterative process of estimating a clean sequence and re-noising it at progressively lower diffusion levels is repeated until k′=0k =0, at which point no additional noise is added and the clean estimate becomes the final joint sample of robot plan velocities and entity-prediction velocities. Hence, the reverse sampling process provides a practical mechanism for generating samples from the learned conditional distribution pθ(S()∣C())p_θ\! (S^( out) C^( ctx) ), corresponding to the sampling requirement in the objective of formulation P in (7). However, the resulting samples are not guaranteed to satisfy constraints (C1)–(C5) as (27) represents the unguided reverse sampling process. To address this, Phase 3 incorporates safety-social guidance into the reverse sampling process by modifying only the robot component of each clean estimate S^k() S_k^( out) before re-noising. IV-C Phase 3: Safety-Social Energy Guidance Since Phase 2 samples are not guaranteed to satisfy constraints (C1)–(C5), Phase 3 guides the reverse sampling process toward constraint-compliant robot trajectories. To this end, following gradient-guided diffusion [8, 18], we define a differentiable energy function J over each clean joint velocity estimate (Sec. IV-C1), where lower energy indicates greater compliance, and correct only the robot components before re-noising (see Appendix A, Fig. 5); entity predictions remain unchanged. IV-C1 Differentiable energy function design We define J as a weighted differentiable energy function corresponding to constraints (C1)–(C5), evaluated on the integrated robot plans and predicted entity trajectories: (S())=w()+w()+w()+w(). aligned J(S^( out))=w_ colJ^( col)+w_ socJ^( soc)+w_ kinJ^( kin)+w_ goalJ^( goal). aligned (28) The weights w,w,w,w≥0w_ col,w_ soc,w_ kin,w_ goal≥ 0 control the importance of collision avoidance term ()J^( col), social compliance term ()J^( soc), kinematic feasibility term ()J^( kin), and task progress term ()J^( goal), respectively. These four terms are discussed next. (i) Physical anti-collision potential energy (()J^( col)). To capture constraints (C1) and (C2), we define a differentiable potential that penalizes proximity between each robot–entity pair and each pair of robots. Specifically, for a robot rjr_j and entity oio_i, let dj,it=‖j(,t)−i(,t)‖2d_j,i^t=\|p_j^( r,t)-p_i^( o,t)\|_2 denote their distance at time t, and for robots rjr_j and rj′r_j , let dj,j′t=‖j(,t)−j′(,t)‖2d_j,j ^t=\|p_j^( r,t)-p_j ^( r,t)\|_2. We define the collision potential ϕ(d;m)=exp[−d2/(2σ2)]+ReLU(m−d)2 _ col(d;m)= [-d^2/(2 _ col^2)]+ReLU(m-d)^2, which provides smooth repulsion as the pairwise distance d decreases and imposes an additional penalty when d falls below the safety margin m. The resulting collision energy is then defined as follows: ()=∑t=1T()[∑rj∈ℛ[∑oi∈0ϕ(dj,it;mj,i)+w∑j<j′ϕ(dj,j′t;mj,j′)]], aligned J^( col)= -2.27621pt _t=1^T^( prd) -1.99168pt [ -0.85358pt _r_j -0.85358pt [ -2.81683pt _o_i _N^0 -3.41432pt _ col(d_j,i^t;m_j,i)+w_ R2R _j<j _ col(d_j,j ^t;m_j,j ) ] -0.85358pt ] -1.42262pt, aligned (29) where mj,im_j,i and mj,j′m_j,j are the safety margins defined in (C1) and (C2), and w≥0w_ R2R≥ 0 controls the relative importance of robot-to-robot collision avoidance (the summation index j<j′j<j ensures that each distinct robot pair is counted only once). (i) Heterogeneous anisotropic social potential (()J^( soc)). To capture the role-conditioned social compliance constraint in (C4), we define a differentiable potential that penalizes robots for entering the social comfort regions of neighboring entities in their forward half-plane. Specifically, for each robot rjr_j and entity oio_i, we define the normalized relative position i,jt=(di,j(,t)/σi,di,j(,t)/σi) δ_i,j^t=(d_i,j^( lon,t)/ _i lon,d_i,j^( lat,t)/ _i lat) where σi=σ(i(,)) _i lon=σ lon(C_i^( o,role)) and σi=σ(i(,)) _i lat=σ lat(C_i^( o,role)) are defined in (C4). The resulting social compliance energy is then defined as: ()=∑t=1T()∑rj∈ℛ∑oi∈0[di,j(,t)≥0]exp(−‖i,jt‖22), aligned J^( soc)= -0.56905pt _t=1^T^( prd) _r_j _o_i _N^0I\! [d_i,j^( lon,t)≥ 0 ] \! (-\| δ_i,j^t\|_2^2 ), aligned (30) where the term exp(−‖i,jt‖22) (-\| δ_i,j^t\|_2^2) assigns a larger penalty when the robot is closer to the entity within its normalized social space, while [di,j(,t)≥0]I[d_i,j^( lon,t)≥ 0] restricts this penalty to entities located in the robot’s forward half-plane. (i) Kinematic potential energy (()J^( kin)). To capture the kinematic feasibility constraint in (C3), we penalize violations of the prescribed translational-speed, lateral-velocity, and angular-velocity limits. Specifically, each penalty remains zero when the corresponding kinematic quantity is within its allowable range and increases quadratically once the limit is exceeded. The resulting kinematic energy is defined as follows: ()=∑t=1T()∑rj∈ℛ[[‖jt‖2−vmax]+2+[|vj(,t)|−vmax]+2+[|ωjt|−ωmax]+2], aligned J^( kin)= -3.1298pt _t=1^T^( prd) -2.41849pt _r_j -1.99168pt [ -1.13809pt [\|v_j^t\|_2-v_ ]_+^2+ [|v_j^( lat,t)|-v_ lat ]_+^2+ [| _j^t|- _ ]_+^2 ] -0.56905pt, aligned (31) where [x]+=ReLU(x)=max(x,0)[x]_+=ReLU(x)= (x,0). (iv) Task-progress potential energy (()J^( goal)). To capture the task-progress requirement in (C5), we penalize the terminal distance between each robot and its assigned waypoint. Specifically, the task-progress energy is defined as follows: ()=∑rj∈ℛ‖j(,T())−j‖22. aligned J^( goal)= _r_j \|p_j^( r,T^( prd))-g_j \|_2^2. aligned (32) IV-C2 Guided correction algorithm design We now integrate the energy-guided correction into the reverse sampling process of Phase 2. Recall that, in the unguided reverse update in (27), the denoiser first produces the clean joint velocity estimate S^k() S_k^( out), which is then directly re-noised to obtain Sk′()S_k ^( out). Phase 3 modifies this process by inserting an energy-guided correction between these two operations: S^k() S_k^( out) is first corrected to obtain S~k() S_k^( out), and this guided estimate, rather than S^k() S_k^( out), is subsequently re-noised to continue the reverse sampling process. Specifically, we use the energy function J defined in (28) to guide the clean velocity estimates generated during the reverse sampling process toward greater constraint compliance. Specifically, at diffusion level k, SAGE first evaluates (S^k())J( S_k^( out)) on the clean joint robot–entity velocity estimate S^k() S_k^( out) obtained from (27). The corresponding gradient ∇S^k()(S^k()) _ S_k^( out)J( S_k^( out)) indicates how changes to the estimated velocities affect the total energy. Since only robot trajectories are controllable, we apply a mask ℳℛM_R that retains the gradient components corresponding to robot velocities and sets the entity components to zero, and then modify the robot velocities in the negative-gradient direction to reduce J. Specifically, the guided correction and subsequent re-noising are performed as follows: k _k =Clip(ℳℛ∇S^k()(S^k())), =Clip_ grad\! (M_R _ S_k^( out)J( S_k^( out)) ), (33) S~k() S_k^( out) =Π(S^k()−Clip(λk)), = _ vel\! ( S_k^( out)-Clip_ step( _k) ), Sk′() S_k ^( out) =α¯k′S~k()+1−α¯k′z,k′>0,S~k(),k′=0. = In (33), the operator Clip(⋅)Clip_ grad(·) applies trajectory-wise norm clipping to the robot-only gradient ℳℛ∇S^k()(S^k())M_R _ S_k^( out)J( S_k^( out)), resulting in the bounded gradient kg_k. The parameter λ controls the correction strength, while ClipClip_ step bounds the resulting correction step. Also, the operator Π(⋅) _ vel(·) caps the corrected robot speeds at γvmax _ velv_ , with γ=1.25 _ vel=1.25, while leaving the entity velocities unchanged, resulting in the guided clean joint velocity estimate S~k() S_k^( out). Finally, z∼(0,I)z (0,I) denotes the Gaussian noise used to re-noise S~k() S_k^( out) when k′>0k >0. In words, at each reverse sampling step, SAGE computes how the total energy changes with the clean joint velocity estimate, retains only the gradient corresponding to the controllable robots, and adjusts their velocities in the negative-gradient direction to reduce the energy. The resulting correction and robot speeds are bounded to prevent excessive changes, while the predicted entity velocities remain unchanged. The corrected estimate is then re-noised to continue the reverse sampling process when k′>0k >0; when k′=0k =0, it becomes the final joint velocity sample. V Experiments In the following, we numerically evaluate SAGE in terms of task effectiveness, safety, social compliance, guidance controllability, and R2R coordination. V-A Experimental Setup We consider two evaluation regimes: real-world trajectory data for safety and social compliance, and controlled heterogeneous simulations for task performance and scalability. ∙ Datasets and rationale. We employ three data sources, each serving a distinct purpose. (i) ETH/UCY [2] contains five pedestrian scenes (ETH, HOTEL, UNIV, ZARA1, and ZARA2). Since this dataset lacks semantic labels and robot waypoints, we treat a target pedestrian as the robot and its future endpoint as the waypoint. (i) SDD provides aerial trajectories with official object annotations [29]. We map Pedestrians to PHAs, Bikers/Skaters/Carts to HNAAs, and Cars/Buses to SSAAs, making SDD the primary source for role-aware social-compliance evaluation. (i) Controlled heterogeneous simulations complement recorded data by varying robot population, entity density, role composition, and start-waypoint conflicts. Robots receive independently assigned waypoints across open, crossing, corridor, bottleneck, and intersection layouts, enabling task and scalability evaluation in Sec. V-D. ∙ Evaluation protocol. We set T()=8T^( obs)=8, T()=12T^( prd)=12, and Δt=0.4s t=0.4s. Unless noted, we use Nsamp=20N_samp=20 stochastic samples and a reverse-diffusion stride of 10. ∙ Safety and social-compliance metrics. For physical safety, collision rate (CR) metric measures the fraction of robot-entity and robot-robot interactions across the prediction horizon with separation below the collision margin, while minimum separation (MD) metric measures the minimum separation over the prediction horizon. For social compliance, personal-space intrusion rate (PIR) metric measures the fraction of robot–entity interactions across the prediction horizon within the personal-space margin, while social-violation rate (SVR) metric measures the fraction within the role-conditioned ellipse in (11). Speed-violation rate (Sp-VR) metric measures the fraction of robot states exceeding the maximum speed, and Energy measures the total guidance objective J in (28), averaged over evaluation scenes and generated trajectory samples. ∙ Task/accuracy metrics. For task progress, Goal-FDE measures the terminal distance to the assigned waypoint, while Goal-SR measures the fraction of robots reaching within d()d^( goal) of their assigned waypoints. ADE/FDE measure the mean/final Euclidean errors against recorded trajectories. The prefixes R- and E- denote robot and entity quantities, respectively (e.g., R-ADE and E-ADE). ADE/FDE and goal metrics report the best of 20 trajectory samples, while safety-social metrics are averaged over all 20 samples. Lower values are better except for MD and Goal-SR. Details of preprocessing, comfort margins, thresholds, and model settings are provided in Appendix A-A. TABLE I: Results of real-world scenes. ETH/UCY values are averaged over five scenes; SDD uses official semantic annotations. !25 Dataset Method R-ADE↓ Goal-FDE↓ CR↓ MD↑ PIR↓ SVR↓ Energy↓ ETH/UCY Constant Velocity 0.4760 1.0092 0.0471 0.0398 0.3407 0.1456 20.4553 Trajectron++ 0.1461 0.0968 0.0416 0.0469 0.3440 0.1355 20.0778 MID 0.1163 0.0929 0.0409 0.0510 0.3547 0.1482 20.1621 SAGE w/o guidance 0.1152 0.0937 0.0441 0.0471 0.3560 0.1479 20.3761 SAGE w/ guidance 0.1174 0.0947 0.0353 0.0623 0.3457 0.1248 19.1257 SDD Constant Velocity 0.7290 1.6818 0.1701 0.0572 0.1560 0.0892 28.7860 Trajectron++ 0.1391 0.1188 0.1798 0.0875 0.1645 0.0965 27.6866 SAGE w/o guidance 0.1151 0.0700 0.1763 0.0582 0.1612 0.0925 26.1494 SAGE w/ guidance 0.1241 0.0813 0.1729 0.0694 0.1576 0.0679 24.4608 ∙ Benchmarks. On ETH/UCY dataset, we compare against three methods under the same proxy-navigation protocol: Constant Velocity, Trajectron++ [34], and MID [11]. On SDD, we compare against Constant Velocity and Trajectron++. We also evaluate SAGE with and without Phase 3 guidance discussed in Sec. IV-C (SAGE w/ guidance and SAGE w/o guidance). All simulations use an NVIDIA RTX 4090 GPU. V-B Safety/Social Compliance on ETH/UCY and SDD We first evaluate SAGE on real-world trajectory data, with Table I summarizing the results on ETH/UCY and SDD. Two findings emerge from comparing SAGE with and without guidance. First, compared with SAGE w/o guidance, SAGE consistently improves safety and social compliance. On ETH/UCY, SAGE reduces CR by 20.0% (0.0441→ 0.0353), SVR by 15.6%, and Energy by 6.1%, while increasing MD by 1.32× when compared with SAGE w/o guidance. On SDD, SAGE reduces SVR by 26.6% and Energy by 6.5%, with a smaller reduction in CR (1.9%) when compared with SAGE w/o guidance. Second, these improvements come with a modest accuracy trade-off compared with SAGE w/o guidance. On ETH/UCY, SAGE w/o guidance achieves an R-ADE of 0.1152, compared with 0.1163 for MID and 0.1461 for Trajectron++, while SAGE achieves an R-ADE of 0.1174, corresponding to only a 1.9% increase. Complete per-scene and role-wise results are provided in Appendix A-B. V-C Guidance Mechanism: Safety–Accuracy Trade-off We next evaluate the effect of the guidance scale λ, which controls the strength of the safety-social energy gradient applied during reverse diffusion dictated by (33). As shown in Fig. 2, increasing λ from 0 to 0.05 reduces CR and SVR by 20.0% and 15.6%, respectively, while R-ADE increases by only 1.9% (0.1152→ 0.1174). Thus, λ provides inference-time control over the safety–accuracy trade-off without retraining. Unless specified, we use λ=0.05λ=0.05 in experiments. TABLE I: Multi-robot scalability (Ne=50N_e=50, Nsamp=20N_samp=20). Metric values are averaged over three seeds, each with 1,024 scenes. !25 NrN_r Guidance Goal-FDE↓ R2E-CR↓ R2R-CR↓ R2E-MD↑ R2R-MD↑ SVR↓ Energy↓ Runtime 3 w/o 5.872 0.0418 0.0851 0.0206 0.0578 0.0216 292.2 85.0 ms 3 w/ 5.928 0.0394 0.0750 0.0241 0.0838 0.0198 274.7 92.1 ms 6 w/o 5.813 0.0372 0.0781 0.0136 0.0308 0.0185 594.9 90.5 ms 6 w/ 5.868 0.0349 0.0693 0.0171 0.0465 0.0169 557.9 99.6 ms 9 w/o 5.430 0.0397 0.0924 0.0122 0.0194 0.0202 966.3 96.6 ms 9 w/ 5.465 0.0375 0.0812 0.0146 0.0286 0.0187 904.3 106.1 ms 12 w/o 5.420 0.0399 0.0951 0.0106 0.0138 0.0205 1369.8 103.1 ms 12 w/ 5.448 0.0379 0.0839 0.0117 0.0207 0.0191 1279.9 113.8 ms 16 w/o 5.319 0.0383 0.0910 0.0090 0.0116 0.0193 1881.1 123.0 ms 16 w/ 5.330 0.0365 0.0801 0.0111 0.0165 0.0180 1752.6 137.0 ms 20 w/o 5.432 0.0387 0.0869 0.0078 0.0094 0.0196 2473.6 132.0 ms 20 w/ 5.444 0.0368 0.0769 0.0093 0.0141 0.0183 2297.5 147.2 ms Figure 2: Guidance effects across evaluation regimes. (a) Safety, social compliance, and trajectory accuracy versus guidance scale λ on ETH/UCY (five-scene average); CR, PIR, and SVR use the left axis, while R-ADE uses the right axis. (b) Relative safety-social improvement in the multi-robot scalability sweep; bars report reductions over unguided sampling averaged over three seeds, and error bars denote standard deviation. V-D Multi-Robot Evaluation: Coordination and Scalability We use controlled simulations to evaluate task performance and scalability as the robot team grows. We evaluate (i) the effectiveness of guidance with increasing team size and (i) the individual contributions of R2R graph edges and R2R collision guidance to safety and waypoint progress. ∙ Scalability of guided sampling. Table I evaluates Nr=|ℛ|=3N_r=|R|=3 to 2020 with Ne=||=50N_e=|O|=50. First, guidance improves safety and social compliance-related metrics across all team sizes (further visualized in Fig. 2): averaged over all NrN_r, it reduces R2E-CR by 5.35%, R2R-CR by 11.75%, SVR by 7.43%, and Energy by 6.52%, while increasing R2E-MD by 1.19× and R2R-MD by 1.48×. Second, the improvement is particularly pronounced for R2R safety: at Nr=20N_r=20, guidance reduces R2R-CR from 0.0869 to 0.0769 and increases R2R-MD from 0.0094 to 0.0141. Third, these improvements have modest task and computational costs: Goal-FDE changes by less than 2% across all team sizes, while guided runtime increases from 92.1 ms at Nr=3N_r=3 to 147.2 ms at Nr=20N_r=20, remaining below the 0.4s sampling interval. ∙ Roles of the two R2R mechanisms in SAGE. Table I evaluates the individual contributions of R2R graph edges in Phase 1 and R2R collision guidance in Phase 3 of SAGE. Compared with using neither component, R2R collision guidance alone reduces R2R-CR by 10.4% at Nr=12N_r=12 and 11.0% at Nr=20N_r=20, while increasing R2R-MD by 1.47× at both team sizes. Complete SAGE achieves the best (i.e., the lowest) Goal-FDE in both settings, indicating that R2R graph edges complement collision guidance in maintaining waypoint progress. TABLE I: Ablation of R2R graph edges and collision guidance. !25 NrN_r Variant Goal-FDE↓ R2R-CR↓ R2R-MD↑ 12 Neither component 5.574 0.0892 0.0144 12 Graph edges only 5.488 0.0934 0.0146 12 R2R guidance only 5.534 0.0799 0.0212 12 Both (SAGE) 5.448 0.0839 0.0207 20 Neither component 5.557 0.0858 0.0095 20 Graph edges only 5.510 0.0859 0.0094 20 R2R guidance only 5.486 0.0764 0.0140 20 Both (SAGE) 5.444 0.0769 0.0141 Figure 3: Controlled cases: bottleneck yielding, multi-robot crossing, vehicle-aware clearance, and dense merging. Dashed red: unguided; solid green: guided; blue circles: robot starts. V-E Qualitative Analysis Fig. 3 shows representative multi-robot cases in controlled simulations, where only the guidance setting changes. Three patterns emerge. First, guidance primarily adjusts trajectories near social fields or collision boundaries, while leaving other trajectories largely unchanged. Second, robots maintain larger clearance from PHAs than from SSAAs, reflecting the role-conditioned social fields. Third, guided trajectories remain close to their assigned waypoints, indicating that improved safety and social compliance do not substantially compromise waypoint progress. Fig. 4 shows similar behavior on SDD dataset, where guidance redirects robot trajectories away from nearby heterogeneous entities. Figure 4: Qualitative comparisons in SDD scenes, where hyang_0, coupa_0, and coupa_1 are official SDD video-scene identifiers. The first picture shows three robots in hyang_0; the last two pictures show a single robot in coupa_0 and coupa_1. Blue circles and yellow stars denote robots and waypoints, respectively; black, red, and green lines denote recorded futures, unguided trajectories, and SAGE-guided trajectories, respectively. Colored markers and thin traces show surrounding role-labeled entities. V-F Additional Ablations and Robustness Studies To further assess the robustness of SAGE and isolate its key design choices, Appendix A provides additional ablations and diagnostics in Tables IV–XIII. In the following, we briefly discuss the key findings and refer interested readers to Appendix A for further details and discussion. In brief, the per-scene ETH/UCY results in Table IV, the complete SDD results in Table V, and the role-wise SDD results in Table VI show that, when comparing SAGE w/ and w/o guidance, guidance consistently improves safety and social compliance across different scenes and semantic categories, while largely preserving trajectory prediction accuracy and task performance. Table VII further shows that replacing HGT with a homogeneous graph-attention encoder slightly improves displacement accuracy (i.e., R-ADE and Goal-FDE) but worsens CR, SVR, and Energy, confirming the benefit of role-dependent interaction modeling. Moreover, Table VIII shows that jointly generating robot and entity trajectories achieves better waypoint-directed performance as the robot team grows compared with a two-stage approach that first predicts entity trajectories and then plans robot trajectories based on the fixed predictions, highlighting the benefit of coupling trajectory prediction and robot planning within a unified generative framework. Additional results demonstrate the controllability and generalizability of SAGE: Table IX shows that increasing wgoalw_goal improves waypoint-reaching performance at the cost of higher safety and social-compliance violations, revealing an interpretable task-safety trade-off. Table X confirms that the safety benefits of guided sampling remain consistent under different pseudo-role construction strategies on ETH/UCY. Compared with the Social Force planner [14] in Table XI, SAGE achieves substantially better task progress and trajectory accuracy, while Social Force attains lower violation rates through more conservative repulsion. Finally, the multi-robot SDD results in Table XII and Table XIII demonstrate that SAGE generalizes beyond the primary controlled-simulation setting, with guidance consistently improving robot-entity safety, inter-robot safety, and social compliance. VI Conclusion and Future Work In this work, we introduced SAGE for safe and socially compliant robot navigation in heterogeneous multi-agent environments. SAGE models role-dependent and asymmetric agent interactions through a directed heterogeneous graph and an HGT-based scene encoder. Using the resulting interaction-aware representations, a conditional diffusion model jointly generates predicted entity trajectories and planned robot trajectories. SAGE further integrates a training-free safety-social energy guidance mechanism that incorporates collision avoidance, kinematic feasibility, task progress, and role-dependent social norms into the reverse sampling process, enabling robot trajectories to be refined at inference time without retraining. Our evaluations in real-world and controlled heterogeneous multi-robot scenarios demonstrated that SAGE improves physical safety and social compliance while largely preserving trajectory accuracy and task performance. Promising future directions include (i) extending SAGE to decentralized multi-robot navigation under partial observability and communication constraints and (i) adapting role-dependent social priors to capture context-, culture-, and individual-dependent preferences through adaptive or human-in-the-loop mechanisms. References [1] A. Ajay, Y. Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal (2023) Is conditional generative modeling all you need for decision-making?. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: §I-B. [2] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese (2016) Social LSTM: human trajectory prediction in crowded spaces. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 961–971. Cited by: §I-A, §I, §I-A, §V-A. [3] C. Bai, Y. Peng, H. Piao, W. Pan, and J. Guo (2024-01) Learning-based multi-UAV flocking control with limited visual field and instinctive repulsion. IEEE Trans. Cybern. 54 (1), p. 462–475. External Links: Document Cited by: §I-C. [4] A. Biswas, A. Wang, G. Silvera, A. Steinfeld, and H. Admoni (2022-09) SocNavBench: a grounded simulation testing framework for evaluating social navigation. ACM Trans. Hum.-Robot Interact. 11 (3), p. 1–24. External Links: Document Cited by: §I-C. [5] G. Campion, G. Bastin, and B. Dandrea-Novel (1996-02) Structural properties and classification of kinematic and dynamic models of wheeled mobile robots. IEEE Trans. Robot. Autom. 12 (1), p. 47–62. External Links: Document Cited by: §I-A. [6] C. Chen, Y. Liu, S. Kreiss, and A. Alahi (2019) Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), p. 6015–6022. Cited by: §I-C. [7] Y. Chen, F. Zhao, and Y. Lou (2022-04) Interactive model predictive control for robot navigation in dense crowds. IEEE Trans. Syst., Man, Cybern., Syst. 52 (4), p. 2289–2301. External Links: Document Cited by: §I, §I-C. [8] P. Dhariwal and A. Nichol (2021) Diffusion models beat GANs on image synthesis. In Adv. Neural Inf. Process. Syst., Vol. 34, p. 8780–8794. Cited by: §IV-C. [9] M. Faroni, A. Umbrico, M. Beschi, A. Orlandini, A. Cesta, and N. Pedrocchi (2024-06) Optimal task and motion planning and execution for multiagent systems in dynamic environments. IEEE Trans. Cybern. 54 (6), p. 3366–3377. External Links: Document Cited by: §I-A. [10] Y. Gao and C. Huang (2022-01) Evaluation of socially-aware robot navigation. Front. Robot. AI 8, p. 721317. External Links: Document Cited by: §I-C. [11] T. Gu, G. Chen, J. Li, C. Lin, Y. Rao, J. Zhou, and J. Lu (2022) Stochastic trajectory prediction via motion indeterminacy diffusion. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 17113–17122. Cited by: §I-A, §I-B, §V-A. [12] A. Gupta, J. Johnson, L. Fei-Fei, S. Savarese, and A. Alahi (2018) Social GAN: socially acceptable trajectories with generative adversarial networks. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 2255–2264. Cited by: §I-A, §I, §I-A. [13] E. T. Hall (1966) The hidden dimension. Doubleday, Garden City, NY, USA. Cited by: §A-A, §I-A, §I-C, §I-C1. [14] D. Helbing and P. Molnar (1995-05) Social force model for pedestrian dynamics. Phys. Rev. E 51 (5), p. 4282–4286. External Links: Document Cited by: §A-A, §A-D1, §I, §I-C, §V-F. [15] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Adv. Neural Inf. Process. Syst., Vol. 33, p. 6840–6851. Cited by: §I-A, §I-B, §IV-B. [16] Z. Hu, Y. Dong, K. Wang, and Y. Sun (2020) Heterogeneous graph transformer. In Proc. Web Conf. (W), p. 2704–2710. Cited by: §I-A, §I-A. [17] M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine (2022) Planning with diffusion for flexible behavior synthesis. In Proc. 39th Int. Conf. Mach. Learn. (ICML), p. 9902–9915. Cited by: §I-B. [18] C. M. Jiang, A. Cornman, C. Park, B. Sapp, Y. Zhou, and D. Anguelov (2023) MotionDiffuser: controllable multi-agent motion prediction using diffusion. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 9644–9653. Cited by: §I-B, §IV-C. [19] A. I. Karoly, P. Galambos, J. Kuti, and I. J. Rudas (2021-01) Deep learning in robotics: survey on model structures and training strategies. IEEE Trans. Syst., Man, Cybern., Syst. 51 (1), p. 266–279. External Links: Document Cited by: §I. [20] V. Kosaraju, A. Sadeghian, R. Martín-Martín, I. Reid, H. Rezatofighi, and S. Savarese (2019) Social-BiGAT: multimodal trajectory forecasting using Bicycle-GAN and graph attention networks. In Adv. Neural Inf. Process. Syst., Vol. 32, p. 137–146. Cited by: §I-A, §I-A. [21] H. Kretzschmar, M. Spies, C. Sprunk, and W. Burgard (2016-09) Socially compliant mobile robot navigation via inverse reinforcement learning. Int. J. Robot. Res. 35 (11), p. 1289–1307. External Links: Document Cited by: §I. [22] T. Kruse, A. K. Pandey, R. Alami, and A. Kirsch (2013-12) Human-aware robot navigation: a survey. Robot. Auton. Syst. 61 (12), p. 1726–1743. External Links: Document Cited by: §I-C. [23] M. Li, J. Qin, J. Li, Q. Liu, Y. Shi, and Y. Kang (2024-10) Game-based approximate optimal motion planning for safe human-swarm interaction. IEEE Trans. Cybern. 54 (10), p. 5649–5660. External Links: Document Cited by: §I. [24] Z. Lin, Z. C. Chen, G. Zhu, J. Chen, and J. Li (2026-03) GALC: guided amplified learning with Lipschitz constraint for robust trajectory generation. IEEE Trans. Cybern., p. 1–11. Note: early access, doi: 10.1109/TCYB.2026.3668987 External Links: Document Cited by: §I-B. [25] C. Mavrogiannis, F. Baldini, A. Wang, D. Zhao, A. Steinfeld, P. Trautman, and J. Oh (2023-09) Core challenges of social robot navigation: a survey. ACM Trans. Hum.-Robot Interact. 12 (3), p. 1–39. External Links: Document Cited by: §I, §I, §I-C. [26] B. Miloradovic, B. Curuklu, M. Ekstrom, and A. V. Papadopoulos (2022-10) GMP: a genetic mission planner for heterogeneous multirobot system applications. IEEE Trans. Cybern. 52 (10), p. 10627–10638. External Links: Document Cited by: §I-A. [27] A. Mohamed, K. Qian, M. Elhoseiny, and C. Claudel (2020) Social-STGCNN: a social spatio-temporal graph convolutional neural network for human trajectory prediction. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 14424–14432. Cited by: §I-A, §I, §I-A. [28] M. Niedoba, J. W. Lavington, Y. Liu, V. Lioutas, J. Sefas, X. Liang, D. Green, S. Dabiri, B. Zwartsenberg, A. Scibior, and F. Wood (2023) A diffusion-model of joint interactive navigation. In Adv. Neural Inf. Process. Syst., Vol. 36, p. 27440–27459. Cited by: §I-B. [29] A. Robicquet, A. Sadeghian, A. Alahi, and S. Savarese (2016) Learning social etiquette: human trajectory understanding in crowded scenes. In Proc. Eur. Conf. Comput. Vis. (ECCV), p. 549–565. Cited by: §V-A. [30] A. Rudenko, L. Palmieri, M. Herman, K. M. Kitani, D. M. Gavrila, and K. O. Arras (2020-07) Human motion trajectory prediction: a survey. Int. J. Robot. Res. 39 (8), p. 895–935. External Links: Document Cited by: §I. [31] S. Saadatnejad, M. Bahari, P. Khorsandi, M. Saneian, S. Moosavi-Dezfooli, and A. Alahi (2022-08) Are socially-aware trajectory prediction models really socially-aware?. Transp. Res. C, Emerg. Technol. 141, p. 103705. External Links: Document Cited by: §I-C. [32] S. Saadatnejad, Y. Gao, K. Messaoud, and A. Alahi (2023) Social-Transmotion: promptable human trajectory prediction. Note: arXiv:2312.16168 External Links: Link Cited by: §I-C. [33] A. Sadeghian, V. Kosaraju, A. Sadeghian, N. Hirose, H. Rezatofighi, and S. Savarese (2019) SoPhie: an attentive GAN for predicting paths compliant to social and physical constraints. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), p. 1349–1358. Cited by: §I-A. [34] T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone (2020) Trajectron++: dynamically-feasible trajectory forecasting with heterogeneous data. In Proc. Eur. Conf. Comput. Vis. (ECCV), p. 683–700. Cited by: §I-A, §I-A, §V-A. [35] W. Schwarting, A. Pierson, J. Alonso-Mora, S. Karaman, and D. Rus (2019-12) Social behavior for autonomous vehicles. Proc. Natl. Acad. Sci. USA 116 (50), p. 24972–24978. External Links: Document Cited by: §I. [36] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli (2015) Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. Int. Conf. Mach. Learn. (ICML), p. 2256–2265. Cited by: §I-B. [37] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: §I-A. [38] J. Turner, Q. Meng, G. Schaefer, A. Whitbrook, and A. Soltoggio (2018-09) Distributed task rescheduling with time constraints for the optimization of total task allocations in a multirobot system. IEEE Trans. Cybern. 48 (9), p. 2583–2597. External Links: Document Cited by: §I-C. [39] J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha (2011) Reciprocal n-body collision avoidance. In Proc. 14th Int. Symp. Robot. Res. (ISRR), p. 3–19. Cited by: §I, §I-C. [40] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio (2018) Graph attention networks. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: footnote 3. [41] G. Wen, X. Yu, W. Yu, and J. Lu (2021-10) Coordination and control of complex network systems with switching topologies: a survey. IEEE Trans. Syst., Man, Cybern., Syst. 51 (10), p. 6342–6357. External Links: Document Cited by: §I-C. [42] J. Wu, C. Luo, Y. Luo, and K. Li (2022-10) Distributed UAV swarm formation and collision avoidance strategies over fixed and switching topologies. IEEE Trans. Cybern. 52 (10), p. 10969–10979. External Links: Document Cited by: §I-C. [43] J. Wu and S. Yu (2026-06) JPPD: joint prediction–planning diffusion with differentiable safety guidance for dynamic obstacle avoidance in intelligent transportation systems. arXiv preprint arXiv:2606.20686. Cited by: §I-B. [44] W. Xiao, T. Wang, C. Gan, R. Hasani, M. Lechner, and D. Rus (2025) SafeDiffuser: safe planning with diffusion probabilistic models. In Proc. Int. Conf. Learn. Represent. (ICLR), Cited by: §I-B. [45] Y. Yuan, X. Weng, Y. Ou, and K. M. Kitani (2021) AgentFormer: agent-aware transformers for socio-temporal multi-agent forecasting. In Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), p. 9813–9823. Cited by: §I, §I-A, §IV-B. [46] C. Zhang, D. Song, C. Huang, A. Swami, and N. V. Chawla (2019) Heterogeneous graph neural network. In Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., p. 793–803. Cited by: §I-A. [47] Z. Zhong, D. Rempe, D. Xu, Y. Chen, S. Veer, T. Che, B. Ray, and M. Pavone (2023) Guided conditional diffusion for controllable traffic simulation. In Proc. IEEE Int. Conf. Robot. Autom. (ICRA), p. 3560–3566. Cited by: §I-B. [48] Y. Zhou, H. Hu, Y. Liu, and Z. Ding (2017-07) Collision and deadlock avoidance in multirobot systems: a distributed approach. IEEE Trans. Syst., Man, Cybern., Syst. 47 (7), p. 1712–1726. External Links: Document Cited by: §I, §I-C. Appendix A Additional Experimental Details and Results This appendix collects implementation settings and additional results. A-A Implementation and Guidance Settings We retain up to eight neighboring entities in ETH/UCY and up to 12 in SDD. ETH/UCY and the controlled simulations use metric coordinates, whereas SDD coordinates are divided by 50 before training and evaluation; accordingly, all distance-related thresholds are specified in their respective coordinate systems. The safety distance, maximum velocity, and goal-reaching threshold (d(),vmax,d())(d^( safe),v_ ,d^( goal)) are set to (0.45,2.5,0.5)(0.45,2.5,0.5) for ETH/UCY, (0.30,3.0,0.5)(0.30,3.0,0.5) for the controlled simulations, and (0.35,3.0,0.6)(0.35,3.0,0.6) for SDD. Across all settings, we use vmax=0.15v_ lat=0.15 and ωmax=1.2 _ =1.2. Following proxemics theory [13] and prior social-navigation literature [14], the role-dependent longitudinal and lateral social margins (σ,σ)(σ lon,σ lat) are set to (2.0,1.0)(2.0,1.0) for PHAs, (2.6,1.2)(2.6,1.2) for HNAAs, and (1.2,0.7)(1.2,0.7) for SSAAs. Unless otherwise stated, the guidance weights are set to (w,w,w,w)=(2.0,1.0,0.1,0.3)(w_ col,w_ soc,w_ kin,w_ goal)=(2.0,1.0,0.1,0.3), with σ=0.8 _ col=0.8. The gradient-clipping and guidance-step-clipping norms are set to 1.01.0 and 0.050.05, respectively, while the robot-to-robot interaction weight w_ R2R is set to 1.51.5 in the controlled simulations and 1.01.0 in the SDD multi-robot experiments. For model training, we use AdamW with a learning rate of 3×10−43× 10^-4, a weight decay of 10−410^-4, and a batch size of 128. The ETH/UCY and SDD models are trained for 100 and 80 epochs, respectively. Each controlled-simulation model is trained for 50 epochs using 8,192 episodes and evaluated on 1,024 held-out episodes per seed. The HGT consists of L=2L=2 layers with hidden dimension d=128d=128 and four attention heads, while the denoiser consists of four temporal Transformer layers and uses 100 diffusion steps. A-B Complete Recorded-Scene Diagnostics TABLE IV: Full ETH/UCY per-scene proxy-navigation results. !25 Dataset Method R-ADE↓ Goal-FDE↓ Goal-SR↑ E-ADE↓ E-FDE↓ CR↓ MD↑ PIR↓ SVR↓ Energy↓ ETH SAGE w/o guidance 0.1338 0.0550 1.0000 0.7319 1.2722 0.0723 0.0362 0.3580 0.1494 10.5465 ETH SAGE w/ guidance 0.1339 0.0531 1.0000 0.7183 1.2350 0.0591 0.0549 0.3538 0.1221 9.9119 HOTEL SAGE w/o guidance 0.0977 0.1132 0.9691 0.5364 1.0041 0.0491 0.0551 0.3884 0.1000 10.0647 HOTEL SAGE w/ guidance 0.0997 0.1116 0.9783 0.5452 1.0207 0.0392 0.0684 0.3733 0.0802 9.2323 UNIV SAGE w/o guidance 0.1567 0.1441 0.9681 0.7837 1.4014 0.0393 0.0213 0.2574 0.1298 35.4965 UNIV SAGE w/ guidance 0.1588 0.1467 0.9686 0.7835 1.4011 0.0341 0.0258 0.2482 0.1170 33.5777 ZARA1 SAGE w/o guidance 0.0954 0.0661 0.9996 0.2899 0.4944 0.0240 0.0707 0.4006 0.1869 16.9963 ZARA1 SAGE w/ guidance 0.0993 0.0711 1.0000 0.2860 0.4893 0.0179 0.0942 0.3849 0.1722 15.9966 ZARA2 SAGE w/o guidance 0.0926 0.0900 0.9995 0.3076 0.5763 0.0358 0.0520 0.3753 0.1733 28.7766 ZARA2 SAGE w/ guidance 0.0955 0.0908 0.9993 0.3057 0.5707 0.0265 0.0680 0.3684 0.1326 26.9102 Avg. SAGE w/o guidance 0.1152 0.0937 0.9873 0.5299 0.9497 0.0441 0.0471 0.3560 0.1479 20.3761 Avg. SAGE w/ guidance 0.1174 0.0947 0.9892 0.5278 0.9434 0.0353 0.0623 0.3457 0.1248 19.1257 TABLE V: Full SDD semantic diagnostics with official annotations (Nsamp=20N_samp=20 samples). !25 Method R-ADE↓ Goal-FDE↓ Goal-SR↑ E-ADE↓ E-FDE↓ CR↓ MD↑ PIR↓ SVR↓ Sp-VR↓ Energy↓ SAGE w/o guidance 0.1151 0.0700 0.9945 0.4769 0.9274 0.1763 0.0582 0.1612 0.0925 0.0298 26.1494 SAGE 0.1241 0.0813 0.9913 0.4768 0.9274 0.1729 0.0694 0.1576 0.0679 0.0300 24.4608 TABLE VI: SDD role-wise semantic diagnostics. !25 Role Guidance ADE↓ FDE↓ CR↓ MD↑ PIR↓ SVR↓ Count/Scene PHA w/o 0.2781 0.5223 0.0141 0.1329 0.1786 0.0933 2.6130 PHA w/ 0.2787 0.5230 0.0119 0.1476 0.1748 0.0710 2.6130 HNAA w/o 0.7382 1.4622 0.0134 0.1915 0.1040 0.0752 1.7265 HNAA w/ 0.7385 1.4631 0.0123 0.2098 0.1008 0.0508 1.7265 SSAA w/o 0.1428 0.2569 0.0058 0.6455 0.0327 0.0103 0.0697 SSAA w/ 0.1406 0.2526 0.0057 0.6475 0.0320 0.0042 0.0697 Table IV reports the full per-scene ETH/UCY proxy-navigation results underlying the averages in Table I. Across all five scenes, guidance consistently reduces CR, PIR, SVR, and Energy while increasing MD, with only minor changes in trajectory accuracy and goal-reaching performance. On average, CR decreases from 0.0441 to 0.0353 and SVR from 0.1479 to 0.1248, while MD increases from 0.0471 to 0.0623. Meanwhile, R-ADE changes only from 0.1152 to 0.1174, and Goal-SR slightly improves from 0.9873 to 0.9892. These per-scene results confirm that the safety and social-compliance gains reported in Table I are consistent across the ETH/UCY scenes rather than driven by a particular environment. Table V provides the complete aggregate results on SDD, including entity displacement metrics. Guidance reduces CR from 0.1763 to 0.1729, PIR from 0.1612 to 0.1576, SVR from 0.0925 to 0.0679, and Energy from 26.1494 to 24.4608, while increasing MD from 0.0582 to 0.0694. Notably, entity trajectory prediction remains essentially unchanged, with E-ADE changing from 0.4769 to 0.4768 and E-FDE remaining at 0.9274. These safety and social-compliance improvements come with a modest degradation in robot trajectory and goal-reaching metrics, as R-ADE increases from 0.1151 to 0.1241, Goal-FDE from 0.0700 to 0.0813, and Goal-SR decreases slightly from 0.9945 to 0.9913. Finally, Table VI decomposes the SDD results across the three semantic agent categories. Guidance reduces role-wise CR by 15.6% for PHAs (0.0141→ 0.0119) and 8.2% for HNAAs (0.0134→ 0.0123), with a marginal reduction for SSAAs (0.0058→ 0.0057). More pronounced improvements are observed in social compliance: SVR decreases by 23.9% for PHAs (0.0933→ 0.0710), 32.4% for HNAAs (0.0752→ 0.0508), and 59.2% for SSAAs (0.0103→ 0.0042). Also, upon using guidance, MD increases and PIR decreases for all three categories, while their ADE and FDE remain nearly unchanged. Thus, the benefits of guidance extend across all semantic categories rather than being concentrated in a single dominant class. Since SSAAs are relatively rare in the evaluated split (0.07 per scene), their role-wise results should be interpreted as diagnostic rather than conclusive. Figure 5: Detailed illustration of one safety-social guidance step during reverse diffusion. The role-conditioned social field, geometry-aware R2E/R2R collision margins, kinematic limits, and waypoint progress jointly define the guidance energy. Its clipped gradient applies a bounded correction only to the robot trajectory before re-noising, while entity predictions remain unchanged. A-C Additional Ablations and Diagnostics A-C1 Heterogeneous encoder To assess the contribution of heterogeneous interaction modeling, we replace the HGT with a homogeneous graph-attention encoder that uses a single shared relation type and omits role embeddings, thereby removing relation-specific attention projections. As shown in Table VII, the homogeneous encoder achieves better displacement accuracy, with R-ADE and Goal-FDE decreasing from 0.1241 to 0.1170 and from 0.0813 to 0.0621, respectively. In contrast, HGT provides better safety and social-compliance performance, reducing CR from 0.1806 to 0.1729, SVR from 0.0743 to 0.0679, and Energy from 24.8181 to 24.4608. These results indicate that explicitly modeling role-dependent and relation-specific interactions primarily improves interaction quality and social compliance, albeit with a modest trade-off in raw trajectory accuracy. TABLE VII: HGT versus homogeneous graph-attention encoding on SDD under guided sampling. !25 Variant R-ADE↓ Goal-FDE↓ CR↓ SVR↓ Energy↓ Homogeneous encoder 0.1170 0.0621 0.1806 0.0743 24.8181 HGT 0.1241 0.0813 0.1729 0.0679 24.4608 A-C2 Joint vs. two-stage generation To evaluate the benefit of jointly generating robot and surrounding-entity trajectories, Table VIII compares SAGE with a two-stage variant in which entity trajectories are first generated and then fixed while robot trajectories are subsequently generated using the same Phase 3 guidance settings. The two-stage variant achieves slightly better local safety and social-compliance metrics, with lower R2E-CR, R2R-CR, and SVR and notably higher R2R-MD. In contrast, joint generation consistently achieves better Goal-FDE, reducing it from 5.725 to 5.448 for Nr=12N_r=12 and from 5.675 to 5.444 for Nr=20N_r=20. These results suggest that, while decoupling prediction and planning can favor local collision avoidance, jointly modeling the evolution of surrounding entities and robot actions better preserves goal-directed task performance as multiple robots plan simultaneously. TABLE VIII: Multi-robot two-stage decoupling in controlled simulation. !25 Setting NrN_r Variant Guidance Goal-FDE↓ R2E-CR↓ R2R-CR↓ R2R-MD↑ SVR↓ Controlled sim. 12 Joint (i.e., SAGE) w/ 5.448 0.0379 0.0839 0.0207 0.0191 Controlled sim. 12 Two-stage w/ 5.725 0.0369 0.0788 0.0626 0.0178 Controlled sim. 20 Joint (i.e., SAGE) w/ 5.444 0.0368 0.0769 0.0141 0.0183 Controlled sim. 20 Two-stage w/ 5.675 0.0360 0.0757 0.0391 0.0173 A-C3 Task-progress recovery via w_ goal To characterize the trade-off between goal-directed task performance and safety/social compliance, Table IX sweeps the task-progress weight w_ goal on SDD under guided sampling. As w_ goal increases from 0 to 2.02.0, R-ADE decreases from 1.4740 to 0.8267, Goal-FDE decreases from 2.4349 to 1.1677, and Goal-SR increases from 0.2120 to 0.3169, indicating progressively stronger goal-directed behavior. This improvement comes at the cost of higher CR (0.0118→ 0.0146), PIR (0.0087→ 0.0106), SVR (0.0025→ 0.0044), and Energy (5.5359→ 23.5397). Thus, w_ goal provides an explicit control knob for balancing goal-directed task performance against safety and social compliance. Note that this table reports results in raw coordinate units; therefore, the relative trends, rather than direct comparisons with normalized results reported elsewhere, are of primary interest. TABLE IX: SDD w_ goal sweep under guided sampling. !25 w_ goal R-ADE↓ Goal-FDE↓ Goal-SR↑ CR↓ PIR↓ SVR↓ Energy↓ 0.0 1.4740 2.4349 0.2120 0.0118 0.0087 0.0025 5.5359 0.3 1.3487 2.1951 0.2247 0.0122 0.0090 0.0027 11.4233 0.6 1.2335 1.9741 0.2387 0.0126 0.0092 0.0029 15.9795 1.0 1.0945 1.7031 0.2595 0.0131 0.0096 0.0033 19.8706 2.0 0.8267 1.1677 0.3169 0.0146 0.0106 0.0044 23.5397 A-C4 Pseudo-role construction robustness Since ETH/UCY does not provide semantic agent labels, Table X evaluates whether the effectiveness of guidance depends on the pseudo-role construction strategy. We consider three alternatives: speed_threshold, which assigns roles using fixed speed thresholds; speed_quantile, which uses speed quantiles; and all_pha, which assigns every entity to the PHA category. Across all three strategies, guidance consistently and substantially reduces CR, PIR, SVR, and Energy. For example, CR decreases from approximately 0.0460.046–0.0470.047 without guidance to 0.00300.0030–0.00340.0034 with guidance, while SVR decreases from 0.16730.1673–0.20830.2083 to 0.02180.0218–0.02610.0261. Similarly, PIR decreases from 0.36250.3625–0.38850.3885 to 0.09610.0961–0.10180.1018, and Energy from 25.761725.7617–27.543227.5432 to 9.43709.4370–9.66359.6635. These consistent improvements demonstrate that the effectiveness of guidance is robust to the particular pseudo-role construction strategy used for ETH/UCY. Since these experiments use separate task-conditioned checkpoints with λ=0.03λ=0.03, only guided-versus-unguided comparisons within each strategy should be made; their absolute values are not directly comparable with those in Table I. TABLE X: Proxy role construction ablation on ETH/UCY (five-scene average, Nsamp=20N_samp=20 samples). !25 Role Strategy Guidance R-ADE↓ Goal-FDE↓ CR↓ PIR↓ SVR↓ Energy↓ speed_threshold w/o 0.5149 0.8900 0.0456 0.3885 0.1673 26.4692 speed_threshold w/ 0.6980 1.0756 0.0030 0.1018 0.0218 9.4370 speed_quantile w/o 0.6054 1.0353 0.0459 0.3625 0.1714 25.7617 speed_quantile w/ 0.6748 1.0259 0.0034 0.0961 0.0261 9.6635 all_pha w/o 0.5519 0.9574 0.0471 0.3810 0.2083 27.5432 all_pha w/ 0.7024 1.0839 0.0031 0.0989 0.0252 9.6558 A-D Social Force Baseline and Multi-Robot Diagnostics A-D1 Social Force comparison Table XI compares SAGE with the deterministic Social Force planner [14], which uses hand-crafted attractive and repulsive potentials to regulate robot motion. In the controlled simulation, Social Force achieves lower R2E-CR (0.0136 vs. 0.0371), R2R-CR (0.0172 vs. 0.0772), and SVR (0.0049 vs. 0.0185) than SAGE, but at a substantial cost in goal-directed and trajectory performance: its Goal-FDE is 1.68×1.68× higher (9.371 vs. 5.583) and R-ADE is 3.33×3.33× higher (2.688 vs. 0.808). A similar trend is observed in the SocialGym-style setting444Here, SocialGym-style refers to locally generated doorway, hallway, intersection, roundabout, and open-crowd layouts designed to represent diverse social-navigation scenarios; these experiments do not use rollouts from the SocialGym simulator. , where Social Force again achieves lower collision and social-violation rates, while SAGE reduces Goal-FDE from 7.766 to 6.605 and R-ADE from 2.322 to 0.696. Notably, SAGE also achieves zero Sp-VR in both settings, compared with 0.0474 and 0.0366 for Social Force. These results highlight the task–safety trade-off between conservative hand-crafted repulsion and SAGE’s learned, task-conditioned trajectory generation. TABLE XI: Comparison with deterministic Social Force planner. !25 Setting Method Goal-FDE↓ R-ADE↓ R2E-CR↓ R2R-CR↓ SVR↓ Sp-VR↓ Controlled sim. Social Force 9.371 2.688 0.0136 0.0172 0.0049 0.0474 Controlled sim. SAGE 5.583 0.808 0.0371 0.0772 0.0185 0.0000 SocialGym-style Social Force 7.766 2.322 0.0083 0.0198 0.0043 0.0366 SocialGym-style SAGE 6.605 0.696 0.0292 0.0823 0.0168 0.0000 A-D2 SDD multi-robot setting Table XII evaluates the SAGE pipeline in multi-robot navigation settings constructed from SDD scenes using the official semantic annotations. As the number of robots increases from Nr=3N_r=3 to 1212, Goal-FDE increases from 0.1260 to 0.3254 and Goal-SR decreases from 0.9716 to 0.8919, reflecting the increasing difficulty of goal-directed navigation with larger robot teams. Notably, the normalized interaction metrics improve with team size (because these metrics are normalized over the increasing number of robot-entity or robot-robot interactions, rather than reporting the total number of violations): R2E-CR decreases from 0.0639 to 0.0130, R2R-CR from 0.2547 to 0.0590, PIR from 0.0567 to 0.0105, and SVR from 0.0296 to 0.0062. Meanwhile, the aggregate Energy increases from 71.46 to 224.74 as more robots contribute to the overall interaction energy. While the controlled simulations in Sec. V-D provide the primary evidence for scalability, these results demonstrate that the same SAGE pipeline can be extended to multi-robot navigation in real-world scenes with semantic agent annotations. TABLE XII: Guided SDD multi-robot proxy sweep with official semantic annotations. !25 NrN_r Goal-FDE↓ Goal-SR↑ R2E-CR↓ R2R-CR↓ PIR↓ SVR↓ Energy↓ 3 0.1260 0.9716 0.0639 0.2547 0.0567 0.0296 71.46 6 0.2229 0.9338 0.0249 0.1228 0.0215 0.0128 131.63 12 0.3254 0.8919 0.0130 0.0590 0.0105 0.0062 224.74 A-D3 SocialGym-style scale and density To further stress-test SAGE beyond the primary controlled-simulation setting, we evaluate whether the benefits of guidance persist across a broader range of structured and crowded navigation scenarios and under varying robot-team sizes and entity densities. To this end, we construct SocialGym-style environments using our local generator, which produces doorway, hallway, intersection, roundabout, and open-crowd layouts representative of diverse social-navigation scenarios; these experiments do not use rollouts from the SocialGym simulator (see Footnote 4). The corresponding SAGE model is trained for 60 epochs on 8,192 generated episodes and evaluated on 1,024 held-out episodes for each of three seeds. Table XIII evaluates the effect of guidance across different robot-team sizes and environment densities. Across all configurations, guidance consistently reduces R2E-CR, R2R-CR, SVR, and Energy, with only modest increases in Goal-FDE. For example, at Nr=6N_r=6 and Ne=80N_e=80, guidance reduces R2E-CR from 0.0319 to 0.0296, R2R-CR from 0.0970 to 0.0872, SVR from 0.0194 to 0.0176, and Energy from 789.5 to 740.0, while Goal-FDE increases only from 6.462 to 6.533. Similar trends are observed as the robot team grows from Nr=3N_r=3 to 99 with Ne=40N_e=40, demonstrating that the safety and social-compliance benefits of guidance persist across different environment structures, team sizes, and entity densities. TABLE XIII: SocialGym-style scale and density diagnostics (three evaluation seeds). !25 NrN_r NeN_e Guidance Goal-FDE↓ R2E-CR↓ R2R-CR↓ SVR↓ Energy↓ 3 40 w/o 6.675 0.0315 0.1050 0.0188 248.4 3 40 w/ 6.707 0.0292 0.0922 0.0169 233.7 6 40 w/o 6.543 0.0313 0.0907 0.0185 529.6 6 40 w/ 6.573 0.0291 0.0807 0.0168 497.3 6 80 w/o 6.462 0.0319 0.0970 0.0194 789.5 6 80 w/ 6.533 0.0296 0.0872 0.0176 740.0 9 40 w/o 6.518 0.0308 0.0853 0.0184 864.6 9 40 w/ 6.548 0.0288 0.0765 0.0168 815.2