Paper deep dive
IMR: Iterative Mode-World Weighted Regression for Multi-Agent Trajectory Prediction
Honglin Wang, Shiyao Pan, Yun-Fu Liu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/8/2026, 2:52:37 AM
Summary
This paper introduces IMR, a novel framework for multi-agent trajectory prediction in automated driving that addresses the trade-off between mode diversity and prediction accuracy. It proposes an iterative decoder for recurrent, segmented trajectory generation and a mode-world weighted regression loss to mitigate mode collapse while enhancing world ranking and confidence. The method achieves state-of-the-art performance on the Argoverse 2 benchmark.
Entities (11)
Relation Signals (12)
Yun-Fu Liu → authored → IMR
confidence 99% · Yun-Fu Liu EACON Fujian, China yunfuliu@gmail.com
Shiyao Pan → authored → IMR
confidence 99% · Shiyao Pan EACON Fujian, China dcspsy@gmail.com
Honglin Wang → authored → IMR
confidence 99% · Honglin Wang EACON Fujian, China wanghonglin921@gmail.com
EACON → affiliatedwith → Honglin Wang
confidence 95% · Honglin Wang EACON Fujian, China
IMR → evaluatedon → Argoverse 2
confidence 95% · Experimental results show the proposed method ranks first in the Argoverse 2 multi-agent motion forecasting benchmark
IMR → solves → Multi-Agent Motion Prediction
confidence 95% · Multi-agent motion prediction is essential for automated vehicles to understand the intentions of surrounding vehicles.
Iterative Decoder → improves → Prediction Accuracy
confidence 94% · Furthermore, the proposed iterative decoder improves prediction accuracy by recurrently and segmentally generating trajectories.
IMR → uses → Iterative Decoder
confidence 94% · Furthermore, the proposed iterative decoder improves prediction accuracy by recurrently and segmentally generating trajectories.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent motion prediction is essential for automated vehicles to understand the intentions of surrounding vehicles. However, previous prediction-based and anchor-based methods have limitations in mode diversity and prediction accuracy, respectively. These limitations may cause inadequate safety assessments and behavioral deviations in automated vehicles. To address this issue, a mode-world weighted regression loss is proposed to bridge the gap between these features. Specifically, this approach mitigates mode collapse while simultaneously improving world ranking and top-1 confidence. Furthermore, the proposed iterative decoder improves prediction accuracy by recurrently and segmentally generating trajectories. Experimental results show the proposed method ranks first in the Argoverse 2 multi-agent motion forecasting benchmark against other methods.
Tags
Links
- Source: https://arxiv.org/abs/2607.05705v1
- Canonical: https://arxiv.org/abs/2607.05705v1
Trouble viewing inline? Open PDF directly →
Full Text
16,095 characters extracted from source content.
Expand or collapse full text
IMR: Iterative Mode-World Weighted Regression for Multi-Agent Trajectory Prediction Honglin Wang EACON Fujian, China wanghonglin921@gmail.com Shiyao Pan EACON Fujian, China dcspsy@gmail.com Yun-Fu Liu EACON Fujian, China yunfuliu@gmail.com Abstract Multi-agent motion prediction is essential for automated vehicles to understand the intentions of surrounding vehicles. However, previous prediction-based and anchor-based methods have limitations in mode diversity and prediction accuracy, respectively. These limitations may cause inadequate safety assessments and behavioral deviations in automated vehicles. To address this issue, a mode-world weighted regression loss is proposed to bridge the gap between these features. Specifically, this approach mitigates mode collapse while simultaneously improving world ranking and top-1 confidence. Furthermore, the proposed iterative decoder improves prediction accuracy by recurrently and segmentally generating trajectories. Experimental results show the proposed method ranks first in the Argoverse 2 multi-agent motion forecasting benchmark against other methods. 1 Introduction Motion prediction is crucial for autonomous driving technology, establishing the foundation for achieving high reliability and safety in autonomous vehicles [2], [3]. Through accurate prediction of future trajectories of road entities (e.g., vehicles, bicycles, pedestrians), it facilitates real-time environmental perception and dynamic analysis. This capability enhances path planning and obstacle avoidance precision, mitigates accident risks, and improves overall driving safety, contributing to smoother navigation. Previous prediction-based methods (e.g., QCNet [10] and Forecast-MAE [1]) in multi-agent motion prediction are prone to mode collapse in complex scenarios. Anchor-based approaches (e.g., MTR [5] and TNT [9]) mitigate this issue at the expense of prediction accuracy. To resolve this trade-off, we propose a mode-world weighted regression loss within the prediction-based framework. Experiments demonstrate enhanced mode diversity without compromising accuracy, thereby bridging the performance gap across paradigms. Crucially, it mitigates mode collapse while concurrently improving the accuracy of world ranking and top-1 confidence. Furthermore, we note that the previous state-of-the-art method QCNeXt [11] on the Argoverse 2 [7] multi-agent motion forecasting benchmark, employs a proposal-refinement decoding architecture. This method further refines proposed trajectories to generate more accurate predictions. However, when initial trajectories substantially deviate from ground truth, refinement layers struggle to effectively capture correct offsets due to excessive errors. Consequently, we propose an iterative decoder, and each iteration directly outputs trajectory coordinates rather than offsets. Simultaneously, encoded features, decoded outputs, and predicted trajectory from each iteration is propagated to next iteration, maximizing intermediate information utilization. This recurrent and segmented method facilitates iterative trajectory optimization. Figure 1: The architecture of the proposed iterative decoder. The arrows shown around the top-left corner indicate the transmission of decoding features between different decoding stages. Figure 2: Details of the interaction block shown in Fig. 1, in which the number of layers is denoted as L. 2 Approach The proposed method is introduced in this section. First, the scene representation approach and encoder design are described. Second, the architecture and components of the iterative decoder are elaborated. Finally, the mode-world weighted regression loss is presented. 2.1 Scene representation and encoder We treat the historical trajectory of each agent at every timestep as an independent element. Using the element as the origin, we construct a polar coordinate system and select agents within a preset radius as interaction objects. For each interaction pair, we compute 3D spatial relationships (relative distance, azimuth, orientation) to construct a dynamic attention weighted matrix. The graph attention network-based encoder models each element and their interactions as a graph structure. Each element is represented as a node, with inter-node connections denoting interaction weights. By modeling multi-source interactions, this captures implicit relationships between different elements. 2.2 Iterative decoder Figure 1 illustrates the architecture of the iterative decoder. The agent and map features represent the outputs from the encoder. The iterative decoder executes N iterations. Each iteration comprises S decoding stages designed to progressively decode trajectory segments, which are then concatenated into a full trajectory. The predicted trajectory from each iteration is processed by a temporal attention layer and then combined with the decoding feature to serve as input for the subsequent iteration. As depicted in Fig. 2, each interaction block comprises L stacked interaction layers and it outputs predicted trajectory through a prediction head as formulated below, Fi∈RK×D,i=0F_i∈ R^K× D,\>i=0 (1) Fi=Fi−1′+LSTM(yi−1),i>0F_i=F^ _i-1+LSTM(y_i-1),\>i>0 (2) where FiF_i and Fi′F^ _i denote the input and output features of the i-th iteration, respectively. K is the number of predicted world, D is the hidden size, LSTMLSTM indicates the LSTM layer, and yiy_i represents the predicted trajectory of the i-th iteration. In each iteration, predicted trajectory is executed in segments. After the prediction of each segment of the trajectory is completed each time, the segments are connected to generate the complete trajectory. The equation shows below, Fij′=GAT(Fi,A,M),j=0F^ j_i=GAT(F_i,A,M),\>j=0 (3) Fij′=GAT(Fij′−1,A,M),j>0F^ j_i=GAT(F^ j-1_i,A,M),\>j>0 (4) where GATGAT denotes the hierarchical multivariate Graph Attention Networks layer, A and M represent the encoding feature of agents and maps, respectively. Fij′F^ j_i denotes the decoding feature for the j-th decoding stage in the i-th iteration. Segmented processing can break down the complex trajectory prediction task into multiple simple sub-tasks, making the prediction process more controllable. Meanwhile, each prediction segment is known with the result of the previous segment, forming a dynamic interaction relationship. The iterative process model continuously explores the potential patterns in the data by constantly reusing the complete trajectory generated last time, and further optimizes the prediction results. It is worth noting that we set the output of each decoding stage to the position coordinate value of the predicted trajectory instead of the offset relative to the previous decoding stage. This helps to avoid the uncorrectable consequences of the incorrect prediction of the previous decoding stage on the current decoding stage. Method avgMinFDE6avgMinFDE_6 avgMinFDE1avgMinFDE_1 actorMR6actorMR_6 avgMinADE6avgMinADE_6 avgMinADE1avgMinADE_1 avgBrierMinFDE6avgBrierMinFDE_6 actorCR6actorCR_6 LRP 1.21 2.57 0.16 0.58 1.05 1.85 0.01 DONUT 1.17 2.73 0.15 0.58 1.13 1.83 0.01 SEPT [4] 1.14 3.22 0.15 0.55 1.27 1.80 0.01 LOF [6] 1.25 2.34 0.18 0.58 0.96 1.68 0.02 QCNeXt [11] 1.02 2.29 0.13 0.50 0.94 1.65 0.01 IMR (ours) 1.08 2.29 0.15 0.51 0.92 1.59 0.01 Table 1: Quantitative results on the Argoverse 2 multi-agent motion forecasting benchmark. Method MinFDE6MinFDE_6 MinFDE1MinFDE_1 MinADE6MinADE_6 MinADE1MinADE_1 MR6MR_6 MR1MR_1 brier-MinFDE6brier-MinFDE_6 DeMo [8] 1.11 3.70 0.60 1.49 0.12 0.55 1.73 Polaris 1.11 3.70 0.61 1.53 0.12 0.54 1.71 NDPNet 1.09 3.71 0.58 1.47 0.13 0.53 1.71 SEPT++ [4] 1.09 3.51 0.57 1.41 0.13 0.51 1.65 LOF [6] 1.07 3.63 0.58 1.46 0.12 0.51 1.63 IMR (ours) 1.08 3.54 0.57 1.42 0.12 0.51 1.63 Table 2: Quantitative results on the Argoverse 2 single agent motion forecasting benchmark. (a) Case 1: mode-wise loss (b) Case 1: world-wise loss (c) Case 1: mode-world loss (d) Case 2: mode-wise loss (e) Case 2: world-wise loss (f) Case 2: mode-world loss Figure 3: Visualization comparison of different types of regression loss on Argoverse 2 validation set. Each row represents a different case, and the results of three types of loss involving mode-wise, world-wise, and the proposed mode-world regression loss are shown in different columns. 2.3 Mode-world weighted regression loss In order to generate the world-wise confidence scores to adapt to the joint trajectory prediction task, each world should contain the joint trajectories of all agents as well as a corresponding confidence score to this world. During the training process, we use laplace loss and focal loss as the regression loss function and classification loss function respectively. Among them, the final regression loss value is obtained by the weighted method. The mode-wise regression loss and the world-wise regression loss are calculated together and weighted. The equation is formulated as follows, Lreg=ωmode⋅Lregmode+ωworld⋅LregworldL_reg= _mode· L^mode_reg+ _world· L^world_reg (5) where LregL_reg represents the final regression loss value, ωmode _mode and ωworld _world respectively represent the mode-wise and world-wise regression loss weights, LregmodeL^mode_reg and LregworldL^world_reg represent the mode-wise regression loss function value and the world-wise regression loss value. For mode regression, a mode winner-takes-all strategy is adopted to minimize losses. The winner is defined as the best mode-wise trajectory in each agent. Among them, the selection method of the optimal mode-wise trajectory is obtained by weighting the minimum maximum displacement error and the minimum average displacement error, as shown in the following equation: m=ωADE⋅ADE+ωMDE⋅MDEm= _ADE· ADE+ _MDE· MDE (6) where m represents the optimal mode-wise trajectory. ωADE _ADE and ωMDE _MDE represent the weights corresponding to the two kinds of errors, and ADEADE and MDEMDE are the average and maximum displacement error, respectively. 3 Experiments 3.1 Implementation details The length of the predicted trajectory is supposed to 6s as defined in Argoverse 2. The hidden size D is set to 128, and the number of iterations N is set to 6. For each iteration, the predicted trajectory is set to 3 segments at equal time intervals in a chronological order, and the number of stacked interaction layers L is set to 3 as well. Notably, the initial decoding feature is randomly initialized for a cold start. The number of predicted worlds K is 6. ωmode _mode and ωworld _world are 0.5 and 1, respectively. We train the model using AdamW for 60 epochs on the Argoverse 2 motion forecasting dataset with batch size 32. The initial learning rate is 5×10−45× 10^-4, with weight decay and dropout both at 0.1. Through cosine annealing, the learning rate decays to 0. 3.2 Ensemble We train 10 models with different random seeds, producing a total of 60 predicted worlds. These world-wise predictions are then integrated using the weighted k-means algorithm introduced in QCNeXt [11]. 3.3 Quantitative results We compare our approach to other methods on the Argoverse 2 multi-agent motion forecasting benchmark. As shown in Table 1, our method achieves a 0.06 improvement in avgBrierMinFDE6avgBrierMinFDE_6 over the previous SOTA QCNeXt. Furthermore, we also verified the effectiveness of the proposed method in the Argoverse 2 single agent motion forecasting benchmark as listed in Table 2. Our model achieves a competitive performance in contrast to LOF [6] as well. 3.4 Qualitative results Figure 3 presents a visual comparison of different types of regression losses. In these figures, green lines depict historical trajectories of orange-colored interested agents. Orange and blue lines indicate ground truth and the predicted future trajectories, respectively. As shown in the mode-wise loss results of Figs. 3(a) and (d), the predicted trajectories exhibit high diversity yet a lower accuracy is introduced due to the length inconsistency with the ground truth. Figures 3(b) and (e) present the results of the world-wise regression and they reveal significant mode collapse as represented by absence of predictions matching ground truth. In contrast, the proposed mode-world weighted regression loss mitigates mode collapse while improving prediction accuracy as shown in Figs. 3(c) and (f). 4 Conclusions In this work, a multi-agent motion prediction method is proposed to address insufficient mode diversity and low prediction accuracy. Experimental results demonstrate that the weighted regression loss mitigates mode collapse and resolves the diversity-accuracy trade-off. Furthermore, it enhances world ranking accuracy and top-1 confidence. By generating trajectories through a recurrent and segmented method, the decoder enables the model to learn from previous information, gradually improving the prediction accuracy. However, this approach incurs high computational complexity in decoding, limiting real-time applicability. As demonstrated in experimental results, our method ranks first on the Argoverse 2 multi-agent motion forecasting benchmark and achieves first-tier performance on the single-agent benchmark. References [1] J. Cheng, X. Mei, and M. Liu (2023) Forecast-mae: self-supervised pre-training for motion forecasting with masked autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 8679–8689. Cited by: §1. [2] R. Huang, G. Zhuo, L. Xiong, S. Lu, and W. Tian (2023) A review of deep learning-based vehicle motion prediction for autonomous driving.. Sustainability (2071-1050) 15 (20). Cited by: §1. [3] Y. Huang, J. Du, Z. Yang, Z. Zhou, L. Zhang, and H. Chen (2022) A survey on trajectory-prediction methods for autonomous driving. IEEE Transactions on Intelligent Vehicles 7 (3), p. 652–674. Cited by: §1. [4] Z. Lan, Y. Jiang, Y. Mu, C. Chen, and S. E. Li (2023) Sept: towards efficient scene representation learning for motion prediction. arXiv preprint arXiv:2309.15289. Cited by: Table 1, Table 2. [5] S. Shi, L. Jiang, D. Dai, and B. Schiele (2022) Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems 35, p. 6531–6543. Cited by: §1. [6] M. Wang, X. Ren, R. Jin, M. Li, X. Zhang, C. Yu, M. Wang, and W. Yang (2024) FutureNet-lof: joint trajectory prediction and lane occupancy field prediction with future context encoding. arXiv preprint arXiv:2406.14422. Cited by: Table 1, Table 2, §3.3. [7] B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. K. Pontes, et al. (2023) Argoverse 2: next generation datasets for self-driving perception and forecasting. arXiv preprint arXiv:2301.00493. Cited by: §1. [8] B. Zhang, N. Song, and L. Zhang (2024) Decoupling motion forecasting into directional intentions and dynamic states. arXiv preprint arXiv:2410.05982. Cited by: Table 2. [9] H. Zhao, J. Gao, T. Lan, C. Sun, B. Sapp, B. Varadarajan, Y. Shen, Y. Shen, Y. Chai, C. Schmid, et al. (2021) Tnt: target-driven trajectory prediction. In Conference on Robot Learning, p. 895–904. Cited by: §1. [10] Z. Zhou, J. Wang, Y. Li, and Y. Huang (2023) Query-centric trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 17863–17873. Cited by: §1. [11] Z. Zhou, Z. Wen, J. Wang, Y. Li, and Y. Huang (2023) Qcnext: a next-generation framework for joint multi-agent trajectory prediction. arXiv preprint arXiv:2306.10508. Cited by: §1, Table 1, §3.2.