Paper deep dive
IMM-based Multiple Object Tracking using a State Prediction Neural Network
Chan-Bin Lim, Dong-Hee Paek, Seung-Hyun Kong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/16/2026, 7:51:41 AM
Summary
This paper proposes PR-IMM, a multiple object tracking method that integrates a Transformer-based state prediction model (PR) into the Interacting Multiple Model (IMM) framework. The PR model uses radar Doppler measurements to predict object displacement, addressing the limitations of traditional physics-based models (CV, CA, CT) in handling nonlinear motion. Experimental results on the RadarScenes dataset demonstrate that PR-IMM significantly reduces position-estimation error and ID switches compared to standard IMM and the standalone PR model.
Entities (9)
Relation Signals (8)
PR-IMM → integrates → IMM
confidence 95% · PR model is integrated into the IMM as a mode
PR-IMM → uses → Transformer
confidence 95% · The proposed method employs a transformer-based PRediction model (PR)
PR-IMM → outperforms → IMM
confidence 92% · PR-IMM reduces position-estimation error by 57.3% over the IMM
PR-IMM → evaluatedon → RadarScenes
confidence 90% · Using the large-scale radar point-cloud dataset RadarScenes, we demonstrate that the proposed method significantly improves
PR-IMM → includes → Constant Acceleration (CA)
confidence 90% · PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models
PR-IMM → includes → Constant Turn Rate (CT)
confidence 90% · PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models
PR-IMM → includes → Constant Velocity (CV)
confidence 90% · PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models
Radar → utilizes → Doppler Effect
confidence 85% · Radar maintains detection performance... and can measure relative velocity through the Doppler effect
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Object tracking is essential for autonomous vehicles to avoid obstacles and plan routes. Radar maintains detection performance even in adverse weather and can measure relative velocity through the Doppler effect, making it well suited for object tracking. In this paper, we propose a data-driven state PRedictor-based Interacting Multiple Model tracking method (PR-IMM) that improves nonlinear object-motion representation while preserving the stability and interpretability of physics-based motion models. The proposed method employs a transformer-based PRediction model (PR) that incorporates radar Doppler measurements to predict object displacement. The PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models, and their prior positions are dynamically combined according to the mode probabilities. Experimental results show that PR-IMM reduces position-estimation error by 57.3% over the IMM and by 16.5% over the PR, while reducing ID switches by 25.3% and improving IDF1 by 9.6% over the IMM.
Tags
Links
- Source: https://arxiv.org/abs/2609.13307v1
- Canonical: https://arxiv.org/abs/2609.13307v1
Trouble viewing inline? Open PDF directly →
Full Text
43,473 characters extracted from source content.
Expand or collapse full text
IMM-based Multiple Object Tracking using a State Prediction Neural Network 1st Chan-Bin Lim Robotics Program Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea, 34051 2nd Dong-Hee Paek Mechanical Engineering Research Institute Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea, 34141 3rd Seung-Hyun Kong* Cho Chun Shik Graduate School of Mobility Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea, 34051 †thanks: *Corresponding author: skong@kaist.ac.kr Abstract Object tracking is essential for autonomous vehicles to avoid obstacles and plan routes. Radar maintains detection performance even in adverse weather and can measure relative velocity through the Doppler effect, making it well suited for object tracking. In this paper, we propose a data-driven state PRedictor-based Interacting Multiple Model tracking method (PR-IMM) that improves nonlinear object-motion representation while preserving the stability and interpretability of physics-based motion models. The proposed method employs a Transformer-based PRediction model (PR) that incorporates radar Doppler measurements to predict object displacement. The PR model is integrated into the IMM as a mode alongside the CV, CA, and CT motion models, and their prior positions are dynamically combined according to the mode probabilities. Experimental results show that PR-IMM reduces position-estimation error by 57.3% over the IMM and by 16.5% over the PR, while reducing ID switches by 25.3% and improving IDF1 by 9.6% over the IMM. Index Terms: Radar, multiple object tracking, Kalman filter, interacting multiple model, Transformer. I Introduction Object tracking detects objects from sensor data and continuously estimates their locations. With the development of autonomous driving and advanced driver assistance systems (ADAS), it has become a core technology for safe driving decisions[1, 2, 3]. Cameras, LiDAR, radar, and other sensors are widely used for this purpose. Among them, radar has attracted particular attention for object tracking because it maintains robust detection performance in adverse weather such as rain and fog and can directly measure relative velocity through the Doppler effect[4, 5, 6]. Accordingly, various approaches have been proposed for radar-based object tracking, and early studies mainly relied on motion models implemented with Kalman filters (KF)[7, 8]. Representative models include Constant Velocity (CV), Constant Acceleration (CA), and Constant Turn Rate (CT). Each assumes a specific motion pattern and defines a state-transition model to predict the next object position. Because a single motion model cannot flexibly represent diverse motion patterns, the Interacting Multiple Model (IMM), which dynamically combines multiple motion models according to their mode probabilities, has been widely adopted[9]. However, motion-model-based approaches depend on predefined motion patterns and therefore have limited ability to represent the nonlinear object motions that occur in real road environments[10]. To overcome these limitations, recent studies have actively applied deep-learning-based trajectory prediction models to object tracking[11]. In particular, Transformer-based models[12] effectively model long-range dependencies in time-series data through self-attention and have shown strong performance in predicting future positions from past trajectories[13]. Because such data-driven models do not assume a particular motion pattern, they can learn nonlinear motions that are difficult to describe with CV, CA, or CT models. On the other hand, their predictions can become unstable when observation noise is large or when the input lies outside the training distribution[14]. Motion-model-based and data-driven approaches therefore have complementary strengths. Data-driven models can directly learn diverse motion patterns without predefining dynamic assumptions, while the Kalman filters underlying physics-based motion models provide stable state estimation by explicitly handling measurement uncertainty through covariance. Combining these strengths is thus a promising way to improve object-tracking performance. From this perspective, we propose a PR (PRediction model)-IMM tracker that integrates a Transformer-based prediction model into the IMM framework. The proposed method uses a data-driven prior state from a Transformer predictor learned from radar measurements together with physics-based prior states from CV, CA, and CT models. These priors are probabilistically combined through IMM mode probabilities so that the tracker can adaptively exploit the most suitable information under each motion condition, targeting robust tracking performance that cannot be achieved by a single motion model or a data-driven predictor alone. The main contributions of this paper are as follows: • We propose a Transformer-based network that considers radar Doppler measurements for accurate state prediction. • We propose a PR-IMM multiple-object-tracking framework in which a data-driven Transformer predictor and physics-based CV, CA, and CT motion models operate as IMM modes. • Using the large-scale radar point-cloud dataset RadarScenes[15], we demonstrate that the proposed method significantly improves both position-estimation accuracy and object-tracking performance over single models and a conventional IMM (CV+CA+CT). The remainder of this paper is organized as follows. Section I reviews related work. Section I describes the proposed PR-IMM tracker in detail. Section IV presents the experimental setup and results, and Section V concludes the paper. I Related Works This section reviews representative object-tracking methods based on Kalman filters, IMM, deep learning, and combinations of Kalman filtering and neural networks. I-A Kalman-Filter-Based Object Tracking CV, CA, and CT motion models are among the most common choices in Kalman-filter-based tracking. SORT combines a CV-model Kalman filter with the Hungarian algorithm for real-time multiple-object tracking[16], while CA-model Kalman filtering has also been used to estimate vehicle motion states from automotive radar measurements[17]. The CV model is robust for constant-speed motion but weak under abrupt acceleration. The CA model mitigates this limitation but can accumulate larger inertial errors in the lateral direction. The CT model is advantageous for turning maneuvers but may exhibit larger errors during straight constant-speed motion. Thus, a single motion model is optimized for a particular motion assumption and inevitably involves a trade-off; prediction errors can increase in real traffic where objects combine straight motion, acceleration, deceleration, and turning. I-B IMM-Based Object Tracking IMM-based tracking commonly combines CV, CA, and CT models. Previous studies have applied IMM to 3D multiple-object tracking to reduce ID switches[18] and to improve tracking under diverse motion patterns[19]. By dynamically weighting several motion models according to their mode probabilities, IMM can handle a broader range of motions than a single model. Nevertheless, when an object performs a maneuver outside the scope of all predefined models, the likelihoods of all modes decrease. The subsequent weighted combination is then formed from models that each approximate the true motion poorly, resulting in increased state-estimation error. I-C Deep-Learning-Based Object Tracking Deep-learning-based object-tracking methods frequently predict the next position with a neural network and then use the prediction for data association[20, 21]. Recurrent networks such as LSTMs were initially used as predictors that replace the motion model of a Kalman filter[22]. More recently, Transformer-based predictors have attracted attention because self-attention can capture longer-range temporal dependencies more effectively than recurrent models[13]. However, these predictors output a future position without explicitly providing how reliable that prediction is. Consequently, when a prediction deviates substantially from the actual observation, no intrinsic uncertainty-based mechanism is available to correct it, and the error can accumulate throughout tracking. I-D Existing Combinations of Kalman Filtering and Deep Learning Recent work has also combined Kalman filtering with deep learning. Representative examples include a neural network coupled to the Kalman-filter state model[23] and an LSTM-based error-compensation method for Kalman filtering[24]. In these approaches, however, deep learning is used primarily as an auxiliary component of the Kalman filter. When the learned prediction becomes inaccurate, there is no independent alternative mode that can replace or compensate for it, which can degrade tracking performance. I PR-IMM-Based Multiple Object Tracking This section presents the proposed PR-IMM tracker, which integrates a Transformer-based prediction model and Kalman-filter-based motion models into a single IMM framework. The Transformer predictor learns nonlinear object motion from data, while the CV, CA, and CT models provide physics-based motion priors that can compensate when the learned prediction becomes unreliable. The goal is to achieve robust position estimation even under noisy observations. I-A Overall System Pipeline The proposed object-tracking pipeline is shown in Fig. 1. We assume that object detection has already been completed[25, 26] and focus on the subsequent tracking stage. Instead of running a detector, we use the point-wise class IDs and track IDs provided by RadarScenes[15]. Points with the same track ID are grouped into one object point cloud, and the point values are averaged to obtain an object-level radar measurement vector. At time step k, this vector is defined as k=[xk,yk,vr,k,RCSk]T,m_k= [x_k,\;y_k,\;v_r,k,\;RCS_k ]^T, (1) where xkx_k and yky_k denote the longitudinal and lateral distances of the object, respectively, vr,kv_r,k is radial velocity in the radar coordinate system, and RCSkRCS_k is the Radar Cross Section. Fig. 1: Overall framework of PR-IMM. I-B Kalman-Filter Prediction with Motion Models This subsection describes the Kalman-filter prediction stage based on the CV, CA, and CT motion models. All three models use the two-dimensional object position as the Kalman-filter observation, with k=[xk,yk]T.z_k= [x_k,\;y_k ]^T. (2) Each motion model i∈CV,CA,CTi∈\CV,CA,CT\ uses its own state vector and transition function. The Kalman prediction consists of state prediction i,k−=fi(i,k−1),x^-_i,k=f_i\! (x_i,k-1 ), (3) and covariance prediction i,k−=i,ki,k−1i,kT+i,P^-_i,k=F_i,kP_i,k-1F_i,k^T+Q_i, (4) where i,k−x^-_i,k is the prior state predicted by model i from the previous posterior state i,k−1x_i,k-1. The covariance i,k−P^-_i,k represents the uncertainty of the prior state after incorporating the predefined process-noise covariance iQ_i. The superscript (⋅)−(·)^- denotes a value before the Kalman correction. For CV and CA, fi()=if_i(x)=F_ix is linear, whereas CT uses the nonlinear transition in Eqs. (10)–(13) and propagates covariance using its Jacobian. I-B1 CV Model The CV model assumes straight-line motion at constant velocity. Its state vector contains longitudinal and lateral positions and velocities: CV,k−=[xk−,vx,k−,yk−,vy,k−]T.x^-_CV,k= [x_k^-,\;v_x,k^-,\;y_k^-,\;v_y,k^- ]^T. (5) The initial velocity components vxv_x and vyv_y are initialized from the measured radar radial velocity vrv_r and subsequently refined by Kalman correction. Using the time interval Δtk t_k, the state-transition matrix is CV,k=[1Δtk000100001Δtk0001].F_CV,k= bmatrix1& t_k&0&0\\ 0&1&0&0\\ 0&0&1& t_k\\ 0&0&0&1 bmatrix. (6) I-B2 CA Model The CA model assumes constant acceleration and augments the CV state with acceleration components: CA,k−=[xk−,vx,k−,ax,k−,yk−,vy,k−,ay,k−]T.x^-_CA,k= [x_k^-,\;v_x,k^-,\;a_x,k^-,\;y_k^-,\;v_y,k^-,\;a_y,k^- ]^T. (7) The initial accelerations are set to zero and are subsequently corrected using measurements. The corresponding transition matrix is CA,k=[1Δtk12Δtk200001Δtk0000010000001Δtk12Δtk200001Δtk000001].F_CA,k= bmatrix1& t_k& 12 t_k^2&0&0&0\\ 0&1& t_k&0&0&0\\ 0&0&1&0&0&0\\ 0&0&0&1& t_k& 12 t_k^2\\ 0&0&0&0&1& t_k\\ 0&0&0&0&0&1 bmatrix. (8) I-B3 CT Model The CT model assumes a constant turn rate ω and uses CT,k−=[xk−,vx,k−,yk−,vy,k−,ωk−]T.x^-_CT,k= [x_k^-,\;v_x,k^-,\;y_k^-,\;v_y,k^-,\; _k^- ]^T. (9) The initial turn rate is set to zero and is later refined through Kalman correction. Defining ϕk=ωk−1Δtk _k= _k-1 t_k, the nonlinear transition for ωk−1≠0 _k-1≠ 0 is xk−=xk−1+sinϕkωk−1vx,k−1−1−cosϕkωk−1vy,k−1,x_k^-=x_k-1+ _k _k-1v_x,k-1- 1- _k _k-1v_y,k-1, (10) vx,k−=cosϕkvx,k−1−sinϕkvy,k−1,v_x,k^-= _k\,v_x,k-1- _k\,v_y,k-1, (11) yk−=yk−1+1−cosϕkωk−1vx,k−1+sinϕkωk−1vy,k−1,y_k^-=y_k-1+ 1- _k _k-1v_x,k-1+ _k _k-1v_y,k-1, (12) vy,k−=sinϕkvx,k−1+cosϕkvy,k−1.v_y,k^-= _k\,v_x,k-1+ _k\,v_y,k-1. (13) When ωk−1 _k-1 is close to zero, a Taylor-series approximation is used to avoid numerical divergence, and covariance propagation is performed using the Jacobian of Eqs. (10)–(13). I-C Transformer-Based Prediction This subsection describes the Transformer-based prediction model, denoted PR, which predicts the displacement of an object at the next time step from radar observations. The input feature at historical index l is l=[xl−xk−1,yl−yk−1,vr,l,RCSl,tl−tk]T,s_l= [x_l-x_k-1,\;y_l-y_k-1,\;v_r,l,\;RCS_l,\;t_l-t_k ]^T, (14) where l=k−L,…,k−1l=k-L,…,k-1. The PR model uses the previous L radar observations before prediction time k, with a maximum sequence length of L=5L=5. Each ls_l consists of the relative position with respect to the most recent observation at k−1k-1, radial velocity, RCS, and relative time to prediction time k. Fig. 2: Architecture of the PR model. I-C1 PR Model Architecture Figure 2 shows the proposed PR architecture. To make the model learn motion patterns rather than absolute scene coordinates, all historical positions are represented relative to the latest observation k−1z_k-1. Radial velocity and RCS provide motion and reflection characteristics of the object, and the actual time interval is used to estimate displacement precisely. Because a Transformer processes an input sequence in parallel, temporal order is not explicitly encoded. Positional embeddings are therefore added to each historical input so that the network receives the temporal ordering of the past trajectory. Two learnable query tokens, qxq_x and qyq_y, are also appended to the input to predict the x- and y-direction displacements at the next time step. The input sequence and query tokens pass through a Transformer encoder composed of Multi-Head Attention and Feed-Forward layers with residual connections. From the final outputs corresponding to the two query tokens, the model predicts Δ^PR,k=[Δx^k,Δy^k]T p_PR,k=[ x_k, y_k]^T. I-C2 PR Filter The PR prior position at time k is obtained by adding the predicted displacement to the posterior PR position at time k−1k-1: PR,k−=PR,k−1+Δ^PR,k.p^-_PR,k=p_PR,k-1+ p_PR,k. (15) As in the CV, CA, and CT modes, the covariance of the PR prior state represents prediction uncertainty and is defined as PR,k−=PR,k−1+PR.P^-_PR,k=P_PR,k-1+Q_PR. (16) I-D Prior State Fusion This subsection explains how the motion-model predictions in Section I-B and the PR prediction in Section I-C are combined. PR-IMM contains four modes: PR, CV, CA, and CT. Although their complete state vectors have different dimensions, the position component of each prior state is projected through the observation matrix iH_i and used for state fusion: i,k−=ii,k−=[xi,k−,yi,k−]Tp^-_i,k=H_ix^-_i,k=[x^-_i,k,y^-_i,k]^T. I-D1 Input Mixing Before the PR, CV, CA, and CT modes predict the next state, each performs input mixing using the previous mode probabilities and mode-transition probabilities. PR-IMM maintains the modes independently, but an object’s motion pattern can change over time. To preserve continuity when the dominant mode changes, the posterior positions and position covariances of all modes at the previous time step are combined and used as the initial values for each destination mode. Thus, mode j begins prediction from the mixed position ¯j,k−1 p_j,k-1, which reflects not only its own previous state but also the posterior states of other modes. The overbar denotes a mode-weighted mean. Let πij _ij denote the transition probability from mode i to mode j, and let μi,k−1 _i,k-1 be the posterior mode probability at the previous time step. The prior probability that mode j is selected at the current time step is cj,k=∑r=1Mπrjμr,k−1,c_j,k= _r=1^M _rj _r,k-1, (17) where M is the total number of modes, namely PR, CV, CA, and CT. The mixing probability that quantifies the contribution of previous mode i to the initial state of current mode j is βij,k−1=πijμi,k−1cj,k. _ij,k-1= _ij _i,k-1c_j,k. (18) Using Eq. (18), the mixed initial position and position covariance of mode j are ¯j,k−1=∑i=1Mβij,k−1i,k−1, p_j,k-1= _i=1^M _ij,k-1p_i,k-1, (19) ¯pj,k−1=∑i=1Mβij,k−1[pi,k−1+(i,k−1−¯j,k−1)×(i,k−1−¯j,k−1)T]. split P^p_j,k-1= _i=1^M _ij,k-1 [P^p_i,k-1+ (p_i,k-1- p_j,k-1 )\\ × (p_i,k-1- p_j,k-1 )^T ]. split (20) Equation (20) includes the dispersion caused by disagreement among the mode positions, so the initial position covariance becomes more conservative when the modes disagree strongly. Because the complete states of the modes have different dimensions and structures, input mixing is performed only for the common position and position covariance. Each mode independently preserves its non-position state variables. The mixed full state is therefore written as ¯j,k−1=[¯j,k−1,j,k−1]T, x_j,k-1= [ p_j,k-1,\;h_j,k-1 ]^T, (21) where j,k−1h_j,k-1 contains the non-position states of destination mode j at time k−1k-1. To maintain consistency between the mixed position covariance and the full covariance, the cross-covariance between position and the remaining states and the covariance of the remaining states are updated using the conditional relationship of the original covariance. Before input mixing, the full covariance is partitioned as j,k−1=[pp,j,k−1ph,j,k−1hp,j,k−1hh,j,k−1],P_j,k-1= bmatrixP_p,j,k-1&P_ph,j,k-1\\ P_hp,j,k-1&P_h,j,k-1 bmatrix, (22) where ppP_p is the position covariance, ph=hpTP_ph=P_hp^T is the position/non-position cross-covariance, and hhP_h is the covariance of the remaining states. We define j,k−1=hp,j,k−1(pp,j,k−1)†,B_j,k-1=P_hp,j,k-1 (P_p,j,k-1 ) , (23) j,k−1=hh,j,k−1−j,k−1pp,j,k−1j,k−1T,C_j,k-1=P_h,j,k-1-B_j,k-1P_p,j,k-1B^T_j,k-1, (24) and reconstruct the mixed full covariance as ¯j,k−1=[¯j,k−1p¯j,k−1pj,k−1Tj,k−1¯j,k−1pj,k−1+j,k−1¯j,k−1pj,k−1T]. 21705675$ P_j,k-1= bmatrix P^p_j,k-1& P^p_j,k-1B^T_j,k-1\\ B_j,k-1 P^p_j,k-1&C_j,k-1+B_j,k-1 P^p_j,k-1B^T_j,k-1 bmatrix$. (25) Here, (⋅)†(·) denotes the pseudoinverse. I-D2 PR-IMM Prior State Fusion Starting from the mixed initial values in Eqs. (21) and (25), each mode independently predicts its prior state. The PR-IMM prior position at time k is then obtained by weighting the predicted position of each mode by its current prior mode probability: PR−IMM,k−=∑i=1Mci,ki,k−.p^-_PR\!-\!IMM,k= _i=1^Mc_i,kp^-_i,k. (26) The fused prior position in Eq. (26) is used to match tracks and observations in the data-association stage. I-E Data Association This subsection describes how the PR-IMM prior state is used to associate existing tracks with new observations. Radar-based multiple-object tracking receives multiple observations at each time step, making correct observation-to-track assignment essential. For track a and observation j, we define an association cost using the Euclidean distance between the PR-IMM prior position PR−IMM,a,k−p^-_PR\!-\!IMM,a,k and observation position j,k=[xj,k,yj,k]Tz_j,k=[x_j,k,y_j,k]^T together with their radial-velocity difference: da,j,k= d_a,j,k= ‖j,k−PR−IMM,a,k−‖2 \|z_j,k-p^-_PR\!-\!IMM,a,k \|_2 (27) +λvmin(|vr,j,k−vr,alast|,vmax). + _v \! ( |v_r,j,k-v_r,a^last |,v_ ). Here, vr,alastv_r,a^last is the radial velocity of the radar measurement most recently associated with track a. This term helps distinguish spatially close objects moving at different velocities. To prevent velocity from dominating the total association cost, the velocity difference is capped by vmaxv_ and weighted by λv _v. We use λv=0.1 _v=0.1 and vmax=5v_ =5 m/s. Because RCS varies substantially with object pose, viewing angle, and material composition, it is used only as an input feature to the PR model and is not included in the data-association cost. For each observation j, Eq. (27) is used to construct a cost matrix between tracks and observations belonging to the same object class. The Hungarian algorithm then performs a global one-to-one assignment. Associated tracks proceed to the posterior-state-fusion stage using the assigned observation, while observations not associated with any existing track initialize new tracks. I-F Posterior State Fusion This subsection explains how each mode is corrected after data association, how the posterior mode probabilities are updated, and how the final position is obtained. I-F1 Mode Probability Update Once an observation is assigned to a track, each mode computes the innovation i,k ν_i,k between its prior prediction and the observation and the corresponding innovation covariance i,kS_i,k. These quantities are used both for Kalman correction and for computing the mode likelihood and posterior mode probability. For PR, CV, CA, and CT, i,k=k−i,k−, ν_i,k=z_k-p^-_i,k, (28) i,k=ii,k−iT+i,S_i,k=H_iP^-_i,kH_i^T+R_i, (29) where iH_i maps the prior state of mode i to the position observation and iR_i is the radar measurement-noise covariance. For the PR mode, PRQ_PR is set smaller than PRR_PR so that the learned prior state can be treated as relatively reliable compared with the raw observation. The Gaussian likelihood measuring how well mode i explains the current observation is Λi,k=1(2π)2|i,k|exp(−12i,kTi,k−1i,k). _i,k= 1 (2π)^2|S_i,k| \! (- 12 ν_i,k^TS_i,k^-1 ν_i,k ). (30) Using this likelihood, the mode probability is updated by μi,k=Λi,kτci,k∑j=1MΛj,kτcj,k, _i,k= _i,k^τc_i,k _j=1^M _j,k^τc_j,k, (31) where τ controls the influence of the likelihood. We use τ=0.1τ=0.1. I-F2 Correction Using the PR-IMM prior state and the associated observation kz_k, each mode updates its posterior state i,kx_i,k and covariance i,kP_i,k. The Kalman gain is i,k=i,k−iTi,k−1,K_i,k=P^-_i,kH_i^TS_i,k^-1, (32) followed by the posterior-state update i,k=i,k−+i,ki,k,x_i,k=x^-_i,k+K_i,k ν_i,k, (33) and the Joseph-form posterior-covariance update i,k=(−i,ki)i,k−(−i,ki)T+i,kii,kT. splitP_i,k=& (I-K_i,kH_i )P^-_i,k (I-K_i,kH_i )^T\\ &+K_i,kR_iK_i,k^T. split (34) Although the observation vector contains only position, the Kalman gain in Eq. (32) is computed from each mode’s full prior covariance. Therefore, the position innovation also updates velocity, acceleration, and turn-rate states through their cross-covariances with position. I-F3 PR-IMM Posterior State Fusion The final PR-IMM position is obtained by weighting the posterior position of every mode by the posterior mode probability from Eq. (31): PR−IMM,k=∑i=1Mμi,kii,k.p_PR\!-\!IMM,k= _i=1^M _i,kH_ix_i,k. (35) Thus, the PR, CV, CA, and CT position estimates are dynamically combined according to the motion situation at each time step. IV Experiments This section quantitatively and qualitatively evaluates the proposed PR-IMM tracker and analyzes the contribution of its components. IV-A Experimental Setup IV-A1 Dataset We train the PR model and evaluate the tracker on RadarScenes[15], a large-scale radar point-cloud dataset collected in autonomous-driving environments. RadarScenes provides point-wise segmentation labels and track IDs for five object classes: Car, Large Car, Pedestrian, Pedestrian Group, and Two Wheeler. We merge Car and Large Car into a Car category and Pedestrian and Pedestrian Group into a Pedestrian category, and evaluate only these Car and Pedestrian categories. RadarScenes contains 158 sequences and is split by sequence for training and evaluation. The validation set uses sequences 6, 42, 58, 85, 99, and 122. The evaluation set uses sequences 1, 2, 3, 4, 16, 33, 38, 41, 47, 49, 57, 71, 81, 98, 105, 112, 113, 120, 123, 125, 131, and 146. The remaining 130 sequences form the training set. Table I summarizes the split and class composition. TABLE I: Dataset split and class composition. Split Sequences Frames Car object frames Pedestrian object frames Train 130 32,555 49,349 60,216 Validation 6 1,291 1,591 1,744 Evaluation 22 4,784 11,307 6,559 IV-A2 Compared Methods To evaluate PR-IMM, we quantitatively compare six methods in total. The single-motion-model baselines are CV, CA, and CT Kalman filters. The conventional IMM (CV+CA+CT) serves as the multi-model baseline. We also include the standalone Transformer predictor PR to contrast the limitations of purely physics-based and purely data-driven approaches with the proposed hybrid framework. IV-A3 Evaluation Metrics Position-prediction performance is evaluated using Root Mean Squared Error (RMSE), while identity preservation in multiple-object tracking is evaluated using ID switches (IDSW) and IDF1. The longitudinal error RMSExRMSE_x, lateral error RMSEyRMSE_y, and planar distance error RMSExyRMSE_xy are RMSEx=1N∑k=1N(xk−x^k)2,RMSE_x= 1N _k=1^N (x_k- x_k )^2, (36) RMSEy=1N∑k=1N(yk−y^k)2,RMSE_y= 1N _k=1^N (y_k- y_k )^2, (37) RMSExy=1N∑k=1N[(xk−x^k)2+(yk−y^k)2],RMSE_xy= 1N _k=1^N [ (x_k- x_k )^2+ (y_k- y_k )^2 ], (38) where N is the total number of samples. IDSW is incremented by one when the estimated track ID associated with a ground-truth object g at time k differs from the estimated track ID most recently associated with that object. To compute IDF1, an overlap matrix is first formed for each sequence from the number of object frames in which a ground-truth track ID and an estimated track ID are simultaneously matched. The Hungarian algorithm establishes a one-to-one assignment between the two ID sets that maximizes the total overlap. The overlap sum of the assigned ID pairs is defined as IDTP. Estimated object frames not included in IDTP are IDFP, while ground-truth object frames not included in IDTP are IDFN. IDF1 is then IDF1=2IDTP2IDTP+IDFP+IDFN.IDF1= 2\,IDTP2\,IDTP+IDFP+IDFN. (39) The IDTP, IDFP, and IDFN values are summed over all evaluation sequences before the final IDF1 is computed. Lower IDSW and higher IDF1 indicate better identity-preservation performance. IV-B Quantitative Performance Comparison Table I compares one-step prior prediction accuracy. PR-IMM achieves the lowest error among all methods. This result indicates that the general physics-based motion models compensate when the data-driven PR prediction becomes unstable, while the mode probabilities automatically assign greater weight to the mode that best explains the current motion. TABLE I: One-step prior prediction RMSE. Model ↓RMSE_x [m] ↓RMSE_y [m] ↓RMSE_xy [m] CV 4.884 3.801 6.189 CA 2.241 2.068 3.050 CT 4.872 3.545 6.025 PR 0.971 0.822 1.272 IMM (CV+CA+CT) 1.827 1.685 2.485 PR-IMM (Ours) 0.799 0.698 1.062 RadarScenes does not provide labels for motion scenarios, so scenario categories are derived from the previous ten ground-truth positions, timestamps, and the object’s heading. Objects that satisfy none of the scenario conditions are excluded from scenario-wise evaluation. As shown in Table I, PR-IMM obtains the lowest RMSExyRMSE_xy in all four motion scenarios. In particular, it outperforms the standalone PR model even in the Nonlinear scenario, which is difficult to explain with a single motion assumption. This demonstrates that combining data-driven prediction with physics-based motion models remains effective for complex motion. TABLE I: Scenario-wise prior prediction RMSE. Scenario CV CA CT PR IMM Ours Straight 4.007 1.308 5.119 0.633 1.038 0.584 Accel./Decel. 8.387 2.323 8.694 1.029 2.099 0.984 Left/Right Turn 7.055 2.999 6.781 1.024 2.291 0.904 Nonlinear 7.433 3.896 7.164 1.270 3.143 1.106 Table IV reports overall IDSW and IDF1. Compared with the standalone PR model, PR-IMM reduces IDSW from 915 to 519 and increases IDF1 from 0.8795 to 0.9020. This shows that the proposed method improves not only position RMSE but also the ability to preserve track IDs when multiple objects are close to one another. TABLE IV: Overall multiple-object-tracking performance (GT object frames = 17,866). Model ↓IDSW ↑IDF1 CV 954 0.7583 CA 700 0.8075 CT 901 0.7571 PR 915 0.8795 IMM (CV+CA+CT) 695 0.8228 PR-IMM (Ours) 519 0.9020 Fig. 3: Qualitative tracking results on the RadarScenes dataset. (a) Radar point cloud with annotated vehicle points. (b) CV/CA/CT-based IMM result. (c) PR model result. (d) Proposed PR-IMM result. In (b)–(d), light thick lines denote ground-truth trajectories, whereas dark thin lines denote estimated trajectories. To analyze tracking performance under more difficult conditions, we additionally construct scenarios in which data association becomes ambiguous because objects are spatially close or their paths cross. Table V summarizes these results. Close Proximity denotes frames in which the nearest object is within 4 m. Crossing Motion denotes cases in which two objects are within 8 m and their heading difference is between 45∘45 and 135∘135 . Missed-detection scenarios are generated by removing 10% or 20% of the measurements in each sequence using the same random seed. PR-IMM achieves the lowest IDSW and the highest IDF1 in every scenario. Thus, it remains robust even when data association is more ambiguous or observations are partially missing. TABLE V: Scenario-wise multiple-object-tracking performance. Scenario (GT object frames) Metric PR IMM PR-IMM (Ours) Close Proximity (1,284) IDSW↓ 54 43 34 IDF1↑ 0.8769 0.8100 0.8941 Crossing Motion (687) IDSW↓ 20 21 11 IDF1↑ 0.9127 0.8486 0.9258 10% Miss (17,866) IDSW↓ 1,245 836 711 IDF1↑ 0.7316 0.7185 0.7610 20% Miss (17,866) IDSW↓ 1,952 1,197 1,146 IDF1↑ 0.5476 0.5629 0.5849 IV-C Qualitative Analysis This subsection qualitatively analyzes PR-IMM through mode-probability statistics and trajectory visualizations. IV-C1 Mode Probability Visualization Figure 4 shows the average probability of each PR-IMM mode over the entire test dataset. PR has the highest mean probability at 0.348, followed by CA at 0.304, CV at 0.185, and CT at 0.162. PR receives the largest share because the Transformer predictor frequently provides a prior state close to the observation based on the recent trajectory and therefore tends to obtain a relatively high likelihood. Fig. 4: Mean mode probability of PR-IMM. However, the PR probability is only 0.348 and is therefore not dominant by itself; more than half of the total probability is distributed among CV, CA, and CT. The CA mode has a higher average probability than CV and CT, suggesting that the dataset contains a substantial number of accelerating and decelerating intervals in which CA complements the PR prediction. These results support the reason why PR-IMM outperforms PR alone: tracking does not rely on a single mode, but rather combines the data-driven predictor and physics-based motion models in a complementary manner. IV-C2 Object Trajectory Visualization Figure 3 presents qualitative object-tracking results. The conventional IMM (CV+CA+CT) shows the largest deviations of multi-object trajectories from the ground truth, with particularly noticeable dispersion where objects are close to one another. The PR model follows the ground truth more closely than the conventional IMM, but its estimated positions still fluctuate in some intervals. In contrast, PR-IMM produces trajectories that most closely match the ground truth throughout the sequence and tracks each object stably even when multiple objects are adjacent. These observations are consistent with the RMSE results and demonstrate that PR-IMM robustly combines data-driven and physics-based prior states in both quantitative and qualitative evaluations. V Conclusion This paper proposed a PR-IMM tracker for radar-point-cloud-based multiple-object tracking. The PR model converts radar observations into relative displacements with respect to the most recent observation and uses a Transformer encoder to predict the next displacement, allowing nonlinear object motion to be learned directly from data without a fixed motion assumption. The PR model is then integrated as an independent IMM mode together with CV, CA, and CT. Because the four modes have different state dimensions, input mixing and state fusion are performed using the two-dimensional position and position covariance shared by all modes, enabling data-driven and physics-based prior states to be dynamically combined according to the current motion situation. Experiments show that PR-IMM achieves the best performance among the CV, CA, and CT single-motion models, the conventional IMM (CV+CA+CT), and the standalone PR model. Analysis of mode probabilities and tracking trajectories further confirms the complementary operation of the PR mode and the physics-based modes. These results demonstrate that integrating a learned state predictor into the probabilistic IMM framework can improve both position-estimation accuracy and identity-preserving multiple-object tracking under diverse motion and observation conditions. Acknowledgment This work was supported by the Ministry of SMEs and Startups, Republic of Korea, in 2026 under Grant RS-2025-24535910. References [1] G. Han and S. Kee (2025) A hybrid approach multi-object tracking with optical flow and kalman filter. Transactions of the Korean Society of Automotive Engineers 33 (9), p. 755–762. External Links: Document Cited by: §I. [2] D. Yoon, K. Jang, J. Won, and Y. Kang (2024) A multi-task deep learning method for road environment recognition and object tracking. Transactions of the Korean Society of Automotive Engineers 32 (1), p. 77–82. External Links: Document Cited by: §I. [3] H. Jung, D. Paek, and S. Kong (2025) Open-source autonomous driving software platforms: comparison of Autoware and Apollo. arXiv preprint arXiv:2501.18942. Cited by: §I. [4] S. J. Cho, B. S. Kim, T. S. Kim, and S. Kong (2019) Enhancing GNSS performance and detection of road crossing in urban area using deep learning. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), p. 2115–2120. Cited by: §I. [5] S. Kong and B. Kim (2013) Two-dimensional compressed correlator for fast PN code acquisition. IEEE Transactions on Wireless Communications 12 (11), p. 5859–5867. External Links: Document Cited by: §I. [6] S. Kong (2014) Fast multi-satellite ML acquisition for A-GPS. IEEE Transactions on Wireless Communications 13 (9), p. 4935–4946. Cited by: §I. [7] S. Jan and Y. Kao (2013) Radar tracking with an interacting multiple model and probabilistic data association filter for civil aviation applications. Sensors 13 (5), p. 6636–6650. External Links: Document Cited by: §I. [8] A. Manjunath, Y. Liu, B. Henriques, and A. Engstle (2018) Radar based object detection and tracking for autonomous driving. In 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), p. 1–4. Cited by: §I. [9] H. A. P. Blom and Y. Bar-Shalom (1988) The interacting multiple model algorithm for systems with markovian switching coefficients. IEEE Transactions on Automatic Control 33 (8), p. 780–783. Cited by: §I. [10] W. Lv, Y. Huang, N. Zhang, R. S. Lin, M. Han, and D. Zeng (2024) DiffMOT: a real-time diffusion-based multiple object tracker with non-linear prediction. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 19321–19330. Cited by: §I. [11] D. Kim, D. Paek, S. Song, and S. Kong (2025) Bayesian approximation-based trajectory prediction and tracking with 4d radar. In 2025 IEEE Intelligent Vehicles Symposium (IV), p. 1380–1385. Cited by: §I. [12] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30, p. 5998–6008. Cited by: §I. [13] M. Kim, B. I. Kwak, J. U. Hou, and T. Kim (2024) Robust long-term vehicle trajectory prediction using link projection and a situation-aware transformer. Sensors 24 (8), p. 2398. External Links: Document Cited by: §I, §I-C. [14] X. Tang, X. Cheng, and N. Xu (2024) A robust multiobject tracking method based on 4-d millimeter-wave radar and monocular vision fusion. IEEE Sensors Journal 24 (22), p. 37764–37774. Cited by: §I. [15] O. Schumann, M. Hahn, N. Scheiner, F. Weishaupt, J. F. Tilly, J. Dickmann, and C. Wöhler (2021) RadarScenes: a real-world radar point cloud data set for automotive applications. In 2021 IEEE 24th International Conference on Information Fusion (FUSION), p. 1–8. Cited by: 3rd item, §I-A, §IV-A1. [16] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft (2016) Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP), p. 3464–3468. External Links: Document Cited by: §I-A. [17] Y. Li, C. Liang, M. Lu, X. Hu, and Y. Wang (2019) Cascaded kalman filter for target tracking in automotive radar. The Journal of Engineering 2019 (19), p. 6264–6267. Cited by: §I-A. [18] Q. Li, Z. Zhang, G. He, X. Hu, and X. Kang (2024) 3D multi-object tracking for autonomous driving based on IMM-EKF and re-identification. In 2024 IEEE International Conference on Unmanned Systems (ICUS), p. 1197–1202. Cited by: §I-B. [19] X. Liu, X. Zhao, G. Liu, Z. Wu, T. Wang, L. Meng, and Y. Wang (2025) IMM-MOT: a novel 3d multi-object tracking framework with interacting multiple model filter. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 20540–20547. Cited by: §I-B. [20] T. Meinhardt, A. Kirillov, L. Leal-Taixe, and C. Feichtenhofer (2022) TrackFormer: multi-object tracking with transformers. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 8844–8854. Cited by: §I-C. [21] P. Wang, Y. Wang, H. Cao, W. Chen, and D. Li (2025) LA-MOTR: end-to-end multi-object tracking by learnable association. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 12438–12448. External Links: Document Cited by: §I-C. [22] L. Cheng, A. Sengupta, and S. Cao (2024) Deep learning-based robust multi-object tracking via fusion of mmwave radar and camera sensors. IEEE Transactions on Intelligent Transportation Systems 25 (11), p. 17218–17233. Cited by: §I-C. [23] Y. Bai, B. Yan, W. Dong, X. Jin, T. Su, and H. Ma (2025) A novel adaptive state estimation model: kalman filter coupled with neural networks. International Journal of Adaptive Control and Signal Processing 39 (5), p. 914–926. External Links: Document Cited by: §I-D. [24] S. Kim, J. Choi, D. Jeong, and S. Lee (2025) LSTM-based kalman filter error compensation for enhanced moving object tracking. Journal of Institute of Control, Robotics and Systems 31 (1), p. 68–76. External Links: Document Cited by: §I-D. [25] S. Kong, D. Paek, and S. Lee (2025) RTNH+: enhanced 4d radar object detection network using two-level preprocessing and vertical encoding. IEEE Transactions on Intelligent Vehicles 10 (2), p. 1427–1440. External Links: Document Cited by: §I-A. [26] Z. Pan, F. Ding, H. Zhong, and C. X. Lu (2024) RaTrack: moving object detection and tracking with 4d radar point cloud. In 2024 IEEE International Conference on Robotics and Automation (ICRA), p. 4480–4487. Cited by: §I-A.