Paper deep dive
MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception
Yuhao Li, Louie Hong Yao, Tianyi Shi, Hanqun Cao, Hongxia Hao, Zhen Zhao, Shengchao Liu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures. The former provide stronger priors but may constrain flexibility, whereas the latter are more flexible but leave the spatiotemporal coupling largely implicit. We therefore seek an approach that combines flexible learning with an explicit geometric bias for jointly modeling time and space. To this end, we propose Minkowski Positional Encoding (MinkowskiPE), which uses joint temporal and spatial coordinates to parameterize Lorentz transformations applied to query and key features. With MinkowskiPE, the query-key attention score depends on position only through the relative spacetime displacement between the two tokens and is therefore invariant to global translation of the coordinates. This paradigm retains the standard dot-product attention interface and remains compatible with efficient attention implementations. We evaluate MinkowskiPE on microscopic molecular dynamics and macroscopic video prediction tasks, achieving the best results on all nine multi-trajectory molecular evaluations and reducing KTH video-prediction MSE by 9.9% relative to the best baseline while using roughly one-tenth as many parameters.
Tags
Links
- Source: https://arxiv.org/abs/2609.33804v1
- Canonical: https://arxiv.org/abs/2609.33804v1
Trouble viewing inline? Open PDF directly →
Full Text
74,191 characters extracted from source content.
Expand or collapse full text
MinkowskiPE: Minkowski Positional Encoding for Spatiotemporal Perception Yuhao Li Affiliation: Wave Intelligence Lab Affiliation: Department of Computer Science and Engineering, The Chinese University of Hong Kong Email: scliu@cuhk.edu.hk Louie Hong Yao Affiliation: Wave Intelligence Lab Tianyi Shi Affiliation: Wave Intelligence Lab Affiliation: Department of Computer Science and Engineering, The Chinese University of Hong Kong Hanqun Cao Affiliation: Department of Computer Science and Engineering, The Chinese University of Hong Kong Hongxia Hao Affiliation: Shanghai Artificial Intelligence Laboratory* liyh5366@gmail.com Zhen Zhao Affiliation: Shanghai Artificial Intelligence Laboratory* liyh5366@gmail.com Shengchao Liu Affiliation: Wave Intelligence Lab Affiliation: Department of Computer Science and Engineering, The Chinese University of Hong Kong September 27, 2026 Abstract Modeling spatiotemporal coupling is a key challenge in building physical intelligence across scales, from microscopic to macroscopic. Existing models capture such structure broadly through physics-motivated dynamical formulations or learning-motivated architectures. The former provide stronger priors but may constrain flexibility, whereas the latter are more flexible but leave the spatiotemporal coupling largely implicit. We therefore seek an approach that combines flexible learning with an explicit geometric bias for jointly modeling time and space. To this end, we propose Minkowski Positional Encoding (MinkowskiPE), which uses joint temporal and spatial coordinates to parameterize Lorentz transformations applied to query and key features. With MinkowskiPE, the query–key attention score depends on position only through the relative spacetime displacement between the two tokens and is therefore invariant to global translation of the coordinates. This paradigm retains the standard dot-product attention interface and remains compatible with efficient attention implementations. We evaluate MinkowskiPE on microscopic molecular dynamics and macroscopic video prediction tasks, achieving the best results on all nine multi-trajectory molecular evaluations and reducing KTH video-prediction MSE by 9.9% relative to the best baseline while using roughly one-tenth as many parameters. 1 Introduction Space and time are fundamental to the description of physical systems across scales, from microscopic molecular motion to macroscopic scene dynamics. Accordingly, physical intelligence requires the ability to capture their joint spatiotemporal structure. In molecular dynamics, each atomic state is associated with a time step and a position in three-dimensional space [36, 21]. In video, each image patch is associated with a frame and a spatial location within that frame [4, 1]. Despite their differences in scale, modality, and data structure, both settings can therefore be viewed as collections of observations with spatiotemporal coordinates. Existing methods for spatiotemporal modeling broadly fall into two categories. The first research line is motivated by the view of temporal modeling as dynamical evolution, as in Figure 1(a). This includes recurrent and convolutional propagation [35, 44, 14], as well as continuous-time evolution parameterized by differential equations [6, 21]. Other models incorporate stronger dynamical structure through physical priors or constraints [30, 16, 27]. Across these methods, temporal structure is organized through a prescribed propagation or evolution mechanism, introducing stronger inductive structure but potentially limiting the model’s flexibility in learning spatiotemporal dependencies from data. The second research line is learning-motivated, focusing on the data-driven modeling of spatiotemporal coupling, as in Figure 1(b). Transformer-based models usually capture the coupling of time and space through the structure of learned attention, such as factorized spatial and temporal attention, multiscale attention, and trajectory-aware attention [4, 1, 11, 26]. Such models provide more flexibility in modeling how information interacts across space and time, but the relation between temporal and spatial coordinates generally remains implicit. These two directions highlight a trade-off between physical structure and data-driven flexibility in spatiotemporal coupling: physics-motivated methods often limit expressiveness, whereas learning-based methods often lack explicit physical priors. This naturally raises a question: Can we design a spatiotemporal coupling paradigm that combines the flexibility of data-driven learning with a strong physical prior? To explore this question, we focus on positional encoding as a mechanism for modeling temporal and spatial coordinates while preserving their distinct roles [41]. Further, Minkowski geometry provides such a structure, placing time and space within a unified coordinate system while assigning them different geometric roles [5]. Our Contributions. We propose Minkowski Positional Encoding (MinkowskiPE), a relative positional encoding for spatiotemporal modeling, as illustrated in Figure 1(c). Each token is associated with a spacetime coordinate that parameterizes Lorentz transformations applied to its query and key features. For multidimensional spacetime coordinates, we learn multiple projection directions and apply the corresponding transformations across feature blocks. As a result, the pairwise attention depends on position only through the relative spacetime displacement. MinkowskiPE preserves the standard dot-product attention interface, allowing it to directly benefit from efficient attention implementations such as FlashAttention [10]. Empirically, we evaluate MinkowskiPE on two spatiotemporal prediction tasks, molecular dynamics and video prediction. Molecular dynamics involves the three-dimensional motion of atoms over time, whereas video prediction concerns the evolution of two-dimensional visual scenes across frames. Together, these tasks evaluate the same positional formulation across different physical scales, data modalities, and spatial structures. On molecular dynamics prediction, MinkowskiPE ranks among the top two in 26 of 30 evaluations across ten protein–ligand systems in the single-trajectory setting, and achieves the best performance on all nine evaluations across three multi-trajectory datasets. On KTH video prediction, MinkowskiPE achieves the best performance on four of five metrics among the compared methods, reducing MSE by 9.9% while using roughly one-tenth as many parameters as the lowest-MSE baseline. Figure 1: Different perspectives on spatiotemporal modeling. (a) Physics-motivated approaches impose explicit structure on temporal evolution. (b) Learning-motivated approaches learn spatiotemporal dependencies directly from data. (c) MinkowskiPE combines physics-inspired geometric priors with flexible data-driven learning. 2 Related Work Spatiotemporal Modeling. Existing models encode temporal evolution through a range of architectural and dynamical mechanisms. Recurrent and convolutional models organize temporal information through sequential propagation or local temporal transformations [35, 44, 14], while continuous-time approaches parameterize evolution using differential equations [6, 21]. Other methods incorporate stronger dynamical or physical structure through differential-equation constraints [30, 17], Hamiltonian dynamics [16], or graph-based simulation [31, 27]. Transformer-based models instead learn spatiotemporal dependencies through attention mechanisms such as factorized spatial–temporal attention [4, 1], multiscale representations [11], local windows [22, 15], and trajectory-aware attention [26]. Our work differs by introducing inductive bias at the positional representation level rather than prescribing temporal evolution or a specific attention organization. Positional Representations. Transformers incorporate positional information through learned relative embeddings [34], relative attention formulations [9], attention biases [29, 28], and feature-space transformations [37]. These mechanisms were later generalized to multidimensional coordinates in vision and multimodal models [18, 24, 7, 42], and further developed through Lie-group transformations, commuting operators, translation-invariant constructions, and group representations [25, 50, 32, 20, 51, 49]. Related work has also explored hyperbolic feature transformations for one-dimensional positional encoding [8]. More recent work has extended positional representations to spatiotemporal settings, such as video [48, 47, 23]. MinkowskiPE instead uses joint temporal and spatial event coordinates to parameterize Lorentz transformations within a single relative positional mechanism. 3 Preliminaries 3.1 Relative Positional Encoding Let qi,kj∈ℝdemq_i,k_j ^d_em denote the query and key vectors associated with positions xix_i and xjx_j. A relative positional encoding is designed such that their pairwise interaction depends on the positions only through their relative displacement Δxij=xj−xi x_ij=x_j-x_i, i.e., ⟨fq(qi,xi),fk(kj,xj)⟩=g(qi,kj,Δxij), f_q(q_i,x_i),\,f_k(k_j,x_j) =g(q_i,k_j, x_ij), (1) where ⟨⋅,⋅⟩ ·,· denotes the dot product in feature space, fqf_q and fkf_k denote feature transformations, and g denotes the resulting query–key score before attention scaling and softmax. Equivalently, this score is invariant under a common translation xi↦xi+ax_i x_i+a and xj↦xj+ax_j x_j+a. Existing positional encodings realize this property through relative embeddings, attention biases, or position-dependent transformations of query and key features [34, 29, 28, 37]. 3.2 Minkowski Geometry and Lorentz Transformations We next introduce the geometric and algebraic structure used in our construction. A spatiotemporal event in (1+d)(1+d)-dimensional spacetime is represented by x=(t,r)∈ℝ1+d,x=(t,r) ^1+d, (2) where t∈ℝt denotes the temporal coordinate and r∈ℝdr ^d is the spatial coordinate. Minkowski geometry distinguishes temporal and spatial directions through an indefinite metric η=diag(−1,+1,⋯,+1)∈ℝ(1+d)×(1+d).η=diag(-1,+1,·s,+1) ^(1+d)×(1+d). (3) The metric η defines the indefinite bilinear form u⊤ηvu η v on spacetime coordinates. For two events xix_i and xjx_j, the relative displacement Δxij=xj−xi x_ij=x_j-x_i has the associated quadratic form Δxij⊤ηΔxij=−(tj−ti)2+‖rj−ri‖2. x_ij η x_ij=-(t_j-t_i)^2+\|r_j-r_i\|^2. (4) The signature of η distinguishes the temporal direction from the spatial directions, causing them to enter this form with opposite signs. The transformations preserving this bilinear form are Lorentz transformations. A matrix Λ is Lorentz if Λ⊤ηΛ=η, η =η, (5) or equivalently, Λ⊤η=ηΛ−1. η=η ^-1. (6) This identity will be the key algebraic ingredient for constructing a relative positional interaction. Notice that in the proposed MinkowskiPE method, the event coordinates themselves are not transformed by Λ . Instead, they parameterize Lorentz transformations applied to query–key features. We therefore use Minkowski geometry as a positional-representation structure rather than imposing Lorentz symmetry on the underlying data. 4 Method Building on the relative position construction in Section 3, we formulate spatiotemporal tokens as events and construct a Lorentzian positional transformation whose pairwise interaction depends only on relative event displacement. Then we extend this construction to practical multi-head attention. Event-Token Formulation. We consider a sequence of tokens (hi,xi)i=1N\(h_i,x_i)\_i=1^N, where hi∈ℝdemh_i ^d_em denotes the embedding of the token i, and xix_i denotes the associated spatiotemporal coordinate. We write the coordinate as xi=(ti,ri)x_i=(t_i,r_i), with ti∈ℝt_i being the temporal coordinate, ri∈ℝdr_i ^d being the spatial coordinate. For two events, we define their relative displacement as Δxij=xj−xi x_ij=x_j-x_i. This event-based representation applies to a broad range of spatiotemporal applications. For example, in video, a token may correspond to an image patch at a particular frame, with ri∈ℝ2r_i ^2. In molecular dynamics, a token may correspond to an atom at a particular time, with ri∈ℝ3r_i ^3. 4.1 Minkowski Positional Encoding We seek to use the spatiotemporal coordinate xix_i to parameterize a Lorentz transformation acting on query and key features. Following Equation 6, we have Λ(xi)⊤ηΛ(xj)=ηΛ(xi)−1Λ(xj). (x_i) η (x_j)=η (x_i)^-1 (x_j). (7) To ensure that the positional interaction depends only on relative displacement Δxij=xj−xi x_ij=x_j-x_i, a sufficient condition is Λ(x1+x2)=Λ(x1)Λ(x2),Λ(0)=I, (x_1+x_2)= (x_1) (x_2), (0)=I, (8) which gives Λ(xi)−1Λ(xj)=Λ(xj−xi) (x_i)^-1 (x_j)= (x_j-x_i). For multidimensional coordinates, a general Lorentz representation satisfying Equation 8 requires mutually commuting transformations across coordinate directions. To obtain a simple and tractable construction, we restrict it to a one-parameter Lorentz subgroup. The event coordinate xi∈ℝDx_i ^D, with D=1+dD=1+d, is first mapped to a scalar parameter ρi=w⊤xi, _i=w x_i, (9) by a learnable linear projection w∈ℝDw ^D, so that ρj−ρi=w⊤Δxij _j- _i=w x_ij. We therefore parameterize the transformation associated with event xix_i by Λ(ρi) ( _i). Appendix A gives a group-theoretic characterization of the construction, showing that the homomorphic formulation above imposes commuting-generator structure and that, for 2D Lorentzian feature blocks, any smooth homomorphism from the additive group into SO+(1,1)SO^+(1,1) reduces to the scalar-projection one-parameter form used above. MinkowskiPE. We then instantiate the one-parameter construction above using the standard (1+1)-dimensional Lorentz boost acting on two-dimensional query and key feature blocks Λ(ρ)=[coshρsinhρsinhρcoshρ],η=[−1001]. (ρ)= bmatrix ρ& ρ\\ ρ& ρ bmatrix, η= bmatrix-1&0\\ 0&1 bmatrix. (10) Let qi,ki∈ℝ2q_i,k_i ^2 denote a query and key feature block associated with event xix_i, with ρi=w⊤xi _i=w x_i defined as above. We define the position-encoded query and key as q~i=Λ(ρi)qi,k~i=ηΛ(ρi)ki. q_i= ( _i)q_i, k_i=η ( _i)k_i. (11) The metric factor η in the key transformation allows the standard Euclidean dot product to realize the corresponding Minkowski bilinear interaction. Specifically, q~i⊤k~j=qi⊤Λ(ρi)⊤ηΛ(ρj)kj=qi⊤ηΛ(ρj−ρi)kj=qi⊤ηΛ(w⊤Δxij)kj. q_i k_j=q_i ( _i) η ( _j)k_j=q_i η ( _j- _i)k_j=q_i η \! (w x_ij )k_j. (12) Thus, although each token is transformed according to its own event coordinate, the positional contribution to the pairwise query–key interaction depends on xix_i and xjx_j only through their relative displacement Δxij x_ij, as shown in Equation 12. 4.2 Practical Parameterization The single-block construction in Section 4.1 extends directly to a full attention head by assigning a separate projection direction to each two-dimensional query–key feature block. For a head dimension dhd_h, we partition the query and key features into P=dh/2P=d_h/2 two-dimensional blocks and introduce a learnable projection matrix W=[w1,…,wP]∈ℝD×P,W=[w_1,…,w_P] ^D× P, (13) where each column wp∈ℝDw_p ^D defines the positional projection for the p-th block. The same projection matrix W is shared across all Transformer layers and attention heads. For coordinate xix_i, this produces the block-wise parameters ρi=W⊤xi∈ℝP _i=W x_i ^P, with ρi,p=wp⊤xi _i,p=w_p x_i. By linearity, ρj,p−ρi,p=wp⊤Δxij _j,p- _i,p=w_p x_ij, so each block receives a different learned scalar projection of the same relative event displacement. For efficient implementation, the block-wise Lorentz transformations can be written equivalently in element-wise form. For qi=(qi,1,…,qi,dh)⊤q_i=(q_i,1,…,q_i,d_h) , we have q~i=[qi,1qi,2qi,3qi,4qi,dh−1qi,dh]⊙[coshρi,1coshρi,1coshρi,2coshρi,2coshρi,Pcoshρi,P]+[qi,2qi,1qi,4qi,3qi,dhqi,dh−1]⊙[sinhρi,1sinhρi,1sinhρi,2sinhρi,2sinhρi,Psinhρi,P], q_i= bmatrixq_i,1\\ q_i,2\\ q_i,3\\ q_i,4\\ \\ q_i,d_h-1\\ q_i,d_h bmatrix bmatrix _i,1\\ _i,1\\ _i,2\\ _i,2\\ \\ _i,P\\ _i,P bmatrix+ bmatrixq_i,2\\ q_i,1\\ q_i,4\\ q_i,3\\ \\ q_i,d_h\\ q_i,d_h-1 bmatrix bmatrix _i,1\\ _i,1\\ _i,2\\ _i,2\\ \\ _i,P\\ _i,P bmatrix, (14) where ⊙ denotes element-wise multiplication. The key features are transformed analogously, together with the Minkowski metric factor defined in Section 4.1. Figure 2: Experimental overview. (a) Molecular dynamics and (b) video predictions instantiate spatiotemporal forecasting at different physical scales. (c) Both tasks follow the same task-agnostic modeling scheme, in which task-specific features are paired with spatiotemporal coordinates, processed by a Transformer with MinkowskiPE, and mapped to future states by task-specific decoders. 5 Experiments As illustrated in Figure 2, we evaluate MinkowskiPE on molecular dynamics and video prediction, two spatiotemporal prediction tasks that differ in physical scale and data structure. Molecular dynamics represents atomic motion in three-dimensional space, whereas video prediction represents visual evolution on a two-dimensional image plane. Despite these differences, both domains can be modeled within the same task-agnostic spatiotemporal framework. We describe the task formulation, model instantiation, and experimental protocol for each domain below. 5.1 Molecular Dynamics Prediction Task Formulation. We consider the protein–ligand binding dynamics prediction under the semi-flexible setting, where the protein is treated as a fixed environment while the ligand evolves dynamically over time. Given the surrounding protein structure and an observed segment of ligand motion, the task is to predict the future three-dimensional positions of all ligand heavy atoms over multiple time steps. This requires the model to capture both the temporal evolution of individual atoms and the spatial interactions that constrain the ligand trajectory within its molecular environment. Model Instantiation. For molecular dynamics, each ligand atom at each observed time step is treated as an event token with spatiotemporal coordinate (t,x,y,z)(t,x,y,z), where t denotes the time step and (x,y,z)(x,y,z) specifies its three-dimensional position. The token features encode molecular identity and motion information, while the protein is represented as static residue-level structural context. Each Transformer block first performs frame-wise cross-attention from ligand atoms to protein residues, followed by global self-attention over ligand event tokens across the observed trajectory. MinkowskiPE is applied within the attention modules using the associated spatiotemporal coordinates. After the final Transformer block, the representation of each ligand atom at the last observed frame is passed to an MLP decoder, which predicts the displacements over all future frames. Experimental Setup. We evaluate on the MISATO dataset [36] under both single-trajectory and multi-trajectory settings. For single-trajectory prediction, a separate model is trained for each of the ten protein–ligand systems, using 10 observed frames to predict the subsequent 20 frames. For multi-trajectory prediction, a separate model is trained on each of MISATO-100, MISATO-1000, and MISATO-Full, which represent progressively larger collections of protein–ligand trajectories, using 80 observed frames to predict the subsequent 20 frames. We evaluate prediction quality using MAE, Matching, and Stability, which measure coordinate accuracy and structural consistency from complementary perspectives. We compare against VerletMD, GNN-MD, and DenoisingLD, three representative baselines implemented in NeuralMD [21] and motivated by numerical integration, autoregressive graph-based simulation [13], and diffusion-based molecular dynamics [2], respectively. We further include NeuralMD-ODE, NeuralMD-SDE [21], and the recent generative model BioMD [12]. More experimental details are provided in Appendix B. Table 1: Results on single-trajectory predictions (in10/out20). All results are averaged over three random seeds. The best result in each column is shown in bold, and the second-best is underlined. Method Metric 5WIJ 4ZX0 3EOV 4K6W 1KTI 1XP6 4YUR 4G3E 6B7F 3B9S VerletMD MAE 14.629 21.278 27.960 15.428 18.157 13.753 16.764 5.111 31.934 19.473 Matching 5.459 7.971 13.588 7.505 7.467 4.672 9.555 3.388 21.691 0.923 Stability 24.360 19.168 13.067 15.441 19.352 28.129 16.542 31.852 11.050 57.801 GNN-MD MAE 2.280 2.370 3.512 3.695 6.641 2.378 7.031 2.709 4.136 2.578 Matching 0.803 0.555 1.216 1.038 0.386 0.966 0.920 0.893 1.194 1.414 Stability 54.475 68.613 40.984 42.480 81.831 49.239 47.555 61.802 39.067 49.306 NeuralMD-ODE MAE 2.252 1.878 3.858 3.656 6.675 1.924 6.957 2.191 3.921 3.039 Matching 0.464 0.428 1.062 0.928 0.337 0.537 0.584 0.505 0.459 0.659 Stability 82.046 81.401 47.328 49.438 86.430 75.533 69.775 71.436 75.692 76.065 NeuralMD-SDE MAE 2.260 2.158 3.395 3.765 6.646 2.061 7.038 2.345 3.842 3.132 Matching 0.615 0.696 0.962 1.076 0.167 0.615 0.749 0.521 0.741 0.444 Stability 67.464 59.109 50.108 49.700 98.508 69.423 60.344 68.729 57.917 77.801 DenoisingLD MAE 2.501 3.138 4.055 3.942 7.051 2.218 7.128 3.588 4.431 2.811 Matching 0.815 1.072 1.209 0.839 0.268 0.676 0.834 1.069 0.672 0.472 Stability 52.418 44.228 41.469 53.820 91.986 64.951 49.676 40.823 61.583 71.852 BioMD MAE 3.191 2.200 4.193 3.362 4.251 2.273 5.163 2.535 5.156 4.243 Matching 0.386 0.424 0.712 0.778 0.153 0.544 0.477 0.228 0.189 0.095 Stability 87.150 83.080 64.430 57.680 98.320 78.050 78.840 94.420 96.530 100.000 MinkowskiPE MAE 2.143 1.982 3.491 2.477 6.669 1.911 6.491 1.818 3.899 2.486 Matching 0.392 0.441 0.989 0.742 0.129 0.509 0.422 0.190 0.205 0.071 Stability 86.882 81.138 52.287 58.499 99.383 79.397 81.934 97.325 95.825 100.000 Table 2: Results on multi-trajectory predictions (in80/out20). All results are averaged over three random seeds. The best result in each column is shown in bold, and the second-best is underlined. Method MISATO-100 MISATO-1000 MISATO-Full MAE Matching Stability MAE Matching Stability MAE Matching Stability VerletMD 14.317 6.927 19.78 27.374 10.042 20.10 24.242 10.101 18.76 GNN-MD 4.198 0.489 77.67 4.500 0.553 75.06 4.434 0.509 77.59 NeuralMD-ODE 4.210 0.407 84.76 4.548 0.446 84.69 4.448 0.428 84.90 NeuralMD-SDE 4.215 0.407 84.71 4.555 0.446 84.59 4.455 0.430 84.70 DenoisingLD 4.184 0.396 85.39 4.467 0.430 85.42 4.400 0.412 85.64 BioMD 4.747 0.471 80.28 4.522 0.472 82.47 4.390 0.420 85.15 MinkowskiPE 4.181 0.395 85.45 4.367 0.415 86.01 4.315 0.392 86.29 Figure 3: Representative molecular dynamics predictions. Final-step ligand configurations predicted by MinkowskiPE for 4K6W, 1XP6, 4G3E, and 3B9S, compared with the corresponding reference configurations. The surrounding protein surface is locally clipped for visualization clarity. Results. Table 1 reports the single-trajectory results. Across the ten protein–ligand systems and three evaluation metrics, MinkowskiPE achieves the best result in 17 of 30 evaluations and ranks among the top two in 26 of 30. In particular, it achieves the best performance across all three metrics on 4K6W, 1XP6, 4G3E, and 3B9S. Its strong performance on both Matching and Stability indicates that the gains are not confined to coordinate accuracy, but also extend to the structural quality of the predicted ligand configurations. Figure 3 visualizes the final predicted configurations for these four systems alongside the corresponding ground truth, providing a direct view of the resulting molecular structures. Table 2 reports the multi-trajectory results. Across all three dataset scales, MinkowskiPE achieves the best performance on all nine evaluations. The advantage is consistent across both coordinate and structural metrics and persists as the number and diversity of training trajectories increase. Together with the single-trajectory results, these findings show that MinkowskiPE performs well not only when modeling individual molecular systems, but also when learning shared dynamics across heterogeneous protein–ligand complexes. 5.2 Video Prediction Task Formulation. We next consider video prediction, where a model observes a sequence of past frames and forecasts the subsequent visual evolution. Unlike molecular dynamics, where spatial coordinates correspond to continuous three-dimensional atom positions, video observations are organized on a regular two-dimensional image plane. The task therefore requires modeling both temporal changes across frames and spatial structure within each frame, including the motion and deformation of visual patterns over time. Model Instantiation. Each video frame is first encoded by a convolutional encoder into a 16×1616× 16 latent feature grid. Every latent feature is treated as an event token with spatiotemporal coordinate (t,u,v)(t,u,v), where t denotes the frame index and (u,v)(u,v) specifies its spatial location on the latent grid. The observed tokens are concatenated with future tokens initialized from the last observed latent map, and augmented with learnable future-step embeddings. All tokens are then jointly processed by an eight-layer global Transformer equipped with MinkowskiPE. The resulting future latent features are reshaped into spatial feature maps and passed through a convolutional decoder to reconstruct all future frames. Experimental Setup. We evaluate on the KTH human-action dataset following the OpenSTL protocol [40]. The model takes 10 grayscale frames at 128×128128× 128 resolution as input and predicts the subsequent 20 frames. We evaluate prediction quality using MSE, MAE, SSIM, PSNR, and LPIPS, covering pixel-level accuracy, structural similarity, and perceptual quality. We compare against representative video-prediction baselines reported in OpenSTL, including ConvLSTM [35], MIM [46], PredRNN variants [44, 43, 45], SimVP variants [14, 38], and TAU [39]. To match its reporting protocol, which selects the best model over three trials, we use our lowest-MSE run among three random seeds for the primary comparison with OpenSTL baselines. We additionally report the mean performance across the three seeds for completeness. More experimental details are provided in Appendix B. Table 3: Results on KTH video prediction (in10/out20). We adopt the baseline results and experimental settings from OpenSTL [40], which report the best result across three random seeds. For protocol-matched comparison, bold and underline denote the best and second-best results using our best run. The three-seed mean of MinkowskiPE is additionally reported for reference. Method Params (M) MSE ↓ MAE ↓ SSIM ↑ PSNR ↑ LPIPS ↓ ConvLSTM-S 14.9 47.65 445.5 0.8977 26.99 0.26686 MIM 39.8 40.73 380.8 0.9025 27.78 0.18808 PredRNN 23.6 41.07 380.6 0.9097 27.95 0.21892 PredRNN++ 38.3 39.84 370.4 0.9124 28.13 0.19871 PredRNN.V2 23.6 39.57 368.8 0.9099 28.01 0.21478 SimVP+IncepU 12.2 41.11 397.1 0.9065 27.46 0.26496 SimVP+gSTA 15.6 45.02 417.8 0.9049 27.04 0.25240 TAU 15.0 45.32 421.7 0.9086 27.10 0.22856 MinkowskiPE (best) 2.5 35.66 347.7 0.9078 28.18 0.18490 MinkowskiPE (mean) 2.5 35.78 347.9 0.9079 28.15 0.18342 Results. Table 3 reports the results on the KTH video prediction task. Under the protocol-matched comparison, MinkowskiPE achieves the best performance on MSE, MAE, PSNR, and LPIPS, while remaining competitive on SSIM. Compared with the strongest baseline result for each metric, it reduces MSE, MAE, and LPIPS by 9.9%, 5.7%, and 1.7%, respectively, while using only 2.5M parameters. The three-seed mean remains better than the reported baseline results on the same four metrics. Overall, these results show that the same event-based positional formulation used for molecular dynamics also performs effectively on dense visual prediction with a different spatial dimensionality and token structure. 5.3 Effect of MinkowskiPE Table 4: Controlled ablation of MinkowskiPE on molecular dynamics and video prediction. We compare the same model architecture with MinkowskiPE and with positional encoding removed, while keeping all others unchanged. All results are averaged over three random seeds. Method MISATO-1000 KTH MAE ↓ Matching ↓ Stability ↑ MSE ↓ MAE ↓ SSIM ↑ PSNR ↑ LPIPS ↓ No PE 4.488 0.432 84.96 37.93 360.8 0.9045 27.93 0.19203 MinkowskiPE 4.367 0.415 86.01 35.78 347.9 0.9079 28.15 0.18342 To isolate the contribution of the proposed positional mechanism from the underlying task-specific architectures, we conduct controlled ablations on both molecular dynamics and video prediction. In each case, we remove only the positional encoding from the corresponding attention modules, and keep the model architecture, optimization protocol, and training configuration unchanged. As shown in Table 4, MinkowskiPE improves all evaluated metrics in both domains. On MISATO-1000, it reduces MAE and Matching by 2.7% and 3.9% respectively, and improves Stability by 1.2%. On KTH, it reduces MSE and MAE by 5.7% and 3.6% respectively, and improves LPIPS by 4.5%, with consistent gains in SSIM and PSNR. These controlled comparisons show that the improvements extend across multiple evaluation metrics and two different spatiotemporal domains, supporting that the observed gains are not solely attributable to the task-specific architectures. 6 Discussion Minkowski Structure as Representation Bias. MinkowskiPE does not assume that the underlying data obey relativistic dynamics. Rather, it uses the algebraic structure of Minkowski geometry as a representation-level inductive bias for spatiotemporal attention. Joint temporal and spatial coordinates parameterize Lorentz transformations in query–key feature space. Together with the linear coordinate projections, preservation of the indefinite bilinear form ensures that the positional interaction depends only on relative event displacement. This construction provides a geometric mechanism for jointly modeling temporal and spatial information without prescribing a dynamical law or requiring Lorentz symmetry of the underlying data. Cross-domain behavior. A notable property of MinkowskiPE is that the same positional construction can be applied across substantially different spatiotemporal domains without changing its underlying formulation. Molecular dynamics and video prediction differ not only in physical scale, but also in token semantics, spatial structure, and prediction architecture. Nevertheless, both can be represented through the same event-based view of time and space and modeled with the same positional mechanism. This suggests that MinkowskiPE provides a flexible geometric representation for spatiotemporal relations across different Transformer-based prediction settings. Limitations and Outlook. MinkowskiPE adopts a fixed geometric bias rather than learning the geometry from data, and different applications may benefit from different geometric inductive biases. Importantly, this choice does not assume that the underlying dynamics obey Lorentz symmetry. If one instead seeks a positional encoding that explicitly respects the spacetime symmetry of a physical system, Galilean symmetry provides a natural candidate for many non-relativistic settings [19]. In Appendix C, we analyze one such construction based on finite-dimensional unitary token-wise transformations and show that exact Galilean invariance imposes a strong structural constraint: the resulting pairwise interaction cannot retain the Galilean-invariant spatial component of the relative group element. This result does not rule out Galilean positional encodings more generally, but highlights a nontrivial obstruction for this particular class of constructions. A second limitation is that our current construction uses learned scalar projections followed by one-parameter Lorentz transformations. More expressive multi-parameter constructions may capture richer interactions among temporal and spatial directions, but require mutually commuting Lorentz generators and are therefore substantially more constrained. We provide further analysis in Appendix A. Finally, our experiments focus on molecular and visual forecasting. Extending the framework to embodied dynamics, weather forecasting, and generative models, as well as exploring data-adaptive geometric structures, are natural directions for future work. 7 Conclusion We introduced MinkowskiPE, a relative positional encoding that represents Transformer tokens as events in joint time–space coordinates. Using Lorentz transformations in query–key feature space, it couples temporal and spatial positional information, with attention scores depending on position only through relative spacetime displacement. The construction requires only a lightweight modification to query–key transformations and applies unchanged across different spatiotemporal tokenizations. Across molecular dynamics and video prediction, MinkowskiPE achieves strong performance across distinct spatiotemporal settings. Together, these results highlight positional geometry as a key design axis for spatiotemporal Transformers. AI use statement The authors used ChatGPT to polish the writing and assist in literature discovery and interpretation. CodeX was used for limited code review, debugging, and utility scripts, while the core implementation was developed by the authors. During experimentation, CodeX assisted with monitoring, aggregating, and organizing experimental results. All AI-assisted material was reviewed and verified by the authors, who take full responsibility for the final manuscript, code, and reported results. References [1] A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lučić, and C. Schmid (2021) ViViT: a video vision transformer. In 2021 IEEE/CVF international conference on computer vision (ICCV), Cited by: §1, §1, §2. [2] M. Arts, V. Garcia Satorras, C. Huang, D. Zugner, M. Federici, C. Clementi, F. Noé, R. Pinsler, and R. van den Berg (2023) Two for one: diffusion models and force fields for coarse-grained molecular dynamics. Journal of Chemical Theory and Computation 19 (18), p. 6151–6159. Cited by: §5.1. [3] V. Bargmann (1954) On unitary ray representations of continuous groups. Annals of Mathematics 59 (1), p. 1–46. Cited by: Appendix C. [4] G. Bertasius, H. Wang, and L. Torresani (2021) Is space-time attention all you need for video understanding?. In Proceedings of the 38th International Conference on Machine Learning, Cited by: §1, §1, §2. [5] S. M. Carroll (2019) Spacetime and geometry: an introduction to general relativity. Cambridge University Press. Cited by: §1. [6] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud (2018) Neural ordinary differential equations. In Advances in Neural Information Processing Systems, Vol. 31. Cited by: §1, §2. [7] X. Chu, J. Su, B. Zhang, and C. Shen (2024) VisionLLaMA: a unified llama backbone for vision tasks. In European Conference on Computer Vision, Cited by: §2. [8] C. Dai, H. Shan, M. Song, and D. Liang (2025) HoPE: hyperbolic rotary positional encoding for stable long-range dependency modeling in large language models. arXiv preprint arXiv:2509.05218. Cited by: §2. [9] Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. Le, and R. Salakhutdinov (2019) Transformer-xl: attentive language models beyond a fixed-length context. In Proceedings of the 57th annual meeting of the association for computational linguistics, p. 2978–2988. Cited by: §2. [10] T. Dao, D. Y. Fu, S. Ermon, A. Rudra, and C. Re (2022) FlashAttention: fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems, Cited by: §1. [11] H. Fan, B. Xiong, K. Mangalam, Y. Li, Z. Yan, J. Malik, and C. Feichtenhofer (2021) Multiscale vision transformers. In 2021 IEEE/CVF international conference on computer vision (ICCV), Cited by: §1, §2. [12] B. Feng, J. Zhang, X. Zhang, Z. Liu, and Y. Li (2026) BioMD: all-atom generative model for biomolecular dynamics simulation. In The Fourteenth International Conference on Learning Representations, Cited by: §5.1. [13] X. Fu, T. Xie, N. J. Rebello, B. Olsen, and T. S. Jaakkola (2023) Simulate time-integrated coarse-grained molecular dynamics with multi-scale graph networks. Transactions on Machine Learning Research. External Links: ISSN 2835-8856 Cited by: §5.1. [14] Z. Gao, C. Tan, L. Wu, and S. Z. Li (2022) SimVP: simpler yet better video prediction. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2, §5.2. [15] Z. Gao, X. Shi, H. Wang, Y. Zhu, Y. (. Wang, M. Li, and D. Yeung (2022) Earthformer: exploring space-time transformers for earth system forecasting. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), Cited by: §2. [16] S. Greydanus, M. Dzamba, and J. Yosinski (2019) Hamiltonian neural networks. In Advances in Neural Information Processing Systems, Cited by: §1, §2. [17] V. L. Guen and N. Thome (2020) Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Cited by: §2. [18] B. Heo, S. Park, D. Han, and S. Yun (2024) Rotary position embedding for vision transformer. In European Conference on Computer Vision, Cited by: §2. [19] J. Lévy-Leblond (1971) Galilei group and galilean invariance. In Group theory and its applications, p. 221–299. Cited by: Appendix C, Appendix C, §6. [20] H. Liu, L. Lin, J. Sun, Z. Shangguan, M. A. Alvarez, and H. Zhou (2025) Rethinking rope: a mathematical blueprint for n-dimensional positional embedding. arXiv preprint arXiv:2504.06308. Cited by: §2. [21] S. Liu, W. Du, H. Xu, Y. Li, Z. Li, V. Bhethanabotla, D. Yan, C. Borgs, A. Anandkumar, H. Guo, et al. (2026) A multi-grained symmetric differential equation model for learning protein-ligand binding dynamics. Nature Communications 17 (1), p. 1049. Cited by: §B.2, §1, §1, §2, §5.1. [22] Z. Liu, J. Ning, Y. Cao, Y. Wei, Z. Zhang, S. Lin, and H. Hu (2022) Video Swin Transformer. In 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Cited by: §2. [23] Z. Liu, L. Guo, Y. Tang, T. Yue, J. Cai, K. Ma, Q. Liu, X. Chen, and J. Liu (2025) VRoPE: rotary position embedding for video large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Cited by: §2. [24] Z. Lu, Z. Wang, D. Huang, C. Wu, X. Liu, W. Ouyang, and L. BAI (2024) FiT: flexible vision transformer for diffusion model. In Forty-first International Conference on Machine Learning, Cited by: §2. [25] S. Ostmeier, B. Axelrod, M. Varma, M. Moseley, A. S. Chaudhari, and C. Langlotz (2025) LieRE: lie rotational positional encodings. In Forty-second International Conference on Machine Learning, Cited by: §2. [26] M. Patrick, D. Campbell, Y. Asano, I. Misra, F. Metze, C. Feichtenhofer, A. Vedaldi, and J. F. Henriques (2021) Keeping your eye on the ball: trajectory attention in video transformers. In Advances in Neural Information Processing Systems, Cited by: §1, §2. [27] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. Battaglia (2021) Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, Cited by: §1, §2. [28] O. Press, N. Smith, and M. Lewis (2022) Train short, test long: attention with linear biases enables input length extrapolation. In International Conference on Learning Representations, Cited by: §2, §3.1. [29] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21 (140), p. 1–67. Cited by: §2, §3.1. [30] M. Raissi, P. Perdikaris, and G.E. Karniadakis (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics 378, p. 686–707. Cited by: §1, §2. [31] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia (2020) Learning to simulate complex physics with graph networks. In Proceedings of the 37th International Conference on Machine Learning, Cited by: §2. [32] C. Schenck, I. Reid, M. G. Jacob, A. Bewley, J. Ainslie, D. Rendleman, D. Jain, M. Sharma, K. A. Dubey, A. Wahid, S. Singh, R. Wagner, T. Ding, C. Fu, A. Byravan, J. Varley, A. A. Gritsenko, M. Minderer, D. Kalashnikov, J. Tompson, V. Sindhwani, and K. M. Choromanski (2025) Learning the roPEs: better 2D and 3D position encodings with STRING. In Forty-second International Conference on Machine Learning, Cited by: §2. [33] C. Schuldt, I. Laptev, and B. Caputo (2004) Recognizing human actions: a local svm approach. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., Vol. 3, p. 32–36. External Links: Document Cited by: §B.3. [34] P. Shaw, J. Uszkoreit, and A. Vaswani (2018) Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), p. 464–468. Cited by: §2, §3.1. [35] X. Shi, Z. Chen, H. Wang, D. Yeung, W. Wong, and W. Woo (2015) Convolutional lstm network: a machine learning approach for precipitation nowcasting. In Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, Cited by: §1, §2, §5.2. [36] T. Siebenmorgen, F. Menezes, S. Benassou, E. Merdivan, K. Didi, A. S. D. Mourão, R. Kitel, P. Liò, S. Kesselheim, M. Piraud, et al. (2024) MISATO: machine learning dataset of protein–ligand complexes for structure-based drug discovery. Nature computational science 4 (5), p. 367–378. Cited by: §B.2, §1, §5.1. [37] J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu (2024) Roformer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. Cited by: §2, §3.1. [38] C. Tan, Z. Gao, S. Li, and S. Z. Li (2025) SimVPv2: towards simple yet powerful spatiotemporal predictive learning. IEEE Transactions on Multimedia 27 (), p. 5170–5184. Cited by: §5.2. [39] C. Tan, Z. Gao, L. Wu, Y. Xu, J. Xia, S. Li, and S. Z. Li (2023) Temporal attention unit: towards efficient spatiotemporal predictive learning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §5.2. [40] C. Tan, S. Li, Z. Gao, W. Guan, Z. Wang, Z. Liu, L. Wu, and S. Z. Li (2023) OpenSTL: a comprehensive benchmark of spatio-temporal predictive learning. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, Cited by: §B.3, §5.2, Table 3. [41] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Cited by: §1. [42] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin (2024) Qwen2-VL: enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191. Cited by: §2. [43] Y. Wang, Z. Gao, M. Long, J. Wang, and P. S. Yu (2018) PredRNN++: towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. In International conference on machine learning, Cited by: §5.2. [44] Y. Wang, M. Long, J. Wang, Z. Gao, and P. S. Yu (2017) PredRNN: recurrent neural networks for predictive learning using spatiotemporal lstms. In Advances in Neural Information Processing Systems, Cited by: §1, §2, §5.2. [45] Y. Wang, H. Wu, J. Zhang, Z. Gao, J. Wang, P. S. Yu, and M. Long (2023) PredRNN: a recurrent neural network for spatiotemporal predictive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2), p. 2208–2225. Cited by: §5.2. [46] Y. Wang, J. Zhang, H. Zhu, M. Long, J. Wang, and P. S. Yu (2019) Memory in memory: a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §5.2. [47] X. Wei, X. Liu, Y. Zang, X. Dong, P. Zhang, Y. Cao, J. Tong, H. Duan, Q. Guo, J. Wang, X. Qiu, and D. Lin (2025) VideoRoPE: what makes for good video rotary position embedding?. In Forty-second International Conference on Machine Learning, Cited by: §2. [48] Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, et al. (2025) Cogvideox: text-to-video diffusion models with an expert transformer. In International Conference on Learning Representations, Cited by: §2. [49] Y. Yao and B. Yang (2025) GeoPE:a unified geometric positional embedding for structured tensors. arXiv preprint arXiv:2512.04963. Cited by: §2. [50] H. Yu, T. Jiang, S. Jia, S. Yan, S. Liu, H. Qian, G. Li, S. Dong, and C. Yuan (2025) ComRoPE: scalable and robust rotary position embedding parameterized by trainable commuting angle matrices. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2. [51] Y. Zhang, Z. Chen, Y. Liu, Z. Qin, H. Yuan, K. Xu, Y. Yuan, Q. Gu, and A. C. Yao (2026) Group representational position encoding. In The Fourteenth International Conference on Learning Representations, Cited by: §2. Appendix A Additional Derivations of MinkowskiPE This appendix develops the group-theoretic structure underlying the construction in Section 4. We first formalize how a homomorphic Lorentzian representation yields an exact relative-position interaction. And then we characterize general smooth homomorphisms from the additive spatiotemporal coordinate group into a Lorentz group, and show that for the (1+1)(1+1)-dimensional feature blocks used by MinkowskiPE, the construction necessarily reduces to a linear scalar projection followed by a one-parameter Lorentz boost. A.1 Relative Lorentzian Construction Let x∈ℝDx ^D denote an event coordinate, and let Γ:(ℝD,+)→SO+(1,ds) :(R^D,+)→ SO^+(1,d_s) denote a position-dependent Lorentz representation acting on a (1+ds)(1+d_s)-dimensional feature space. We write ηs=diag(−1,1,…,1) _s=diag(-1,1,…,1) (15) for the Minkowski metric on this feature space. By definition, Γ(x)⊤ηsΓ(x)=ηs, (x) _s (x)= _s, (16) and hence Γ(x)⊤ηs=ηsΓ(x)−1. (x) _s= _s (x)^-1. (17) For query and key features qiq_i and kjk_j associated with event coordinates xix_i and xjx_j, define q~i=Γ(xi)qi,k~j=ηsΓ(xj)kj. q_i= (x_i)q_i, k_j= _s (x_j)k_j. (18) Their Euclidean dot product becomes q~i⊤k~j=qi⊤Γ(xi)⊤ηsΓ(xj)kj=qi⊤ηsΓ(xi)−1Γ(xj)kj, q_i k_j=q_i (x_i) _s (x_j)k_j=q_i _s (x_i)^-1 (x_j)k_j, (19) where the second equality follows from Equation 17. Suppose that Γ is a homomorphism from the additive coordinate group into the Lorentz group, Γ(x1+x2)=Γ(x1)Γ(x2),Γ(0)=I. (x_1+x_2)= (x_1) (x_2), (0)=I. (20) Then Γ(−x)=Γ(x)−1 (-x)= (x)^-1, and therefore Γ(xi)−1Γ(xj)=Γ(xj−xi)=Γ(Δxij). (x_i)^-1 (x_j)= (x_j-x_i)= ( x_ij). (21) Substituting this relation into Equation 19 gives q~i⊤k~j=qi⊤ηsΓ(Δxij)kj, q_i k_j=q_i _s ( x_ij)k_j, (22) so the positional contribution depends on the event coordinates only through their relative displacement. The metric factor ηs _s in the key transformation is essential to this construction. If both query and key were transformed simply as Γ(xi)qi (x_i)q_i and Γ(xj)kj (x_j)k_j, their interaction would instead contain Γ(xi)⊤Γ(xj) (x_i) (x_j). Lorentz transformations are generally not orthogonal under the ordinary Euclidean inner product, so Γ(x)⊤≠Γ(x)−1 (x) ≠ (x)^-1. The insertion of ηs _s converts the transpose into the group inverse through Equation 17, allowing the two absolute transformations to reduce exactly to a transformation of the relative coordinate. A.2 Group-Theoretic Characterization We next characterize smooth Lorentz representations satisfying Equation 20. Let Γ:(ℝD,+)→SO+(1,ds) :(R^D,+)→ SO^+(1,d_s) be a smooth homomorphism, where D is the dimension of the event coordinate and 1+ds1+d_s is the dimension of the Lorentzian feature representation. For each coordinate direction, define the infinitesimal generator Gμ=∂xμΓ(x)|x=0,μ=1,…,D.G_μ= . ∂ x_μ (x) |_x=0, μ=1,…,D. (23) Since every Γ(x) (x) preserves ηs _s, each generator belongs to the Lorentz Lie algebra and satisfies Gμ⊤ηs+ηsGμ=0.G_μ _s+ _sG_μ=0. (24) Because (ℝD,+)(R^D,+) is Abelian, the images of different coordinate directions commute Γ(seμ)Γ(teν)=Γ(teν)Γ(seμ) (se_μ) (te_ν)= (te_ν) (se_μ) (25) for arbitrary s,t∈ℝs,t . Differentiating at the identity gives [Gμ,Gν]=0for all μ,ν.[G_μ,G_ν]=0 all μ,ν. (26) Each coordinate direction therefore generates a one-parameter subgroup, Γ(seμ)=exp(sGμ). (se_μ)= (sG_μ). (27) Since x=∑μ=1Dxμeμx= _μ=1^Dx_μe_μ, the homomorphism property gives Γ(x)=∏μ=1Dexp(xμGμ). (x)= _μ=1^D (x_μG_μ). (28) Using Equation 26, this becomes Γ(x)=exp(∑μ=1DxμGμ). (x)= ( _μ=1^Dx_μG_μ ). (29) Conversely, any collection of mutually commuting Lorentz generators Gμ=1D\G_μ\_μ=1^D defines a smooth homomorphism through Equation 29. Thus, within this homomorphic construction, an exact multidimensional relative Lorentzian representation is characterized by a mutually commuting set of Lorentz generators. The (1+1)-dimensional Case. MinkowskiPE operates on two-dimensional Lorentzian feature blocks. For such a block, the relevant connected Lorentz group is SO+(1,1)SO^+(1,1), whose Lie algebra (1,1) so(1,1) is one-dimensional. A basis generator is G=[0110],η1,1=[−1001],G= bmatrix0&1\\ 1&0 bmatrix, _1,1= bmatrix-1&0\\ 0&1 bmatrix, (30) which satisfies Equation 24. Because (1,1) so(1,1) is one-dimensional, every generator GμG_μ must be proportional to G, i.e., Gμ=wμG_μ=w_μG (31) for some scalar coefficient wμw_μ. Substituting this relation into Equation 29 gives Γ(x)=exp(∑μ=1DxμwμG)=exp[(w⊤x)G]. (x)= ( _μ=1^Dx_μw_μG )= [(w x)G ]. (32) Defining ρ=w⊤xρ=w x, we therefore obtain Γ(x)=Λ(ρ)=Λ(w⊤x), (x)= (ρ)= (w x), (33) where Λ denotes the corresponding one-parameter Lorentz boost. Since G2=IG^2=I, Λ(ρ)=eρG=cosh(ρ)I+sinh(ρ)G=[coshρsinhρsinhρcoshρ]. (ρ)=e^ρ G= (ρ)I+ (ρ)G= bmatrix ρ& ρ\\ ρ& ρ bmatrix. (34) Equation (33) gives a structural characterization of the scalar-projection form used in MinkowskiPE. Within a (1+1)-dimensional Lorentzian feature block, any smooth homomorphism from the additive event-coordinate group into SO+(1,1)SO^+(1,1) is of the form x⟼ρ=w⊤x⟼Λ(ρ),x~ ~ρ=w x~ ~ (ρ), including the trivial case w=0w=0. Thus, the learned scalar projection in MinkowskiPE is not merely an implementation convenience but the general smooth homomorphic parameterization within this feature-block class. Higher-dimensional Lorentz Representations. The one-parameter construction above is not the only possible homomorphic Lorentz representation. In higher-dimensional feature spaces, genuinely multi-parameter constructions can be obtained from higher-dimensional commuting subalgebras of (1,ds) so(1,d_s). However, the commutation requirement places strong structural constraints on the allowed generators. For example, let KaK_a denote the Lorentz boost generator along the a-th positive-signature direction and JabJ_ab the generator of a rotation in the (a,b)(a,b) plane. For a≠ba≠ b, [Ka,Kb]=−Jab,[K_a,K_b]=-J_ab, (35) up to convention-dependent signs. Hence independent boosts along different directions do not commute and cannot directly serve as independent generators of a homomorphism from (ℝD,+)(R^D,+). More generally, for noncommuting generators A and B, the Baker–Campbell–Hausdorff formula gives eAeB=exp(A+B+12[A,B]+⋯),e^Ae^B= (A+B+ 12[A,B]+·s ), (36) so composition introduces additional commutator terms and the simple additive relation Γ(x1)Γ(x2)=Γ(x1+x2) (x_1) (x_2)= (x_1+x_2) (37) does not generally hold. Higher-dimensional commuting constructions remain possible, including subgroups that act only within positive-signature directions. Our choice of (1+1)(1+1)-dimensional Lorentzian blocks instead gives the minimal feature-space realization that mixes the two metric signatures while admitting an exact one-parameter homomorphic form. More expressive multi-parameter Lorentzian positional encodings would require additional choices of commuting subalgebras and are left for future work. Appendix B Additional Experimental Details B.1 Implementation Details of MinkowskiPE MinkowskiPE Implementation. In all experiments, MinkowskiPE is applied to the query and key features, while the value features remain unchanged. For each attention head, adjacent feature dimensions are grouped into two-dimensional blocks and transformed using the Lorentzian construction described in Section 4. The same projection matrix W is shared across all Transformer layers and attention heads. For molecular dynamics, the hidden dimension is 128128 with 88 attention heads, giving a head dimension of 1616 and P=8P=8 Lorentz blocks. For video prediction, the hidden dimension is 144144 with 88 attention heads, giving a head dimension of 1818 and P=9P=9 blocks. Coordinate Normalization. Before applying MinkowskiPE, event coordinates are normalized to task-dependent ranges. For molecular dynamics, all spatial coordinates within an input window are first centered by the mass-weighted ligand center of mass (COM) at the last observed frame. Let rmax=max∈ℛ‖−COMlast‖2,r_ = _r \|r-r_COM^last \|_2, (38) where ℛR contains all valid ligand positions in the window and the alpha-carbon (Cα) positions of the cropped protein residues. The spatial coordinates used by MinkowskiPE are then PE=−COMlastmax(10,rmax/2),r_PE= r-r_COM^last (10,r_ /2), (39) while the temporal coordinates over the observed frames are linearly mapped to [−1,1][-1,1]. For video, the temporal and two spatial coordinates are directly constructed on a regular grid in [−1,1]3[-1,1]^3. This normalization keeps the inputs in a numerically well-behaved regime for the hyperbolic functions. Numerical Implementation. The coordinate projection and the evaluation of cosh and sinh are performed in float32, after which the transformed query and key features are cast back to the attention dtype. We implement attention using PyTorch scaled dot-product attention. On the NVIDIA H200 GPUs used in our experiments, this operation is dispatched to the FlashAttention backend. B.2 Molecular Dynamics Prediction Figure S1: Architecture used for molecular dynamics prediction. Each block performs frame-wise cross-attention from ligand atoms to the surrounding protein residues, followed by global self-attention over ligand event tokens across the observed trajectory. MinkowskiPE is applied to the query–key interactions using the corresponding spatiotemporal event coordinates. The representations at the last observed ligand frame are decoded into future atomic displacements. Data and Preprocessing. We use protein–ligand trajectories from MISATO [36] and follow the preprocessing pipeline of NeuralMD [21]. Each preprocessed trajectory contains 100100 stored frames. Ligand inputs contain heavy atoms only, whereas the protein is represented at the residue level using backbone N, Cα, and C atoms. For each input window, we retain at most 128128 protein residues whose Cα atoms are closest to the mass-weighted ligand center of mass at the last observed frame. The same center is used to translate the observed ligand, target ligand, and protein coordinates. Ligand content features include atomic number, atomic mass, centered coordinates, normalized time, frame-to-frame velocity, and a ligand-type embedding. Protein content features include residue identity, the local backbone vectors N–Cα and C–Cα, the centered Cα coordinate, and a protein-type embedding. No geometric data augmentation is applied. Although the protein structure is static, its positional coordinate in frame-wise cross-attention is paired with the temporal coordinate of the corresponding ligand frame. Specifically, a protein residue with Cα position (xj,yj,zj)(x_j,y_j,z_j) is assigned the event coordinate (ti,xj,yj,zj)(t_i,x_j,y_j,z_j) when attending to ligand atoms at frame tit_i. Single- and Multi-trajectory Settings. For single-trajectory prediction, we follow the NeuralMD protocol and train a separate model for each of the ten protein–ligand systems reported in Table 1. Training windows contain 1010 observed frames and 2020 target frames, following the same in10/out20 split and protocol as NeuralMD. For multi-trajectory prediction, we use MISATO-100, MISATO-1000, and MISATO-Full. After preprocessing, the three datasets contain 80/10/1080/10/10, 800/100/100800/100/100, and 13,066/1,357/1,35713,066/1,357/1,357 training/validation/test complexes, respectively. Each trajectory contributes one window containing 8080 observed frames followed by 2020 target frames. Model Architecture. The molecular model consists of four repeated blocks, each containing frame-wise ligand-to-protein cross-attention followed by global self-attention over all ligand event tokens in the observed trajectory, as illustrated in Figure S1. The hidden dimension is 128128, with 88 attention heads and a 512512-dimensional feed-forward layer. Both the attention and feed-forward sublayers use pre-normalization and residual connections, with dropout 0.10.1. A final LayerNorm is applied after the Transformer stack. After the final block, only the representation of each ligand atom at the last observed frame is passed to the prediction head. The displacement decoder consists of 128→512→60128→ 512→ 60 with a GELU nonlinearity and dropout 0.10.1. The 6060 outputs correspond to the three-dimensional displacements of the 2020 future frames. All future frames are predicted in parallel, and the predicted displacements are added to the last observed ligand positions. Optimization Objective. The model is trained using a combination of coordinate prediction error and pairwise structural consistency. For a predicted ligand trajectory ^f,i r_f,i and target trajectory f,ir_f,i, we use ℒ=ℒcoord+2ℒpair,whereℒcoord=∑f,i‖^f,i−f,i‖1∑fNf,L=L_coord+2L_pair, _coord= _f,i \| r_f,i-r_f,i \|_1 _fN_f, (40) and ℒpairL_pair is the mean SmoothL1 loss, with β=0.5β=0.5, between predicted and target inter-atomic distances over all valid unordered atom pairs i<ji<j and future frames. Training Details. We train all molecular dynamics models with Adam, zero weight decay, and a linear warmup followed by cosine learning-rate decay. The backbone is warmed up for two epochs, while the decoder learning rate is applied without warmup. For the single-trajectory setting, both the backbone and decoder use a learning rate of 10−410^-4, and four temporal windows are accumulated per optimizer step. For the multi-trajectory setting, the backbone learning rate is 3×10−53× 10^-5, the decoder learning rate remains 10−410^-4, and the batch size is eight complexes. Training uses BF16 mixed precision for at most 100100 epochs with early-stopping patience 2020. No gradient clipping is applied. All reported results use random seeds 00, 4242, and 123123. Evaluation Metrics. We follow the evaluation definitions of NeuralMD. Let ef,ij=d^f,ij−df,ije_f,ij= d_f,ij-d_f,ij denote the error between predicted and target pairwise distances at future frame f. The coordinate error is measured by MAE=∑f,i‖^f,i−f,i‖1∑fNf,MAE= _f,i \| r_f,i-r_f,i \|_1 _fN_f, (41) where the ℓ1 _1 norm sums the errors over the three Cartesian coordinates. For each evaluated sample and frame, Matching is the root-mean-square error over all valid ordered atom pairs, Matchingf=1|f|∑(i,j)∈fef,ij 2,Matching_f= 1|P_f| _(i,j) _fe_f,ij^\,2, (42) where fP_f includes the diagonal and both (i,j)(i,j) and (j,i)(j,i). The reported Matching score is averaged over all evaluated sample–frame pairs. Stability measures the fraction of pairwise distances whose absolute error does not exceed 0.50.5 Å: Stabilityf=100×∑(i,j)∈f[|ef,ij|≤0.5Å]|f|.Stability_f=100× _(i,j) _f 1\! [|e_f,ij|≤ 0.5~ ]|P_f|. (43) The final Stability score is the average over all evaluated sample–frame pairs. For the single-trajectory comparison in Table 1, the VerletMD, GNN-MD, NeuralMD-ODE, NeuralMD-SDE, and DenoisingLD results are taken from NeuralMD, while BioMD and MinkowskiPE are evaluated in our implementation under the same data split and evaluation protocol. For the multi-trajectory comparison in Table 2, all baselines are retrained under the in80/out20 protocol described above. B.3 Video Prediction Data and Preprocessing. We follow the OpenSTL [40] preprocessing and evaluation protocol for the KTH human-action dataset [33]. Persons 01–16 are used for training and persons 17–25 for testing, producing 5,2005,200 training clips and 3,1673,167 test clips. Each clip contains 3030 consecutive grayscale frames, with the first 1010 used as observations and the subsequent 2020 as prediction targets. All frames are resized to 128×128128× 128 and pixel intensities are scaled to [0,1][0,1]. During training, the same spatial augmentation is applied synchronously to all frames in a clip. The frames are first bilinearly resized by a factor of 1/0.951/0.95, followed by a random 128×128128× 128 crop and horizontal flipping with probability 0.50.5. Following the OpenSTL preprocessing protocol, 30-frame clips are extracted with start-index strides of 3 frames for jogging and running, and 30 frames for boxing, handclapping, handwaving, and walking. Model Architecture. Each observed grayscale frame is concatenated with its frame difference, giving a two-channel input. The convolutional encoder consists of three stages, 2→36→72→1442→ 36→ 72→ 144, reducing the spatial resolution from 128×128128× 128 to 16×1616× 16. Each stage uses a 4×44× 4 convolution with stride 22, followed by GroupNorm and GELU. The resulting latent feature dimension is 144144. The 1010 observed latent maps are combined with 2020 future latent maps, initialized from the final observed latent map. Each future token is augmented with time-step and token-type embeddings, together with a velocity embedding computed from the difference between the final two observed latent maps. The velocity embedding is modulated by a zero-initialized learned gate. The resulting 30×16×16=7,68030× 16× 16=7,680 event tokens are jointly processed by an eight-layer global Transformer equipped with MinkowskiPE. The Transformer has hidden dimension 144144, 88 attention heads, and a 576576-dimensional feed-forward layer. The final 2020 latent maps are decoded through progressive bilinear upsampling from 16×1616× 16 to 32×3232× 32, 64×6464× 64, and 128×128128× 128. The first two decoding stages use skip connections from the corresponding encoder features of the last observed frame. The convolutional channel dimensions of the three stages are 216→72→72,108→36→36,36→18→18,216→ 72→ 72, 108→ 36→ 36, 36→ 18→ 18, followed by a final 3×33× 3 convolution from 1818 channels to one output channel. The decoder predicts residuals relative to the last observed frame, and all 2020 future frames are generated in parallel. The resulting model contains 2,482,0862,482,086 parameters in total, reported as 2.52.5M in Table 3. Training and Evaluation. The model is trained using the mean squared error between predicted and target frames. We use Adam with a learning rate of 10−310^-3, zero weight decay, and a OneCycle learning-rate schedule. The global batch size is 1616, the gradient norm is clipped at 1.01.0, and training uses BF16 mixed precision. We train for at most 100100 epochs with an early-stopping patience of 2020 epochs. We follow the OpenSTL evaluation protocol. Within each run, we select the checkpoint with the lowest MSE on the test dataset. We report MSE, MAE, SSIM, PSNR, and LPIPS following the metric definitions used in OpenSTL, with LPIPS computed using the AlexNet backbone. All experiments use random seeds 0, 42, and 123. Appendix C Constraints on Galilean-Invariant Positional Encoding Our use of Lorentz transformations provides a representation-level geometric bias and does not impose Lorentz symmetry on the underlying dynamics. If one instead seeks a positional encoding that respects the physical symmetry of non-relativistic dynamics, Galilean symmetry is a natural starting point [19]. This appendix examines a particular construction based on finite-dimensional unitary token-wise transformations. We show that exact invariance under common Galilean transformations forces the resulting pairwise interaction to discard the spatial component of the relative group element. This obstruction depends on the unitary assumption, which is not satisfied by the Lorentz boosts used in MinkowskiPE. It therefore identifies a constraint on this specific route to Galilean-invariant positional encoding. Galilean Relative Structure. Consider first the rotation-free Galilei group G0G_0, generated by spatial translations, time translations, and Galilean boosts. An element is written as g=(a,b,v),g=(a,b,v), (44) where a∈ℝda ^d is a spatial translation, b∈ℝb is a time translation, and v∈ℝdv ^d is a boost. The group law is (a1,b1,v1)(a2,b2,v2)=(a1+a2+v1b2,b1+b2,v1+v2).(a_1,b_1,v_1)(a_2,b_2,v_2)=(a_1+a_2+v_1b_2,\,b_1+b_2,\,v_1+v_2). (45) We denote the corresponding subgroups by T(a)=(a,0,0),S(b)=(0,b,0),B(v)=(0,0,v).T(a)=(a,0,0), S(b)=(0,b,0), B(v)=(0,0,v). (46) For spacetime coordinates (x,t)(x,t) alone, a common Galilean boost acts as x↦x+vt,t↦t,x x+vt, t t, (47) and therefore transforms a relative displacement according to Δxij↦Δxij+vΔtij. x_ij x_ij+v t_ij. (48) Thus, unlike in the translational setting used by MinkowskiPE, Δxij x_ij itself is not invariant under a common Galilean boost. To obtain a boost-invariant relative spatial quantity, one may augment each token with a velocity u and write z=(x,t,u).z=(x,t,u). (49) Identifying such a state with the group element g(z)=(x,t,u),g(z)=(x,t,u), (50) the relative group element is g(zi)−1g(zj)=(rij,Δtij,Δuij),g(z_i)^-1g(z_j)=(r_ij, t_ij, u_ij), (51) where rij=xj−xi−ui(tj−ti),Δtij=tj−ti,Δuij=uj−ui.r_ij=x_j-x_i-u_i(t_j-t_i), t_ij=t_j-t_i, u_ij=u_j-u_i. (52) Under a common Galilean boost, x↦x+vt,u↦u+v,x x+vt, u u+v, (53) the quantity rijr_ij remains unchanged. Hence rijr_ij is the natural spatial component of the relative Galilean group element. A Finite-dimensional Obstruction. We now consider a direct rotary-style token-wise encoding. Let U:G0→U(M)U:G_0→ U(M) be a continuous finite-dimensional unitary encoding, where M is the feature dimension, and define the pairwise operator A(gi,gj)=U(gi)†U(gj).A(g_i,g_j)=U(g_i) U(g_j). (54) We require invariance under a common Galilean transformation, A(hgi,hgj)=A(gi,gj)for all h,gi,gj∈G0.A(hg_i,hg_j)=A(g_i,g_j) all h,g_i,g_j∈ G_0. (55) This is the direct group-theoretic analogue of the relative-position property underlying rotary positional encodings. Proposition 1. Let U:G0→U(M)U:G_0→ U(M) be continuous, with M finite, and suppose that the pairwise operator in Equation 54 satisfies Equation 55. Then A(gi,gj)A(g_i,g_j) is independent of the spatial component rijr_ij of the relative group element gi−1gjg_i^-1g_j. It may depend only on Δtij t_ij and Δuij u_ij. Proof. Define F(g)=A(e,g).F(g)=A(e,g). (56) Choosing h=gi−1h=g_i^-1 in Equation 55 gives A(gi,gj)=F(gi−1gj).A(g_i,g_j)=F(g_i^-1g_j). (57) Moreover, the token-wise factorization in Equation 54 implies A(gi,gj)A(gj,gk)=A(gi,gk).A(g_i,g_j)A(g_j,g_k)=A(g_i,g_k). (58) Combining Equation 57 and Equation 58, and writing a=gi−1gja=g_i^-1g_j, c=gj−1gkc=g_j^-1g_k, we obtain F(a)F(c)=F(ac).F(a)F(c)=F(ac). (59) Thus F is a continuous finite-dimensional unitary representation of G0G_0. Let PrP_r, H, and KrK_r denote the Hermitian generators of spatial translations, time translations, and boosts: F(T(a))=e−ia⋅P,F(S(b))=e−ibH,F(B(v))=e−iv⋅K.F(T(a))=e^-ia· P, F(S(b))=e^-ibH, F(B(v))=e^-iv· K. (60) The Galilei group law in Equation 45 gives B(v)S(b)B(v)−1S(b)−1=T(vb),B(v)S(b)B(v)^-1S(b)^-1=T(vb), (61) and therefore the generators satisfy [Kr,H]=iPr.[K_r,H]=iP_r. (62) Spatial translations are central in the rotation-free subgroup G0G_0, so [Pr,Ks]=0,[Pr,H]=0.[P_r,K_s]=0, [P_r,H]=0. (63) Using Equation 62 and Equation 63, tr(Pr2)=−itr(Pr[Kr,H])=−itr([PrKr,H])=0.tr(P_r^2)=-i\,tr\! (P_r[K_r,H] )=-i\,tr\! ([P_rK_r,H] )=0. (64) Because PrP_r is Hermitian, tr(Pr2)tr(P_r^2) is the sum of the squared eigenvalues of PrP_r. Hence Pr=0P_r=0 for every r=1,⋯,dr=1,·s,d. It follows that F(T(r))=Ifor all r∈ℝd.F(T(r))=I all r ^d. (65) Using Equation 51 and Equation 57 A(gi,gj)=F(T(rij)S(Δtij)B(Δuij))=F(S(Δtij)B(Δuij)),A(g_i,g_j)=F\! (T(r_ij)S( t_ij)B( u_ij) )=F\! (S( t_ij)B( u_ij) ), (66) which is independent of rijr_ij. □ Implications for Positional Encoding. The proposition does not imply that Galilean positional encoding is impossible in general. Rather, it identifies an obstruction specific to exact, finite-dimensional, unitary, token-wise factorizations of the form in Equation 54. Such an encoding may still represent relative time and relative velocity. For example, one may define U(z)=⨁k=1M/2R(ωkt+κk⊤u),U(z)= _k=1^M/2R\! ( _kt+ _k u ), (67) where R(⋅)R(·) is a planar rotation. Its pairwise interaction depends on ωkΔtij+κk⊤Δuij, _k t_ij+ _k u_ij, (68) but contains no dependence on the Galilean-invariant spatial quantity rijr_ij, in agreement with Proposition 1. Other approaches, including explicitly pairwise constructions based on Galilean invariants, non-unitary representations, and infinite-dimensional representations, fall outside the assumptions of Proposition 1. Projective representations and central extensions provide further directions that require a separate analysis [3, 19]. These alternatives remain open possibilities for incorporating Galilean structure into positional encoding.