Paper deep dive
TimeRoute: Time-Aware Modality Routing and Diffusion for Multi-Modal Recommendation
Pengyu Zhang, Yangqin Jiang, Klim Zaporojets, Congfeng Cao, Paul Groth
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/16/2026, 3:39:57 AM
Summary
The paper introduces TimeRoute, a time-aware multi-modal recommender system that addresses the 'modality time-scale mismatch' where the relevance of item modalities (text, image, audio) drifts over time. TimeRoute employs a temporal-aware modal router to generate personalized modality fusion weights based on user behavioral profiles and a diffusion-based graph reconstructor conditioned on temporal signals via Feature-wise Linear Modulation (FiLM) to suppress outdated modality edges. Experiments on TikTok, Amazon-Baby, and Amazon-Sports datasets show significant improvements in Recall@K, Precision@K, and NDCG@K over baselines.
Entities (10)
Relation Signals (8)
TimeRoute → addresses → Modality Time-Scale Mismatch
confidence 95% · We address both challenges within a unified diffusion-based recommender, TimeRoute.
TimeRoute → containscomponent → Temporal-Aware Modal Router
confidence 95% · A temporal-aware modal router maps each user's aggregated behavioral features to a personalized modality distribution
TimeRoute → containscomponent → Diffusion-Based Graph Reconstructor
confidence 95% · The diffusion-based graph reconstructor is then conditioned on the same temporal profile
TimeRoute → evaluatedon → TikTok
confidence 95% · Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements
TimeRoute → evaluatedon → Amazon Baby
confidence 95% · Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements
TimeRoute → evaluatedon → Amazon-Sports
confidence 95% · Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements
TimeRoute → outperforms → DiffMM
confidence 90% · Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements... over strong baselines
Diffusion-Based Graph Reconstructor → usestechnique →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-modal recommenders fuse collaborative signals with item modalities such as text, images, and audio, but the usefulness of each drifts over time and at different rates. For example, chocolate purchases typically guided by textual ingredient cues can shift toward visual packaging and ambient audio around Valentine's Day. This modality time-scale mismatch gives rise to two coupled challenges: (1) users require different modality proportions across temporal contexts, and (2) less relevant modalities are more likely to introduce outdated or misleading signals into the recommender. We address both challenges within a unified diffusion-based recommender, TimeRoute. A temporal-aware modal router maps each user's aggregated behavioral features to a personalized modality distribution, replacing the globally shared fusion weights used in prior work. The diffusion-based graph reconstructor is then conditioned on the same temporal profile through Feature-wise Linear Modulation (FiLM) with dual-stream long- and short-term denoising heads, suppressing outdated modality edges before they enter the propagation graph. Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements of up to 9.8\% in Recall@K, Precision@K, and NDCG@K over strong baselines across 10-seed paired tests. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.10983v1
- Canonical: https://arxiv.org/abs/2608.10983v1
Trouble viewing inline? Open PDF directly →
Full Text
72,976 characters extracted from source content.
Expand or collapse full text
TimeRoute: Time-Aware Modality Routing and Diffusion for Multi-Modal RecommendationDOI: X.XXXXXXXConference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NYISBN: 978-1-4503-X-X/2018/06CCS: Information systems Recommender systemsCCS: Computing methodologies Neural networksCCS: Computing methodologies Probabilistic reasoning Pengyu Zhang email: p.zhang@uva.nl OrcID: 0000-0001-5111-4487 Affiliation: University of Amsterdam , Amsterdam , The Netherlands , Yangqin Jiang OrcID: 0009-0002-3936-1703 Affiliation: University of Hong Kong , Hong Kong , China , Klim Zaporojets OrcID: 0000-0003-4988-978X Affiliation: Aarhus University , Aarhus , Denmark , Congfeng Cao OrcID: 0000-0001-9011-3807 Affiliation: University of Amsterdam , Amsterdam , The Netherlands and Paul Groth OrcID: 0000-0003-0183-6910 Affiliation: University of Amsterdam , Amsterdam , The Netherlands 2018; © , 2018; Received 5 June 2009 Abstract. Multi-modal recommenders fuse collaborative signals with item modalities such as text, images, and audio, but the usefulness of each drifts over time and at different rates. For example, chocolate purchases typically guided by textual ingredient cues can shift toward visual packaging and ambient audio around Valentine’s Day. This modality time-scale mismatch gives rise to two coupled challenges: (1) users require different modality proportions across temporal contexts, and (2) less relevant modalities are more likely to introduce outdated or misleading signals into the recommender. We address both challenges within a unified diffusion-based recommender, TimeRoute. A temporal-aware modal router maps each user’s aggregated behavioral features to a personalized modality distribution, replacing the globally shared fusion weights used in prior work. The diffusion-based graph reconstructor is then conditioned on the same temporal profile through Feature-wise Linear Modulation (FiLM) with dual-stream long- and short-term denoising heads, suppressing outdated modality edges before they enter the propagation graph. Experiments on TikTok, Amazon-Baby, and Amazon-Sports demonstrate consistent improvements of up to 9.8% in Recall@K, Precision@K, and NDCG@K over strong baselines across 10-seed paired tests. Code is available at https://anonymous.4open.science/r/TimeRoute. Keywords: Multi-modal Recommendation, Time-aware Recommendation, Diffusion Models, Modality Fusion 1. Introduction Multi-modal recommender systems power large-scale platforms such as e-commerce marketplaces and short-video applications, where rich item content from images, text, and audio helps overcome sparse user-item feedback (32; 6). Recent graph-based approaches that fuse modality-specific signals with collaborative filtering through graph neural networks achieve strong performance across benchmarks (24). However, the usefulness of each modality is not static. User interests and item popularity drift over time (33), and the drift speed varies across modalities (37; 34). We refer to this phenomenon as modality time-scale mismatch: the relevance of each modality shifts over time, but not uniformly across modalities. Figure 1 illustrates this point. Chocolate is purchased year-round, but around Valentine’s Day a user’s decision shifts from textual cues such as ingredients toward visual packaging and atmospheric audio, then reverts after the holiday. The key observation is that different modalities become more or less relevant at different moments and evolve at different rates, requiring the same user to rely on different modality proportions over time (15). Figure 1. An example of modality time-scale mismatch. The same user purchasing chocolates relies primarily on textual cues such as ingredients during normal periods (Scenario A), and shifts toward visual packaging and audio around Valentine’s Day (Scenario B). Modality relevance varies with the user’s temporal context and evolves at different rates.fig:modality_timescale_mismatch This mismatch gives rise to two interrelated problems. The first problem (Problem 1) is that the same user requires different modality fusion proportions in different temporal contexts. Since modality importance shifts at different rates, the optimal fusion depends on which temporal context the user is currently in (17). A user’s temporal behavior features, such as interaction frequency, recency, and distribution, encode this temporal state. However, existing methods, including DiffMM (6), rely on a globally shared fusion weight, assigning the same modality mixture to a user during a holiday shopping spree and routine purchases months later. To address this, we introduce a temporal-aware modal router that infers each user’s temporal context from aggregated behavioral features and produces a per-user softmax distribution over modalities. The router output determines how modality-specific embeddings are fused before GCN propagation, enabling the model to emphasize the most relevant modalities for each user at each point in time. The second problem (Problem 2) is the flip side of the first: a modality that is highly relevant in one temporal context can become a source of noise in another. Signals that were once informative may persist long after they have lost their relevance (20; 12); for example, Valentine-themed visual cues may remain in the graph shortly after Valentine’s Day, even when they are no longer useful for routine purchases. Modalities that are less relevant in the current context are more likely to contribute such outdated signals. While the modal router partially mitigates this by down-weighting less-relevant modalities, the second problem also manifests at the graph structure level, where modality-induced edges built from outdated signals propagate noise through message passing. To address this structural dimension, we condition the diffusion-based graph reconstruction (6) on temporal signals through Feature-wise Linear Modulation (FiLM) (14), which applies learned feature-wise scale and shift parameters to modulate the user context. We combine this conditioning with dual-stream long- and short-term denoising heads (31; 28). The reconstructor thus accounts for temporal context when rebuilding modality-specific graphs, suppressing outdated edges before they propagate outdated noisy signals downstream. Together with the modal router, the framework provides temporal awareness at two complementary levels: the reconstruction of modality-specific graphs (Problem 2) and the fusion of their resulting embeddings (Problem 1). We evaluate TimeRoute on TikTok, Amazon-Baby, and Amazon-Sports with 10-seed paired tests, and summarize our contributions as follows: • We identify modality time-scale mismatch, the phenomenon that different modalities shift in relevance at different rates over time, as the root cause of two interrelated problems: (1) the same user needs different modality fusion proportions in different temporal contexts, and (2) modalities that are less relevant in the current context are more likely to carry outdated, misleading signals. • We propose TimeRoute, a framework with two complementary components. A temporal-aware modal router personalizes modality fusion for Problem 1; a time-conditioned diffusion reconstructor uses FiLM modulation and dual-stream long- and short-term heads to suppress outdated modality edges for Problem 2. • Across three benchmarks, TimeRoute consistently outperforms strong baselines. Controlled noise-input ablations show that temporal signals are a key driver of the router’s gains (p<0.001p<0.001); quartile analysis confirms meaningful per-user differentiation (p<10−6p<10^-6). 2. Related Work Graph-based Multi-Modal Recommendation Multi-modal recommendation leverages item content such as images, text, and audio to alleviate interaction sparsity and improve robustness. Early approaches treated modalities as side information, fusing content features into matrix factorization or ranking models; VBPR exemplifies this line by injecting visual features into Bayesian personalized ranking for implicit feedback (2). Building on this, attention-based methods such as ACF reweight user preferences across modalities at the user-item-component level (1). More recent work adopts graph neural networks as the backbone: MMGCN and GRCN propagate modality signals over user-item graphs to capture high-order dependencies (25; 24), LATTICE extends this idea by mining latent content-induced structures from raw modality features (30), and BM3 employs bootstrap-style self-supervision to learn from imperfect multi-modal signals (35). Despite this progress, modality fusion in graph-based multi-modal recommendation is typically treated as time-invariant: a single fusion weight is learned and applied uniformly across users and time. This formulation is at odds with the modality time-scale mismatch identified in Section 1, where the users with different temporal behavioral profiles require different modality proportions. To address this limitation, our temporally aware modality router generates personalized modality weights from each user’s temporal behavior profile, replacing globally shared fusion with user- and time-adaptive weighting. Temporal Modeling for Recommendation Temporal modeling has long been recognized as essential for recommender systems. Koren first introduced temporal dynamics into collaborative filtering by modeling drift in user preferences and item popularity (7). Building on this idea, sequential recommenders such as TiSASRec inject time-interval information into self-attention to distinguish events that are sequentially close but temporally distant (8). More recent work confronts temporal distribution shift in deployed systems, with retrieval-and-distill paradigms and probabilistic generalization frameworks designed to keep predictions stable as data drifts (37; 34). In multi-modal settings, XSMoE handles distribution shifts in streaming recommendation by adding expandable side experts that absorb new modalities or items as they arrive (15). Yet these temporal models remain largely disconnected from multi-modal fusion. Temporal collaborative filtering and sequential recommenders model user and item dynamics, but typically operate on unimodal interaction sequences. Recent work on temporal distribution shift treats prediction as a single evolving signal rather than a fusion of multiple modalities. Likewise, XSMoE’s per-modality experts are designed to absorb newly arriving modalities over time, rather than reweight existing modalities as their relevance changes within a user. Our work occupies this intersection: the modal router uses each user’s temporal profile to drive per-user multi-modal fusion, while dual-stream long- and short-term heads add temporal modulation within each per-modality denoiser. Diffusion Models for Recommendation Generative models offer an alternative to discriminative ranking. Early work used GANs and VAEs for interaction modeling, representation learning, and data augmentation under sparsity (19; 9). Diffusion models were subsequently adopted for their training stability and capacity to model complex interaction distributions and have since been applied to interaction generation, sequential recommendation, and conditional denoising under contextual signals (20; 22; 11). Building on this line, DiffMM extends diffusion to multi-modal recommendation by reconstructing modality-aware user-item graphs and combining them with cross-modal contrastive learning for alignment (6). DiffMM and related multi-modal diffusion recommenders condition the denoiser on a user context that is itself invariant to temporal context: the same context vector drives reconstruction whether the user has shifted into a new behavioral regime or remains in a stable long-term pattern. The reconstructed graphs therefore cannot reflect when a modality’s relevance has changed, and outdated modality-induced edges continue to propagate through GCN message passing. We address this by conditioning per-modality denoisers on temporal signals through FiLM modulation with dual-stream long- and short-term heads, so that reconstructed graphs respond to shifts in each user’s temporal context. 3. Methodology We propose TimeRoute (Figure 2), a diffusion-based multi-modal recommender that addresses the two challenges from Section 1. To address Problem 1, that the same user requires different modality fusion proportions in different temporal contexts, we introduce a temporal-aware modal router that produces personalized modality fusion weights from each user’s temporal behavior profile. To address Problem 2, that modalities less relevant in the current context tend to carry outdated, misleading signals, we condition the diffusion-based reconstruction of modality-specific user-item graphs on temporal information through FiLM modulation (14) with dual-stream long- and short-term heads. Together, these two components provide temporal awareness at two complementary levels: the fusion of modality-specific embeddings (Problem 1) and the reconstruction of the modality-specific graphs that produce those embeddings (Problem 2). We first present the preliminaries (Section 3.1) and the pipeline (Section 3.2); we then detail the modal router (Section 3.3) and the time-conditioned diffusion reconstructor (Section 3.4); finally, we describe the joint training objective and the three-phase per-epoch schedule (Section 3.5). Figure 2. Overview of TimeRoute. From each user’s interaction timestamps, a per-user temporal profile is constructed and fed into two parallel pathways. The temporal-aware modal router (bottom) produces per-user modality fusion weights for the GCN. The time-conditioned diffusion reconstructor (top) modulates the user context via dual-stream FiLM heads and reconstructs modality-specific user-item graphs. In the graph recommendation framework, the reconstructed graphs are used together with the base interaction graph in the per-modality propagation block, after which router-guided user fusion and global item-side fusion produce representations for GCN-based top-K recommendation.fig:timeroute_overview 3.1. Preliminaries and Problem Formulation Task formulation. We consider implicit-feedback top-K recommendation, where user preferences are inferred from observed interactions such as clicks, purchases, or views, and the goal is to rank the top-K items most relevant to each user. Let U and V denote the sets of users and items, with interactions encoded by a binary matrix ∈0,1||×||R∈\0,1\^|U|×|V|, where Ru,v=1R_u,v=1 indicates an observed interaction. We write u∈0,1||r_u∈\0,1\^|V| for the u-th row of R (the same vector is denoted 0x_0 when discussing diffusion in Section 3.4). Following (6), we score user-item pairs by the inner product of their final embeddings, y^u,v=u⊤v y_u,v=h_u h_v, and return the top-K items per user. Multi-modal inputs. Each item v is associated with raw modality features ^vm∈ℝdm f^m_v ^d_m for each modality m in the modality set ℳM. In our experiments, ℳ=image,textM=\image,text\ on Amazon-Baby and Amazon-Sports, and ℳ=image,text,audioM=\image,text,audio\ on TikTok. Temporal inputs. Each observed interaction (u,v)(u,v) is associated with a unix timestamp τu,v _u,v. From the full sequence of timestamps for user u, we derive a per-user temporal profile from a common set of semantic fields (interaction recency, calendar context, inter-event gaps, weekday/hour patterns). The two downstream pathways consume this profile in different vectorized forms: the modal router uses a 1616-dimensional form ~u∈ℝ16 t_u ^16 in which cyclic fields are sin-cos encoded (Section 3.3.1), and the diffusion conditioner uses a 1212-dimensional raw form u∈ℝ12t_u ^12 that is then processed by per-field encoders (Section 3.4.2). 3.2. Graph Recommendation Framework We introduce the pipeline before the two main components, so that each component’s role in the larger framework is clear. Modality feature projection. To compare and combine modalities of differing dimensionality, we first project raw modality features into a shared d-dimensional space via learnable linear projectors ϕm _m (a linear layer with leaky-ReLU): vm=ϕm(^vm)∈ℝdf^m_v= _m( f^m_v) ^d for m∈ℳm . We let m∈ℝ||×dF^m ^|V|× d denote the matrix stacking vmf^m_v over items. User context embedding. Both diffusion denoising and graph learning benefit from a compact representation that summarizes a user’s history. We introduce learnable ID embedding matrices U∈ℝ||×dE_U ^|U|× d and V∈ℝ||×dE_V ^|V|× d, and define the user context embedding for user u as: (1) =(U)u+uV∈ℝd,u=(E_U)_u+r_u\,E_V ^d, where (U)u(E_U)_u is the user’s ID embedding and uVr_u\,E_V aggregates the embeddings of items the user has interacted with. The vector u serves as the time-modulation target inside the diffusion denoiser: it is modulated by the FiLM heads to produce the dual-stream user contexts longu^long and shortu^short (Equation (10)) used for denoising. Modality-specific propagation and fusion. The diffusion reconstructor (Section 3.4) produces, for each modality, a reconstructed user-item graph ^m R^m. Concretely, for each modality m, we obtain modality-specific user and item embeddings (um,vm)(e^m_u,\,e^m_v) via two complementary propagation steps following (6): (i) two-step LightGCN-style propagation (4) on the base interaction graph R initialized with the normalized modality features normalize(m)normalize(F^m) stacked under the user ID embeddings, producing a base modality embedding (⋅)m,basee^m,base_(·); and (i) one-step propagation on the reconstructed graph ^m R^m initialized with the ID embeddings [U;V][E_U;E_V], producing a reconstruction-based embedding (⋅)m,rece^m,rec_(·). The two are combined via (2) (⋅)m=(⋅)m,base+λadj(⋅)m,rec,e^m_(·)=e^m,base_(·)+ _adj\,e^m,rec_(·), where λadj _adj controls the contribution of the reconstructed graph relative to the base graph. The modality-specific embeddings are then fused into a single representation per node by the temporal-aware modal router defined in Section 3.3, producing ufusee^fuse_u and vfusee^fuse_v. A final stack of L GCN layers (4) on the base graph then refines (⋅)fusee^fuse_(·) by summing layer-wise outputs and adding a normalized residual term controlled by λres _res, producing the final embeddings (⋅)h_(·) used for scoring. 3.3. Temporal-Aware Modal Router The modal router is the model’s mechanism for addressing Problem 1. It infers each user’s typical temporal context from their interaction history and produces personalized modality fusion weights. We describe its five components: temporal profile construction, the routing function, the weight floor, per-user modality fusion, and a diversity regularizer. 3.3.1. User Temporal Profile Construction For each user u, we extract 1212 raw temporal fields from the user’s interaction timestamps. The same set of fields feeds two downstream pathways with different vectorization: a 1616-dimensional router input ~u t_u used in this section, and the 1212-dimensional raw form ut_u that the diffusion conditioner processes through per-field encoders (Section 3.4.2). The 12 fields are organized into three semantic groups: Long-term context (4 fields): global_days, user_days, year_offset, month, encoding the user’s overall position in time and historical span. Short-term context (6 fields): delta_log (log-scaled inter-event gap), delta_bucket (a quantile-based gap id, used only by the diffusion conditioner and not part of the router input), pos_norm (normalized recency), weekday, hour, is_weekend, encoding the local temporal structure of recent activity. Auxiliary fields (2 fields): day of month and unix_timestamp, used by the router to disambiguate users with otherwise identical aggregated signatures. Each field is aggregated across the user’s interactions using a recency-weighted average, with weights proportional to interaction position so that more recent interactions contribute more. Cyclic fields (month, day, weekday, hour) are aggregated via sin -cos averaging followed by angular reconstruction, ensuring that, for example, the average of 23:0023:00 and 01:0001:00 is 00:0000:00 rather than 12:0012:00. Vectorization for the router. For the router, the four cyclic fields are converted to (sin,cos)( , ) pairs to preserve circular proximity, so that adjacent values such as December and January or 23:00 and 01:00 remain close in representation space (8 dimensions). The four continuous temporal fields (global_days, user_days, year_offset, unix_timestamp) are normalized to a comparable range to prevent large-scale timestamp values from dominating the router MLP (4 dimensions). We clip delta_log and pos_norm to [0,1][0,1] to bound extreme gap and recency values (2 dimensions), and encode is_weekend as a 2-dimensional one-hot indicator to represent the binary calendar context without imposing an ordinal scale. This yields the 1616-dimensional router input ~u∈ℝ16 t_u ^16, which encodes when each user is active, how recently, and how their activity is distributed. The encoding is lightweight, adding only 610 parameters on Amazon-Baby and Amazon-Sports and 643 on TikTok (see Appendix11 1 https://anonymous.4open.science/r/TimeRoute/TimeRoute_appendix.pdf). 3.3.2. Routing Function Given ~u t_u, the router produces a probability distribution over modalities through a two-layer MLP followed by a softmax: (3) u=Softmax(2σ(1~u+1)+2),w_u=Softmax\! (W_2\,σ(W_1 t_u+b_1)+b_2 ), where 1∈ℝ32×16W_1 ^32× 16, 2∈ℝ|ℳ|×32W_2 ^|M|× 32, and σ is the ReLU activation. The output u∈Δ|ℳ|−1w_u∈ ^|M|-1 is a per-user modality distribution. We initialize 2W_2 and 2b_2 to zero so that u=1/|ℳ|w_u=1/|M| at initialization; this warm start prevents aggressive routing decisions before the modality-specific embeddings have meaningfully differentiated. As training progresses, the router gradually departs from uniform toward dataset- and user-appropriate weights. 3.3.3. Weight Floor A naive softmax can produce degenerate solutions in which one modality is entirely suppressed across all users, particularly on datasets where one modality dominates the collaborative signal. Such global modality collapse would undermine Problem 1: instead of learning user-specific modality proportions, the router would simply remove a modality from the fusion process. To prevent this, we rescale the softmax output to enforce a minimum weight floor: wu,m′=ϵ+(1−|ℳ|ϵ)wu,mw _u,m=ε+(1-|M|ε)\,w_u,m with ϵ=0.05ε=0.05, which preserves the simplex structure (∑mwu,m′=1 _mw _u,m=1) while guaranteeing wu,m′≥ϵw _u,m≥ε for every modality. Even modalities deemed less relevant by the router still contribute a small but non-trivial signal, which reduces sensitivity to miscalibrated routing decisions and ensures that all modality branches continue to receive training signal. 3.3.4. Per-User Modality Fusion Modality fusion is applied differently on the user and item sides: (4) ufuse ^fuse_u =∑m∈ℳwu,m′⋅um,u∈, = _m w _u,m·e^m_u, u , (5) vfuse ^fuse_v =∑m∈ℳβm⋅vm,v∈, = _m _m·e^m_v, v , where =Softmax(~) β=Softmax( β) is a globally learned modality weight shared across all items. This difference reflects an information asymmetry: user-level temporal profiles provide informative signals for personalized modality weighting (e.g., activity level and recency can shift the balance between visual and textual signals; see Section 4.4), whereas items lack an analogous temporal context. We therefore retain a global item-side modality weight, allowing β to capture dataset-level modality importance without introducing unnecessary item-level routing parameters. 3.3.5. Diversity Regularization The weight floor (Section 3.3.3) prevents any single modality from being fully suppressed, but does not by itself prevent a more subtle failure mode: the router can converge to a near-uniform global weight that is essentially shared across users, recovering DiffMM-like behavior with no meaningful per-user adaptation. To explicitly encourage per-user differentiation, we add a diversity regularizer. This directly supports Problem 1: if the router collapses to nearly identical weights for all users, it reduces to a global modality mixture and can no longer adapt modality fusion to user-specific temporal behavior: (6) ℒdiv=1|ℬ|∑u∈ℬH(u)⏟per-user entropy−H(1|ℬ|∑u∈ℬu)⏟batch-mean entropy,L_div= 1|B| _u H(w_u)_per-user entropy\;-\; H\! ( 1|B| _u w_u )_batch-mean entropy, where H()=−∑mpmlogpmH(p)=- _mp_m p_m is the Shannon entropy, ℬB is the current minibatch of users, and uw_u denotes the pre-floor softmax output (Equation (3)) to preserve gradient flow through the router during optimization. Minimizing ℒdivL_div encourages each user toward a confident routing decision (low per-user entropy) while keeping the batch-averaged distribution broadly balanced across modalities (high batch-mean entropy). This encourages differentiated user-specific modality proportions while preventing collapse to a single globally dominant modality. The regularizer is added to the joint loss with a single hyperparameter λdiv _div, tuned per dataset (see our released code at https://anonymous.4open.science/r/TimeRoute for full values). 3.4. Time-Conditioned Diffusion Reconstruction To address Problem 2, we extend the diffusion-based graph reconstruction pipeline of DiffMM (6) along two orthogonal axes. Time conditioning is introduced via FiLM-modulated user context with dual-stream long- and short-term heads (Section 3.4.2). Modality grounding is introduced via an alignment regularizer that ties modality-specific reconstructions to the modality feature space (Section 3.4.3). The reconstructed graphs ^m R^m produced by this module serve as inputs to the modality-specific propagation described in Section 3.2. 3.4.1. Diffusion Background We follow the standard DDPM formulation (5; 6): the forward process gradually corrupts 0=ux_0=r_u via a Gaussian transition kernel q(t∣0)=(α¯t0,(1−α¯t))q(x_t _0)=N( α_tx_0,(1- α_t)I) with variance schedule α¯t\ α_t\, and a per-modality MLP denoiser DθmD_ _m predicts ^0m=Dθm(t,t,′) x_0^m=D_ _m(x_t,t,u ), where ′u is the time-modulated user context defined in the next subsection. Up to this point, the formulation follows DiffMM. TimeRoute differs in two aspects. First, the denoiser is conditioned on each user’s temporal profile through dual-stream FiLM modulation: long- and short-term temporal encoders generate user-specific scale and shift parameters for the denoising context, and a gate combines the two reconstructions. Second, the denoiser is trained with modality-aware alignment and a time-reweighted reconstruction loss, encouraging reconstructed graphs to remain both modality-consistent and sensitive to recent behavioral shifts. 3.4.2. Time Conditioning via FiLM and Dual-Stream Heads Each modality m∈ℳm has its own denoiser DθmD_ _m, with independently learned field-level encoders, long- and short-term fusion MLPs, FiLM heads, and a gating network. For clarity, we describe the architecture generically below; the same formulation applies to every modality, with parameters learned independently for each m. A standard diffusion-based recommender conditions the denoiser on a single user context u, which remains fixed across temporal contexts. As a result, the denoiser tends to reconstruct similar interaction patterns even when the user’s recent behavior shifts, increasing the risk of propagating outdated modality signals (Problem 2). We address this limitation by time-modulating u via FiLM and equipping the denoiser with separate long- and short-term streams, which capture slowly and rapidly evolving behavioral dynamics, respectively. Field-level temporal encoding. Each of the 1212 fields in ut_u is encoded individually before fusion. Continuous temporal fields, such as global_days, user_days, year_offset, and unix_timestamp, are projected through a linear layer. Cyclic fields, including month, day, weekday, and hour, are transformed into (sin,cos)( , ) representations to preserve periodic structure, keeping values such as December and January or 23:00 and 01:00 close in representation space; the inter-event gap field additionally provides a quantile-based bucket id bub_u that is mapped through an embedding table to capture non-linear gap effects. The per-field representations are then routed into two streams based on temporal scope: (7) ulong ^long_u =MLPlong([ϕ(t)]t∈long), =MLP_long\! (\,[φ(t)]_t _long\, ), (8) ushort ^short_u =MLPshort([ϕ(t)]t∈short), =MLP_short\! (\,[φ(t)]_t _short\, ), where ϕ(⋅)φ(·) denotes the appropriate per-field encoder, [⋅][\,·\,] denotes concatenation, and the long- and short-term field sets are long=tuglb,tuusd,tuyr,tumoS_long=\t^glb_u,t^usd_u,t^yr_u,t^mo_u\ and short=tudlt,bu,tuhr,tuwd,tuwe,tuposS_short=\t^dlt_u,b_u,t^hr_u,t^wd_u,t^we_u,t^pos_u\. The partition into longS_long and shortS_short reflects complementary temporal scales: longS_long captures slowly evolving signals, such as long-term usage history and seasonal patterns, whereas shortS_short models rapidly changing interaction dynamics, including inter-event timing and time-of-day effects. Dual FiLM parameters. Each stream produces its own scale and shift parameters that modulate the user context u defined in Equation (1). For stream s∈long,shorts∈\long,short\: (9) (us,us) ( γ^s_u,\, β^s_u) =ℱs(us), =F_s(h^s_u), (10) s ^s =⊙us+us, =u γ^s_u+ β^s_u, where ℱlongF_long and ℱshortF_short are linear heads. The FiLM shift parameters us β^s_u are distinct from the item-side fusion weights βm _m in Equation (5); we follow the standard FiLM notation despite this overlap. Following (14), we parameterize =+ELU(⋅) γ=1+ELU(·) and =tanh(⋅) β= (·) so that the modulation is centered around an identity transformation, preventing the conditioner from destabilizing early training. Dual-stream denoising and gated combination. The denoiser is run twice per training step, once with longu^long and once with shortu^short as the conditioning user context, producing two reconstructions ^0m,long x_0^m,long and ^0m,short x_0^m,short. The two are combined via a per-user gate: (11) αu _u =σ(MLPgate([ulong,ushort,t])), =σ\! (MLP_gate([\,h^long_u,\,h^short_u,\,e_t\,]) ), (12) ^0m x_0^m =αu⋅^0m,long+(1−αu)⋅^0m,short, = _u· x_0^m,long+(1- _u)· x_0^m,short, where σ is the sigmoid function and te_t is the diffusion-step embedding. This design enables the denoiser to adaptively balance complementary temporal dynamics on a per-user basis, interpolating between slowly evolving long-term patterns and rapidly changing short-term behavior. The user-specific gate αu _u adjusts this balance according to the temporal behavior profile. 3.4.3. Modality-aware Alignment Regularization The denoiser does not take the modality feature matrix mF^m as input directly. To inject modality semantics, we introduce an alignment regularizer that ties the reconstructed user-item scores to the modality feature space. Concretely, for user u, the denoiser predicts a clean interaction vector ^0,um x_0,u^m over items, where the subscript 00 denotes the denoised diffusion state rather than a user index. Projecting this vector through the modality feature matrix mF^m should be consistent with the observed-interaction summary 0,uVx_0,uE_V: (13) ℒalignm=u[‖^0,umm−0,uV‖22].L^m_align=E_u [ \| x_0,u^m\,F^m-x_0,u\,E_V \|_2^2 ]. The term ^0,umm x_0,u^mF^m projects the denoiser’s predicted interactions into modality-m feature space, producing a modality-aware representation of user u. In contrast, 0,uVx_0,uE_V projects the user’s observed interactions into the shared ID embedding space. Minimizing the distance between the two aligns the modality-specific reconstruction with observed interaction behavior, encouraging the denoiser to remain faithful to the user’s preferences while respecting the geometry of modality m. This keeps the denoiser architecture compact while still enabling modality-differentiated graphs ^m R^m. 3.4.4. Time-Reweighted Reconstruction Loss and Graph Assembly A standard diffusion reconstruction loss treats all of a user’s interactions equally. However, more recent interactions are typically more diagnostic of the user’s current preferences, and inaccurate reconstruction near the temporal frontier is precisely where outdated modality signals are most likely to mislead the recommender. We therefore introduce a time-reweighted variant of the per-modality reconstruction loss: (14) ℒdiffm=u,t,ϵ[wsnr(t)⋅(1+λt⋅pos_normu)⋅‖0−^0m‖22],L^m_diff=E_u,\,t,\, ε\! [\,w_snr(t)· (1+ _t·pos\_norm_u )· \|x_0- x_0^m \|_2^2 ], where wsnr(t)w_snr(t) is the SNR-derived diffusion-step weight (standard in DDPM training), pos_normu∈[0,1]pos\_norm_u∈[0,1] is the normalized recency of u’s most recent interaction (11 corresponds to the most recent), and λt _t is a hyperparameter tuned per dataset. The reweighting biases the denoiser to prioritize accurate reconstruction near the temporal frontier, where modality relevance is most likely to have shifted, directly addressing the structural side of Problem 2. Once the per-modality denoisers are trained, we sample ^0m x_0^m via reverse diffusion under the FiLM conditioning of Section 3.4.2 and assemble the modality-specific reconstructed graph ^m R^m by taking the top-k predicted items per user as edges. These reconstructed graphs are then passed to the downstream per-modality propagation block, as illustrated in the rightmost panel of Figure 2. 3.5. Training Objective and Optimization 3.5.1. Joint Loss Function The model is optimized with an alternating three-phase schedule under a composite objective. that combines BPR-based ranking, cross-modality contrastive learning, the per-modality diffusion losses, the router diversity regularizer, and weight regularization: (15) ℒ=ℒBPR+λclℒCL+∑m∈ℳ(ℒdiffm+λeℒalignm)+λdivℒdiv+λreg‖Θ‖22,L=L_BPR+ _clL_CL+\! _m \! (L^m_diff+ _e\,L^m_align )+ _div\,L_div+ _reg\,\| \|_2^2, where ℒBPRL_BPR (16) is the standard pairwise ranking loss computed from the GCN-fused embeddings; ℒCLL_CL is the multi-view contrastive loss inherited from DiffMM (6), which contrasts modality-specific user/item views against an aggregated view that does not pass through the router; ℒdiffmL^m_diff is the time-reweighted diffusion loss (Equation (14)); ℒalignmL^m_align is the modality-aware alignment regularizer (Equation (13)); ℒdivL_div is the router diversity regularizer (Equation (6)); and Θ collects all trainable parameters. The hyperparameters λcl _cl, λe _e, λdiv _div, λt _t, and λreg _reg are tuned per dataset (see the released code for full values). 3.5.2. Three-Phase Per-Epoch Schedule Each training epoch proceeds in three phases: Diffusion training. For each minibatch of users, we run a forward-reverse pass through every per-modality denoiser, conditioned on the user’s temporal profile and the FiLM-modulated user context (Equations (7)-(12)). Only the denoiser-side parameters θmm\ _m\_m are updated, including the temporal encoders, FiLM heads, and denoising MLPs, via ∑m(ℒdiffm+λeℒalignm) _m(L^m_diff+ _eL^m_align). This isolates graph generation from ranking optimization, so the denoisers first learn temporally conditioned modality-specific reconstructions before the recommendation modules are trained on the assembled graphs. Graph reconstruction. Using the just-updated denoisers, we sample ^0m x_0^m for every user and modality and assemble ^mm∈ℳ\ R^m\_m via the top-k procedure in Section 3.4.4. GCN and recommendation training. Using ^m\ R^m\ as inputs to modality-specific propagation, we compute the fused user and item embeddings via the modal router (Equations (4)-(5)) and update the remaining parameters (UE_U, VE_V, projectors ϕmm\ _m\_m, ~ β, and θrouter _router) via ℒBPR+λclℒCL+λdivℒdiv+λreg‖Θ‖22L_BPR+ _clL_CL+ _divL_div+ _reg\| \|_2^2. Although ℒCLL_CL appears in the Phase 3 loss, the contrastive views are constructed by per-modality propagation without routing. This way, ℒCLL_CL does not flow gradients through the router. This isolation prevents the contrastive objective, which can favor near-uniform modality weights, from pulling the router away from ranking-optimized per-user fusion. This is important for Problem 1: the router must preserve user-specific modality distributions rather than collapse toward a global mixture. 4. Experiments We evaluate TimeRoute on three widely used multi-modal benchmarks: TikTok, Amazon-Baby, and Amazon-Sports.22 2 https://github.com/HKUDS/DiffMM/tree/main/Datasets The three datasets span different temporal ranges, timestamp coverages, and user activity distributions, which allow us study modality time-scale mismatch under diverse settings. Full dataset details and the full list of compared baselines are provided in Appendix.33 3 https://anonymous.4open.science/r/TimeRoute/TimeRoute_appendix.pdf We organize the experiments around five research questions: • RQ1 (Overall Effectiveness). Does TimeRoute consistently improve top-K recommendation over strong baselines across multiple benchmarks and evaluation metrics? • RQ2 (Temporal Necessity). Is the performance improvement driven by temporal information in the router input, or merely by the additional parameters introduced by the router MLP? • RQ3 (User-Level Routing). Does the modal router learn meaningful per-user differentiation in modality preferences, and do these preferences vary with users’ temporal behavior patterns? • RQ4 (Component Contributions). What is the individual contribution of each component, including the modal router, dual-stream denoising, FiLM conditioning, and time-weighted loss, to the overall performance? • RQ5 (Noise Robustness). Does learned modality routing improve robustness to modality noise, particularly under uneven corruption across modalities (e.g., stronger Gaussian noise on image features than text features)? 4.1. Experimental Setup 4.1.1. Evaluation metrics. We report three top-K ranking metrics at K=20K=20: Recall@20 (R@20), Precision@20 (P@20), and NDCG@20 (N@20). R@20 measures whether the ground-truth items appear in the top-20 list; P@20 measures the fraction of correct items among the top-20; and N@20 further weights hits by their rank, rewarding correct items near the top of the list. To highlight gains over a strong reference, we report the relative improvement of TimeRoute over the strongest baseline in each table: (16) Δ (%)=(Our Result−Strongest BaselineStrongest Baseline)×100%.$ $ (\%)= ( Our Result-Strongest BaselineStrongest Baseline )× 100\%. Full metric definitions and dataset preprocessing are in appendix.44 4 https://anonymous.4open.science/r/TimeRoute/TimeRoute_appendix.pdf 4.1.2. Statistical testing. To ensure reliability, every experiment is run with 10 different seeds (1, 7, 42, 123, 456, 666, 789, 888, 999, 2026), and we report the mean and standard deviation. Statistical significance is assessed with paired two-sided t-tests at the p<0.05p<0.05 threshold. The same protocol is applied to all comparisons against baselines and to all ablation comparisons. 4.1.3. Data splits. We use the Random split (main setting) protocol for train/validation/test and report top-K recommendation performance, matching prior diffusion-based multi-modal recommenders. To rule out concerns about “future” interactions appearing in training and to assess temporal generalization, we additionally report results under a Chronological split (supplementary setting): for each user, earlier interactions are used for training and later ones are held out for validation and test, with the evaluation protocol unchanged. 4.2. Overall Effectiveness (RQ1) We first examine whether TimeRoute delivers consistent gains across all datasets and evaluation protocols. We report results under two settings: a standard random split evaluated against a full set of strong baselines, and a deployment-aligned per-user chronological split, which probes whether the gains hold under temporal distribution shift. ∙ Random split. Table 1 summarizes the random-split results. TimeRoute outperforms every baseline on every dataset and metric. On TikTok, R@20 rises from the strongest baseline value of 0.11290.1129 to 0.12080.1208 (+7.00%+7.00\%), and N@20 rises from 0.04600.0460 to 0.04930.0493 (+7.17%+7.17\%). On Amazon-Baby and Amazon-Sports, P@20 improves over the strongest baseline by +9.80%+9.80\% and +5.36%+5.36\%, respectively. Compared with the diffusion-based baselines DiffMM and KDiffE, TimeRoute explicitly uses temporal profiles for both modality routing and graph reconstruction, allowing the model to adapt fusion and reconstructed graphs to user-level temporal behavior. The gains are not limited to recall: TimeRoute also improves precision and ranking quality, with the largest relative gains observed on P@20 for the Amazon datasets and N@20 for TikTok. Table 1. Overall performance on the Random split. We report R@20, P@20, and N@20 on TikTok, Amazon-Baby, and Amazon-Sports. Δ(%) (\%) denotes the relative improvement of TimeRoute over the best baseline in each column. Results for TimeRoute are reported as the mean over 10 random seeds. Best results are in bold; second-best are underlined. TikTok Amazon-Baby Amazon-Sports R@20 P@20 N@20 R@20 P@20 N@20 R@20 P@20 N@20 General collaborative filtering MF-BPR (16) 0.0346 0.0017 0.0130 0.0440 0.0024 0.0200 0.0430 0.0023 0.0202 NGCF (21) 0.0604 0.0030 0.0238 0.0591 0.0032 0.0261 0.0695 0.0037 0.0318 LightGCN (4) 0.0653 0.0033 0.0282 0.0698 0.0037 0.0319 0.0782 0.0042 0.0369 SGL (26) 0.0603 0.0030 0.0238 0.0678 0.0036 0.0296 0.0779 0.0041 0.0361 NCL (10) 0.0658 0.0034 0.0269 0.0703 0.0038 0.0311 0.0765 0.0040 0.0349 HCCF (27) 0.0662 0.0029 0.0267 0.0705 0.0037 0.0308 0.0779 0.0041 0.0361 Multi-modal recommendation VBPR (3) 0.0380 0.0018 0.0134 0.0486 0.0026 0.0213 0.0582 0.0031 0.0265 LGCN-M (4) 0.0679 0.0034 0.0273 0.0726 0.0038 0.0329 0.0705 0.0035 0.0324 MMGCN (25) 0.0730 0.0036 0.0307 0.0640 0.0032 0.0284 0.0638 0.0034 0.0279 GRCN (24) 0.0804 0.0036 0.0350 0.0754 0.0040 0.0336 0.0833 0.0044 0.0377 LATTICE (30) 0.0843 0.0042 0.0367 0.0829 0.0044 0.0368 0.0915 0.0048 0.0424 CLCRec (23) 0.0621 0.0032 0.0264 0.0610 0.0032 0.0284 0.0651 0.0035 0.0301 MMGCL (29) 0.0799 0.0037 0.0326 0.0758 0.0041 0.0331 0.0875 0.0046 0.0409 SLMRec (18) 0.0845 0.0042 0.0353 0.0765 0.0043 0.0325 0.0829 0.0043 0.0376 BM3 (36) 0.0957 0.0048 0.0404 0.0839 0.0044 0.0361 0.0975 0.0051 0.0442 DiffMM (6) 0.1129 0.0056 0.0456 0.0975 0.0051 0.0411 0.1017 0.0054 0.0458 KDiffE (13) 0.1120 0.0056 0.0460 0.0950 0.0050 0.0400 0.1020 0.0056 0.0460 TimeRoute 0.1208 0.0059 0.0493 0.1023 0.0056 0.0451 0.1094 0.0059 0.0496 Δ (%) +7.00 +5.36 +7.17 +4.92 +9.80 +9.73 +7.25 +5.36 +7.83 ∙ Chronological split. Under the random split, the model observes modality signals from the full temporal range during training, which can obscure the impact of outdated signals at test time. We therefore re-evaluate under a per-user chronological split, training on earlier interactions and testing on later ones. Given the computational cost of rerunning all methods under this protocol, we compare against DiffMM, the closest diffusion-based backbone and the strongest baseline re-evaluated under the chronological split. Figure 3 reports the relative gains. TimeRoute improves every metric on every dataset, with particularly large gains on N@20: +16.11%+16.11\% on TikTok, +11.02%+11.02\% on Amazon-Baby, and +12.59%+12.59\% on Amazon-Sports. This pattern reinforces the findings under the random split: when test interactions reflect a different temporal regime than training, globally shared modality weights cannot adapt to shifting user behavior, whereas a per-user router can. (a) R@20. (b) P@20. (c) N@20. Figure 3. Stability under the chronological split, comparing TimeRoute with DiffMM, the closest diffusion-based multimodal backbone baseline, on TikTok, Amazon-Baby, and Amazon-Sports. We report (a) R@20, (b) P@20, and (c) N@20. Numbers above each pair of bars give the relative improvement of TimeRoute over DiffMM.Three grouped bar charts comparing TimeRoute against the DiffMM baseline on TikTok, Amazon-Baby, and Amazon-Sports under the chronological split, one chart per metric (R@20, P@20, N@20). 4.3. Temporal Necessity (RQ2) The results of RQ1 establish that TimeRoute outperforms the baseline, but leave an important question unanswered: are the gains driven by the temporal signal in the router input, or merely by the additional parameters introduced by the MLP? To disentangle the two effects, we conduct a controlled ablation on Amazon-Baby. The router’s temporal input is replaced with fixed random noise of identical dimensionality (16-dimensional Gaussian, with a seed independent of the training seed), keeping the architecture and parameter count unchanged. Two outcomes are then possible: if the gains arise primarily from MLP capacity, the noise-input router should match the temporal-input router; if they arise from the temporal signal, the noise-input router should perform similarly to removing the router entirely. Table 2 reports both predictions tested side by side. The noise-input router reaches R@20 = 0.09530.0953, statistically indistinguishable from the no-router baseline at 0.09550.0955 (p=0.50p=0.50). The temporal-input router reaches 0.10230.1023, significantly outperforming both (p<0.001p<0.001). Together, these findings indicate that the router’s gains are driven by temporal information rather than additional MLP capacity. Table 2. Noise-input ablation on Amazon-Baby (Random split). We compare three configurations: the full model with temporal features, a router with random noise input, and no router. Results are reported as the mean ± std over 10 seeds. p-values are from paired t-tests against the full model. Best results per metric in bold. Configuration R@20 N@20 p vs Full Full (time features) 0.1023±0.00050.1023± 0.0005 0.0451±0.00030.0451± 0.0003 - Noise input 0.0953±0.00070.0953± 0.0007 0.0405±0.00030.0405± 0.0003 <0.001<0.001 No router 0.0955±0.00040.0955± 0.0004 0.0405±0.00030.0405± 0.0003 <0.001<0.001 Noise vs No router: p=0.50p=0.50 (not significant) 4.4. User-Level Routing (RQ3) The results of RQ2 confirm that the temporal signal drives the router’s gain. The next question is whether the router learns meaningful per-user differentiation, or simply converges to a single global weighting that improves over DiffMM’s default. To investigate this, we partition users on Amazon-Baby and TikTok into four interaction-count quartiles, where Q1 contains the sparsest users and Q4 the densest. For each quartile, we report the number of users n, the mean per-user routing entropy H=u[H(u)]H=E_u[H(w_u)], and the learned modality weights wimgw_img, wtxtw_txt (and waudw_aud for TikTok). Figure 4 reveals a clear upward trend on both datasets. On Amazon-Baby, the mean image weight rises from 0.5240.524 at Q1 to 0.5880.588 at Q4, a relative increase of 12.2%12.2\%. The trend is more pronounced on TikTok: wimgw_img increases from 0.5990.599 at Q1 to a near-saturated 0.8300.830 at Q4, while text and audio weights decline from around 0.200.20 to below 0.090.09. At the same time, within-group variability decreases: on TikTok, the per-user standard deviation of wimgw_img drops from 0.3630.363 at Q1 to 0.1940.194 at Q4 (a 47%47\% reduction), and the mean per-user entropy decreases from 0.6500.650 to 0.4760.476. Together, these patterns suggest that highly active users exhibit increasingly confident, image-dominant routing, whereas sparse users retain more heterogeneous and uncertain modality preferences. The systematic variation across quartiles indicates that the router learns differentiated user-specific modality weighting rather than collapsing to a single global solution. (a) Amazon-Baby (N=19,445N=19,445). (b) TikTok (N=9,308N=9,308). Figure 4. Per-user routing analysis by user activity quartile on (a) Amazon-Baby and (b) TikTok. For each quartile, we plot the mean modality weights wimgw_img, wtxtw_txt (and waudw_aud for TikTok) together with the mean per-user routing entropy. Q1 contains users with the fewest interactions, whereas Q4 contains the most active users.Two grouped bar charts showing per-user modality routing weights across user activity quartiles Q1 to Q4 on Amazon-Baby and TikTok. The pattern is consistent with the mechanism underlying Problem 1. Sparse users provide limited collaborative signal, leading the router to distribute weight more evenly across modalities so that textual or audio cues can compensate for weak visual evidence. As interaction histories grow, the visual modality becomes increasingly informative for these users, and the router shifts weight toward image features that better capture their preferences. Because this adaptation is driven entirely by each user’s temporal behavior profile, the resulting fusion is personalized rather than globally shared. 4.5. Component Contributions (RQ4) RQ2 and RQ3 establish that the router’s gains are driven by temporal information and that the learned routing is genuinely user-specific. We now ask which components of the framework actually contribute to these gains. Four ablations are evaluated, each removing one component while keeping the rest intact: w/o Modal Router, w/o Dual-Stream, w/o FiLM, and w/o Time-Weight. Figure 5 reports R@20 on the random split. Full dataset details are provided in Appendix.55 5 https://anonymous.4open.science/r/TimeRoute/TimeRoute_appendix.pdf Grouped bar chart of R@20 on Amazon-Baby, Amazon-Sports, and TikTok comparing the full TimeRoute model against four ablations (w/o Modal Router, w/o Dual-Stream, w/o FiLM, w/o Time-Weight). Figure 5. Component ablation on the Random split. Each bar group compares the full model against four ablations (w/o Modal Router, w/o Dual-Stream, w/o FiLM, w/o Time-Weight) on Amazon-Baby, Amazon-Sports, and TikTok. R@20 is reported as the mean over 10 seeds.Grouped bar chart of R@20 on Amazon-Baby, Amazon-Sports, and TikTok comparing the full TimeRoute model against four ablations (w/o Modal Router, w/o Dual-Stream, w/o FiLM, w/o Time-Weight). ∙ Every component contributes. Removing any single component degrades R@20 on every dataset, with relative drops ranging from −2.19%-2.19\% to −4.06%-4.06\%. The four temporal mechanisms therefore act as distinct contributors, rather than redundant components for a single dominant module. ∙ The modal router is the largest contributor on Baby and TikTok. On Amazon-Baby, removing the router reduces R@20 by −3.71%-3.71\%, the largest single-component effect on this dataset. On TikTok the drop is similarly large at −3.56%-3.56\%, tied with the time-weighted loss. Even on Amazon-Sports, where timestamp coverage is the lowest at 63%63\%, removing the router still reduces R@20 by −2.93%-2.93\%, suggesting that it extracts useful user-specific temporal signals rather than collapsing to a uniform weight when timestamps are sparse. ∙ Reconstruction-level components dominate on Sports and TikTok. The dual-stream denoiser is the single largest contributor on Amazon-Sports (−3.29%-3.29\%) and TikTok (−4.06%-4.06\%), while the time-weighted loss also contributes substantially on Amazon-Baby (−3.62%-3.62\%). Together with the modal router, these results show that fusion-level temporal awareness (the router, addressing Problem 1) and reconstruction-level temporal awareness (dual-stream denoising, and time-weighted loss, addressing Problem 2) operate as complementary mechanisms; both layers are required for full performance. ∙ FiLM contributes consistently though not dominantly. Removing FiLM degrades performance by −2.19%-2.19\% to −2.93%-2.93\% across the three datasets, the smallest single-component effect on Sports and TikTok but still well above the across-seed noise floor. The consistent degradation indicates that FiLM’s per-user temporal modulation contributes meaningfully to reconstruction-level temporal awareness, rather than redundantly duplicating the role of the router or dual-stream denoising. ∙ All ablations remain above the strongest baseline. Even with any single component removed, the model still outperforms the strongest R@20 baseline in Table 1 on all datasets (Amazon-Baby: 0.09850.0985 vs. DiffMM 0.09750.0975; Amazon-Sports: 0.10580.1058 vs. KDiffE 0.10200.1020; TikTok: 0.11590.1159 vs. DiffMM 0.11290.1129). These results suggest that the framework’s gains are distributed across complementary components rather than driven by any single dominant mechanism. 4.6. Noise Robustness (RQ5) RQ4 confirms that the framework’s gains hold even when individual components are removed. We next examine robustness. The intuition behind Problem 2 is that temporal routing should be especially beneficial when modalities differ in noise level: by down-weighting less relevant modalities, the model reduces exposure to outdated or noisy signals. To evaluate this, we inject Gaussian noise into modality features at controlled per-modality ratios (noise std == ratio × feature std) and compare our full model (Ours) against a variant with all temporal components disabled (NoTime). Table 3 reports the four configurations on Amazon-Baby. Ours consistently outperforms NoTime across all settings, with the absolute performance gap remaining stable in the range 0.00450.0045-0.00530.0053, including under uneven modality noise. Table 3. Uneven noise stability on Amazon-Baby (R@20, mean over 10 seeds). img_noisy: noise on image only (ratio = 0.5). txt_noisy: noise on text only (ratio = 0.5). Mixed: noise_img = 0.5, noise_txt = 0.1. Clean: noise-free runs. The absolute advantage of Ours over NoTime remains in the range 0.0045-0.0053 across all four settings, indicating that the temporal router preserves its benefits when noise levels differ across modalities. Best results per column in bold. Model Clean img_noisy txt_noisy Mixed Ours 0.1023 0.1020 (−-0.29%) 0.1007 (−-1.56%) 0.1019 (−-0.39%) NoTime 0.0978 0.0967 (−-1.12%) 0.0962 (−-1.64%) 0.0967 (−-1.12%) Under img_noisy (image noise at ratio 0.50.5, text clean), Ours drops by only 0.29%0.29\% while NoTime drops by 1.12%1.12\%, widening the performance gap from +0.0045+0.0045 (clean) to +0.0053+0.0053. The limited degradation suggests that temporal routing mitigates the impact of corrupted image features by reducing reliance on less relevant signals, whereas NoTime remains more sensitive due to its fixed fusion strategy. Under txt_noisy (text noise ratio 0.50.5, image clean), both models degrade by similar amounts (Ours −1.56%-1.56\%, NoTime −1.64%-1.64\%) and the performance gap remains at +0.0045+0.0045. Although Ours is more exposed to corrupted text signals due to its stronger reliance on text, the dual-stream denoiser and time-weighted reconstruction loss provide additional graph-level robustness, allowing the model to retain its advantage rather than sacrificing performance under increased noise. Under Mixed (image noise at 0.50.5, text noise at 0.10.1), an asymmetric corruption setting, the performance gap reaches its largest value of +0.0052+0.0052. This setting does not assume that one modality is always noisier than another; instead, it tests whether TimeRoute remains robust when modality reliability is uneven. This configuration more closely reflects real-world modality imbalance, and the widening gap provides strong evidence that adaptive temporal routing is particularly beneficial when modality quality is uneven. Taken together, RQ5 supports the intuition behind Problem 2 from two complementary angles: per-user routing reduces exposure to noisy signals in down-weighted modalities, while the temporal components in the diffusion reconstructor protect against noise in dominant modalities at the graph level. The net effect is a consistent improvement across all four noise settings, with the largest gain observed under the more realistic uneven noise setting. 5. Conclusion We proposed TimeRoute, a diffusion-based multi-modal recommender for modality time-scale mismatch. TimeRoute combines a temporal-aware modal router, which maps each user’s temporal context to personalized modality weights, with time-conditioned diffusion reconstruction, which uses FiLM-modulated dual-stream denoisers to suppress outdated graph-level signals. Across TikTok, Amazon-Baby, and Amazon-Sports, 10-seed paired tests show consistent improvements over strong baselines. Controlled ablations confirm that the gains come from temporal signals rather than extra router parameters, while quartile analysis shows systematic per-user routing differentiation. Noise-robustness experiments further show effectiveness under uneven modality quality. TimeRoute has limitations. Routing depends on timestamp coverage; on Amazon-Sports, reconstruction-level components contribute more than the router. When one modality dominates, as visual features do on TikTok, highly active users may converge toward near-single-modality weights. Adaptive routing reduces exposure to down-weighted noisy modalities but increases reliance on the dominant one. Future work could add modality-quality estimation and sequential recommendation. GenAI Usage Disclosure The authors used generative AI tools for English proofreading, minor grammar edits, sentence rewriting, and clarity improvements. All content was reviewed and approved by the authors, who take full responsibility for the work. Acknowledgements. The first author is supported by the China Scholarship Council (NO. 202206540007) and the University of Amsterdam. This funding source had no influence on the study design, data collection, analysis, or manuscript preparation and approval. References Chen et al. (2017) J. Chen, H. Zhang, X. He, L. Nie, W. Liu, and T. Chua Attentive collaborative filtering: multimedia recommendation with item- and component-level attention. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’17, New York, NY, USA, p. 335–344. External Links: ISBN 9781450350228, Link, Document Cited by: §2. He and McAuley (2016a) R. He and J. McAuley VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, p. 144–150. Cited by: §2. He and McAuley (2016b) R. He and J. McAuley VBPR: visual bayesian personalized ranking from implicit feedback. In AAAI, Vol. 30. Cited by: Table 1. He et al. (2020) X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang Lightgcn: simplifying and powering graph convolution network for recommendation. In SIGIR, p. 639–648. Cited by: §3.2, §3.2, Table 1, Table 1. Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. External Links: ISBN 9781713829546 Cited by: §3.4.1. Jiang et al. (2024) Y. Jiang, L. Xia, W. Wei, D. Luo, K. Lin, and C. Huang DiffMM: multi-modal diffusion model for recommendation. In Proceedings of the 32nd ACM International Conference on Multimedia, M ’24, New York, NY, USA, p. 7591–7599. External Links: ISBN 9798400706868, Link, Document Cited by: §1, §1, §1, §2, §3.1, §3.2, §3.4.1, §3.4, §3.5.1, Table 1. Koren (2010) Y. Koren Collaborative filtering with temporal dynamics. Commun. ACM 53 (4), p. 89–97. External Links: ISSN 0001-0782, Link, Document Cited by: §2. Li et al. (2020) J. Li, Y. Wang, and J. McAuley Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining, WSDM ’20, New York, NY, USA, p. 322–330. External Links: ISBN 9781450368223, Link, Document Cited by: §2. Liang et al. (2018) D. Liang, R. G. Krishnan, M. D. Hoffman, and T. Jebara Variational autoencoders for collaborative filtering. In Proceedings of the 2018 World Wide Web Conference, W ’18, Republic and Canton of Geneva, CHE, p. 689–698. External Links: ISBN 9781450356398, Link, Document Cited by: §2. Lin et al. (2022) Z. Lin, C. Tian, Y. Hou, and W. X. Zhao Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In W, p. 2320–2329. Cited by: Table 1. Liu et al. (2023) Q. Liu, F. Yan, X. Zhao, Z. Du, H. Guo, R. Tang, and F. Tian Diffusion augmentation for sequential recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM ’23, New York, NY, USA, p. 1576–1586. External Links: ISBN 9798400701245, Link, Document Cited by: §2. Liu et al. (2025) S. Liu, A. Zhang, G. Hu, H. Qian, and T. Chua Preference diffusion for recommendation. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1. Mo et al. (2025) X. Mo, F. Liu, R. Tang, J. Gao, and H. Liu Knowledge-aware diffusion-enhanced multimedia recommendation. IEEE Transactions on Multimedia 27 (), p. 9136–9146. External Links: Document Cited by: Table 1. Perez et al. (2018) E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville FiLM: visual reasoning with a general conditioning layer. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, AAAI’18/IAAI’18/EAAI’18. External Links: ISBN 978-1-57735-800-8 Cited by: §1, §3.4.2, §3. Qu et al. (2025) Y. Qu, L. Qu, T. Chen, Q. V. H. Nguyen, and H. Yin Efficient multimodal streaming recommendation via expandable side mixture-of-experts. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, New York, NY, USA, p. 2460–2470. External Links: ISBN 9798400720406, Link, Document Cited by: §1, §2. Rendle et al. (2009) S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme BPR: bayesian personalized ranking from implicit feedback. In UAI, p. 452–461. Cited by: §3.5.1, Table 1. Siraudin et al. (2025) A. Siraudin, F. D. Malliaros, and C. Morris Cometh: a continuous-time discrete-state graph diffusion model. External Links: Link Cited by: §1. Tao et al. (2022) Z. Tao, X. Liu, Y. Xia, X. Wang, L. Yang, X. Huang, and T. Chua Self-supervised learning for multimedia recommendation. Transactions on Multimedia (TMM). Cited by: Table 1. Wang et al. (2017) J. Wang, L. Yu, W. Zhang, Y. Gong, Y. Xu, B. Wang, P. Zhang, and D. Zhang IRGAN: a minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’17, New York, NY, USA, p. 515–524. External Links: ISBN 9781450350228, Link, Document Cited by: §2. Wang et al. (2023) W. Wang, Y. Xu, F. Feng, X. Lin, X. He, and T. Chua Diffusion recommender model. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, New York, NY, USA, p. 832–841. External Links: ISBN 9781450394086, Link, Document Cited by: §1, §2. Wang et al. (2019) X. Wang, X. He, M. Wang, F. Feng, and T. Chua Neural graph collaborative filtering. In SIGIR, p. 165–174. Cited by: Table 1. Wang et al. (2024) Y. Wang, Z. Liu, L. Yang, and P. S. Yu Conditional denoising diffusion for sequential recommendation. In Advances in Knowledge Discovery and Data Mining: 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Taipei, Taiwan, May 7–10, 2024, Proceedings, Part V, Berlin, Heidelberg, p. 156–169. External Links: ISBN 978-981-97-2264-8, Link, Document Cited by: §2. Wei et al. (2021) Y. Wei, X. Wang, Q. Li, L. Nie, Y. Li, X. Li, and T. Chua Contrastive learning for cold-start recommendation. In M, p. 5382–5390. Cited by: Table 1. Wei et al. (2020) Y. Wei, X. Wang, L. Nie, X. He, and T. Chua Graph-refined convolutional network for multimedia recommendation with implicit feedback. In Proceedings of the 28th ACM International Conference on Multimedia, M ’20, New York, NY, USA, p. 3541–3549. External Links: ISBN 9781450379885, Link, Document Cited by: §1, §2, Table 1. Wei et al. (2019) Y. Wei, X. Wang, L. Nie, X. He, R. Hong, and T. Chua MMGCN: multi-modal graph convolution network for personalized recommendation of micro-video. In Proceedings of the 27th ACM International Conference on Multimedia, M ’19, New York, NY, USA, p. 1437–1445. External Links: ISBN 9781450368896, Link, Document Cited by: §2, Table 1. Wu et al. (2021) J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, and X. Xie Self-supervised graph learning for recommendation. In SIGIR, p. 726–735. Cited by: Table 1. Xia et al. (2022) L. Xia, C. Huang, Y. Xu, J. Zhao, D. Yin, and J. Huang Hypergraph contrastive collaborative filtering. In SIGIR, p. 70–79. Cited by: Table 1. Yi et al. (2025) K. Yi, Q. Zhang, W. Fan, L. Cao, S. Wang, H. He, G. Long, L. Hu, Q. Wen, and H. Xiong A survey on deep learning based time series analysis with frequency transformation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD ’25, New York, NY, USA, p. 6206–6215. External Links: ISBN 9798400714542, Link, Document Cited by: §1. Yi et al. (2022) Z. Yi, X. Wang, I. Ounis, and C. Macdonald Multi-modal graph contrastive learning for micro-video recommendation. In SIGIR, p. 1807–1811. Cited by: Table 1. Zhang et al. (2021) J. Zhang, Y. Zhu, Q. Liu, S. Wu, S. Wang, and L. Wang Mining latent structures for multimedia recommendation. In Proceedings of the 29th ACM International Conference on Multimedia, M ’21, New York, NY, USA, p. 3872–3880. External Links: ISBN 9781450386517, Link, Document Cited by: §2, Table 1. Zhang et al. (2025a) Q. Zhang, Y. Sun, H. Wen, P. Yang, X. Li, M. Li, K. Lam, S. Yiu, and H. Yin Time series analysis in frequency domain: a survey of open challenges, opportunities and benchmarks. External Links: 2504.07099, Link Cited by: §1. Zhang et al. (2025b) S. Zhang, L. Chen, D. Shen, C. Wang, and H. Xiong Hierarchical time-aware mixture of experts for multi-modal sequential recommendation. In Proceedings of the ACM on Web Conference 2025, W ’25, New York, NY, USA, p. 3672–3682. External Links: ISBN 9798400712746, Link, Document Cited by: §1. Zhang et al. (2022) Z. Zhang, X. Wang, Z. Zhang, H. Li, Z. Qin, and W. Zhu Dynamic graph neural networks under spatio-temporal distribution shift. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. External Links: ISBN 9781713871088 Cited by: §1. Zheng et al. (2025) L. Zheng, N. Li, W. Zhang, and Y. Yu Retrieval and distill: a temporal data shift-free paradigm for online recommendation system. External Links: 2404.15678, Link Cited by: §1, §2. Zhou et al. (2023a) X. Zhou, H. Zhou, Y. Liu, Z. Zeng, C. Miao, P. Wang, Y. You, and F. Jiang Bootstrap latent representations for multi-modal recommendation. In Proceedings of the ACM Web Conference 2023, W ’23, New York, NY, USA, p. 845–854. External Links: ISBN 9781450394161, Link, Document Cited by: §2. Zhou et al. (2023b) X. Zhou, H. Zhou, Y. Liu, Z. Zeng, C. Miao, P. Wang, Y. You, and F. Jiang Bootstrap latent representations for multi-modal recommendation. In W, p. 845–854. Cited by: Table 1. Zhu et al. (2025) Y. Zhu, C. Fu, Y. Ni, A. Zeng, and Y. Fang A probabilistic framework for temporal distribution generalization in industry-scale recommender systems. External Links: 2511.21032, Link Cited by: §1, §2.