Paper deep dive
DeepAJM: Deep Association Joint Model for Irregularly Sampled data
Barsha Halder, Jeffrey A. Thompson
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/8/2026, 6:18:02 AM
Summary
The paper introduces DeepAJM, a deep learning-based joint model for irregularly sampled longitudinal and survival data. Unlike classical parametric models or existing deep learning approaches like TransformerJM and DA-LSTM, DeepAJM uses an LSTM-based sequence-to-sequence architecture with cross-attention to model longitudinal trajectories and explicitly links them to survival outcomes via an interpretable association structure modulated by baseline covariates. It employs a Cox-based survival head with the Breslow estimator for baseline hazard. The model was evaluated on cardiovascular EHR, PBC2, and simulated datasets, demonstrating superior performance in discrimination metrics (C-index, AUROC, AUPRC) compared to baselines.
Entities (12)
Relation Signals (11)
DeepAJM → evaluatedon → Cardiovascular Diseases (CVD) dataset
confidence 95% · The model was evaluated on three datasets ( a cardiovascular-disease EHR cohort...
DeepAJM → evaluatedon → PBC2 dataset
confidence 95% · The model was evaluated on three datasets (... a primary biliary cirrhosis (PBC2) dataset...
DeepAJM → usescomponent → LSTM
confidence 95% · Our proposed model, DeepAJM, comprises four components: an LSTM-based seq2seq network...
DeepAJM → usesmetric → AUROC
confidence 95% · All models were assessed using ... time-dependent AUROC...
DeepAJM → usesmetric → C-index
confidence 95% · All models were assessed using C-index...
DeepAJM → usesmetric → AUPRC
confidence 95% · All models were assessed using ... time-dependent AUPRC.
DA-LSTM → istype → RNN-based Model
confidence 90% · DA-LSTM [10] also models longitudinal and time-to-event data using RNN-based architecture...
TransformerJM → istype → Transformer-based Model
confidence 90% · TransformerJM [11], a transformer-based architecture that jointly models longitudinal and survival data...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Joint Models simultaneously model longitudinal and survival outcomes, leveraging patterns in patients' longitudinal trajectory to improve the prediction of survival outcomes. The classical parametric joint models, however, rely on fixed parametric assumptions, making them susceptible to bias under model misspecification and smaller sample sizes. We propose a deep joint model, DeepAJM, that does not require any parametric assumptions, while retaining a partially interpretable, per-longitudinal-outcome association structure. The joint model uses an encoder-decoder (sequence-to-sequence) architecture to learn the latent structure in patients' time-varying covariate trajectories. The model links the longitudinal processes to the survival processes through a learned interpretable association structure, in which each longitudinal output from the decoder gets remodulated by baseline covariates before it contributes to the risk scores from the survival head of the architecture. The model was evaluated on three datasets ( a cardiovascular-disease EHR cohort, a primary biliary cirrhosis (PBC2) dataset, and a simulated dataset) against a classical parametric joint model, TransformerJM, DA-LSTM and a Cox-based survival-only model. All models were assessed using C-index, integrated brier score (IBS), time-dependent AUROC, and time-dependent AUPRC. Our model achieved the best discrimination in terms of the C-index, time-dependent AUROC, and AUPRC across all datasets.
Tags
Links
- Source: https://arxiv.org/abs/2610.07388v2
- Canonical: https://arxiv.org/abs/2610.07388v2
Trouble viewing inline? Open PDF directly →
Full Text
42,351 characters extracted from source content.
Expand or collapse full text
DeepAJM: Deep Association Joint Model for Irregularly Sampled data Barsha Halder Jeffrey A. Thompson Affiliation: Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, KS, USA Abstract Joint Models simultaneously model longitudinal and survival outcomes, leveraging patterns in patients’ longitudinal trajectory to improve the prediction of survival outcomes. The classical parametric joint models, however, rely on fixed parametric assumptions, making them susceptible to bias under model misspecification, smaller sample sizes. We propose a deep joint model, DeepAJM that does not require any parametric assumptions, while retaining a partially interpretable, per-longitudinal-outcome association structure. The joint model uses an encoder-decoder (sequence-to-sequence) architecture to learn the latent structure in patients’ time-varying covariate trajectories. The model links the longitudinal processes to the survival processes through a learned interpretable association structure, in which each longitudinal output from the decoder gets remodulated by baseline covariates before it contributes to the risk scores from the survival head of the architecture. The model was evaluated on three datasets- a cardiovascular-disease EHR cohort, a primary biliary cirrhosis (PBC2) dataset, and a simulated dataset- against a classical parametric joint model, TransformerJM, DA-LSTM and a Cox-based survival-only model. All models were assessed using C-index, integrated brier score (IBS), time-dependent AUROC, and time-dependent AUPRC. Our model achieved the best discrimination in terms of the C-index, time-dependent AUROC, and AUPRC across all datasets. 1 Introduction A Joint model analyzes longitudinal and survival data simultaneously, rather than modeling them separately, in order to capture how longitudinal information influences the risk of an event. The two processes −- longitudinal and survival are linked through a shared latent effect, such that changes in a variable trajectory directly inform, and are reflected in, the survival outcomes. In clinical studies, joint models play a significant role in predicting survival outcomes. Wulfsohn and Tsiatis (1997) first proposed jointly modeling longitudinal and time-to-event data [22]. Later, Rizopoulos developed practical tools in R [17, 16] to make this computationally intensive framework accessible and extended it to accommodate non-linear time-varying trajectories and different functional forms for the association linking the longitudinal and survival processes. However, the traditional parametric joint model relies heavily on rigid parametric assumptions and conditional independence assumption between the two processes given the shared random effects [17, 5]. Furthermore, with higher dimensionality, the parametric approach suffers from computationally complexity as numerical integrations are required for the likelihood estimation. Non-parametric deep-learning approaches have been proposed to overcome these parametric constraints. Generally, these methods use a shared temporal layer −-recurrent (RNN/LSTM) or transformer-based that exploits the temporal information in the longitudinal trajectory, including its shape, and feeds both longitudinal and survival submodels. To date, a few studies have explored joint models using deep learning. TransformerJM [11], a transformer-based architecture that jointly models longitudinal and survival data to predict survival probabilities. It discretizes time into a fixed set of intervals and uses a binary discrete-time survival loss. However, this approach can be dominated by multiple follow-up visits with no events, skewing predictions toward higher survival and leading to poor discrimination. TransformerLSR, a continuous-time transformer model was proposed to jointly model longitudinal measurements, survival and recurrent events, and modeling survival and recurrent-event as flexible temporal point processes [23]. However, rather than using an explicit association structure that link submodels as in classical joint models, TransformerLSR captures dependence implicitly through shared representations. DA-LSTM [10] also models longitudinal and time-to-event data using RNN-based architecture with a multi-task learning approach but relies on discrete-step RNN and does not explicitly handle continuous, irregular time gaps between consecutive observations. All these deep-learning methods [10, 11, 23, 19, 9] jointly model longitudinal and survival data, but none link the longitudinal process to survival through an explicit shared effect as in a classical joint model [17], and none models baseline covariates as outcome-specific modulation weights. Our study addresses both gaps and designs a joint model that processes longitudinal outcomes using a seq2seq architecture and survival processes with a semi-parametric cox formulation without discretizing event time into fixed intervals, modulating each longitudinal outcome by baseline-covariates-specific weights, and linking the two processes through an interpretable association structure. Our proposed model, DeepAJM, comprises four components: an LSTM-based seq2seq network with cross-attention mechanism, a baseline-modulation network, an association structure producing per outcome association weight, and a survival sub-network that maps the association-weighted longitudinal outcomes to survival risk scores. 2 Methods The proposed joint model, DeepAJM, couples an attention-based sequence-to-sequence longitudinal head with a baseline covariate modulation network and a survival head. The longitudinal head, an LSTM-based [6] encoder-decoder structure, processes the longitudinal measurements and predicts the outcomes. These predicted outcomes are then readjusted using output-specific weights learned by the modulation network and linked to the survival head through an association structure. 2.1 Data preprocessing Input sequences: The input sequences for the longitudinal head were constructed as four time points per subject, consisting of X longitudinal features, of shape (N,T,X)(N,\ T,\ X), where N is the number of subjects, T is the sequence length and X is the number of features. Both the encoder and decoder of the longitudinal head received the same input sequences of shape (T,X)(T,\ X) per subject. The longitudinal features consisted of clinical, physiological and functional biomarkers, along with the time at which each set of biomarkers was sampled. The time feature was encoded as time elapsed (in years) from the subject’s baseline visit, allowing the model to account for irregular sampling intervals between visits. To ensure the longitudinal head received sufficient temporal information, each input sequence comprised four time points up to the landmark time (tL) (t_L ). During training, tLt_L was subject’s second-to-last visit (tn,T−1) (t_n,\ T-1 ); during evaluation at fixed horizons (tpred) (t_pred ), tLt_L was the last visit preceding the earliest evaluation horizon, restricting the cohort to subjects with at least four visits up to tLt_L who remained at risk at tLt_L. Figure 1: Construction of training and test sequences used across all evaluated methods. Baseline covariate representation: The baseline covariates in our study were age (b1) (b_1 ) and sex (b2) (b_2 ) and were processed through baseline covariate modulation network. In longitudinal studies, baseline covariates exert distinct effects on both longitudinal and survival outcomes; although a deep joint model learns those feature interactions internally across timesteps, it may not fully capture the interaction between baseline covariates and longitudinal outcomes. Age (OPENb1)b_1), a continuous scalar was projected through a dense layer with ReLU activation to explicitly capture its non-linear effects on longitudinal trajectories, thereby improving survival predictions; whereas sex (b2)(b_2) was retained as a binary indicator. The projection dimension for age was selected via hyperparameter tuning b~1=ReLU(Wgb1+cg) b_1=ReLU (W_gb_1+c_g ) (1) 2.2 Proposed Model 2.2.1 Longitudinal head The longitudinal head was designed as an LSTM-based sequence-to-sequence architecture augmented with a multi-head cross-attention mechanism, used to extract dynamic temporal patterns from longitudinal trajectories. LSTMs are well-suited to process sequential data as they capture long-term dependencies. A seq2seq architecture is composed of two components: an encoder and a decoder [20, 3]. The encoder maps the input sequences into a latent representation and a context vector, which are passed to the decoder. Encoder(E)(E): The encoder processes fixed-length input sequences of shape (T,X)(T,\ X). It consists of two unidirectional LSTM layers; the first is followed by batch normalization. The number of units in each layer was selected via hyperparameter tuning during model training. The second LSTM layer generates two outputs: a latent representation (Henc(2)) (H_enc^(2) ), summarizing hidden states across all time steps, and a context vector (Hencf(2)) (H_enc_f^(2) ), representing the final hidden and cell state. (ht(1),ct(1))=LSTMELU(1)(xt,ht−1(1),ct−1(1)),t=1,…,T (h_t^(1),\ c_t^(1) )=LSTM_ELU^(1) (x_t,\ h_t-1^(1),\ c_t-1^(1) ), t=1,…,T (2) (ht(2),ct(2))=LSTMELU(2)(ht~(1),ht−1(2),ct−1(2)),t=1,…,T (h_t^(2),\ c_t^(2) )=LSTM_ELU^(2) ( h_t^(1),\ h_t-1^(2),\ c_t-1^(2) ), t=1,…,T (3) Henc(2)=(h1(2),h2(2),…,hT(2))∈ℝT×dH_enc^(2)= (h_1^(2),\ h_2^(2),\ …,\ h_T^(2) ) ^T× d (4) Hencf(2)=(hT(2),cT(2))∈ℝT×dH_enc_f^(2)= (h_T^(2),\ c_T^(2) ) ^T× d (5) Decoder(D)(D): The decoder receives same input sequences as the encoder, of shape (T,X)(T,\ X); consists of two unidirectional LSTM layers. The initial hidden state of first layer is initialized with Hencf(2)H_enc_f^(2). Figure 2: DeepAJM Model Architecture The output from this layer (Hdec(1)) (H_dec^(1) ) serves as the query (Q)(Q), one of the three inputs requires by a multiheaded cross-attention layer. The attention layer is followed by a unidirectional LSTM layer and two dense layers – the first with an ELU (exponential linear unit) activation function and the second with a linear activation function. (ht(dec, 1),ct(dec, 1))=LSTMELU(dec, 1)(xt,ht−1(dec, 1),ct−1(dec, 1)),t=1,…,T (h_t^(dec,\,1),\ c_t^(dec,\,1) )=LSTM_ELU^(dec,\,1) (x_t,\ h_t-1^(dec,\,1),\ c_t-1^(dec,\,1) ), t=1,…,T (6) with h0(dec, 1)=hT(2),c0(dec, 1)=cT(2)h_0^(dec,\,1)=h_T^(2), c_0^(dec,\,1)=c_T^(2) Hdec(1)=(h1(dec, 1),h2(dec, 1),…,hT(dec, 1))∈ℝT×dH_dec^(1)= (h_1^(dec,\,1),\ h_2^(dec,\,1),\ …,\ h_T^(dec,\,1) ) ^T× d (7) Multi-headed cross-attention Multi-head cross-attention establishes relationship between two sequences by enabling one sequence to attend to key elements of the other [21]. This attention mechanism enforces the model to extract salient information from Henc(2)H_enc^(2) to enrich the decoder’s output. The layer received three inputs −- query (Q)(Q), key (K)(K), and value (V)(V); the query was derived from Hdec(1)H_dec^(1), while Henc(2)H_enc^(2)\ served as the key and value, as defined in Eq.(8)(8). The resulting attention scores from this layer was concatenated with Hdec(1)H_dec^(1) and passed to the second LSTM layer. The number of heads was set to 2 and per-head dimension dkd_k to 16, selected empirically after multiple trials. The multi-head attention scores and the concatenation layer are given as: Qi=Hdec(1)WQ(i),Ki=Henc(2)WK(i),Vi=Henc(2)WV(i)Q_i=H_dec^(1)W_Q^(i), K_i=H_enc^(2)W_K^(i), V_i=H_enc^(2)W_V^(i) (8) headi=Attention(Qi,Ki,Vi),i∈1,2head_i=Attention (Q_i,\ K_i,\ V_i ), i∈\1,2\ (9) Zct=Concat(Multihead(Q,K,V),Hdec(1))Z_ct=Concat (Multihead(Q,K,V),\ H_dec^(1) ) (10) 2.2.2 Baseline-covariate modulation network The baseline modulation network was incorporated to readjust the longitudinal outputs by baseline covariates, such that each adjusted output carries individual baseline-specific effects together with the temporal patterns. Inputs to this network are processed baseline covariates and the outputs generated by the longitudinal head. For each L longitudinal output, baseline covariates are each passed through a separate fully connected layer without activation, i.e., L parallel projections per covariate, producing output-specific effects (γjb1,γjb2) ( _j^b_1,\ \ γ_j^b_2 ). The adjusted longitudinal outputs are obtained as: y~n,j(adj)=yn,j+(γjb1⋅b~1,n+cjb1)+(γjb2⋅b2,n+cjb2),j=1,…,L;n=1,…,N y_n,j(adj)=y_n,j+ ( _j^b_1· b_1,n+c_j^b_1 )+ ( _j^b_2· b_2,n+c_j^b_2 ), j=1,…,L;\ n=1,…,N (11) 2.2.3 Longitudinal-survival association structure DeepAJM was devised with an interpretable, explicit association structure that linked the longitudinal process to the survival process. It assigns a unique associative weight to each adjusted longitudinal output from the modulation network, generating an association-weighted value per output. Each weight is output-specific and shared across subjects. These association-weighted longitudinal outputs are concatenated and passed to the event-process head, mapping to survival risk scores. sn,j=αjy~n,j(adj)+βj,j=1,…,L;n=1,…,Ns_n,j= _j y_n,j(adj)+ _j, j=1,…,L;\ n=1,…,N (12) sn=concat(sn,1,sn,2,…,sn,L)s_n=concat (s_n,1,\ s_n,2,…,\ s_n,L ) (13) 2.2.4 Survival head The association-weighted outputs were reshaped into a sequence sn′∈ℝL×1s_n ^L× 1 and processed by an LSTM layer, enabling the model to capture cross-biomarker interactions. The LSTM layer was followed by two fully connected layers, with ReLU and linear activations respectively, producing log-hazard score rter^te at tn,Tt_n,T. Survival probabilities were obtained at tpredt_pred using rter^te and baseline hazard, as described below. rn=fsurv(sn′)r_n=f_surv (s_n ) (14) Baseline Cumulative Hazard As the survival component is cox-based and generates survival probabilities at tpredt_pred, the individual relative risk scores required a shared baseline hazard, anchored to an absolute time scale. Therefore, the Breslow estimator [1], a non-parametric estimate of the survival curve, was considered to estimate the baseline cumulative hazard from training log-hazard scores, providing the time-anchored scale that rter^te alone cannot provide. The baseline cumulative hazard was then converted to the baseline survival curve, shared across all subjects. The baseline survival curve, along with the predicted log-hazard scores, was used to compute survival probabilities at the prediction horizon. The incorporation of the Breslow estimator is what makes the proposed joint model a Cox-based neural joint model. The Breslow estimator that yielded the survival probabilities, H^0(t)=∑ti≤tdi∑j∈Rtr(tj)erjtr H_0(t)= _t_i≤ t d_i _j∈ R_tr(t_j)e^r_j^tr (15) where did_i = number of events at time tit_i; rjtr_j^tr = predicted log-hazard score for subject j from the training fold; Rtr(ti)=j:tj≥tiR_tr(t_i)=\j:t_j≥ t_i\ is the risk set built from training subjects. The baseline survival curve at time t: S^0(t)=exp(−H^0(t)) S_0(t)= (- H_0(t) ) (16) The survival probability for subject j at time t: S^(t∣rjte)=S^0(t)exp(rjte) S(t r_j^te)= S_0(t) (r_j^te ) (17) where rjter_j^te = predicted log-hazard score for subject j from the test fold. Survival outcomes at prediction horizon The survival function is a right-continuous step function [4], a piecewise constant between consecutive event times. The survival probability at tpredt_pred was therefore, obtained by evaluating the step function at that horizon, i.e., carrying forward the estimate from the last event time at or before that horizon. The earliest time point (t′) (t ) in the prediction horizon was set to year 5. For the ordered distinct event times t1<t2<t3<…<tj<…<tmt_1<t_2<t_3<…<t_j<…<t_m: S^(t′)=1,t′<t1S^(tj),tj≤t′<tj+1 S(t )= cases1,&t <t_1\\ S(t_j),&t_j≤ t <t_j+1 cases (18) 3 Experimental Setup 3.1 Datasets Cardiovascular Diseases (CVD): This dataset contained electronic health records (EHRs), obtained from the University of Kanas Health System with Institutional Review Board (IRB) approval from The University of Kansas Medical Center. The cohort included subjects with various cardiovascular diseases recorded between 1980 and 2024, comprising longitudinal, time-to-event and baseline covariate information. After preprocessing, 2257 subjects were retained from the cohort, each with a minimum of 5 and maximum of 20 clinical encounters. The longitudinal variables included from this dataset were physiological and clinical biomarkers – diastolic blood pressure, systolic blood pressure, basal metabolic index (BMI) and respiration rate. The time at which all these biomarkers were collectively sampled at each clinical encounter was also extracted and used in the analyses. The time-to-event or time-to-censoring information for each subject was computed based on the provided event time and event indicator information. Primary biliary cirrhosis (PBC2): This dataset contained longitudinal and time-to-event data along with baseline covariates for 312 subjects diagnosed with primary biliary cirrhosis [12]. The subjects were enrolled in a randomized controlled trial conducted by the Mayo Clinic between 1974 and 1984, and the dataset is publicly available. The dataset retained 183 subjects, each with a minimum of 5 and a maximum of 16 clinical encounters. The clinical and functional biomarkers used as longitudinal information were serum bilirubin, albumin, SGOT and prothrombin. The observation time for each clinical encounter was also included. Simulated dataset: The data were simulated for 800 subjects with four longitudinal variables and baseline covariates. The number of follow-up visits per subject was sampled uniformly from [4,10][4,10] , with visit times (in years) sampled irregularly from a continuous uniform distribution on [0,12][0,12], producing between 5 and 11 irregularly spaced encounters per subject. The longitudinal variables were simulated using a nonlinear mixed-effects design. The subject-specific random effects (intercept, slope and curvature) for all variables jointly followed a multivariate normal distribution whose variance-covariance structure induced correlation both within each variable’s subject-specific random effects and across variables. These generated random-effects, combined with variable-specific fixed-effects drawn from uniform distributions and gaussian measurement error simulated the observed longitudinal measurements [Eq. 22][Eq.\ 22]. Baseline covariates were simulated once per subject, with age following a gaussian distribution and sex a Bernoulli distribution. A shared-random-effects joint model then linked the underlying true (noise-free) latent trajectories to a Weibull baseline hazard (shape=1.2)(shape=1.2), through a variable-specific association parameter αk _k, shared across subjects. Event times were simulated using simsurv package in R [2]. After preprocessing, the dataset retained 538 subjects, with a minimum of 5 and a maximum of 11 encounters per subject. For subject j, the underlying trajectory for kthk^th biomarker at time t, Yjk(t)=β0k+b0jk+(β1k+b1jk)t+(β2k+b2jk)t2+ϵjk,k=1,2,…,4Y_jk(t)= _0k+b_0jk+ ( _1k+b_1jk )t+ ( _2k+b_2jk )t^2+ _jk, k=1,2,…,4 (19) Yjk(t)=μjk(t)+ϵjkY_jk(t)= _jk(t)+ _jk (20) bjk=b0jk,b1jk,b2jk,bj∼MVN(0,Σ)b_jk= \b_0jk,\ b_1jk,\ b_2jk \, b_j MVN(0, ) (21) For subject j, the hazard function is: hj(t)=h0(t)exp(∑k=14αkμjk(t))h_j(t)=h_0(t) ( _k=1^4 _k _jk(t) ) (22) αk∼Uniform(0.01,0.03) _k Uniform(0.01,0.03) (23) 3.2 Model Training and Optimization A 5-fold nested cross-validation with a 3-fold inner cross-validation was employed to train and evaluate all models, including baseline methods. Within each inner loop, the hyperparameters were tuned using Bayesian optimization [13], which selected the best configuration of hyperparameters based on the results of past trials. As a result, this optimization method required fewer trials than an exhaustive search to converge to the optimal set of hyperparameters. The number of optimization trials varied by dataset and was specified in the supplementary section. Prior to training, longitudinal features including age were scaled to [0,1][0,1] using scikit-learn’s MinMaxScaler [14]. Loss Functions: The model was trained by jointly optimizing longitudinal and survival loss. The longitudinal loss is the mean absolute error between the observed and predicted longitudinal outputs, and cox proportional-hazards partial-likelihood loss is defined as the survival loss. The relative contribution of survival loss to joint objective is controlled by a weighting coefficient λ, which was tuned as an additional hyperparameter. Longitudinal Loss: Llong=1NL∑i=1N∑k=1L|yi,k−y^i,k|L_long= 1NL _i=1^N _k=1^L |y_i,k- y_i,k | (24) Survival Loss: For each sample i, at time tit_i with δi∈0,1 _i∈\0,1\ as the event indicator, and the predicted log-hazard score riter_i^te, the survival loss was computed as: Lsurv=−1N∑i=1Nδi[rite−log∑j∈R(ti)erjte]L_surv=- 1N _i=1^N _i [r_i^te- _j∈ R(t_i)e^r_j^te ] (25) where R(ti)=j:tj≥tiR(t_i)=\j:t_j≥ t_i\ is the risk set at time tit_i. Joint Loss: The final training objective is the combination of both losses: Lj=Llong+λsurv⋅LsurvL_j=L_long+ _surv· L_surv (26) 3.2.1 Evaluation metrics Four metrics were used to evaluate the performance of survival predictions: concordance index (C-index), integrated Brier score (IBS), time-dependent AUC (t-AUROC) and time-dependent AUPRC (t-AUPRC). The IBS and t-AUROC employed inverse probability of censoring weighting (IPCW) [18], as implemented in scikit-survival [15], to account for right-censoring, with censoring weights estimated by the Kaplan-Meier (KM) method. The same KM-based IPCW was applied to account for censoring when computing t-AUPRC. 3.2.2 Calibration and metric computation The Breslow estimator produces survival probabilities by imposing the proportional hazards assumption, and therefore, maintains a relative risk ranking shared across all prediction horizons. However, IBS is sensitive to absolute predicted values, so calibration of survival probabilities is essential for computing this metric. To address that, the flexible hazard regression model (hare) [8] from the polspline package in R [7] was used to calibrate the predicted survival probabilities at the prediction horizon. As hare models the hazard as a flexible function of time and risk scores, it relaxes the proportional hazards assumption and causes the ranking order to vary across horizons. As a result, uncalibrated survival probabilities by Breslow’s were used to compute discrimination-sensitive metrics, C-index and t-AUROC. However, calibrated survival probabilities by hare improved t-AUPRC, which showed its greater sensitivity to ranking at the high-precision region. Therefore, calibrated probabilities were used for IBS and t-AUPRC, and uncalibrated survival probabilities for C-index and t-AUROC. 3.3 Baseline Methods Our proposed model was compared against four baselines: the parametric joint model, the TransformerJM model, the DA-LSTM model and Cox-based survival-only model. All models were trained and evaluated under identical conditions: same inputs, data splits, outer folds, inner cross-validation for hyperparameter optimization, and evaluation metrics. Parametric Joint Model (PJM): This baseline comprised of two submodels: a Cox proportional hazard model as the survival submodel and a linear mixed-effects as the longitudinal submodel [17]. The model was trained and evaluated using the JMBayes2 package [16] in R. TransformerJM: TransformerJM jointly models longitudinal and time-to-event data using a sequence-to-sequence architecture [11]. The author’s implementation was used and was evaluated under identical conditions as our proposed model. DA-LSTM: This method was proposed as a deep learning approach for jointly modeling longitudinal and time-to-event data [10]. The used both a recurrent neural network (RNN) with ELU activation function (DA-N) and a long short-term memory (LSTM) network (DA-LSTM) as the shared temporal layer within the same architecture. Upon evaluation, they selected DA-LSTM as the primary model because it outperformed DA-N. Accordingly, DA-LSTM was adopted as the baseline from comparison with our proposed model. Since neither DA-LSTM nor DA-N architecture incorporated time of longitudinal measurements as an additional input, and to evaluate DA-LSTM under identical conditions as our proposed model, we added time as additional feature alongside longitudinal and baseline features, yielding DA-LSTM(t), a reimplementation that incorporates time as the time elapsed from subject’s baseline visit. The author’s original implementation was not publicly available, and therefore, we reproduced it following the descriptions in their paper. The implementation details for DA-LSTM are provided in the supplementary section. Cox-based survival-only (Surv-only): This baseline model consisted of the survival head, the same survival head as in the proposed model but without the LSTM layer as it received only baseline features with no longitudinal features as inputs. The baseline features were processed and trained in the same manner as done in our proposed joint model, but the model was trained using only the survival loss. 4 Results DeepAJM was compared against four methods: a parametric joint model (PJM), a transformer-based joint model (TransformerJM), an RNN-based joint model (DA-LSTM) and a Cox-based survival-only model. All methods were evaluated under an identical protocol across two real-world datasets and one simulated dataset. For each method, survival predictions were obtained at the t_predt\_pred horizon, with the first evaluated time point at t′=5t =5, and subsequent evaluation at t′+1,t′+2t +1,\ t +2. Table 1: Concordance index (C-index) across datasets for DeepAJM and benchmarked models. Dataset DeepAJM PJM TransformerJM DA-LSTM DA-LSTM (t) Surv-only CVD 0.9114 0.5124 0.6609 0.6621 0.8474 0.6166 PBC2 0.8073 0.5031 0.5876 0.7188 0.6817 0.5543 Simulated 0.6410 0.6422 0.5897 0.5895 0.6316 0.4609 Table 2: Integrated Brier Score (IBS) across datasets for DeepAJM and benchmarked models. Dataset DeepAJM PJM TransformerJM DA-LSTM DA-LSTM (t) Surv-only CVD 0.1122 0.3070 0.2592 0.2138 0.1733 0.2203 PBC2 0.1012 0.1584 0.1322 0.5895 0.3860 0.1360 Simulated 0.1330 0.0743 0.1107 0.1342 0.1956 0.1066 Each reported metric is the average across all evaluated time points from the 5-fold nested cross-validation. On both real-world datasets, DeepAJM achieved the highest C-index, t-AUROC, t-AUPRC, and the lowest IBS. On the simulated dataset, it outperformed all baselines on t-AUROC and t-AUPRC, and achieved a C-index equivalent to best-perfomring baseline (PJM). Further, DA-LSTM (t) showed improved C-index, t-AUROC and t-AUPRC over the original DA-LSTM on the CVD and simulated datasets. Table 3: Time-dependent AUROC across datasets for DeepAJM and benchmarked models. Dataset DeepAJM PJM TransformerJM DA-LSTM DA-LSTM (t) Surv-only CVD 0.9374 0.5127 0.6884 0.7028 0.8898 0.6190 PBC2 0.8878 0.4497 0.6600 0.7758 0.7928 0.5623 Simulated 0.7483 0.5551 0.5034 0.5194 0.6876 0.4333 Table 4: Time-dependent AUPRC across datasets for DeepAJM and benchmarked models. Dataset DeepAJM PJM TransformerJM DA-LSTM DA-LSTM (t) Surv-only CVD 0.8548 0.3635 0.5500 0.5777 0.7988 0.4479 PBC2 0.5270 0.1752 0.3206 0.4090 0.4490 0.1687 Simulated 0.2462 0.1046 0.1276 0.1527 0.2201 0.1261 5 Discussion DeepAJM is a deep joint model that decodes complex temporal dynamics of a patient’s health profile with an LSTM-based seq2seq architecture. A baseline-covariate modulation network captures the distinct, implicit effects of individual baseline covariates such as age and sex, readjusting each longitudinal outcome by covariate-specific weights, so that it carries richer information for downstream analysis. An explicit longitudinal-survival association structure then generates association weights, assigns them to the corresponding adjusted longitudinal outcome, and maps to a Cox-based survival head, which generates log-hazard scores that are combined with the baseline hazard curve to obtain survival probabilities at prediction horizon. DeepAJM achieved the strongest overall performance across all datasets and metrics, outperforming the baselines on both discrimination and calibration. It attained a consistently higher C-index on two real-world datasets and matched the best baseline on simulated dataset, showing DeepAJM’s robust preservation of survival-risk ranking. For calibration, lower IBS values showed improved calibration and consistently higher t-AUROC and t-AUPRC values indicated stronger discrimination for both event and event-free subjects. Its higher t-AUPRC at horizons with low event rates reflects DeepAJM’s accuracy in the regime where survival models struggle most. The purpose of jointly modeling longitudinal and survival processes is to enhance the predictive power of survival outcomes by accounting for hidden, unobserved factors that potentially influence the risk scores. Built on a LSTM-based sequence-to-sequence network, the model strongly captured temporal patterns present in heterogenous longitudinal trajectories across the population. Integrating these temporal latent representations with the baseline covariates further enriched the longitudinal outcomes, since baseline covariates tend to affect subject’s longitudinal trajectories differently. To realize the gains in preserving the interpretable structure while simultaneously enhancing model’s predictive power, the survival head was implemented as a neural network-based cox model and trained on cox partial likelihood loss. The latent trajectory of longitudinal outcomes was linked to the survival process through an association structure, representing the proposed model a semi-parametric joint model. As the survival submodel further employed an LSTM, it learned a cross-biomarker structure from the association-weighted outcomes. To obtain survival probabilities at fixed prediction horizons, the baseline cumulative hazard was computed from training subjects using Breslow estimator [1]. Since the Breslow estimator relies on proportional hazards (PH) assumption, we verified this assumption, and the results indicated no violation of PH assumption on both input covariates and log-hazard scores. Consequently, the proposed model shared structural and partial interpretability properties of the classical parametric joint model while also addressing its constraints. Overall, the model achieved robust discrimination and calibration as compared to the classical parametric joint model while overcoming several of its limitations. Prior work on joint modeling of longitudinal and survival outcomes [11, 23, 10] did not incorporate an association structure, a key mechanism by which a joint model learns how the longitudinal process informs the survival process. TransformerJM [11] employed a transformer-based architecture to predict survival probabilities at prediction horizons and then autoregressively fed these predictions back to predict outcomes at subsequent time points. Their architecture jointly modeled longitudinal and survival data together with baseline covariates and time but it did not learn any explicit association structure linking the longitudinal outcomes to the survival process. Additionally, it discretized event times into fixed bins, which can yield highly skewed survival estimates in bins with fewer events. Thus, although this transformer-based baseline jointly modeled the longitudinal and survival outcomes, it did not explicitly maintain the inherent properties of a joint model. DA-LSTM [10] used an RNN-based architecture to process temporal information and model the hazard rate through exponential distribution, producing piecewise-constant hazard. It is multi-output model; since our study focused solely on survival predictions, we obtained only the survival output and considered it as a baseline. Like TransformerJM, DA-LSTM did not learn any association structure between the longitudinal and survival processes, which is an explicit gap that our proposed model addressed. Furthermore, the original DA-LSTM did not handle the irregular gap between sampled observations, which is one of the challenges in EHR datasets. To evaluate it as baseline under conditions identical to our model, DA-LSTM was additionally trained with time of longitudinal measurements as an input. As their original implementation of DA-LSTM did not include time, we reported DA-LSTM’s performance both with and without measurement time, and the results showed notable difference between two settings. This evidently showed that in real-world datasets with irregular sampling intervals, incorporating measurement times of longitudinal observations enabled the model to capture temporal distance between irregularly sampled observations and enhanced its predictive ability. The model produced cox-style nonlinear log-hazard scores that were exponentiated to compute the baseline hazard curve, and therefore, the ordering of hazards across subjects were preserved over time, which can further be stratified into risk groups. Therefore, the model attained discriminative interpretability as well as an estimable population baseline curve whose shape over time can be studied for clinical interpretation. The model also incorporated a per-biomarker association layer, analogous to classical joint model’s association structure, and which generated outcome-specific association weights that are extractable and interpretable. However, as these adjusted longitudinal outcomes passed through multiple nonlinear layers downstream, they did not correspond to closed-form hazard ratios as a in parametric joint model. The architecture was designed to predict the log-hazard at Tn,τT_n,\ τ; however, as we evaluated the model performance at fixed prediction horizons for all subjects, the baseline cumulative hazard along with the predicted risk scores were used to compute survival probabilities on the event times grid. The survival function, a right-continuous step function, was used to obtain the outcomes at those horizons. This restricts the model to predicting survival outcomes at horizons that fall at or before the last observed event time. Although the model can in principle predict outcomes at any horizon given the history up to the landmarking time, the requirement to evaluate at fixed horizons for irregularly sampled clinical observations made it necessary to employ a shared baseline hazard, a limitation that also required verifying the PH assumption. Our model used four time points as longitudinal inputs, but it can accommodate any number of time points. The joint model carries greater clinical relevance for disease progression prediction than survival-only models, as it explores the underlying biological process that often reflects latent factors driving event occurrence. It thereby enables producing dynamic, individualized predictions and that can be updated at the point of care by forecasting the longitudinal trajectory. References [1] N. Breslow (1974) Covariance analysis of censored survival data. Biometrics 30 (1), p. 89–99. External Links: Document Cited by: §2.2, §5. [2] S. L. Brilleman, R. Wolfe, M. Moreno-Betancur, and M. J. Crowther (2021) Simulating survival data using the simsurv R package. Journal of Statistical Software 97 (3), p. 1–27. External Links: Document Cited by: §3.1. [3] K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), A. Moschitti, B. Pang, and W. Daelemans (Eds.), Doha, Qatar, p. 1724–1734. External Links: Link, Document Cited by: §2.2.1. [4] T. G. Clark, M. J. Bradburn, S. B. Love, and D. G. Altman (2003) Survival analysis part I: basic concepts and first analyses. British Journal of Cancer 89 (2), p. 232–238. External Links: Document, ISSN 0007-0920 Cited by: §2.2. [5] J. Han, E. H. Slate, and E. A. Peña (2007) Parametric latent class joint model for a longitudinal biomarker and recurrent events. Statistics in Medicine 26 (29), p. 5285–5302. External Links: Document Cited by: §1. [6] S. Hochreiter and J. U. Schmidhuber (1997) Long short-term memory. Neural Computation 9, p. 1735–1780. External Links: Link Cited by: §2. [7] C. Kooperberg, C. J. Stone, and Y. K. Truong (1995) Hazard regression. Journal of the American Statistical Association 90 (429), p. 78–94. Cited by: §3.2.2. [8] C. Kooperberg (2024) Polspline: polynomial spline routines. The Comprehensive R Archive Network. Note: R package version X.X.X External Links: Link Cited by: §3.2.2. [9] C. Lee, J. Yoon, and M. van der Schaar (2020) Dynamic-deephit: a deep learning approach for dynamic survival analysis with competing risks based on longitudinal data. IEEE Transactions on Biomedical Engineering 67 (1), p. 122–133. External Links: Document Cited by: §1. [10] B. Lim and M. van der Schaar (2018) Disease-atlas: navigating disease trajectories using deep learning. In Proceedings of the 3rd Machine Learning for Healthcare Conference, F. Doshi-Velez, J. Fackler, K. Jung, D. Kale, R. Ranganath, B. Wallace, and J. Wiens (Eds.), Proceedings of Machine Learning Research, Vol. 85, p. 137–160. External Links: Link Cited by: §1, §3.3, §5. [11] J. Lin and S. Luo (2022) Deep learning for the dynamic prediction of multivariate longitudinal and survival data. Statistics in Medicine 41 (15), p. 2894–2907. External Links: Document Cited by: §1, §3.3, §5. [12] P. A. Murtaugh, E. R. Dickson, G. M. Van Dam, M. Malinchoc, P. M. Grambsch, A. L. Langworthy, and C. H. Gips (1994) Primary biliary cirrhosis: prediction of short-term survival based on repeated patient visits. Hepatology 20 (1), p. 126–134. Cited by: §3.1. [13] T. O’Malley, E. Bursztein, J. Long, F. Chollet, H. Jin, L. Invernizzi, et al. (2019) KerasTuner. Note: https://github.com/keras-team/keras-tuner Cited by: §3.2. [14] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay (2011) Scikit-learn: machine learning in Python. Journal of Machine Learning Research 12, p. 2825–2830. Cited by: §3.2. [15] S. Pölsterl (2020) Scikit-survival: a library for time-to-event analysis built on top of scikit-learn. Journal of Machine Learning Research 21 (212), p. 1–6. External Links: Link Cited by: §3.2.1. [16] D. Rizopoulos, P. Miranda Afonso, and G. Papageorgiou (2025) JMbayes2: extended joint models for longitudinal and time-to-event data. Note: R package version 0.5-2 External Links: Link Cited by: §1, §3.3. [17] D. Rizopoulos (2012) Joint models for longitudinal and time-to-event data: with applications in R. Chapman & Hall/CRC Biostatistics Series, Chapman and Hall/CRC, Boca Raton, FL. Cited by: §1, §1, §3.3. [18] J. M. Robins and A. Rotnitzky (1992) Recovery of information and adjustment for dependent censoring using surrogate markers. In AIDS Epidemiology: Methodological Issues, N. P. Jewell, K. Dietz, and V. T. Farewell (Eds.), p. 297–331. Cited by: §3.2.1. [19] J. Y. Ryu, M. Y. Lee, J. H. Lee, B. H. Lee, and K. Oh (2020) DeepHIT: a deep learning framework for prediction of herg-induced cardiotoxicity. Bioinformatics 36 (10), p. 3049–3055. External Links: ISSN 1367-4803, Document, Link, https://academic.oup.com/bioinformatics/article-pdf/36/10/3049/48990556/bioinformatics_36_10_3049.pdf Cited by: §1. [20] I. Sutskever, O. Vinyals, and Q. V. Le (2014) Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger (Eds.), Vol. 27, p. . External Links: Link Cited by: §2.2.1. [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §2.2. [22] M. S. Wulfsohn and A. A. Tsiatis (1997) A joint model for survival and longitudinal data measured with error. Biometrics 53 (1), p. 330–339. External Links: ISSN 0006-341X Cited by: §1. [23] Z. Zhang, Y. Zhao, and Y. Xu (2025) TransformerLSR: attentive joint model of longitudinal data, survival, and recurrent events with concurrent latent structure. Artificial Intelligence in Medicine 160, p. 103056. External Links: Document, ISSN 0933-3657 Cited by: §1, §5.