Paper deep dive
Mind the Long Tail: Understanding the Difficulty of Delay Detection in Business Processes
Keyvan Amiri Elyasi, Lukas Kirchdorfer, Heiner Stuckenschmidt
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/17/2026, 5:34:32 AM
Summary
This paper investigates the intrinsic difficulty of detecting delayed cases in business processes using Predictive Process Monitoring (PPM). Analyzing 14 event logs, the authors find that remaining time distributions are strongly right-skewed, causing existing deep learning models to perform poorly on high-delay cases despite capturing the mode well. The study reveals pronounced heteroscedasticity, where predictive uncertainty increases with delay magnitude. While imbalanced regression techniques (resampling and reweighting) show limited benefit, exploiting uncertainty-aware modeling substantially improves the identification of delayed cases.
Entities (12)
Relation Signals (9)
Keyvan Amiri Elyasi → affiliatedwith → University of Mannheim
confidence 99% · Affiliation: Data and Web Science Group, University of Mannheim, Germany
Lukas Kirchdorfer → affiliatedwith → SAP Signavio
confidence 99% · Affiliation: SAP Signavio, Walldorf, Germany
Remaining Time Prediction → istaskof → Predictive Process Monitoring
confidence 95% · Among PPM tasks, remaining time prediction is one of the most widely studied.
Predictive Process Monitoring → uses → historical event logs
confidence 95% · PPM supports this task by using historical event logs to predict the remaining time of ongoing cases
Heteroscedasticity → correlateswith → delay magnitude
confidence 94% · predictive uncertainty increases with delay magnitude
BPIC20PTC → exhibits → right-skewed distribution
confidence 93% · The ground-truth distribution of remaining times is strongly right-skewed
Uncertainty-Aware Modeling → improves → delay detection
confidence 92% · uncertainty-aware models can substantially improve delay detection.
SMOGN → istypeof → imbalanced regression
confidence 90% · Hybrid methods such as SMOGN combine both strategies... evaluate established imbalanced regression approaches
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The early detection of delayed cases in business processes is a critical capability for organizations. Predictive process monitoring (PPM) supports this task by using historical event logs to predict the remaining time of ongoing cases, enabling timely interventions to avoid missed deadlines and service level violations. Although remaining time prediction has advanced considerably through sophisticated deep learning architectures, little is known about the intrinsic difficulty of delay detection itself. Since performance is typically assessed using aggregate metrics, prior work provides limited insight into how models perform across the target distribution, especially on the operationally most critical cases with large delays. In this paper, we address this gap by analyzing the difficulty of delay detection. Across 14 event logs, we show that remaining times are typically strongly right-skewed, with only a small fraction of cases exhibiting large delays. Existing models capture the mode of this distribution well but perform poorly on high-delay cases. We further uncover pronounced heteroscedasticity, showing that predictive uncertainty increases with delay magnitude. Based on these findings, we evaluate approaches to mitigate the imbalance problem, but find only limited benefits, suggesting that the key underlying problem may not be imbalance but higher uncertainty associated with delayed cases. We show that this correlation can be exploited to substantially improve the identification of delayed cases. Overall, our work provides new insights into the sources of difficulty in delay detection and identifies uncertainty-aware modeling as a promising direction for future PPM research.
Tags
Links
- Source: https://arxiv.org/abs/2608.14367v1
- Canonical: https://arxiv.org/abs/2608.14367v1
Trouble viewing inline? Open PDF directly →
Full Text
48,187 characters extracted from source content.
Expand or collapse full text
Mind the Long Tail: Understanding the Difficulty of Delay Detection in Business Processes Keyvan Amiri Elyasi Thanks: Equal contribution OrcID: 0009-0007-3016-2392 Affiliation: Data and Web Science Group, University of Mannheim, Germany E-mail keyvan.amiri, heiner.stuckenschmidt@uni-mannheim.de Lukas Kirchdorfer⋆ OrcID: 0000-0003-4713-9328 Affiliation: Data and Web Science Group, University of Mannheim, Germany E-mail keyvan.amiri, heiner.stuckenschmidt@uni-mannheim.de Affiliation: SAP Signavio, Walldorf, Germany E-mail lukas.kirchdorfer@sap.com Heiner Stuckenschmidt OrcID: 0000-0002-0209-3859 Affiliation: Data and Web Science Group, University of Mannheim, Germany E-mail keyvan.amiri, heiner.stuckenschmidt@uni-mannheim.de Abstract The early detection of delayed cases in business processes is a critical capability for organizations. Predictive process monitoring (PPM) supports this task by using historical event logs to predict the remaining time of ongoing cases, enabling timely interventions to avoid missed deadlines and service level violations. Although remaining time prediction has advanced considerably through sophisticated deep learning architectures, little is known about the intrinsic difficulty of delay detection itself. Since performance is typically assessed using aggregate metrics, prior work provides limited insight into how models perform across the target distribution, especially on the operationally most critical cases with large delays. In this paper, we address this gap by analyzing the difficulty of delay detection. Across 14 event logs, we show that remaining times are typically strongly right-skewed, with only a small fraction of cases exhibiting large delays. Existing models capture the mode of this distribution well but perform poorly on high-delay cases. We further uncover pronounced heteroscedasticity, showing that predictive uncertainty increases with delay magnitude. Based on these findings, we evaluate approaches to mitigate the imbalance problem, but find only limited benefits, suggesting that the key underlying problem may not be imbalance but higher uncertainty associated with delayed cases. We show that this correlation can be exploited to substantially improve the identification of delayed cases. Overall, our work provides new insights into the sources of difficulty in delay detection and identifies uncertainty-aware modeling as a promising direction for future PPM research. Keywords: Predictive Process Monitoring Remaining Time Prediction Imbalanced Regression Uncertainty Quantification. 1 Introduction Organizations rely on business processes to deliver services to customers and ensure the timely execution of work. In this context, execution delays are a major concern, as they lead to missed deadlines, violated service level agreements, increased operational costs, and diminished customer satisfaction [23, 29, 12]. Consequently, the early identification of delay-prone cases is essential for effective process management. Such identification can leverage execution data recorded by enterprise information systems in the form of event logs, which document historical process executions and provide a basis for anticipating future behavior. In business process management, this is addressed through predictive process monitoring (PPM), which aims to predict future characteristics of running cases based on historical event data [5]. Among PPM tasks, remaining time prediction is one of the most widely studied. Given the observed prefix of an ongoing case, it estimates the time until completion and provides an actionable signal for identifying potentially delayed cases and enabling timely interventions. In recent years, substantial progress has been made in remaining time prediction, largely driven by advances in deep learning architectures such as LSTMs, Transformers, and graph neural networks [16, 1, 10]. While these approaches improve predictive performance, they are typically evaluated using aggregate metrics such as mean absolute error (MAE), which obscure where models perform well or fail. In particular, such metrics do not capture performance on rare delayed cases, despite their critical operational impact due to their potential to cause service level violations and increased costs. As we show in this paper, distributions of remaining time in business processes are typically highly right-skewed: most cases complete within an expected time frame, whereas a small number exhibit extreme delays. Accurately identifying these rare delayed cases is essential for effective process management, yet it remains unclear whether existing models can reliably detect them. More fundamentally, there is still limited understanding of the intrinsic difficulty of delay detection in business processes, which hinders the development of more effective PPM approaches grounded in process-specific insights rather than relying primarily on advances in general-purpose machine learning models. To address these challenges, this paper makes three contributions. 1. First, we study the intrinsic difficulty of delay detection in remaining time prediction. Our analysis shows that (a) remaining times typically follow a strongly right-skewed distribution, (b) existing approaches capture the mode of this distribution well but fail to predict cases with large delays accurately, and (c) predictive uncertainty is heteroscedastic and increases with remaining time, indicating that high-delay cases are substantially noisier than normal cases. 2. Second, building on findings (a) and (b), we hypothesize that a key reason why existing models fail on high-delay cases is the identified target imbalance. We therefore evaluate established imbalanced regression approaches and find, however, that their impact is limited. 3. Third, based on finding (c), we investigate the utility of heteroscedastic uncertainty estimates for delay detection. Specifically, we study whether the relationship between predictive uncertainty and remaining time magnitude can be exploited to improve the identification of delayed cases. Our results show that uncertainty-aware models can substantially improve delay detection. The remainder of the paper is structured as follows. Section 2 analyzes the difficulty of delay detection across a variety of processes. Section 3 describes the experimental setup, followed by the evaluation of imbalanced regression approaches in Section 4 and the assessment of heteroscedastic uncertainty in Section 5. Finally, we discuss related work in Section 6 and conclude the paper in Section 7. 2 Motivation and Analysis We illustrate the difficulty of delay detection using the Prepaid Travel Cost process from the BPIC20 benchmark and then show that the observed patterns generalize across other event logs. Figure 1: BPIC20PTC distribution of ground-truth and predicted remaining times (in days), with average error and heteroscedastic uncertainty per target range. Illustration. To illustrate the difficulty of delay detection, we analyze the BPIC20PTC process from Eindhoven University of Technology, which contains events related to two years of travel expense claims. Figure 1 shows the distribution of remaining times across all possible prefix positions in the process (orange histogram) and compares it with the distribution of predicted remaining times (blue histogram) produced by a standard LSTM-based approach [16]. The ground-truth distribution of remaining times is strongly right-skewed, with most observations concentrated around 10 days and only a small number extending to 40–100 days. In contrast, the predictions are compressed into a much narrower range and fail to capture high-delay cases. The model never predicts a remaining time above 35 days. As a result, a case that actually requires, for example, 80 days until completion would receive a prediction of only 30–35 days. Such errors render the prediction ineffective for timely delay identification and operational intervention. Consequently, prediction error increases substantially toward the tail of the distribution, as illustrated by the red line. Another key insight is that predictive uncertainty increases with remaining time, exhibiting a clear heteroscedastic pattern. In other words, the figure suggests a correlation between the magnitude of uncertainty and the magnitude of remaining time. Heteroscedastic uncertainty is data-dependent and reflects the irreducible noise inherent in the data generation process, and arises from stochastic and unobserved factors that influence process execution [27]. In our example process, the uncertainty is low around the mode of the distribution, where the majority of normal cases are located, but increases sharply for larger remaining times. It reaches its highest values around 40 days and remains elevated thereafter. This indicates that high-delay cases are not only rare, but also substantially noisier and therefore more difficult to predict than normal cases. Analysis Across Event Logs. To assess whether these observations are the rule rather than the exception, we analyze 14 publicly available event logs, focusing on the skewness of the remaining time distribution and the degree of heteroscedastic uncertainty. Table 1 summarizes the characteristics of the event logs. It reports the Fisher moment coefficient of skewness (γ) of the remaining time distribution, which quantifies its asymmetry. Positive skewness indicates a long right tail, with larger values corresponding to stronger asymmetry (for example, γ=2γ=2 for an exponential distribution). Most logs exhibit pronounced positive skewness, indicating heavy-tailed remaining time distributions where most cases complete within a normal time frame, while a small number experience extremely long durations. Table 1: Characteristics of the 14 event logs used for evaluation. Case duration is in days. γ denotes the skewness of the remaining-time distribution. Case duration Case duration Event log Cases Events Avg. Max γ Event log Cases Events Avg. Max γ P2P 608 9119 21.5 109.4 2.8 Helpdesk 4580 21348 40.9 60.0 -0.1 BPIC17W 30276 240854 12.7 288.9 4.2 Sepsis 1050 15214 28.5 422.3 3.0 BPIC15-1 1199 52217 95.9 1486.0 5.2 BPIC20ID 6449 72151 86.5 742.0 3.5 BPIC15-2 832 44354 160.3 1326.0 3.2 BPIC20D 10500 56437 11.5 469.2 11.5 BPIC15-3 1409 59681 62.2 1512.0 10.0 BPIC20PTC 2099 18246 36.8 328.2 4.0 BPIC15-4 1053 47293 116.9 927.0 3.7 BPIC20TPD 7065 86581 87.4 1202.0 2.6 BPIC15-5 1156 59083 98.0 1344.0 5.2 BPIC20RFP 6886 36796 12.0 410.0 10.6 Table 2: Spearman correlation between ground-truth remaining time y and (i) absolute prediction error |e||e| and (i) prediction interval (PI) width produced by the survival model (Surv). Dataset ρ(y,|e|)ρ(y,|e|) ρ(y,PISurv)ρ(y,PI_Surv) Dataset ρ(y,|e|)ρ(y,|e|) ρ(y,PISurv)ρ(y,PI_Surv) P2P 0.890.89 0.460.46 Helpdesk −0.41-0.41 −0.53-0.53 BPIC17W 0.380.38 −0.43-0.43 Sepsis 0.510.51 0.390.39 BPIC15-1 0.300.30 0.550.55 BPIC20ID 0.640.64 0.800.80 BPIC15-2 −0.20-0.20 −0.22-0.22 BPIC20D 0.470.47 0.210.21 BPIC15-3 0.680.68 0.640.64 BPIC20PTC 0.560.56 0.670.67 BPIC15-4 −0.69-0.69 0.090.09 BPIC20TPD 0.610.61 0.680.68 BPIC15-5 0.600.60 0.320.32 BPIC20RFP 0.550.55 0.220.22 To examine how prediction error and predictive uncertainty relate to remaining time, we analyze their association with the ground-truth remaining time y on the test sets (final 20% of cases) of each event log. Specifically, we compute the Spearman correlation between y and two quantities: (i) the absolute prediction error |y^−y|| y-y| of a standard LSTM model trained with MAE [16], and (i) the width of the prediction interval from an uncertainty-aware model based on a survival formulation [7]. Details on this model are provided in Section 5; here, it suffices to note that, unlike the standard LSTM, it produces both point estimates and associated uncertainty estimates in the form of prediction intervals. Table 2summarizes the correlations. There is a clear tendency toward heteroscedasticity: in 10 out of 14 event logs, both the absolute prediction error and the prediction interval width are positively correlated with the true remaining time. In several logs (e.g., BPIC15-3, BPIC20ID, BPIC20PTC, BPIC20TPD), these correlations are moderate to strong for both measures, indicating that long-running cases are substantially noisier than short-running ones. However, this pattern is not universal. Two datasets (BPIC15-2 and Helpdesk) show negative correlations for both measures, while BPIC17W and BPIC15-4 exhibit mixed behavior. This variation indicates that both the strength and direction of heteroscedasticity differ across processes. Nevertheless, the dominant trend is a positive association between remaining time and both prediction error and uncertainty. Consequently, remaining time prediction is inherently noisy, with certain regions—such as the long tail—being particularly difficult to predict. This noise can arise from a wide range of sources, including resource contention, synchronization effects, batching, task handovers, and system state fluctuations [2, 13]. These results indicate clear heteroscedasticity in the right tail of the remaining time distribution. Long-delay cases are not only rare, due to target imbalance, but also substantially more variable and harder to predict. Prediction difficulty in the tail, therefore, arises from both data scarcity and increased noise. 3 Problem Setting and Experimental Setup In this section, we define the remaining time prediction task and describe the experimental setup used to evaluate the proposed approaches. Problem Definition. We formalize remaining time prediction as a supervised learning task based on event log data. Event Log, Traces, Event Prefixes. Process execution data are recorded in an event log ℰE, which is a collection of traces. Each trace σ=⟨e1,e2,…,en⟩σ= e_1,e_2,...,e_n represents the chronological sequence of events for a single process instance (case). An event is a tuple e=(c,a,t,Δ)e=(c,a,t, ), where c is the case identifier, a the executed activity, t the timestamp, and Δ an attribute-value map with optional data payload. We define the projection πT(e)=t _T(e)=t to extract an event’s timestamp. A prefix of length m∈[1,n−1]m∈[1,n-1] is denoted σm=⟨e1,…,em⟩σ^m= e_1,…,e_m and represents partial execution up to the m-th event. Feature Extraction. Remaining time prediction is formulated as a regression task, starting with feature extraction. Event prefixes of varying lengths are encoded into feature vectors =Γ(σm)x= (σ^m), and target values y=πT(en)−πT(em)y= _T(e_n)- _T(e_m). This yields the training dataset =(i,yi)i=1ND=\(x_i,y_i)\_i=1^N, where i∈ℝdx_i ^d denotes the input and yi∈ℝy_i the target. Problem Statement. Given the training dataset D, the goal is to approximate the function y=f()y=f(x) that maps inputs to remaining time. The predictive model typically follows an encoder–head structure: an encoder =g(,θ)z=g(x,θ) produces a latent representation z, which is mapped to the predicted remaining time y^=h(,ψ) y=h(z,ψ). The parameters θ and ψ are optimized via supervised learning to minimize a regression loss: ℒ(θ,ψ)=1N∑i=1Nwi|y^i−yi|,if absolute error is used,1N∑i=1Nwi(y^i−yi)2,if squared error is used.L(θ,ψ)= cases 1N _i=1^Nw_i\,| y_i-y_i|,&if absolute error is used,\\[8.0pt] 1N _i=1^Nw_i\,( y_i-y_i)^2,&if squared error is used. cases (1) Note that standard loss functions (e.g., MAE, MSE) assign equal weight (wi=1w_i=1) to all training examples, biasing models toward frequent target values. Experimental Setup. For the experiments conducted in the subsequent sections, we will use the following event logs, models, and metrics. The event logs, implementation details, and configurations are available in our repository.11 1 https://zenodo.org/records/20433645 Event Logs and Data Split. We use 14 event logs for evaluation to ensure robust and generalizable results (see Table 1). We apply a temporal hold-out split, dividing each log into 64% training, 16% validation, and 20% test data. Model. We use a data-aware LSTM model proposed by Navarin et al. [16], which predicts remaining time from event prefixes encoded as sequences of feature vectors (control-flow, temporal, and all data attributes), left-padded to a fixed length. The model consists of two LSTM layers (hidden size 150) with layer normalization and dropout (0.1), followed by a dense linear regression head that maps the final hidden representation to the predicted remaining time. Training uses AdamW, batch size 128, early stopping (patience 30), and up to 300 epochs. Hyperparameter Tuning. Hyperparameters are tuned on the validation set using Bayesian optimization. We define approach-specific search spaces, while keeping the model architecture fixed (see details in the supplementary repository). Evaluation Metrics. To analyze prediction difficulty across target regions, we partition test samples into many/medium/few groups using quantile-based splits. Thresholds are computed from the training and validation targets. By default, the few-shot region corresponds to the highest 10% of target values, the medium region to the next 30%, and the many-shot region to the remaining 60%. To enable comparisons across datasets with different cycle time scales, we use a normalized mean absolute error (nMAE) [21]. For a region r, it is defined as nMAEr=1|r|∑i∈r|yi−y^i|1N∑i=1N|yi−median(ytrain)|,nMAE_r= 1|r| _i∈ r|y_i- y_i| 1N _i=1^N|y_i-median(y_train)|, where yiy_i and y^i y_i denote true and predicted remaining times, |r||r| is the number of samples in region r, and N is the total number of test samples. The denominator corresponds to the mean absolute deviation from the training-set median, i.e., the error of a constant median predictor. This normalizes the MAE relative to a trivial baseline, enabling assessment of prediction difficulty and model performance across regions and datasets on a common scale. Consequently, an nMAE of 1 reflects performance comparable to this baseline, values below 1 indicate improved predictions, and values above 1 indicate worse performance. This interpretation enables direct comparison across many-, medium-, and few-shot regions. 4 Assessing the Effectiveness of Imbalanced Regression The remaining time distribution is often highly skewed, with only a small fraction of cases exhibiting large delays. As a result, standard machine learning models tend to perform poorly on these rare cases, which naturally leads to the hypothesis that target imbalance is a key source of the problem. In this section, we therefore evaluate imbalanced regression approaches designed to address this issue, which have shown promising results in time series forecasting, computer vision, and natural language processing [4, 28, 22, 19]. 4.1 Data-level Approaches: Resampling the Training Distribution We describe data-level approaches for imbalanced regression and evaluate their effectiveness for remaining time prediction. Approach. Data-level approaches mitigate imbalance via resampling the training data, either by undersampling frequent target ranges or oversampling rare ones. Oversampling methods generate synthetic samples in sparse regions of the target space, e.g., by interpolating between neighboring observations as in SMOTER [25], or by perturbing rare samples with Gaussian noise [3]. Hybrid methods such as SMOGN combine both strategies by undersampling frequent regions and oversampling rare targets through interpolation and noise injection [4]. Applying data-level approaches to PPM is non-trivial. Remaining time prediction operates on event prefixes encoded as left-padded sequences, where the meaning of an input depends on both feature values and their position. Naive interpolation can produce unrealistic samples by mixing padding with valid events or combining prefixes of different lengths, distorting the sequential structure and violating admissible process behavior. We adapt SMOGN to this setting as a feature-space augmentation procedure applied after prefix encoding. Rare samples are identified using a relevance function derived from the training target distribution, assigning higher importance to large remaining times. To avoid mixing prefixes of different temporal extent, we first group samples by their actual prefix length, computed from the non-padded part of the left-padded input sequence. Nearest neighbors are computed only within these prefix-length groups. For each prefix, the padded rows are discarded, the remaining encoded event-level feature vectors are flattened, and Euclidean distance is used to select neighbors. Thus, padding positions do not influence neighbor selection. Synthetic samples are generated in the encoded numerical input space, rather than from raw trace symbols or learned embedding vectors. For interpolation-based oversampling, we linearly interpolate the non-padded feature vectors of a rare prefix and one of its neighbors from the same prefix-length group; the padded part remains unchanged. The remaining-time target is interpolated analogously. Alternatively, Gaussian noise is added only to the non-padded part of rare prefixes, scaled by the empirical feature-wise standard deviation within the group. Frequent samples are reduced via random undersampling, while all rare samples are retained. Figure 2: BPIC20PTC target distribution for various configurations of SMOGN. Experiments. Figure 2 shows how SMOGN reshapes the training distribution for BPIC20PTC under different configurations. We vary the oversampling factor for rare cases and the undersampling ratio for frequent ones. Increasing both shifts probability mass toward higher remaining times, increasing the density of delayed cases. However, this does not improve predictive performance: the MAE remains between 8.1 and 9.4 days across configurations. We apply SMOGN to all 14 event logs and compare it to the same LSTM trained on the original (imbalanced) data. Averaged over five seeds per log, SMOGN reduces nMAE in the few-shot region by 8.0% but increases error in the medium and many regions by 20.5% and 31.4%, respectively, resulting in a 17.5% higher overall nMAE. Detailed results are provided in the supplementary GitHub repository. Overall, even substantial changes to the training distribution do not improve predictive performance, suggesting that data scarcity in the tail is not the primary bottleneck. We therefore turn to algorithm-level approaches. 4.2 Algorithm-level Approaches: Reweighting the Loss Function Algorithm-level approaches mitigate imbalance by modifying the regression objective in Equation 1 to assign greater weight to underrepresented target regions. We first introduce these approaches and then evaluate their effectiveness for remaining time prediction. Approaches. We consider cost-sensitive weighting, distribution-balanced objectives, error-aware losses, and relevance-based error measures, and refer to the original publications for details. Cost-Sensitive Weighting (CSW). CSW assigns static loss weights to different target ranges based on the empirical target distribution. The continuous target domain is discretized into bins, and their frequencies are estimated from the training data. Samples in less frequent bins receive higher weights, while those in dense regions are down-weighted. In practice, weights are set inversely proportional to the bin frequency or its square root [28]. Balanced Mean Squared Error (BMSE). BMSE incorporates the target distribution directly into the regression objective. We use the batch-based Monte Carlo (BMC) variant proposed in [19]. Instead of applying static weights, BMC contrasts the prediction error of a sample with the errors obtained for other target values within the same mini-batch. This is implemented through a softmax-style normalization over squared errors across batch targets. Consequently, prediction errors associated with dense target regions receive relatively less influence than those in sparse regions. Error-Aware Loss (EAL). EAL adjusts sample contributions based on prediction error magnitude. We consider the Focal-R loss [28], which modulates the error by a nonlinear function that increases with its magnitude. This reduces the contribution of small errors while increasing the relative influence of larger errors. As a result, the optimization focuses more on difficult examples, which are often associated with rare or extreme target values. Squared Error Relevance Area (SERA). SERA [22] is a relevance-based loss for imbalanced regression. It relies on a relevance function ϕ:Y→[0,1]φ:Y→[0,1] that assigns each target value an importance score, allowing non-uniform preference over the continuous target domain [26, 3]. In this work, ϕφ is automatically derived from the training target distribution using box-plot statistics [20], assigning higher relevance to extreme remaining time values. Consequently, prediction errors on highly relevant targets receive greater weight, while errors on low-relevance targets contribute less to the overall loss. Table 3: Comparison of algorithm-level approaches using average ranks (lower is better) and significant pairwise differences identified by the Wilcoxon signed-rank test. Results are reported for nMAE across target regions. Mode Average Ranks (lower is better) Significant Performance Differences (Wilcoxon signed-rank test p−valuesp-values) Many Vanilla (1.64), CSW (2.14), EAL (2.36), BMSE (3.86), SERA (5.00) Vanilla > SERA (0.001); CSW > SERA (0.006); EAL > SERA (0.015) Medium CSW (2.21), BMSE (2.36), EAL (2.64), Vanilla (2.86), SERA (4.93) CSW > SERA (0.012); BMSE > SERA (0.020) Few SERA (1.00), BMSE (2.14), CSW (3.57), EAL (3.79), Vanilla (4.50) Vanilla vs BMSE (0.042); SERA > Vanilla (0.000); SERA > CSW (0.020); SERA > EAL (0.009) All EAL (1.93), CSW (2.00), Vanilla (2.36), BMSE (3.71), SERA (5.00) Vanilla > SERA (0.015); CSW > SERA (0.004); EAL > SERA (0.003) Experiments. We compare the above approaches to a baseline trained with standard MAE (Vanilla), using nMAE across different target regions averaged over five random seeds. A Friedman test reveals significant differences across approaches (p<0.001p<0.001 for all regions: all, many, medium, few), motivating a more detailed comparison. In Table 3, approaches are ranked per dataset by nMAE within each region and averaged across datasets, while significant pairwise differences are identified using the Wilcoxon signed-rank test. In the many-shot region, Vanilla attains the best average rank (1.64), while SERA performs significantly worse than Vanilla, CSW, and EAL. In the medium-shot region, results are comparable, with SERA again underperforming relative to CSW and BMSE. A different pattern emerges in the few-shot region, where SERA achieves the best rank (1.00) and significantly outperforms Vanilla, CSW, and EAL; BMSE ranks second and also improves over Vanilla. However, these gains do not translate to overall performance: SERA performs significantly worse overall, while EAL and CSW achieve the best ranks. (a) Many (b) Medium (c) Few (d) All Figure 3: Distribution of nMAE across datasets for algorithm-level imbalanced regression approaches, analyzed across different target-frequency regions. The boxplots in Figure 3 show nMAE distributions across datasets for each approach. SERA is omitted due to its substantially larger errors in the many- and medium-shot regions and overall. In the many-shot region, Vanilla, CSW, and EAL exhibit nearly identical medians and similar variability, while BMSE shows a slightly higher median and greater spread. A similar pattern appears in the medium-shot region, with increased variability and higher errors across all approaches. In contrast, the few-shot region exhibits substantially higher variability across datasets and consistently large errors, with nMAE often exceeding 1. Since nMAE is normalized by the error of a constant median predictor, values above 1 indicate performance worse than this trivial baseline. This highlights the difficulty and instability of prediction in the long-tail region. We also evaluate feature and label distribution smoothing [28] in combination with these approaches, but observe only marginal and inconsistent effects (see supplementary repository). Overall, algorithm-level approaches do not consistently improve prediction accuracy in the long tail. When improvements occur (e.g., SERA), they come at the cost of worse performance in the many-shot region and overall accuracy. Thus, if the goal is solely to identify high-delay cases, SERA may be effective. However, in most practical settings, models are expected to perform well across all cases. 5 Assessing the Effectiveness of Uncertainty Modeling Building on the observation from Section 2 that heteroscedastic uncertainty increases with remaining time and is particularly pronounced in the long tail of the distribution, we investigate whether this relationship can be exploited to improve delay detection. To this end, we turn the previously used deterministic LSTM into a probabilistic model via survival analysis, enabling it to produce explicit uncertainty estimates. These estimates are then used as input to a downstream classifier that predicts whether a case will exceed a given delay threshold. This reformulation is not intended to replace remaining time regression, but rather to study whether uncertainty estimates can help distinguish highly delayed cases from normal executions. We first introduce the uncertainty modeling framework and then evaluate its utility for delay detection. Modeling Uncertainty via Survival Analysis. To capture predictive uncertainty, we model remaining time using a discrete-time survival formulation, treating process completion as a time-to-event problem [7]. Given an input feature vector x, instead of predicting a single remaining time value y y, the model estimates a distribution over future completion times. Concretely, the remaining time horizon is discretized into K ordered intervals using quantile-based binning of the training targets, where K is treated as a tunable hyperparameter. For each interval k∈1,…,Kk∈\1,…,K\, the model predicts a hazard value hk()h_k(x), corresponding to the conditional probability that the case completes in interval k given that it has not completed earlier. The predictive model uses the same encoder as the Vanilla LSTM model to produce a latent representation =g(,θ)z=g(x,θ), which is mapped to K hazard logits via a dedicated output layer. These logits are transformed into probabilities representing the discrete hazard function. Training minimizes the negative log-likelihood of the observed event time under this formulation. From the predicted hazards, the full discrete remaining time distribution is reconstructed via the corresponding survival and event probabilities. This allows deriving both point estimates and predictive uncertainty measures. In our implementation, we extract summary statistics including the expected remaining time and prediction intervals directly from this distribution. Heteroscedastic uncertainty is quantified by the width of the central 80% prediction interval. Further implementation details are provided in the accompanying repository. Delay Detection via Uncertainty Modeling. To evaluate the utility of uncertainty estimates, we formulate delay detection as a binary classification task. A case is labeled as delayed if its total duration exceeds a predefined quantile threshold (e.g., q=0.8q=0.8) of the cycle times in the training and validation sets. The goal is to predict this label for ongoing cases based on their observed prefixes. As illustrated in Figure 4, we develop an uncertainty-aware model that uses the outputs of the previously described survival model. Specifically, we extract distributional summaries (mean and median remaining time), uncertainty measures (prediction standard deviation and the widths of 80% and 90% prediction intervals), tail mass (i.e., the probability that the remaining time exceeds the modeled horizon), and temporal context features (elapsed time since the start of the case and time since the last event). These features are used as input to a gradient boosting classifier (CatBoost). This design enables the model to incorporate uncertainty signals about future completion times, rather than relying solely on prefix features. Figure 4: Uncertainty-aware delay detection pipeline using survival-based predictions and a tabular classifier. Results. We evaluate the performance of our uncertainty-aware model in detecting high-delay cases using recall, precision, F1-score, and PR-AUC across all 14 event logs in Table 4. We compare our model against a baseline, which shares the same LSTM encoder as the survival model, but—instead of modeling uncertainty—directly produces a point estimate, classifying the prefix either as normal or high-delay. The baseline achieves very low recall and therefore misses most delayed cases. In contrast, the uncertainty-aware model substantially improves recall across all datasets, increasing the average from 0.21 to 0.61. This means that a much larger proportion of delayed cases can be identified in advance, which is essential for timely intervention. For instance, on the BPIC20PTC log, the baseline correctly identifies only 1 out of 175 high-delay cases, whereas the uncertainty-aware model detects 132. A Wilcoxon signed-rank test (p<0.0001p<0.0001) confirms that this improvement is statistically significant. Table 4: Delay detection performance (threshold q=0.8q=0.8). Average Recall, Precision, F1-score, and PR-AUC over five random seeds for baseline and uncertainty-aware models. Baseline Classifier Uncertainty-Aware Model Dataset Recall Precision F1-Score PRAUC Recall Precision F1-Score PRAUC P2P 0.11 0.99 0.20 0.74 0.83 0.95 0.89 0.91 BPIC17W 0.34 0.48 0.40 0.38 0.59 0.77 0.67 0.59 BPIC15-1 0.49 0.34 0.39 0.23 0.66 0.40 0.49 0.30 BPIC15-2 0.38 0.11 0.17 0.07 0.75 0.12 0.21 0.10 BPIC15-3 0.31 0.39 0.34 0.23 0.65 0.65 0.65 0.48 BPIC15-4 0.40 0.04 0.07 0.04 0.75 0.06 0.12 0.05 BPIC15-5 0.33 0.42 0.36 0.24 0.53 0.41 0.46 0.29 HelpDesk 0.02 0.32 0.04 0.21 0.66 0.25 0.36 0.24 Sepsis 0.19 0.59 0.27 0.26 0.51 0.28 0.36 0.24 BPIC20ID 0.05 0.04 0.04 0.02 0.39 0.12 0.18 0.06 BPIC20D 0.11 0.52 0.18 0.30 0.47 0.65 0.55 0.45 BPIC20PTC 0.01 0.18 0.01 0.06 0.76 0.20 0.31 0.16 BPIC20TPD 0.07 0.22 0.11 0.11 0.46 0.28 0.35 0.18 BPIC20RFP 0.18 0.73 0.29 0.42 0.48 0.74 0.58 0.53 Average 0.21 0.38 0.21 0.23 0.61 0.42 0.44 0.33 These gains are achieved while maintaining competitive precision: in most cases, precision remains comparable or improves (e.g., BPIC17W: 0.48 to 0.77), with no statistically significant difference overall (p=0.17p=0.17). Consequently, the F1-score consistently improves across datasets, reflecting a better balance between precision and recall. Similarly, PR-AUC improves in nearly all cases, indicating improved ranking of delayed cases. These improvements are statistically significant (Wilcoxon test: p<0.0001p<0.0001 for F1-score, p<0.0003p<0.0003 for PR-AUC). Overall, these results demonstrate that incorporating predictive uncertainty improves delay detection performance by substantially increasing the ability to identify delayed cases without sacrificing precision. This suggests that uncertainty provides actionable information for prioritizing high-risk cases in practice. We observe similar improvements under a stricter delay definition (q=0.9q=0.9), as reported in the supplementary repository. However, since delay detection abstracts the original regression problem into a threshold-based classification task, these findings should not be interpreted as improvements in remaining time estimation itself. To further assess how early delayed cases can be identified during process execution, we perform an earliness analysis that evaluates model performance at different stages of a running case. Specifically, each prefix is interpreted as a partial observation of an ongoing process instance, and its length is normalized by the total number of events in that case (i.e., its full execution trace). This allows us to group prefixes according to how far the case has progressed (e.g., early, mid, and late stages), and to evaluate performance on subsets of prefixes corresponding to up to 20%, 40%, 60%, and 80% of the process execution. The uncertainty-aware model consistently outperforms the baseline across all prefix ratios, with the most pronounced gains at early stages. In particular, recall is substantially higher even when only a small portion of the case is observed, while precision remains comparable, resulting in consistently improved F1-scores and PR-AUC. Although the performance gap narrows as more of the case becomes available, the uncertainty-aware model remains superior throughout, indicating that incorporating predictive uncertainty enables earlier and more reliable detection of delayed cases. 6 Prior Work In this section, we review prior work on imbalanced learning in predictive process monitoring (PPM) and discuss emerging uncertainty-aware approaches for remaining time prediction. Data imbalance is a recurring challenge across a wide range of PPM tasks [17], including next activity prediction, outcome prediction, and anomaly detection. In these settings, imbalance arises from skewed class distributions, where rare but operationally critical events (e.g., exceptional activities, negative outcomes, or anomalies) are underrepresented. Existing work has addressed imbalance in PPM primarily using techniques from imbalanced classification. In next activity prediction, cost-sensitive learning and focal-style losses have been proposed to emphasize hard and minority examples, improving performance on infrequent activities [11, 18, 9]. At the data level, oversampling techniques have been used to rebalance class distributions [15]. In outcome prediction, resampling approaches such as SMOTE and its variants are commonly applied to mitigate class imbalance [24, 14]. Similar strategies have also been adopted in domain-specific applications, including loan default and payment delay prediction [30, 29, 23]. Finally, in process anomaly and deviation detection, where imbalance is inherent, prior work often combines representation learning with biased sampling or under-sampling to better detect rare behaviors [6, 8]. Overall, these approaches can be broadly categorized into data-level methods (e.g., resampling [14]), algorithm-level methods (e.g., cost-sensitive learning [11]), and hybrid combinations [30]. However, all existing work focuses on classification settings. In contrast, imbalance in regression tasks—such as remaining time prediction—has not been systematically addressed, despite the strongly skewed and long-tailed nature of target distributions in this domain. In parallel, a small body of work has begun to investigate uncertainty-aware modeling for remaining time prediction [27, 2]. These approaches aim to quantify predictive uncertainty, which can provide additional information beyond point estimates. However, they do not explicitly address imbalance, nor do they study the interaction between skewed target distributions and predictive uncertainty. In this work, we address these gaps by systematically studying imbalanced regression in PPM, considering both data-level and algorithm-level approaches, and by investigating uncertainty-aware modeling as a complementary perspective for delay detection. 7 Conclusion In this paper, we investigated the intrinsic difficulty of delay detection in PPM. Our analysis showed that remaining times in business processes are typically strongly right-skewed, with existing models capturing normal cases well but struggling on the rare and operationally critical high-delay cases. We further found that predictive uncertainty increases with delay magnitude, revealing pronounced heteroscedasticity in the tail of the distribution. Thus, our findings identified target imbalance and noise as potential sources of difficulty. While established imbalanced regression approaches yielded only limited improvements, leveraging predictive uncertainty substantially enhanced the identification of delayed cases. These findings highlight the long tail of remaining time prediction as a key challenge for PPM and identify uncertainty-aware modeling as a promising avenue for future work. This work has several limitations that open up avenues for future research. First, while we show that predictive uncertainty can substantially improve delay detection, the underlying sources of this uncertainty are not yet fully disentangled and may stem from multiple factors, such as process variability, missing context, or concept drift. Future work could therefore investigate richer uncertainty modeling approaches, integrate additional contextual information, and study how delay detection methods can be adapted online in changing process environments. Second, reformulating remaining time prediction as a binary delay detection task entails a loss of information and dependence on a threshold choice. A promising direction for future work is therefore to move beyond binary classification and instead model the full conditional distribution of remaining time, enabling delay detection and related problems (e.g., service level agreement violations) to be formulated as downstream decision rules rather than fixed thresholds. Third, our empirical evaluation is based on a set of publicly available event logs and LSTM-based predictive models. Although this reveals consistent patterns, it remains an open question how strongly these findings generalize to other process domains, industrial settings, and other architectures. References [1] K. Amiri Elyasi, H. van der Aa, and H. Stuckenschmidt (2024) PGTNet: a process graph transformer network for remaining time prediction of business process instances. In CAISE, p. 124–140. Cited by: §1. [2] K. Amiri Elyasi, H. van der Aa, and H. Stuckenschmidt (2025) A simple and calibrated approach for uncertainty-aware remaining time prediction. In International Conference on Business Process Management, p. 217–234. Cited by: §2, §6. [3] P. Branco, R. P. Ribeiro, and L. Torgo (2016) UBL: an r package for utility-based learning. arXiv preprint arXiv:1604.08079. Cited by: §4.1, §4.2. [4] P. Branco, L. Torgo, and R. P. Ribeiro (2017) SMOGN: a pre-processing approach for imbalanced regression. In First international workshop on learning with imbalanced domains: Theory and applications, p. 36–50. Cited by: §4.1, §4. [5] C. Di Francescomarino and C. Ghidini (2022) Predictive process monitoring. In Process mining handbook, p. 320–346. Cited by: §1. [6] E. A. Elaziz et al. (2023) Deep reinforcement learning for data-efficient weakly supervised business process anomaly detection. Journal of Big Data 10 (1). Cited by: §6. [7] B. George, S. Seals, and I. Aban (2014) Survival analysis and regression models. Journal of nuclear cardiology 21 (4), p. 686–694. Cited by: §2, §5. [8] M. Grohs, P. Pfeiffer, and J. Rehse (2023) Business process deviation prediction: predicting non-conforming process behavior. In ICPM, p. 113–120. Cited by: §6. [9] X. He, R. Oyamada, J. De Smedt, et al. (2025) Tackling multi-class imbalance in next activity prediction with class-balanced focal loss. In ICPM Workshops, Cited by: §6. [10] M. C. Hennig and R. Schmidt (2025) Leveraging temporal graphs for enhancing transformer-based predictive process monitoring. In BPM, p. 291–307. Cited by: §1. [11] M. Käppel, S. Jablonski, and S. Schönig (2021) Cost-sensitive predictive business process monitoring. In European Conference on Advances in Databases and Information Systems, p. 14–26. Cited by: §6, §6. [12] S. Kim, H. Kim, and Y. Park (2017) Early detection of vessel delays using combined historical and real-time information. J. operational research society 68 (2). Cited by: §1. [13] A. Kraus, K. A. Elyasi, and H. van der Aa (2025) On the use of steady-state detection for process mining: achieving more accurate insights. In CAISE, p. 204–220. Cited by: §2. [14] A. E. Márquez-Chamorro, M. Resinas, A. Ruiz-Cortés, and M. Toro (2017) Run-time prediction of business process indicators using evolutionary decision rules. Expert Systems with Applications 87, p. 1–14. Cited by: §6, §6. [15] N. Mehdiyev, P. Fettke, and J. Evermann (2020) A novel business process prediction model using a deeplearning method. BISE 62 (2), p. 143–157. Cited by: §6. [16] N. Navarin, B. Vincenzi, M. Polato, and A. Sperduti (2017) LSTM networks for data-aware remaining time prediction of business process instances. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI), p. 1–7. Cited by: §1, §2, §2, §3. [17] D. A. Neu, J. Lahann, and P. Fettke (2022) A systematic literature review on state-of-the-art deep learning methods for process prediction. Artificial Intelligence Review 55 (2), p. 801–827. Cited by: §6. [18] A. Nguyen, S. Chatterjee, S. Weinzierl, L. Schwinn, M. Matzner, and B. Eskofier (2020) Time matters: time-aware lstms for predictive business process monitoring. In International Conference on Process Mining, p. 112–123. Cited by: §6. [19] J. Ren, M. Zhang, C. Yu, and Z. Liu (2022) Balanced mse for imbalanced visual regression. In CVPR, p. 7926–7935. Cited by: §4.2, §4. [20] R. P. Ribeiro and N. Moniz (2020) Imbalanced regression and extreme value prediction. Machine Learning 109 (9), p. 1803–1835. Cited by: §4.2. [21] J. Roider, A. Nguyen, D. Zanca, and B. M. Eskofier (2024) Assessing the performance of remaining time prediction methods for business processes. IEEE Access. Cited by: §3. [22] A. Silva, R. P. Ribeiro, and N. Moniz (2022) Model optimization in imbalanced regression. In International Conference on Discovery Science, p. 3–21. Cited by: §4.2, §4. [23] T. Tater, S. Dechu, S. Mani, and C. Maurya (2018) Prediction of invoice payment status in account payable business process. In International Conference on Service-Oriented Computing, p. 165–180. Cited by: §1, §6. [24] I. Teinemaa, M. Dumas, F. M. Maggi, and C. Di Francescomarino (2016) Predictive business process monitoring with structured and unstructured data. In International Conference on Business Process Management, p. 401–417. Cited by: §6. [25] L. Torgo, R. P. Ribeiro, B. Pfahringer, and P. Branco (2013) Smote for regression. In Portuguese conference on artificial intelligence, p. 378–389. Cited by: §4.1. [26] L. Torgo and R. Ribeiro (2007) Utility-based regression. In European conference on principles of data mining and knowledge discovery, p. 597–604. Cited by: §4.2. [27] H. Weytjens and J. De Weerdt (2022) Learning uncertainty with artificial neural networks for predictive process monitoring. Applied Soft Computing 125, p. 109134. External Links: ISSN 1568-4946 Cited by: §2, §6. [28] Y. Yang, K. Zha, Y. Chen, H. Wang, and D. Katabi (2021) Delving into deep imbalanced regression. In ICML, p. 11842–11851. Cited by: §4.2, §4.2, §4.2, §4. [29] S. Zeng, P. Melville, C. A. Lang, I. Boier-Martin, and C. Murphy (2008) Using predictive analysis to improve invoice-to-cash collection. In KDD, p. 1043–1050. Cited by: §1, §6. [30] X. Zhang, T. Zhang, L. Hou, X. Liu, Z. Guo, Y. Tian, and Y. Liu (2025) Data-driven loan default prediction: a machine learning approach for enhancing business process management. Systems 13 (7), p. 581. Cited by: §6, §6.