Paper deep dive
Cross-Dataset Transfer and Reliability of Explainable Artificial Intelligence for RhythmFormer Remote Photoplethysmography
Louis Chen, Torbjörn E. M. Nordling
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/4/2026, 3:43:09 AM
Summary
This study evaluates the cross-dataset transfer and reliability of Explainable Artificial Intelligence (XAI) methods for the RhythmFormer remote photoplethysmography (rPPG) model. Using the NCKU-rPPG and UBFC-rPPG datasets, the authors trained eight condition-specific RhythmFormer models and assessed XAI explanations (Raw attention, rollout, attention flow, Beyond Intuition) using skin coverage and the Salience-guided Faithfulness Coefficient (SaCo). Results indicate that Beyond Intuition performed best in faithfulness metrics, but these metrics did not consistently correlate with model performance (heart-rate error, waveform correlation, SNR) within individual clips. However, across scenarios, Beyond Intuition's coverage correlated with performance measures. The study concludes that attribution faithfulness provides complementary information to performance metrics but does not guarantee accuracy.
Entities (10)
Relation Signals (9)
RhythmFormer → evaluatedon → UBFC-rPPG
confidence 95% · set them beside a UBFC-rPPG reproduction.
Beyond Intuition → measuredby → Salience-guided Faithfulness Coefficient
confidence 95% · Beyond Intuition ranked highest on both datasets, at median coverage 0.789 and SaCo 0.837
Beyond Intuition → measuredby → Skin Coverage
confidence 95% · Beyond Intuition ranked highest on both datasets, at median coverage 0.789
RhythmFormer → trainedon → NCKU-rPPG
confidence 95% · We trained eight condition-specific RhythmFormer models on NCKU-rPPG
NCKU-rPPG → collectedby → National Cheng Kung University
confidence 90% · At the Nordling Lab at National Cheng Kung University (NCKU), we have collected a remote photoplethysmography (rPPG) multi-condition dataset, hereafter NCKU-rPPG
Beyond Intuition → outperforms → Attention Flow
confidence 90% · Beyond Intuition ranked highest on both datasets... lower ranks differed.
Beyond Intuition → outperforms → rollout
confidence 90% · Beyond Intuition ranked highest on both datasets... lower ranks differed.
Beyond Intuition → outperforms → Raw attention
confidence 90% · Beyond Intuition ranked highest on both datasets... lower ranks differed.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Background. Remote photoplethysmography estimates the cardiovascular pulse from facial video, and its explanations have rested on inspecting heatmaps rather than on quantitative evidence about where a model reads it. We quantified the explanations and asked whether such explanations transfer between datasets and track model performance. Method. We trained eight condition-specific RhythmFormer models on NCKU-rPPG, recorded under three illumination levels, speaking, rotation, and cycling, estimated one heart rate per 5.12-second clip, and set them beside a UBFC-rPPG reproduction. Raw attention, rollout, attention flow, and Beyond Intuition were assessed by skin coverage and the Salience-guided Faithfulness Coefficient (SaCo). Results. Beyond Intuition ranked highest on both datasets, at median coverage 0.789 and SaCo 0.837 on Static level 3 against 0.826 and 0.917 on UBFC-rPPG; lower ranks differed. Within one participant of one condition, neither measure was related to a clip's heart-rate error, waveform correlation, or signal-to-noise ratio on either dataset: 186 of the 252 coefficients fell below $|\rho|=0.10$ and 28 reached $p<0.05$ against the 13 expected by chance. Across the eight scenarios only Beyond Intuition's coverage followed the three performance measures, at $\rho=-0.43$, $+0.57$, and $+0.43$, while the attention-only methods' SaCo ran opposite to each. It failed at 40 lux alone, its median coverage falling to 0.180 and its median SaCo to $-0.178$, whereas motion degraded the estimates far more without such a drop. Conclusions. Skin coverage and SaCo carry information complementary to the performance measures rather than a proxy for them: attributing to the skin does not guarantee an accurate estimate. What an attribution reveals about a condition is where the model looks rather than how faithfully its map is ordered.
Tags
Links
- Source: https://arxiv.org/abs/2609.03663v1
- Canonical: https://arxiv.org/abs/2609.03663v1
Trouble viewing inline? Open PDF directly →
Full Text
213,954 characters extracted from source content.
Expand or collapse full text
Cross-Dataset Transfer and Reliability of Explainable Artificial Intelligence for RhythmFormer Remote Photoplethysmography Louis Chen †thanks: Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan. ORCID: 0009-0001-8373-2295. Email: louis.chen@nordlinglab.org. Torbjörn E. M. Nordling †thanks: Department of Mechanical Engineering, National Cheng Kung University, Tainan, Taiwan. ORCID: 0000-0003-4867-6707. Corresponding author: torbjorn.nordling@nordlinglab.org. September 3, 2026 Abstract Background. Remote photoplethysmography estimates the cardiovascular pulse from facial video, and its explanations have rested on inspecting heatmaps rather than on quantitative evidence about where a model reads it. We quantified the explanations and asked whether such explanations transfer between datasets and track model performance. Method. We trained eight condition-specific RhythmFormer models on NCKU-rPPG, recorded under three illumination levels, speaking, rotation, and cycling, estimated one heart rate per 5.12-second clip, and set them beside a UBFC-rPPG reproduction. Raw attention, rollout, attention flow, and Beyond Intuition were assessed by skin coverage and the Salience-guided Faithfulness Coefficient (SaCo). Results. Beyond Intuition ranked highest on both datasets, at median coverage 0.789 and SaCo 0.837 on Static level 3 against 0.826 and 0.917 on UBFC-rPPG; lower ranks differed. Within one participant of one condition, neither measure was related to a clip’s heart-rate error, waveform correlation, or signal-to-noise ratio on either dataset: 186 of the 252 coefficients fell below |ρ|=0.10|ρ|=0.10 and 28 reached p<0.05p<0.05 against the 13 expected by chance. Across the eight scenarios only Beyond Intuition’s coverage followed the three performance measures, at ρ=−0.43ρ=-0.43, +0.57+0.57, and +0.43+0.43, while the attention-only methods’ SaCo ran opposite to each. It failed at 40 lux alone, its median coverage falling to 0.180 and its median SaCo to −0.178-0.178, whereas motion degraded the estimates far more without such a drop. Conclusions. Skin coverage and SaCo carry information complementary to the performance measures rather than a proxy for them: attributing to the skin does not guarantee an accurate estimate. What an attribution reveals about a condition is where the model looks rather than how faithfully its map is ordered. Keywords: remote photoplethysmography; RhythmFormer; cross-dataset transfer; model reliability; attribution faithfulness; sparse attention; physiological signal estimation. 1 Introduction Remote photoplethysmography (rPPG) recovers the cardiac pulse from ordinary facial video, but the pulse-induced skin-colour change is weak and easily confounded by illumination, camera response, and motion. Public rPPG datasets span stationary recordings, deliberate head motion, speaking, exercise, and changes in illumination, but they differ markedly in camera characteristics, physiological references, sampling rates, and participant composition [1, 2, 3, 4, 5]. These differences make dataset adaptation a substantive part of model evaluation and a model trained on one dataset may not generalise to another, even if the datasets are similar in some respects [6, 5]. The best performing rPPG models are deep neural networks (DNNs) [7], but their complexity makes it difficult to understand how they work and whether they can be trusted [8]. In particular in clinical applications, it is important to know when a model’s predictions are reliable and whether its explanations are faithful to the model’s behaviour. The present study investigates whether explainable artificial intelligence (XAI) methods can provide insight into rPPG model behaviour and whether their explanations transfer across datasets and track model reliability. RhythmFormer is a DNN demonstrated to perform well on some datasets. RhythmFormer uses a hierarchical Temporal Periodic Transformer with Periodic Sparse Attention to recover an rPPG waveform from facial video [9]. At the Nordling Lab at National Cheng Kung University (NCKU), we have collected a remote photoplethysmography (rPPG) multi-condition dataset, hereafter NCKU-rPPG, which includes eight scenarios with different illumination, motion, and speaking conditions. This is to overcome the limitations of existing datasets, like the widely used UBFC-rPPG dataset that we also used, and to provide a more comprehensive evaluation of rPPG models. We therefore investigated whether RhythmFormer explanations transfer across datasets and whether their faithfulness tracks model reliability. First, we compare explainable artificial intelligence (XAI) methods on Static level 3 and UBFC-rPPG, the NCKU-rPPG condition and the reference dataset whose recording settings and clip lengths are closest, to test whether attribution rankings and correlation structures recur. Second, we compare each method’s median Salience-guided Faithfulness Coefficient across eight NCKU-rPPG scenarios with scenario-level mean absolute error, waveform Pearson correlation, and signal-to-noise ratio. The conversion, participant-independent split, and eight independently trained checkpoints provide the experimental basis for these comparisons, but the design supports descriptive transfer and reliability evidence rather than significance, causality, or cross-dataset generalisation claims. 2 Related Work Remote photoplethysmography (rPPG) validation datasets pair facial video with synchronised contact cardiac signals, such as electrocardiography (ECG) or photoplethysmography (PPG). Dataset suitability for this study depends first on whether complete facial videos and synchronised physiological signals can be obtained under workable research terms. PURE [1], COHFACE [2], and UBFC-rPPG [4] provide direct research-use acquisition routes, although their publications do not state a named standard public licence. MAHNOB-HCI [10], ECG-Fitness [11], AMIGOS [12], VIPL-HR [3], and MMPD [5] require a signed academic or non-commercial data-use agreement before access and are therefore usable only after that administrative step. The remaining datasets have more restrictive practical roles. LGI Multi-Session [13] has an announced research licence, but a complete working download was not available at the time of this study. Bias-eval-rPPG [14] releases frame-wise cropped skin-region colour values rather than the original facial videos, so it cannot support the present spatial XAI pipeline. SCAMPS [15] is directly obtainable for non-commercial research but contains synthetic rather than recorded participants. No public acquisition route for the complete OBF dataset [16] was identified. NCKU-rPPG is available to the present study under its ethics approvals and institutional controls but is not a public dataset. These access conditions, together with the need for complete facial frames and synchronised waveforms, made UBFC-rPPG the practical public reference and NCKU-rPPG the controlled multi-condition extension. Recent explainable artificial intelligence (XAI) studies in rPPG have mainly used visual attribution to indicate where a model may obtain its physiological evidence. Dual-path TokenLearner, PhysKANNet, D-rPPGNet, and TS-CAN+ respectively report attention maps, gradient saliency, or Gradient-weighted Class Activation Mapping (Grad-CAM) visualisations, while CIN-rPPG examines channel–spatial interaction maps internal to its architecture [17, 18, 19, 20, 21]. These visualisations can reveal plausible facial, non-facial, and background responses, but visual plausibility alone does not establish that the highlighted regions affect an rPPG prediction. In particular, the available studies do not quantitatively test whether the ordering imposed by an attribution map agrees with the effect of perturbing its spatial regions. RhythmFormer was developed to model periodic spatiotemporal patterns with hierarchical Temporal Periodic Transformer stages and Periodic Sparse Attention [9]. Chen and Nordling [22] addressed this limitation for RhythmFormer on UBFC-rPPG by adapting the Salience-guided Faithfulness Coefficient (SaCo) to rPPG regression and pairing perturbation faithfulness with skin-coverage analysis. That evaluation showed that attention aggregation across layers can restore non-zero multi-hop paths that sparse top-k routing had excluded, which weakens the spatial specificity of attention rollout. Beyond Intuition obtained the strongest median skin alignment and SaCo result in that controlled evaluation, but the observation was restricted to one dataset, one model family, and a limited recording setting. It therefore motivates replication rather than a general ranking of XAI methods. The present study extends that experimental framework from UBFC-rPPG to eight scenario-specific models trained and evaluated on NCKU-rPPG, covering controlled variation in illumination, speaking, head rotation, and cycling intensity. It examines whether the previous observations transfer to independently trained RhythmFormer models across these recording scenarios. The analysis also distinguishes SaCo’s within-clip perturbation-faithfulness interpretation from its possible association with reliability across independently trained models, which is not implied by its definition. Finally, comparison across scenarios provides descriptive evidence on whether the multi-hop leakage observed on UBFC-rPPG also appears under differing illumination, speech, head rotation, and cycling conditions. Published dataset results do not remove the need to document adaptation because input duration, frame rate, face preprocessing, participant grouping, and evaluation aggregation can all change the experiment. 3 Methods 3.1 Dataset and experimental conditions We analysed the Nordling Lab at National Cheng Kung University (NCKU) remote photoplethysmography (rPPG) multi-condition dataset, hereafter NCKU-rPPG. We followed the experimental protocol established in the Nordling Lab and presented by Wang [23]. The source cohort submitted to conversion contained 618 Camera 1 recording sessions from 78 participants. The recordings paired facial video with contact photoplethysmography (PPG) and electrocardiography (ECG) under controlled illumination and motion conditions. Because each participant contributed at most one session per scenario, the numbers of participants with a usable session and usable sessions were identical: 76 for Static level 1, 76 for Static level 3, 75 for Static level 5, 76 for Speak, 76 for Rotate, 77 for Bike level 1, 76 for Bike level 3, and 75 for Bike level 5. 3.1.1 Ethical statement The National Cheng Kung University Human Research Ethics Committee approved the relevant experimental procedures under case numbers 106-262, 108-244, and 112-406. The corresponding approval numbers were NCKU HREC-F-107-008-2, NCKU HREC-E-108-244-2, and NCKU HREC-E-112-406-2, respectively. Data collection was conducted in accordance with the approved procedures. 3.1.2 Acquisition system The acquisition system comprised the following devices. Camera 1 recorded the frontal facial view with a Panasonic GX85 and an Olympus M.ZUIKO DIGITAL ED 14–150-m lens at 1920×10801920× 1080 pixels and 50 frames per second as compressed H.264 High Profile video with yuv420p pixel format, and an average bit rate of approximately 26.98 Mb/s (range 25.89–27.08 Mb/s). A Logitech C170 webcam viewed the fluorescent ceiling light and provided a synchronisation cue from the abrupt brightness change at the beginning of each test. A Philips IntelliVue MX400 recorded three-lead ECG at 500 Hz and finger PPG from the SpO2 channel at 125 Hz. The alignment files paired the selected Camera 1 frames with the 125 Hz PPG measurements aligned and supplied at the source video’s 50 Hz frame cadence used in this analysis. Two FOTGA LED504 lights and the fluorescent ceiling light established the illumination conditions, which were checked with a TES 1330A lux meter beside the participant’s face. Table 1: Acquisition devices used for the analysed recordings. Component Device Analytic configuration Camera 1 Panasonic GX85 with Olympus 14–150-m lens Frontal portrait recording; 1920×10801920× 1080 pixels; H.264 High Profile, yuv420p; approximately 26.98 Mb/s (25.89–27.08 Mb/s); 50 frames per second; fixed 5500-K white balance. Synchronisation camera Logitech C170 Viewed the fluorescent light to identify the alignment cue. Physiological monitor Philips IntelliVue MX400 Three-lead ECG at 500 Hz and finger PPG at 125 Hz; the aligned PPG sequence was supplied at 50 Hz. Exercise apparatus BH Fitness H917 flywheel Used for the three six-minute cycling conditions. Illumination Two FOTGA LED504 lights and fluorescent ceiling light Frontal illuminance was measured with a TES 1330A lux meter. Instruction display Television Displayed the text and head-rotation instructions used in the Speak and Rotate scenarios. 3.1.3 Experiment design The protocol comprised five two-minute seated recordings and three six-minute cycling recordings. Static recordings were acquired at illumination levels 1, 3, and 5. Speaking and head rotation were recorded at level 3. The three cycling recordings repeated illumination levels 1, 3, and 5 while participants changed cadence and resistance under researcher supervision. Table 2 summarises the eight analysed scenarios. Table 2: Analysed experimental scenarios. Scenario Illumination level Duration Procedure Static level 1 1 2 min Sit still on the flywheel and look towards Camera 1. Static level 3 3 2 min Sit still on the flywheel and look towards Camera 1. Static level 5 5 2 min Sit still on the flywheel and look towards Camera 1. Speak 3 2 min Read the displayed text aloud while facing the screen. Rotate 3 2 min Follow displayed instructions to rotate the head through multiple poses. Bike level 1 1 6 min Cycle while following instructions to change cadence and resistance. Bike level 3 3 6 min Cycle while following instructions to change cadence and resistance. Bike level 5 5 6 min Cycle while following instructions to change cadence and resistance. Illumination levels 1, 3, and 5 corresponded to frontal illuminance targets of 40±5040± 50, 200±50200± 50, and 700±50700± 50 lux, respectively. The static conditions isolated the planned illumination settings while limiting voluntary movement. Speaking introduced local facial motion, rotation introduced large pose changes, and cycling combined body motion with a changing heart rate. 3.2 Face extraction and resizing Figure 1: Face extraction and resizing for NCKU-rPPG conversion, illustrated with output frame 3176 from participant 798 in the static level 3 condition and the corresponding reproduced preprocessing evidence. We converted the source recordings into cropped facial videos paired with aligned PPG sequences for RhythmFormer training and evaluation. The face-extraction pipeline first generated face candidates using the Multi-task Cascaded Convolutional Network (MTCNN) implementation provided by facenet-pytorch version 2.6.0 [24]. It then estimated 68 facial landmarks using FaceXFormer [25]. For reproducibility, FaceXFormer was pinned to commit 10fe8291f8a64e2ca1daf938e3e0007bd860303b of https://github.com/Kartik-3004/facexformer and to the released checkpoint ckpts/model.pt from https://huggingface.co/kartiknarayan/facexformer. The checkpoint had the SHA-256 digest 327a755849ba64d336fb96589f87b27e84a12be1ecf8bcfaa503d66f803286d. Before processing, the converter verified the source commit and checkpoint digest and terminated if either did not match the specified version. The conversion proceeded as follows. The conversion proceeded as follows. 1. We read each Camera 1 video and its unique alignment CSV file, which supplied the aligned start frame and PPG values. 2. We retained the shorter of the decodable aligned-video sequence and the contiguous finite PPG sequence beginning at the aligned start frame. The processing and verification stages required all expected source frames to be decodable. We retained the 50 Hz video cadence and did not resample the recordings to the 30 Hz rate used by UBFC-rPPG. 3. We applied the participant-specific clockwise or counter-clockwise 90-degree correction to obtain a portrait frame. 4. We ran a Multi-task Cascaded Convolutional Network (MTCNN) face detector [24] on every frame, selected the finite candidate with the highest confidence, and expanded that candidate before landmark inference. 5. FaceXFormer [25] estimated 68 facial landmarks in source-frame coordinates. We converted the landmark extrema to a square rPPG crop with side length s=1.6max(xmax−xmin,ymax−ymin),s=1.6 (x_ -x_ ,y_ -y_ ), (1) centred horizontally on the landmarks and shifted upward by eight percent of the square crop side to include more forehead. Both the side and the shift were recomputed from each frame’s own landmark span, so the displacement varied frame by frame and had no single value in pixels. 6. We linearly interpolated missing crop centres and side lengths. Nearest valid values filled leading and trailing gaps. Each session required a raw face-detection rate of at least 0.80 before interpolation. 7. We smoothed each interpolated centre coordinate and side length with a five-frame median filter followed by an 11-frame, second-order Savitzky–Golay filter. If a recording was shorter than a requested filter window, the implementation used the largest odd integer no greater than both the requested window and the number of frames; it skipped the median stage below three frames and the Savitzky–Golay stage unless the window exceeded the polynomial order. 8. We shifted each square crop inside the frame without changing its side length, extracted it from the rotated source frame, and resized it from approximately 510×640510× 640 pixels to 128×128128× 128 pixels. 9. We wrote a lossless FFV1 vid.avi and a corresponding ground_truth.txt for each session. We required a minimum encode–decode temporal RGB correlation of 0.99 between the source frames and the decoded output, and the converter deleted the output and stopped for any session below it. Every one of the 607 accepted sessions therefore met the threshold. 3.3 Participant split and RhythmFormer preprocessing Of the 78 participants in the source cohort, one had all eight sessions skipped because the required alignment CSV files were absent. The converted dataset therefore contained 607 sessions from 77 participants, which we split before selecting a scenario. The fixed manifest assigned 65 participants and 511 converted sessions to training and 12 participants and 96 converted sessions to testing. Scenario filtering was applied only after this grouping, preventing different sessions from the same participant from entering both partitions. Missing sessions reduced the scenario-specific training set to 63–65 participants, whereas every test scenario contained all 12 test participants. No original participant or converted session appeared in both the training and test partitions in any of the eight scenarios. RhythmFormer loader’s own face cropping was disabled throughout.It read the already cropped 128×128128× 128 FFV1 videos directly and converted OpenCV’s decoded blue–green–red channel order to red–green–blue and did not perform another face detection, dynamic tracking, expanded crop, or change of spatial resolution in the formal experiments. For each complete session, the loader standardised the video with one scalar mean and one scalar standard deviation computed jointly over the time, height, width, and RGB-channel dimensions of the T×H×W×CT× H× W× C tensor. It independently standardised the PPG vector using its mean and standard deviation over time. Standardisation preceded temporal segmentation. We divided each session into non-overlapping 256-frame clips. At 50 Hz, each clip spanned 5.12 s, which approximated but did not exactly match the 5.33 s duration of the 160-frame, 30 Hz clips used to train RhythmFormer on UBFC-rPPG. We discarded the terminal remainder when fewer than 256 frames remained and did not resample time. Video clips and labels were stored as 32-bit floating-point arrays. The loader converted the video to time–channel–height–width order before batching. We excluded 285 training clips and 80 test clips whose PPG-labels contained a non-finite value or had a 32-bit floating-point standard deviation no greater than 10−610^-6, and trained on the remainder. Supplementary Section S4. Low-variance label checks distinguishes this gate from the training-time augmentation fallback. 3.4 RhythmFormer architecture RhythmFormer maps an RGB facial video to a one-dimensional rPPG waveform [9]. The Fusion Stem combines raw frames with temporal frame differences, and a three-dimensional convolution embeds non-overlapping spatial patches. The features pass through three hierarchical Temporal Periodic Transformer (TPT) stages. Each stage contains two Periodic Sparse Attention (PSA) blocks, and stage-specific temporal downsampling and upsampling expose periodic structure at multiple temporal scales. After the six PSA blocks, spatial averaging and a one-dimensional predictor produce one waveform value per input frame. Figure 2: RhythmFormer architecture and Periodic Sparse Attention (PSA). The Fusion Stem and patch embedding precede three Temporal Periodic Transformer (TPT) stages, each containing two PSA blocks. PSA uses coarse scores to route selected keys and values before refined attention, with a parallel local-context branch. For these 256-frame, 50 Hz experiments, the trainer instantiated the three stages with top-k=40k=40 routing. The model applied no stochastic depth: its drop path rate was 0. 3.5 Scenario-specific training We trained an independent RhythmFormer model for each of the eight scenarios in Table 2. Each model used only the training clips from its scenario and was tested only on the corresponding held-out scenario clips. Table 3 gives the shared formal configuration. Table 3: Shared training and inference configuration for the eight scenario-specific models. Setting Value Random seed and precision 100; FP32 Hardware Four NVIDIA RTX A5000 graphics processing units Software PyTorch 2.0.1 with CUDA 11.8 Epochs and checkpoint 30; final checkpoint from epoch 30 Training and test batch sizes 4 and 2 Learning rate and scheduler 0.009; OneCycleLR Optimiser AdamW; weight decay 0; effective ϵ=10−8ε=10^-8 Recorded model drop rate 0.2 Data-loader workers 16 For predicted waveform y y and reference waveform y, the training objective was ℒ=0.2ℒNP(y^,y)+ℒCE(y^,y)+ℒHR−KL(y^,y),L=0.2L_NP( y,y)+L_CE( y,y)+L_HR-KL( y,y), (2) where ℒNPL_NP is the negative-Pearson time-domain loss, ℒCEL_CE is cross-entropy over the frequency-domain heart-rate target, and ℒHR−KLL_HR-KL is the Kullback–Leibler loss between predicted and reference heart-rate distributions. These three components follow the published RhythmFormer training objective [9]. Heart-rate-aware temporal augmentation was sampled during training. Each clip was selected for augmentation independently with probability 0.5. The routine estimated each clip’s reference heart rate from the clip’s own label, by the detrending, 0.75–2.5-Hz bandpass, and Welch procedure described in Section 3.6 applied at the 50-Hz training sampling rate. A clip counted as low-heart-rate below 75 beats per minute and as high-heart-rate above 90 beats per minute. For a selected low-heart-rate clip, the routine retained every second frame and repeated the accelerated sequence; for a selected high-heart-rate clip, it expanded a randomly selected half-length interval by alternating source samples with their interpolated neighbours. Other selected clips, those from 75 to 90 beats per minute inclusive, remained temporally unchanged. A batch-level horizontal flip was sampled independently. Some NCKU-rPPG labels contained a constant terminal suffix. Temporal augmentation could select only that suffix and thereby create a constant target with an undefined Pearson denominator even when the original full clip was valid. After all temporal-augmentation random draws, the trainer therefore checked each augmented label. If any value was non-finite or the standard deviation was no greater than 10−810^-8, it restored the original unaugmented video–label pair. The check did not redraw the augmentation or horizontal-flip decision and therefore did not alter the random sampling sequence. The optimiser was AdamW with ϵ=10−8ε=10^-8 and zero weight decay. The formal runs did not construct a validation loader or select a model by validation performance. Testing instead used the model from the final training epoch. 3.6 Performance evaluation Analysis unit. Two conventions for heart-rate metrics dominate the 2022–2026 remote photoplethysmography literature. The rPPG-Toolbox convention estimates one heart rate per test sample, computes mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), Pearson correlation, and signal-to-noise ratio (SNR) across those samples, and reports every estimate with a standard error [6]. The RhythmNet and PhysFormer convention divides a 30-second recording into three non-overlapping 10-second clips, estimates a heart rate in each, averages the three into one recording-level value, and reports standard deviation, MAE, RMSE, and Pearson correlation across recordings [26, 27]. We follow the rPPG-Toolbox definitions for all five metrics. It is the code this experiment ran, it is the convention behind the published RhythmFormer results we compare with [9], and of the two it is the only one that defines every metric we report, the other reporting neither MAPE nor SNR. The toolbox takes a whole test video as one sample, which in the datasets it was built around lasts about a minute. Our held-out recordings are far longer: 123 s per participant in the static, speaking, and rotation conditions, and 364 s under cycling. A single spectral peak over 364 s is a modal value rather than a heart rate, and the cycling protocol exists precisely to move the heart rate within that interval. The reference finger photoplethysmogram’s median within-participant interquartile range is 4.1–5.1 beats per minute in the five stationary conditions and 13.4–17.5 under cycling, and the change between consecutive clips exceeds 10 beats per minute in 2.5–9.8 % of stationary pairs against 18.7–23.8 % of cycling pairs. We therefore estimate one heart rate per 256-frame clip, which is 5.12 s, as the second convention does on its 10-second clips, but we do not average the clips into a recording-level value. This also makes the comparison with the UBFC-rPPG reproduction possible, because it uses a 160-frame clip, approximately 5.33 s at 30 Hz [22]. The shorter window reduces the native frequency resolution, so we quantified its practical effect. For a 256-sample window at 50 Hz, adjacent discrete Fourier transform bins are separated by 11.7 beats per minute. Zero-padding and peak interpolation permit estimates between these sampled frequencies, although they do not increase the intrinsic spectral resolution. We estimated heart rate from each reference photoplethysmogram clip using two methods: the dominant spectral peak and the median interval between detected systolic peaks. Across the five stationary conditions, the median absolute difference between the two estimates ranged from 0.65 to 1.09 beats per minute. Across the three cycling conditions, the median absolute difference ranged from 1.81 to 2.47 beats per minute, whereas the mean absolute difference ranged from 5.49 to 8.54 beats per minute. The substantially larger means indicate that a minority of cycling clips produced large discrepancies between the two estimators. In addition, 30 of the 2,559 cycling clips produced spectral heart-rate estimates at the upper boundary of the 150-beats-per-minute search band and were therefore censored; no stationary clip reached this boundary. Together, these results show that short-clip reference heart-rate estimation was generally consistent under stationary conditions but less stable during cycling, particularly for outlying or high-heart-rate clips. Heart-rate estimation. Each clip’s predicted and reference waveforms were detrended with smoothness parameter 100 and filtered with a first-order, zero-phase Butterworth bandpass from 0.75 to 2.5 Hz. We estimated the dominant heart rate over the corresponding 45–150 beats-per-minute range with Welch spectra using a discrete Fourier transform length of nfft=200,000nfft=200,000. Definitions. Let P=12P=12 be the held-out participants of a condition, pC_p the clips of participant p, Np=|p|N_p= _p , and N=∑p=1PNpN= _p=1^PN_p. Write h^i h_i and hih_i for the predicted and reference heart rate of clip i in beats per minute, and s^i[t] s_i[t] and si[t]s_i[t] for its predicted and reference waveform samples, t=1,…,Tt=1,…,T with T=256T=256. Every estimate below is a participant-weighted mean: clip values are averaged within a participant first, and the participant means are then averaged. For a clip-level quantity xix_i, x¯p=1Np∑i∈pxi,⟨x⟩=1P∑p=1Px¯p,SE(⟨x⟩)=1P1P−1∑p=1P(x¯p−⟨x⟩)2. x_p= 1N_p _i _px_i, x = 1P _p=1^P x_p, ( x )= 1 P 1P-1 _p=1^P ( x_p- x )^2. (3) The three error measures are then MAE =⟨|h^i−hi|⟩, = \, h_i-h_i \, , (4) RMSE =⟨(h^i−hi)2⟩, = \, ( h_i-h_i )^2\, , (5) MAPE =⟨|h^i−hi|hi⟩×100%, = \, h_i-h_i h_i\, × 100\,\%, (6) with the standard error of the RMSE obtained from that of the mean squared error inside Equation (5) by the delta method, that is by dividing it by 2RMSE2\,RMSE. We report two Pearson correlations, because the manuscript uses the word for two different quantities and they must not be confused. The heart-rate correlation compares the N estimated heart rates of a condition with their references, rHR=∑i=1N(h^i−h^¯)(hi−h¯)∑i=1N(h^i−h^¯)2∑i=1N(hi−h¯)2,SE(rHR)=1−rHR2P−2,r_HR= _i=1^N ( h_i- h ) (h_i- h ) _i=1^N ( h_i- h )^2\; _i=1^N (h_i- h )^2, (r_HR )= 1-r_HR^2P-2, (7) where h^¯ h and h¯ h are the means over the N clips. The waveform correlation is taken within a clip, between the predicted and the reference photoplethysmogram, and then averaged, ri=∑t=1T(s^i[t]−s^i¯)(si[t]−si¯)∑t=1T(s^i[t]−s^i¯)2∑t=1T(si[t]−si¯)2,rwave=⟨ri⟩.r_i= _t=1^T ( s_i[t]- s_i ) (s_i[t]- s_i ) _t=1^T ( s_i[t]- s_i )^2\; _t=1^T (s_i[t]- s_i )^2, r_wave= r_i . (8) It is rwaver_wave, not rHRr_HR, that every clip-level relationship in Sections 4.2 and 4.3 calls the waveform Pearson correlation. Equation (8) is evaluated at a fitted lag rather than at zero. Each recording is synchronised on its own light-off trigger, so a video-to-photoplethysmogram synchronisation error is one constant per participant and recording. We fit that constant by maximising the participant’s mean clip correlation over ±1± 1 second and apply it to every clip of that recording, so that rwaver_wave measures waveform agreement rather than synchronisation error. Supplementary Table 20 reports the correlation before and after the fit for every participant, the fitted lag, and a null in which the same fit is applied to mismatched participant pairs. The fit is worth two to seven times its null on NCKU-rPPG and less than its null on UBFC-rPPG, whose fitted lags are zero for eight of twelve subjects and within two samples for eleven of them. No other quantity in Table 4 changes under it, because a time shift leaves a spectrum’s magnitude, and therefore every heart-rate and signal-to-noise measure, unaltered. Chen and Nordling [22] evaluates the same correlation at zero lag, which is the convention that study states, so its archived per-clip value and ours are not the same quantity. The difference is small in aggregate and concentrated in two subjects: the fit raises the participant-averaged UBFC-rPPG correlation from 0.7850.785 to 0.8280.828, of which those two supply nine tenths, gaining 0.220.22 and 0.250.25 against 0.040.04 or less for the other ten. Recomputed at the fitted lag, that study’s clip-pooled rank correlations between skin coverage and waveform quality move from +0.31+0.31 to +0.29+0.29 for attention rollout and from +0.24+0.24 to +0.19+0.19 for Beyond Intuition, without changing which of its four attribution methods resolve. The signal-to-noise ratio follows De Haan and Van Leest [28] in the form the toolbox implements, SNRi=10log10∑f[Ui(f)Si(f)]2∑f[(1−Ui(f))Si(f)]2,SNR=⟨SNRi⟩,SNR_i=10 _10 _f [U_i(f)\,S_i(f) ]^2 _f [ (1-U_i(f) )S_i(f) ]^2, = _i , (9) where Si(f)S_i(f) is the power spectrum of the predicted waveform of clip i and Ui(f)U_i(f) is a binary template equal to one within ±6± 6 beats per minute of the reference fundamental and of its second harmonic and zero elsewhere. The two sums do not run over the same support. The denominator is confined to 0.75–2.5 Hz, that is 45–150 beats per minute, whereas the numerator is the template’s own neighbourhoods wherever they fall. Above a reference of 75 beats per minute the second harmonic lies beyond 2.5 Hz, outside that range and inside the stop band of the bandpass already applied, so for those clips the second-harmonic term contributes attenuated power rather than signal. This is the common case rather than the exception: 75.2 % of the 4001 NCKU-rPPG clips have a reference above 75 beats per minute, from 59.9 % under rotation to 84.4 % on Bike level 3. The measure is therefore closer to a first-harmonic one than Equation (9) suggests, in the same conservative direction as the resolution argument below. Si(f)S_i(f) is not the spectrum in which the heart rate was located, and the difference matters enough to state. Equation (7) reads the heart rate from a Welch spectrum whose segment is capped at 256 samples and which is zero-padded to nfft=200,000nfft=200,000; Equation (9) takes its own periodogram of the clip, zero-padded to the next power of two above the clip length and averaged over nothing. The split is not an oversight. Locating a peak rewards segment averaging, which lowers the variance of the location, whereas partitioning power against a fixed ±6± 6 beats-per-minute template rewards resolution, because a spectral lobe wider than the template spills the signal’s own power into the noise band. At a 256-frame clip the periodogram resolves 11.7 beats per minute, which is wider than the template, so the reported SNR is a conservative measure of waveform quality at this clip length rather than a neutral one. Equation (9) squares a quantity it calls a power spectrum, so it returns twice the value that a ratio of powers in decibels would give, which is the amplitude convention rather than the 10log1010 _10 conventionally used for a ratio of powers. The underlying ratio is therefore recovered as 10SNR/2010^SNR/20 and not as 10SNR/1010^SNR/10. We retained this convention because it is the one implemented in the rPPG-Toolbox code the experiment used and the one behind the published SNR values we compare with [6, 9], so that changing it would make our values incomparable with theirs. Uncertainty. Our standard errors depart from the toolbox in one respect. It computes σ/nσ/ n over n test samples, and (1−r2)/(n−2) (1-r^2)/(n-2) for the Pearson correlation, both of which treat every test sample as independent [6]. With one estimate per clip the samples are not independent: each participant contributes 12 to 71 clips, and heart rate, skin tone, and facial geometry are properties of a person. A standard error taken over clips would understate the uncertainty by roughly the square root of the number of clips per participant. Equation (3) therefore averages within participant first and takes the standard error over the resulting participant means, and Equation (7) substitutes P for n in the toolbox formula, so that the effective sample size is the number of participants in every column of Table 4. The same treatment applies to the skin-coverage summaries, which are clip-level for the same reason. A pre-attention-to-refined-attention difference is taken between two means of the same clips, so we formed the difference clip by clip before averaging, which gives it a smaller standard error than either mean it is built from. This uncertainty describes between-participant variation for one trained seed; it is neither numerical calculation error nor retraining variability. We report every estimate by the convention of letting the uncertainty set the precision: round the standard error to two significant figures, then round the estimate to that same decimal place. We applied the same procedure to the UBFC-rPPG reproduction of Chen and Nordling [22], so that row of Table 4 is comparable with the scenario rows. Its archived clip evidence records one absolute heart-rate error and one waveform correlation per clip and neither a reference heart rate nor a signal-to-noise ratio, so we did not read that row off the evidence. We cut the archived per-subject prediction and reference waveforms into the same 141 clips of 160 frames at 30 Hz and passed each through the identical estimation used for the NCKU-rPPG scenarios, which yields every column from one code path. Recomputed and archived per-clip errors agree to within 0.03 beats per minute, two orders of magnitude below the precision reported, and the recomputation is checked against the archive whenever the row is generated. The two datasets are matched in clip duration but not in every transform parameter, and the residue falls on one column only. A UBFC-rPPG clip is 160 frames at 30 Hz and an NCKU-rPPG clip 256 frames at 50 Hz, that is 5.33 against 5.12 s, so the resolving power of the two spectra differs by 4 %, at 11.25 against 11.72 beats per minute. Two parameters are nonetheless set by the clip rather than chosen, and differ. The Welch segment is min(N−1,256) (N-1,256), which the UBFC-rPPG clip does not reach, so it is 159 there against 256 here. The SNR periodogram is padded to the next power of two above the clip, which pads UBFC-rPPG by a factor 1.6 and NCKU-rPPG not at all. Neither can be equalised by a choice of window: matching the Welch segment would need 265, above the cap, and matching the padded length would need 427, which is not a power of two. Measured, the padding difference is worth about 1 decibel and runs against UBFC-rPPG, so the SNR advantage reported for it in Table 4 is if anything understated; the heart-rate columns are unaffected, because they depend on the peak’s location rather than on how the power around it is partitioned. 3.7 Explainable artificial intelligence attribution methods We extracted attribution from all six Periodic Sparse Attention (PSA) blocks in the three Temporal Periodic Transformer (TPT) levels, with two blocks per level. Pre-attention produced a 4×44× 4 spatial-region map, whereas refined attention produced an 8×88× 8 spatial-token map. For raw attention, attention rollout, and attention flow, we completely collapsed the temporal query and key axes within every block before any aggregation across blocks or TPT levels. The resulting spatial matrices retained 4×44× 4 pre-attention or 8×88× 8 refined spatial coordinates, but no temporal coordinate. These three methods therefore did not use a T=20T=20 temporal rollout. Beyond Intuition retained the temporal coordinate during cross-layer aggregation, mapped all six refined-attention blocks to a common T=20T=20 representation, and pooled time only after producing the integrated attribution. Its reasoning-feedback term used 20 integrated-gradient steps from a standardised-space zero baseline to the standardised input clip. Zero in this space corresponded to the per-clip mean-intensity state. Raw attention. Raw attention provides a layer-local explanation by measuring how strongly each source position is used as a key, without propagating attribution across layers. Because RhythmFormer has no classification token, we used the received attention obtained by summing the head-averaged attention matrix A(l)A^(l) over query positions: rk(l)=∑qAqk(l).r_k^(l)= _qA_qk^(l). (10) Here, k indexes the source position and l indexes a PSA block; the temporal query and key axes were collapsed before the resulting spatial map was aggregated across blocks. Attention rollout. Attention rollout [29] estimates cross-layer information propagation by augmenting each attention matrix with the residual connection and multiplying the matrices in forward order. With residual weight 1/21/2, the augmented matrix and input-side salience were defined as A^(l)=0.5I+0.5A(l),A~=A^(L)⋯A^(1),ψk=∑qA~qk. A^(l)=0.5I+0.5A^(l), A= A^(L)·s A^(1), _k= _q A_qk. (11) The rollout was applied to the temporally collapsed spatial matrices, so no cross-stage temporal downsampling was used. Attention flow. Attention flow [29] treats the stacked attention matrices and residual connections as a layered capacity graph. Each source position i was assigned the total maximum flow that reached every output position j across the graph: ϕi=∑jMaxFlow(vi(0),vj(L)). _i= _jMaxFlow\! (v_i^(0),v_j^(L) ). (12) The graph used the same temporally collapsed 4×44× 4 pre-attention or 8×88× 8 refined-attention spatial nodes as raw attention and rollout, with the residual connection represented as diagonal capacity. Beyond Intuition. Beyond Intuition [30] combines structural propagation with a gradient-based reasoning-feedback mask. Its final attribution was the Hadamard product of a value-projection-corrected rollout P(L)P^(L) and an integrated-gradient mask FcF^c: T=P(L)⊙Fc,Fc=ReLU(−1Nsteps∑s=1Nsteps∇A(L)ℒ(sNstepsX)),T=P^(L) F^c, F^c=ReLU\! (- 1N_steps _s=1^N_steps _A^(L)L\! ( sN_stepsX ) ), (13) where the negative gradient is used because the regression loss is minimised. The correction scales the rollout contribution of source token j by αj=∥zjW∥2/∥zj∥2 _j= z_jW _2/ z_j _2 before forming P(L)P^(L). Unlike the other three methods, Beyond Intuition retained the temporal coordinate, unified the refined-attention blocks at T=20T=20, and pooled time only after computing the integrated attribution. Mixed-precision attribution for the rotation scenario produced non-finite values. We therefore generated the formal rotation attributions in FP32. This attribution-only change did not alter the rotation checkpoint, model parameters, or training procedure. 3.8 Anatomical plausibility and perturbation faithfulness We quantified anatomical plausibility with skin coverage. A pretrained Bilateral Segmentation Network (BiSeNet) face parser [31] supplied a binary mask containing only the face-skin and nose classes. Each attribution map was min–max normalised before bicubic upsampling to 128×128128× 128 pixels. For normalised attribution hih_i and binary skin mask mim_i at pixel i, skin coverage was Cskin=∑ihimi∑ihi.C_skin= _ih_im_i _ih_i. (14) Skin coverage measured anatomical plausibility and was not treated as a faithfulness measure. We measured perturbation faithfulness with the Salience-guided Faithfulness Coefficient (SaCo) [32]. For each clip, we sorted the spatial positions by salience and divided them into K=8K=8 equally sized groups G1,…,G8G_1,…,G_8. Group salience was the summed salience, s(Gi)=∑p∈Gihps(G_i)= _p∈ G_ih_p. Each group was masked separately in every video frame by replacing its pixels with zero in standardised space, equivalent to the per-clip per-channel mean intensity before standardisation. The perturbation impact was the waveform mean absolute error d(Gi)=MAE(Φ(x),Φ(xGi)),d(G_i)=MAE\! ( (x), (x_G_i) ), (15) where Φ(x) (x) and Φ(xGi) (x_G_i) were the waveforms predicted from the original and independently masked clips. For every pair i<ji<j, wij=s(Gi)−s(Gj)w_ij=s(G_i)-s(G_j) when d(Gi)≥d(Gj)d(G_i)≥ d(G_j) and wij=−[s(Gi)−s(Gj)]w_ij=-[s(G_i)-s(G_j)] otherwise, and F=∑i<jwij∑i<j|wij|∈[−1,1].F= _i<jw_ij _i<j|w_ij|∈[-1,1]. (16) Beyond Intuition supplied its refined 8×88× 8 attribution map to SaCo, whereas raw attention, attention rollout, and attention flow supplied 4×44× 4 source maps. The attribution visualisations in Figure 3 and Supplementary Section S6. Explainable artificial intelligence results averaged all test clips within their scenario. The violin and scatter plots used clip-level observations (Figures 5 and 9). For the skin-coverage violins, each clip was first averaged across TPT1–TPT3. For the scatter plots, the attention-only methods used refined TPT3 skin coverage and Beyond Intuition used its single coverage value. The combined split violins in Figure 5 placed the 141 UBFC-rPPG clips on each left half and the eight NCKU-rPPG scenarios on each right half. We normalised each scenario’s kernel density estimate independently rather than pooling the eight NCKU-rPPG scenarios into one density. We plotted every available clip with deterministic half-constrained jitter and omitted the central boxplots because one box could not represent the nine data groups. Position and explicit sublabels distinguished pre-attention from refined attention; colour represented only the dataset or scenario. The combined scatter plots in Figure 9 used circles for NCKU-rPPG clips and crosses for UBFC-rPPG clips and retained the complete linear data ranges. Within each attribution-method panel, we fitted one ordinary least-squares line to all displayed clips and reported the corresponding all-clips Pearson correlation, nominal two-sided p value, and sample size. The cross-method summaries pooled all test clips and all three TPT levels before calculating each mean. 3.9 Cross-dataset transfer and scenario-level reliability comparisons We defined two complementary levels of comparison. At the cross-dataset level, we compared Static level 3 with the published UBFC-rPPG evaluation of Chen and Nordling [22] because their heart-rate accuracy was similar, while retaining their different participants, cameras, preprocessing, frame rates, and checkpoints. We compared the median skin coverage and SaCo rankings, pre-attention-to-refined-attention changes, and clip-level correlation structures without treating the two evaluations as a controlled cross-dataset experiment. We related three clip-level measures of model performance, the heart-rate mean absolute error, the waveform correlation of Equation (8), and the signal-to-noise ratio of Equation (9), to two clip-level attribution summaries, skin coverage and SaCo. That yielded six relationships, and a seventh between the two attribution summaries themselves, SaCo against skin coverage. For each attribution method we calculated all seven separately within each of the eight NCKU-rPPG scenarios and within UBFC-rPPG, which yielded 252 coefficients in total. The signal-to-noise ratio was recorded per clip in the NCKU-rPPG evidence and was recomputed from the archived waveforms for UBFC-rPPG, whose clip evidence did not carry it, by the same estimation used for every other clip. Each correlation was taken inside a participant and averaged over participants, not pooled over clips. Pooling every clip of every participant would have measured mostly the differences between participants, because heart rate, skin tone and facial geometry are properties of a person rather than of a clip, and a coefficient built from them would not have shown whether a clip with more attributed skin carried a better waveform. We therefore computed the Spearman coefficient within each participant, averaged the coefficients through the Fisher transform, and tested the mean against zero with a one-sample t test over participants. Participants were independent, so this p value was inferential rather than descriptive, which no test pooled over nested clips could have been. The difference was not cosmetic: when clips were pooled, the waveform Pearson–skin coverage relationship of Static level 3 was −0.41-0.41 with p<10−12p<10^-12 for all four methods alike, whereas the same data taken within participants yielded coefficients between −0.16-0.16 and +0.13+0.13, none of them resolved. A participant contributed a coefficient only when it had at least four clips in the relationship, and a relationship was reported only when at least three participants contributed. The same estimator annotated every panel of Supplementary Figure 9, so every correlation reported as a finding was taken within participants; a panel spanning all eight conditions and UBFC summarised the display rather than reproducing one table row and, when restricted to a single condition, returned that row exactly. The two clip-pooled coefficients quoted anywhere in the manuscript were the −0.41-0.41 above and the recomputation of the clip-pooled values of Chen and Nordling [22] in Section 3.6; both were labelled as pooled where they appeared, and neither was reported as a finding. The correlation calculations used the same aligned arrays as the corresponding scatter panels and never pooled clips across the nine data groups. We reported nominal p values descriptively because clips were nested within participants. At the scenario level, each of the eight NCKU-rPPG scenarios was one observation. For each attribution method, we calculated the median SaCo across its evaluated clips within each scenario and related those eight medians to scenario-level heart-rate mean absolute error (MAE), waveform Pearson correlation, and signal-to-noise ratio (SNR) using Spearman’s rank correlation coefficient. If SaCo tracked model reliability, its expected directions were negative with MAE and positive with waveform Pearson correlation and SNR. Because n=8n=8 scenarios were independently trained conditions rather than repeated samples from a common intervention, we report the coefficients descriptively without significance tests or causal interpretation. 4 Results 4.1 Performance across illumination and motion Table 4 reports clip-level performance for the eight final-epoch checkpoints. Every NCKU-rPPG estimate and standard error was calculated from the clips of the 12 held-out participants in the corresponding scenario, averaged within participant before averaging across participants. The UBFC-rPPG row comes from the study of Chen and Nordling [22], which trained and tested RhythmFormer on UBFC-rPPG alone. We did not repeat that experiment. We calculated its estimates and standard errors from the 141 clips of its archived waveforms by the same procedure. Table 4: RhythmFormer performance across the eight NCKU-rPPG conditions, and on UBFC-rPPG from the archived waveforms of Chen and Nordling [22]. One heart rate is estimated per clip, 5.12 s for the NCKU-rPPG rows and approximately 5.33 s for UBFC-rPPG, and each row reports the participant-weighted estimate ± its standard error across the 12 held-out participants, as defined in Section 3.6. rHRr_HR is the correlation between the estimated and reference heart rates of the condition’s clips, Equation (7), and rwaver_wave the mean within-clip correlation between the predicted and reference waveforms, Equation (8). The UBFC-rPPG row is recomputed from that study’s archived waveforms rather than read from its published summary, so that its analysis unit matches the scenario rows. Scenario MAE (bpm) RMSE (bpm) MAPE (%) rHRr_HR rwaver_wave SNR (dB) UBFC-rPPG [22] 0.96±0.160.96± 0.16 1.37±0.281.37± 0.28 0.99±0.180.99± 0.18 0.991±0.0410.991± 0.041 0.828±0.0220.828± 0.022 8.54±0.978.54± 0.97 Static level 1 6.2±1.96.2± 1.9 13.0±3.013.0± 3.0 7.3±2.27.3± 2.2 0.57±0.260.57± 0.26 0.516±0.0760.516± 0.076 −1.7±2.5-1.7± 2.5 Static level 3 1.96±0.341.96± 0.34 3.74±0.753.74± 0.75 2.36±0.422.36± 0.42 0.968±0.0790.968± 0.079 0.626±0.0370.626± 0.037 5.7±1.55.7± 1.5 Static level 5 3.9±1.13.9± 1.1 9.0±2.19.0± 2.1 4.5±1.34.5± 1.3 0.80±0.190.80± 0.19 0.588±0.0550.588± 0.055 3.6±2.33.6± 2.3 Speak 8.9±2.68.9± 2.6 15.4±4.115.4± 4.1 9.4±2.19.4± 2.1 0.43±0.290.43± 0.29 0.324±0.0450.324± 0.045 −4.7±2.3-4.7± 2.3 Rotate 12.9±2.612.9± 2.6 18.3±2.818.3± 2.8 15.6±2.815.6± 2.8 0.22±0.310.22± 0.31 0.300±0.0430.300± 0.043 −11.0±2.5-11.0± 2.5 Bike level 1 18.6±5.518.6± 5.5 30.3±7.630.3± 7.6 18.8±4.018.8± 4.0 −0.15±0.31-0.15± 0.31 0.235±0.0440.235± 0.044 −10.2±3.3-10.2± 3.3 Bike level 3 16.1±2.316.1± 2.3 24.3±2.624.3± 2.6 17.9±2.317.9± 2.3 0.12±0.310.12± 0.31 0.232±0.0470.232± 0.047 −11.9±2.0-11.9± 2.0 Bike level 5 15.1±3.515.1± 3.5 23.9±3.923.9± 3.9 17.5±3.617.5± 3.6 0.25±0.310.25± 0.31 0.319±0.0480.319± 0.048 −8.9±3.2-8.9± 3.2 Among the static conditions, level 3 had the lowest error for MAE, RMSE, and MAPE, the highest heart-rate correlation, and the highest SNR. Static level 5 ranked second on those measures and static level 1 last, and level 1 was the only static condition with a negative mean SNR. Level 3 also had the highest waveform correlation, but the three are unevenly spaced: level 1 falls 7.4 dB below level 3 in SNR and 0.11 below it in waveform correlation, whereas level 5 falls only 2.1 dB and 0.04 below it. Speaking and head rotation both degraded performance relative to the static level 3 model trained and tested at the same nominal illumination level. Rotation degraded both measures further than speaking, but not evenly: its heart-rate error was about 1.4 times as large, while its SNR fell a further 6.3 dB, so the two conditions are separated more by waveform quality than by heart-rate error. The three cycling conditions were the weakest of the eight on every column, with mean absolute errors of 15 to 19 beats per minute and mean SNR from −8.9-8.9 to −11.9-11.9 dB. Bike level 5 was nominally the best of them on all six measures, but which of the other two was worst depends on the measure: bike level 1 had the largest errors and the lowest heart-rate correlation, and bike level 3 the lowest waveform correlation and the lowest SNR. Every pairwise difference among the three is within twice the standard errors of the two values it separates, so the three are not separated by these data. The measures order the eight scenarios closely: across scenario-level values, MAE ranks against the heart-rate correlation at ρ=−0.98ρ=-0.98 (p<.001p<.001), against the waveform correlation at ρ=−0.95ρ=-0.95 (p<.001p<.001), and against SNR at ρ=−0.88ρ=-0.88 (p=.004p=.004), while the waveform correlation ranks against SNR at ρ=+0.98ρ=+0.98 (p<.001p<.001). The two correlations are still not interchangeable. Static level 3 pairs a heart-rate correlation of 0.968±0.0790.968± 0.079 with a waveform correlation of 0.626±0.0370.626± 0.037, because a dominant frequency can be recovered from a waveform that matches the reference only moderately well point by point. Supplementary Figure 6 shows a representative clip from static level 3 and one from bike level 1. The heart rate is easier to predict on UBFC-rPPG than on our data: it stands ahead of Static level 3 on all six measures, at less than half the mean absolute error and 2.8 dB more SNR. The differing clip sample counts do not create that SNR gap, because Equation (9) partitions the power of a periodogram padded to the next power of two above the clip, which pads the 160-sample UBFC-rPPG clip by a factor 1.6 and leaves the 256-sample NCKU-rPPG clip unpadded, and that costs UBFC-rPPG about 1 dB (Section 3.6). 4.2 Attribution summaries for Static level 3 and UBFC-rPPG Static level 3 is the NCKU-rPPG condition closest to the UBFC-rPPG recording setting, so we examine the two together before turning to the whole set of conditions. Figure 3 shows the Static level 3 attributions, and Table 5 the pre-attention and refined-attention skin-coverage means of both evaluations. Most of the attribution falls on the mid-face, but the four methods place their maxima differently. Raw attention peaks on the neck below the chin before refinement and on the forehead and the nose after it; attention rollout and attention flow hold a broad ridge over one cheek, on opposite sides of the face, that after refinement spreads onto the hairline and the surrounding background; and Beyond Intuition contracts to a single spot on one cheek. Figure 4 shows the clip-level SaCo and skin-coverage distributions of the two evaluations, the coverage means of which Table 5 summarises. Beyond Intuition has the highest median SaCo and skin coverage in both evaluations, but the other three methods differ between them. For Static level 3, refinement changed skin coverage by less than its own standard error for every attention-only method, so no change is resolved by these data. The methods differed from each other more than refinement changed any of them: raw attention covered the least skin and attention flow the most, a spread of about 0.08 before refinement and 0.05 after. UBFC-rPPG behaved differently in three respects. Refinement moved coverage by 0.12 to 0.22 there, seven to fourteen times the corresponding standard error, against nothing resolved on Static level 3. It moved the methods in opposite directions, raw attention gaining coverage while attention rollout and attention flow lost it, whereas on Static level 3 all three changes were indistinguishable from zero. And it reordered them: raw attention has the lowest refined coverage on Static level 3 and the highest on UBFC-rPPG, while attention flow moves from highest to lowest. The same implementation of the three methods therefore give a refinement step that is inert on one evaluation and decisive on the other. The whole set of conditions locates that difference in magnitude rather than in direction, however. Across all nine conditions refinement raises the coverage of raw attention and lowers that of attention flow without exception, and lowers attention rollout in eight of the nine, so the directions on UBFC-rPPG are the directions everywhere; what is peculiar to Static levels 1 and 3 is that the changes are too small to resolve (Supplementary Table 23). Table 5: Mean skin coverage for Static level 3 and UBFC-rPPG, pooled over all held-out clips and TPT1–TPT3, as estimate ± its standard error across the 12 test participants. The difference is between the two means of the same clips and therefore carries its own, smaller standard error. Beyond Intuition has no pre-attention/refined-attention pair and is therefore not included. The UBFC-rPPG rows are calculated from the 141 clips of Chen and Nordling [22]. Method Pre Refined Difference UBFC-rPPG Raw Attention 0.342±0.0250.342± 0.025 0.566±0.0230.566± 0.023 0.224±0.0160.224± 0.016 Attention Rollout 0.644±0.0280.644± 0.028 0.527±0.0310.527± 0.031 −0.117±0.017-0.117± 0.017 Attention Flow 0.609±0.0300.609± 0.030 0.442±0.0230.442± 0.023 −0.167±0.016-0.167± 0.016 Static level 3 Raw Attention 0.398±0.0310.398± 0.031 0.416±0.0330.416± 0.033 0.018±0.0180.018± 0.018 Attention Rollout 0.465±0.0390.465± 0.039 0.465±0.0380.465± 0.038 0.0004±0.00680.0004± 0.0068 Attention Flow 0.474±0.0380.474± 0.038 0.470±0.0290.470± 0.029 −0.005±0.014-0.005± 0.014 Figure 3: Static level 3 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Static level 3 test set rather than that representative alone. (a) SaCo faithfulness coefficient F. (b) Skin coverage, before and after refinement. Figure 4: Clip-level split-violin summaries for Static level 3 and UBFC-rPPG. Each left half is UBFC-rPPG (dark grey) and each right half Static level 3 (orange), the two normalised independently rather than pooled. All available clips are plotted as points on their own half: UBFC-rPPG contributes 141 SaCo and 141 skin-coverage observations per method, and Static level 3 contributes 200 and 289. In (b) each clip is averaged across TPT1–TPT3 for the attention-only methods, and Beyond Intuition has a refined value only. Table 6: Two-sided Spearman summaries for Static level 3 and UBFC-rPPG, taken within each participant and averaged over participants. Each cell gives ρ;pρ;p, where p comes from a one-sample t test over participants and is therefore inferential rather than descriptive. Cells in bold satisfy both p<0.05p<0.05 and |ρ|≥0.10|ρ|≥ 0.10, the condition Table 9 counts as directional. The first sample-size pair in each dataset label gives the number of skin-only clips followed by the number of clips in relationships involving SaCo; the second gives the participants contributing to each, which differ because the SaCo run does not cover every held-out participant. Dataset Method MAE–skin MAE–SaCo Pearson–skin Pearson–SaCo SNR–skin SNR–SaCo SaCo–skin Static level 3 (n=289/200n=289/200) (P=12/9P=12/9) raw −0.14;.020-0.14;.020 0.01;.9530.01;.953 0.13;.0510.13;.051 0.11;.2930.11;.293 −0.06;.390-0.06;.390 0.02;.7940.02;.794 0.09;.4270.09;.427 rollout 0.04;.5080.04;.508 −0.02;.821-0.02;.821 0.02;.7500.02;.750 0.12;.2840.12;.284 0.00;.9700.00;.970 0.01;.8740.01;.874 0.03;.8530.03;.853 flow 0.09;.1110.09;.111 −0.00;.944-0.00;.944 0.05;.4220.05;.422 0.02;.8640.02;.864 −0.05;.612-0.05;.612 0.04;.7640.04;.764 −0.13;.125-0.13;.125 BI 0.06;.5130.06;.513 −0.19;.344-0.19;.344 −0.16;.086-0.16;.086 0.02;.8090.02;.809 −0.08;.218-0.08;.218 0.19;.0440.19;.044 −0.13;.264-0.13;.264 UBFC-rPPG (n=141/141n=141/141) (P=12/12P=12/12) raw −0.05;.714-0.05;.714 −0.01;.853-0.01;.853 −0.12;.372-0.12;.372 0.01;.9140.01;.914 −0.08;.471-0.08;.471 −0.03;.821-0.03;.821 −0.08;.656-0.08;.656 rollout −0.09;.405-0.09;.405 −0.05;.490-0.05;.490 0.32;.0470.32;.047 0.06;.4950.06;.495 0.15;.1130.15;.113 0.01;.9310.01;.931 0.01;.8690.01;.869 flow −0.07;.326-0.07;.326 0.02;.8480.02;.848 0.21;.0410.21;.041 0.06;.5330.06;.533 −0.01;.932-0.01;.932 −0.03;.796-0.03;.796 0.14;.0660.14;.066 BI −0.20;.073-0.20;.073 −0.02;.800-0.02;.800 0.25;.0620.25;.062 0.05;.6590.05;.659 0.13;.1450.13;.145 0.04;.7730.04;.773 0.28;.0070.28;.007 Table 6 summarises all five clip-level relationships for Static level 3 and UBFC-rPPG. The complete nine-group results appear in Supplementary Table 22. Once the participant is held fixed, almost nothing survives in either evaluation. Of these twenty-eight relationships, two are directional on Static level 3, the MAE–skin coverage of raw attention at ρ=−0.14ρ=-0.14 (p=.020p=.020) and the SNR–SaCo of Beyond Intuition at ρ=+0.19ρ=+0.19 (p=.044p=.044), and three on UBFC-rPPG: the waveform Pearson–skin coverage of attention rollout at ρ=+0.32ρ=+0.32 (p=.047p=.047) and of attention flow at ρ=+0.21ρ=+0.21 (p=.041p=.041), and the SaCo–skin coverage of Beyond Intuition at ρ=+0.28ρ=+0.28 (p=.007p=.007). All significant but at modest significance levels. The remaining twenty-three are unresolved. The two evaluations therefore do not disagree so much as fail to establish much in either. What the near-absence of resolved relationships establishes is weaker but firmer than a disagreement between the two: within a participant, neither skin coverage nor SaCo predicts how well the model does on a given clip. 4.3 Attribution summaries across the nine conditions Figure 5 compares the clip-level skin-coverage and SaCo distributions of the eight NCKU-rPPG scenarios with UBFC-rPPG. Table 8 reports the medians of Figure 5 for every condition, with the interquartile range as the measure of spread. The skin-coverage rows use every held-out clip of the condition and the SaCo rows the clips the SaCo run evaluated, so the two differ in sample size where the run did not cover the whole condition. Beyond Intuition holds the highest median refined skin coverage in seven of the nine conditions and the highest median SaCo in Static level 3, Static level 5, Speak and UBFC-rPPG, so where the recording is clean it both attributes more inside the skin region and orders the clips more faithfully than any attention-only method. Negative SaCo is confined to a minority of outlier clips in 34 of the 36 condition-method pairs; the exception is Beyond Intuition on Static level 1, where 117 of the 200 clips fall below zero and the median itself is −0.178-0.178, and Bike level 1, where 41% fall below zero. Both measures are lowest under the dimmest illumination. Within the static triple and the cycling triple, which differ only in illuminance, the 40-lux member has the lowest median refined coverage for seven of the eight method–triple pairs and the lowest median SaCo for six of them, the exceptions being raw attention for coverage in the cycling triple and raw attention and attention rollout for SaCo in the static triple. The 40-lux conditions also carry almost every negative clip: 28% of their SaCo values fall below zero, against 0.6% at 200 lux and 5.6% at 700 lux. The four methods agree with one another more closely in what they attribute to than in how faithful their orderings are, and not in the same pairs (Table 7). Raw attention and attention rollout produce almost the same SaCo, at ρ=0.97ρ=0.97, while their refined skin coverage agrees at only 0.180.18; attention rollout and attention flow are the other way round, agreeing at 0.960.96 in coverage and 0.560.56 in SaCo. Beyond Intuition is the least like the others in SaCo, at 0.360.36 to 0.520.52, and closer to them in coverage, at 0.480.48 to 0.650.65. Two methods can therefore place their attribution almost identically and still be ordered differently enough to separate their faithfulness, and the reverse. Table 7: Spearman correlations between the four attribution methods, taken over every clip of every participant of all nine conditions. The lower triangle gives the refined skin coverage, over 4142 clips, and the upper triangle SaCo, over the 1741 clips the SaCo run evaluated. Coverage is the mean over TPT1–TPT3 for the attention-only methods and the single refined value for Beyond Intuition. Each cell gives ρ;pρ;p, and cells in bold satisfy both p<0.05p<0.05 and |ρ|≥0.10|ρ|≥ 0.10, the condition counted as directional; every cell meets it at these sample sizes, and a p that underflows double precision is written at the bound 10−30010^-300. Both triangles pool the conditions and the participants, so differences between them contribute here, unlike the coefficients of Table 10. Raw Attention Attention Rollout Attention Flow Beyond Intuition Raw Attention — 0.97;<−0.97;<10^-300 0.55;<−0.55;<10^-138 0.36;<−0.36;<10^-52 Attention Rollout 0.18;<−0.18;<10^-32 — 0.56;<−0.56;<10^-145 0.36;<−0.36;<10^-53 Attention Flow 0.22;<−0.22;<10^-45 0.96;<−0.96;<10^-300 — 0.52;<−0.52;<10^-122 Beyond Intuition 0.48;<−0.48;<10^-234 0.57;<−0.57;<10^-300 0.65;<−0.65;<10^-300 — (a) SaCo, with all eight NCKU-rPPG scenarios on the right. (b) Skin coverage, with all eight NCKU-rPPG scenarios on the right. Figure 5: Combined clip-level split-violin summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each left half shows UBFC-rPPG, and each right half overlays eight independently normalised NCKU-rPPG scenario kernel densities without pooling them. All available clips are plotted on their corresponding half: UBFC-rPPG contains 141 skin-coverage and 141 SaCo observations per method; Static levels 1, 3, and 5 contain 289 skin-coverage observations each; Speak contains 288; Rotate contains 287; Bike levels 1, 3, and 5 contain 853 each; and each NCKU-rPPG scenario contains 200 SaCo observations per method. For attention-only skin coverage, each clip is averaged across TPT1–TPT3 before plotting; Beyond Intuition has only a refined value. Colours denote Static level 1 (blue, #0173B2), Static level 3 (orange, #DE8F05), Static level 5 (green, #029E73), Speak (red-orange, #D55E00), Rotate (purple, #C78BC), Bike level 1 (brown, #CA9161), Bike level 3 (pink, #FBAFE4), Bike level 5 (light blue, #56B4E9), and UBFC-rPPG (dark grey, #4D4D4D). Table 8: Median skin coverage and SaCo faithfulness (F, K=8K=8) for the eight NCKU-rPPG scenarios and UBFC-rPPG, each given as median with the interquartile range in brackets. For attention-based skin coverage, each clip was first averaged across TPT1–TPT3; Beyond Intuition operates only at the refined level. Condition Method Skin Cov. (pre)↑ Skin Cov. (refined)↑ SaCo F↑ Static level 1 Raw Attention 0.369 [0.276, 0.403] 0.345 [0.316, 0.378] 0.739 [0.445, 0.833] Attention Rollout 0.387 [0.300, 0.451] 0.376 [0.279, 0.423] 0.732 [0.443, 0.827] Attention Flow 0.365 [0.280, 0.428] 0.337 [0.286, 0.376] 0.202 [-0.012, 0.409] Beyond Intuition — 0.180 [0.114, 0.428] -0.178 [-0.523, 0.420] Static level 3 Raw Attention 0.433 [0.311, 0.495] 0.457 [0.308, 0.504] 0.701 [0.563, 0.843] Attention Rollout 0.491 [0.338, 0.566] 0.492 [0.333, 0.557] 0.703 [0.563, 0.839] Attention Flow 0.500 [0.331, 0.563] 0.510 [0.362, 0.545] 0.641 [0.494, 0.722] Beyond Intuition — 0.789 [0.627, 0.875] 0.837 [0.740, 0.974] Static level 5 Raw Attention 0.365 [0.289, 0.420] 0.511 [0.436, 0.554] 0.548 [0.355, 0.727] Attention Rollout 0.492 [0.425, 0.568] 0.458 [0.383, 0.548] 0.543 [0.369, 0.713] Attention Flow 0.475 [0.419, 0.551] 0.435 [0.374, 0.480] 0.416 [0.227, 0.620] Beyond Intuition — 0.779 [0.544, 0.897] 0.708 [0.457, 0.755] Speak Raw Attention 0.394 [0.353, 0.440] 0.507 [0.405, 0.568] 0.841 [0.730, 0.905] Attention Rollout 0.630 [0.457, 0.681] 0.575 [0.455, 0.663] 0.847 [0.718, 0.917] Attention Flow 0.639 [0.480, 0.687] 0.538 [0.431, 0.601] 0.824 [0.735, 0.899] Beyond Intuition — 0.632 [0.495, 0.731] 0.863 [0.714, 0.974] Rotate Raw Attention 0.410 [0.342, 0.461] 0.480 [0.434, 0.523] 0.819 [0.675, 0.901] Attention Rollout 0.478 [0.396, 0.563] 0.392 [0.340, 0.452] 0.816 [0.677, 0.902] Attention Flow 0.419 [0.346, 0.484] 0.353 [0.312, 0.395] 0.652 [0.466, 0.793] Beyond Intuition — 0.620 [0.491, 0.736] 0.751 [0.581, 0.922] Bike level 1 Raw Attention 0.397 [0.336, 0.435] 0.505 [0.424, 0.544] 0.537 [0.003, 0.816] Attention Rollout 0.446 [0.357, 0.527] 0.321 [0.255, 0.385] 0.521 [0.032, 0.817] Attention Flow 0.325 [0.290, 0.444] 0.298 [0.226, 0.360] 0.345 [-0.083, 0.697] Beyond Intuition — 0.369 [0.273, 0.487] 0.201 [-0.289, 0.546] Bike level 3 Raw Attention 0.383 [0.326, 0.421] 0.418 [0.346, 0.472] 0.830 [0.659, 0.900] Attention Rollout 0.629 [0.569, 0.696] 0.612 [0.560, 0.703] 0.821 [0.687, 0.898] Attention Flow 0.620 [0.548, 0.670] 0.572 [0.494, 0.639] 0.665 [0.540, 0.822] Beyond Intuition — 0.695 [0.532, 0.779] 0.822 [0.671, 0.976] Bike level 5 Raw Attention 0.432 [0.335, 0.478] 0.554 [0.454, 0.588] 0.767 [0.555, 0.845] Attention Rollout 0.493 [0.379, 0.545] 0.344 [0.248, 0.427] 0.756 [0.527, 0.832] Attention Flow 0.447 [0.332, 0.499] 0.366 [0.292, 0.411] 0.680 [0.431, 0.821] Beyond Intuition — 0.696 [0.461, 0.815] 0.724 [0.462, 0.883] UBFC-rPPG Raw Attention 0.348 [0.261, 0.404] 0.587 [0.521, 0.624] 0.845 [0.735, 0.936] Attention Rollout 0.671 [0.556, 0.725] 0.567 [0.461, 0.600] 0.850 [0.740, 0.932] Attention Flow 0.638 [0.506, 0.693] 0.457 [0.388, 0.495] 0.845 [0.763, 0.913] Beyond Intuition — 0.826 [0.661, 0.852] 0.917 [0.843, 0.979] Table 9 summarises the direction of all 252 correlations across the nine conditions. A correlation counts as directional only if its p is below 0.05 and |ρ||ρ| is at least 0.10, the second limit being Cohen’s benchmark for a small effect. Twenty-five of the 252 are directional, 16 positive and 9 negative, and 28 reach p<0.05p<0.05 against the 13 expected by chance. Table 9: Direction of the clip-level Spearman correlations across the eight NCKU-rPPG scenarios and UBFC-rPPG. Each row counts the nine conditions, so the three columns sum to nine. A correlation is counted as negative or positive only when its p is below 0.05 and |ρ||ρ| is at least 0.10; everything else is counted as no relationship. Each coefficient is the participant-averaged Spearman correlation of Section 3.9, so its p value is a one-sample t test over participants. Relationship Method Negative None Positive HR MAE vs. skin coverage Raw Attention 1 8 0 Attention Rollout 1 7 1 Attention Flow 0 9 0 Beyond Intuition 0 6 3 HR MAE vs. SaCo Raw Attention 1 8 0 Attention Rollout 0 9 0 Attention Flow 0 8 1 Beyond Intuition 0 9 0 Waveform Pearson vs. skin coverage Raw Attention 0 8 1 Attention Rollout 0 7 2 Attention Flow 0 7 2 Beyond Intuition 1 8 0 Waveform Pearson vs. SaCo Raw Attention 0 9 0 Attention Rollout 0 9 0 Attention Flow 0 8 1 Beyond Intuition 2 7 0 SNR vs. skin coverage Raw Attention 0 9 0 Attention Rollout 0 9 0 Attention Flow 0 9 0 Beyond Intuition 2 7 0 SNR vs. SaCo Raw Attention 0 9 0 Attention Rollout 0 9 0 Attention Flow 0 9 0 Beyond Intuition 1 6 2 SaCo vs. skin coverage Raw Attention 0 9 0 Attention Rollout 0 9 0 Attention Flow 0 9 0 Beyond Intuition 0 6 3 No relationship is directional in more than six of its 36 method–condition pairs, and four of the seven are directional in fewer than four. MAE–skin coverage has 4 positive and 2 negative pairs, the largest being Beyond Intuition on Bike level 1 and on Speak, both at ρ=+0.24ρ=+0.24 (p<10−3p<10^-3), so where it resolves at all more attributed skin accompanies larger error. Waveform Pearson–skin coverage has 5 positive and 1 negative, and its three largest are the two UBFC-rPPG coefficients quoted above together with attention flow on Speak at ρ=+0.19ρ=+0.19 (p=.050p=.050). The two SNR relationships behave like the rest and add nothing that the others did not already show: SNR–skin coverage resolves twice in 36, both negative and both Beyond Intuition, on Speak at ρ=−0.25ρ=-0.25 (p=.001p=.001) and Bike level 1 at ρ=−0.12ρ=-0.12 (p=.011p=.011), and SNR–SaCo three times, at |ρ|≤0.19|ρ|≤ 0.19. MAE–SaCo resolves twice in 36, Waveform Pearson–SaCo three times, and SaCo–skin coverage three times, the last being Beyond Intuition on Static level 5, Rotate and UBFC-rPPG, all positive. Beyond Intuition accounts for 14 of the 25 directional pairs, more than the three attention-only methods together, and every one of the five SNR pairs. Every coefficient reported above is taken inside one condition for one attribution method, over the clips of that condition alone, so the 36 coefficients of a relationship never mix two conditions and the differences between conditions cannot enter any of them. Pooling those clips without regard to who they came from yields 146 nominally significant coefficients of the 252 and reaches |ρ|=0.71|ρ|=0.71; taking the coefficient inside each participant first yields 28 and reaches 0.320.32. The pooled coefficients are therefore dominated by differences between the participants of a condition rather than by anything an attribution map reveals about a clip, and the largest of them reverse sign under the correction: the waveform Pearson–skin coverage of Static level 3 reads −0.41-0.41 pooled and between −0.16-0.16 and +0.13+0.13 within participants. Table 10 splits that pooled coefficient into the two levels it mixes. Its three blocks summarise the same 36 coefficients of each relationship, one per condition and attribution method, in three ways: pooled over every clip of the condition, taken inside each participant of the condition and averaged through the Fisher transform, and taken between the participants of the condition by correlating their means. The between-participant block carries its own denominator because the SaCo run reaches three of the twelve participants in each cycling condition, where a coefficient over three points carries no information and is left out. Between the participants of one condition the medians are the largest of the three levels in six of the seven relationships, and they carry the pooled sign for every relationship pairing skin coverage with a performance measure: +0.20+0.20 against a pooled +0.11+0.11 for MAE–skin coverage, −0.24-0.24 against −0.11-0.11 for the waveform correlation, and −0.28-0.28 against −0.14-0.14 for SNR. Within participants the same three medians sit within 0.030.03 of zero, so what the pooled coefficient measures is the participant and not the clip. The largest coefficients behave the same way: between participants they reach 0.660.66 to 0.770.77 in every relationship, against 0.190.19 to 0.320.32 within them, and on twelve points rather than the two hundred and more a pooled coefficient uses. The level above the condition, where illumination and movement differ, is taken up in Section 4.4. Table 10: Each relationship summarised at three levels–pooled over every clip of the condition, taken inside each participant of the condition and averaged through the Fisher transform, and taken between the participants of the condition by correlating their means–over the same 36 condition-and-method coefficients. Sig. counts those reaching p<0.05p<0.05, Median is their median and Largest the largest of them by magnitude. Pooled over clips Within participants Between participants Relationship Sig. Median Largest Sig. Median Largest Sig. Median Largest MAE–skin 24 +0.11+0.11 0.350.35 6 −0.01-0.01 0.240.24 2/36 +0.20+0.20 0.660.66 MAE–SaCo 16 +0.01+0.01 0.530.53 3 −0.01-0.01 0.190.19 1/24 +0.13+0.13 0.670.67 Pearson–skin 27 −0.11-0.11 0.510.51 7 +0.02+0.02 0.320.32 7/36 −0.24-0.24 0.690.69 Pearson–SaCo 14 +0.06+0.06 0.550.55 3 +0.01+0.01 0.220.22 2/24 +0.08+0.08 0.750.75 SNR–skin 25 −0.14-0.14 0.380.38 3 +0.01+0.01 0.250.25 4/36 −0.28-0.28 0.660.66 SNR–SaCo 17 +0.08+0.08 0.430.43 3 +0.02+0.02 0.190.19 3/24 −0.07-0.07 0.770.77 SaCo–skin 23 +0.02+0.02 0.710.71 3 +0.03+0.03 0.290.29 1/24 −0.06-0.06 0.770.77 4.4 Scenario-level SaCo and model reliability Table 11 takes every relationship to the level above the condition, which is the level at which illumination and movement differ. Each coefficient correlates the scenario medians of the relationship’s two measures over the eight NCKU-rPPG scenarios, one coefficient per attribution method. UBFC-rPPG is left out because it is a different dataset evaluated with a separately trained checkpoint rather than a ninth point on the same axis. The medians are those of Table 8 for the attribution measures and the medians of the same clips for MAE, the waveform Pearson correlation rwaver_wave of Equation (8), and SNR, so they are medians throughout rather than the means of Table 4. The performance side of a relationship does not depend on which attribution method produced the map, so within a row the four coefficients differ only through the attribution measure. Eight points resolve nothing at this level: p falls below 0.050.05 only at |ρ|=0.74|ρ|=0.74 and below 0.100.10 only at 0.640.64, and the largest coefficient of the table is +0.57+0.57. Bold therefore marks |ρ||ρ| above 0.400.40, which says that a pair moves together across the scenarios strongly enough to be worth reading and never that a relationship is established. Table 11: Spearman correlations between the scenario medians of each pair of measures, over the eight NCKU-rPPG scenarios. None is significant at p<0.05p<0.05 due to the small number of scenarios. Each cell gives ρ;pρ;p for one attribution method, and a coefficient in bold has |ρ||ρ| above 0.400.40 before rounding. Relationship Raw Attention Attention Rollout Attention Flow Beyond Intuition MAE–skin −0.21;.610-0.21;.610 +0.02;.955+0.02;.955 −0.02;.955-0.02;.955 −0.43;.289-0.43;.289 MAE–SaCo +0.31;.456+0.31;.456 +0.31;.456+0.31;.456 +0.33;.420+0.33;.420 −0.10;.823-0.10;.823 Pearson–skin +0.05;.911+0.05;.911 +0.12;.779+0.12;.779 +0.14;.736+0.14;.736 +0.57;.139+0.57;.139 Pearson–SaCo −0.24;.570-0.24;.570 −0.24;.570-0.24;.570 −0.17;.693-0.17;.693 +0.05;.911+0.05;.911 SNR–skin +0.21;.610+0.21;.610 −0.02;.955-0.02;.955 +0.02;.955+0.02;.955 +0.43;.289+0.43;.289 SNR–SaCo −0.50;.207-0.50;.207 −0.50;.207-0.50;.207 −0.40;.320-0.40;.320 −0.05;.911-0.05;.911 SaCo–skin −0.33;.420-0.33;.420 +0.50;.207+0.50;.207 +0.48;.233+0.48;.233 +0.50;.207+0.50;.207 The reliability hypothesis predicts a negative coefficient with MAE and positive ones with the waveform correlation and SNR. Only the skin coverage of Beyond Intuition points the hypothesised way on all three performance measures, at −0.43-0.43 with MAE, +0.57+0.57 with the waveform correlation and +0.43+0.43 with SNR, while the three attention-only methods stay within |ρ|=0.21|ρ|=0.21 of zero on those same relationships. The SaCo relationships point the other way: for the three attention-only methods SaCo rises with MAE, at +0.31+0.31 to +0.33+0.33, and falls with the waveform correlation, −0.17-0.17 to −0.24-0.24, and with SNR, −0.40-0.40 to −0.50-0.50, each opposite to the direction a reliability indicator would take, while the three coefficients of Beyond Intuition sit within 0.100.10 of zero. SaCo and skin coverage move together for every method but raw attention, at +0.48+0.48 to +0.50+0.50 against −0.33-0.33. The eight-scenario comparison therefore did not support median SaCo as a consistent tracker of model reliability. 5 Discussion 5.1 Principal findings Within a single participant of a single condition, neither the Salience-guided Faithfulness Coefficient nor the skin coverage of any of the four attribution methods is related to the heart-rate error, the waveform correlation, or the signal-to-noise ratio of a clip (Table 10). The explainable artificial intelligence measures therefore carry information that is complementary to the three performance measures rather than a proxy for them, and attributing to the skin does not by itself guarantee that the heart rate or the waveform is well estimated. The evidence is the near-absence of resolved relationships rather than a small effect. Of the 252 clip-level coefficients, 186 have |ρ||ρ| below 0.100.10, Cohen’s benchmark for a small effect, and only 28 reach p<0.05p<0.05 at all, against the 13 that a nominal 0.05 level would produce by chance in a family of that size (Supplementary Table 22). Both positive and negative relationships occur for the same method pair across the four attentions (Table 9). The largest of the 252 reaches ρ=0.32ρ=0.32, no condition resolves more than five, and no relationship resolves in more than six of its 36 method-and-condition pairs (Table 9). The same clips pooled without regard to who they came from yield 146 coefficients at p<0.05p<0.05 and reach 0.710.71, so what a pooled analysis of this kind measures is the participants of a condition and not the clips (Table 10). Beyond Intuition is the most useful of the four attribution methods on this evidence. It supplies 14 of the 25 relationships that are directional, meaning p<0.05p<0.05 with |ρ|≥0.10|ρ|≥ 0.10 together, which is more than the three attention-only methods supply between them (Table 9); it holds the highest median refined skin coverage in seven of the nine conditions and the highest median SaCo in four (Table 8); and across the eight scenarios it is the only method whose skin coverage follows the reliability measures, falling with the heart-rate error at ρ=−0.43ρ=-0.43 and rising with the waveform correlation and SNR at +0.57+0.57 and +0.43+0.43 (Table 11). The three attention-only methods invite caution for the opposite reason: at the same scenario level their SaCo rises with the heart-rate error, at +0.31+0.31 to +0.33+0.33, and falls with the waveform correlation and SNR, at −0.17-0.17 to −0.24-0.24 and −0.40-0.40 to −0.50-0.50, each the reverse of what a reliability indicator would do (Table 11). They also cannot be treated as interchangeable readings of the same thing, nor as four independent ones. Raw attention and attention rollout order the clips almost identically by SaCo, at ρ=0.97ρ=0.97, while agreeing on where they attribute at only 0.180.18, and attention rollout and attention flow do the reverse, at 0.960.96 in coverage and 0.560.56 in SaCo (Table 7). Two methods can therefore attribute to almost the same place and still be ordered differently enough for their faithfulness to separate, and two that are ordered alike can attribute to different places, so a choice between them is a choice of both properties and not of one. The one condition in which Beyond Intuition fails is the dimmest. At 40 lux its median refined skin coverage falls to 0.1800.180 on Static level 1 against 0.7890.789 and 0.7790.779 at 200 and 700 lux, and its median SaCo falls to −0.178-0.178, with 117 of its 200 clips below zero (Table 8). Coverage and faithfulness dropping together points at where the attribution lands rather than at how it is ordered, so in dim light the model does not concentrate on the skin. Motion produces no comparable drop even though it degrades the estimates far more: speaking, rotation, and cycling at 200 and 700 lux raise the heart-rate error from 1.96±0.341.96± 0.34 to between 8.9±2.68.9± 2.6 and 16.1±2.316.1± 2.3 beats per minute while the median refined coverage of Beyond Intuition stays between 0.6200.620 and 0.6960.696 and its median SaCo between 0.7240.724 and 0.8630.863 (Tables 4 and 8). What an attribution reveals about a condition is therefore where the model looks rather than how faithfully its map is ordered or the quality of the heart rate estimates. Just because a model looks in the right place does not mean the estimate will be accurate. For some clips even a little attention on the skin can be enough to produce a good heart-rate estimate, and for others even a lot of attention on the skin can be insufficient. 5.2 Model performance across NCKU-rPPG The static results were non-monotonic: level 3 performed best, level 5 ranked second, and level 1 was weaker across the reported performance measures (Table 4). This ordering does not show that performance improved or deteriorated monotonically with illumination. The three rows came from independently trained models and scenario-specific samples, so participant availability, recording quality, and training variation could also contribute to their differences. A causal illumination analysis would require paired observations that hold the model and participants constant while changing illumination, together with repeated training seeds. The cycling conditions were ordered by illumination where the static ones were not. Their heart-rate error fell from 18.6±5.518.6± 5.5 to 16.1±2.316.1± 2.3 to 15.1±3.515.1± 3.5 beats per minute from 40 to 200 to 700 lux, and their root mean square error, mean absolute percentage error, and heart-rate correlation ordered the three the same way; the waveform correlation and SNR did not, placing Bike level 3 below Bike level 1 on both (Table 4). The ordering is not a separation, however: the three cycling conditions were not separated from one another at the clip level, every pairwise difference being within twice the standard errors of the values it separated. A monotonic trend in four of the six measures across three conditions that no measure resolves is a direction worth testing with paired recordings, not a demonstration that illumination helps under motion. Speaking and head rotation at illumination level 3 had markedly lower correlations and SNRs than Static level 3 (Table 4). Speaking introduced local deformation around the cheeks and mouth, whereas rotation changed head pose and the visible facial surface. The face-region conversion stabilised a square crop before training, but it could not remove appearance changes or motion-related colour variation within that crop. All three cycling conditions likewise had lower correlations and negative SNRs relative to static levels 3 and 5, indicating that motion robustness remained limited. Static level 3 makes the more durable point, pairing a heart-rate correlation of 0.968±0.0790.968± 0.079 with a waveform correlation of 0.626±0.0370.626± 0.037 (Table 4). This shows why dominant-frequency heart-rate error and spectral waveform quality should be interpreted together: a frequency peak can remain identifiable even when much of the predicted waveform power lies outside the reference fundamental and harmonic bands. These comparisons describe eight independently trained models, each produced with one random seed and evaluated at the final epoch, the thirtieth (Supplementary Table 18). No validation loader or validation-based checkpoint selection was used. The design therefore supports a descriptive baseline but not statistical superiority among scenarios, attribution of between-row differences to illumination or motion alone, or generalisation to another dataset. These results further indicate that RhythmFormer is not robust to variations in illumination. Underexposed conditions substantially degrade its performance, regardless of whether the subject remains stationary or moves. 5.3 Static level 3 versus UBFC-rPPG The Static level 3 and UBFC-rPPG evaluations used the same RhythmFormer architecture, similar observation windows, stationary subjects in controlled settings, and 12 held-out participants in each test set. Static level 3 used 256 frames at 50 Hz, corresponding to 5.12 s, whereas the UBFC-rPPG reproduction used 160 frames at 30 Hz, corresponding to approximately 5.33 s [22]. Static level 3 yielded mean absolute error (MAE) 1.96±0.341.96± 0.34 beats per minute, root mean square error (RMSE) 3.74±0.753.74± 0.75, mean absolute percentage error (MAPE) 2.36±0.422.36± 0.42, heart-rate correlation 0.968±0.0790.968± 0.079, waveform correlation 0.626±0.0370.626± 0.037, and signal-to-noise ratio (SNR) 5.7±1.55.7± 1.5 dB (Table 4). The UBFC-rPPG reproduction yielded 0.96±0.160.96± 0.16, 1.37±0.281.37± 0.28, 0.99±0.180.99± 0.18, 0.991±0.0410.991± 0.041, 0.828±0.0220.828± 0.022, and 8.54±0.978.54± 0.97 dB on the same six measures [22]. UBFC-rPPG is therefore ahead on every column, but by very different margins. The two heart-rate correlations are close, 0.991 against 0.968, because both models order the clips of their evaluation correctly. The waveform correlations are closer than they first appeared, 0.828 against 0.626, but only once a per-recording synchronisation lag is fitted; at zero lag Static level 3 reads 0.28, and the difference between those two figures is a property of the NCKU-rPPG recordings rather than of the model, since the same fit is worth nothing on UBFC-rPPG (Supplementary Table 20). This indicates an error in the synchronisation of the NCKU-rPPG recordings, which is not a property of the model and does not affect the heart-rate estimates and was removed in the waveform correlation reported here by fitting a per-recording lag. What still separates the two evaluations is the heart-rate error, half as large on UBFC-rPPG, and a higher signal-to-noise ratio. The reported difference of 2.8 decibels is a lower bound rather than a like-for-like figure, because the two SNR periodograms are zero-padded differently, and matching their resolution widens it to about 4 decibels. They remain separate within-dataset evaluations of different checkpoints and do not test cross-dataset generalisation. Several differences remain uncontrolled: frame rate and clip length, data splitting, participant composition, cameras, face preprocessing, recorded activities, and the trained checkpoint. These differences prevent a causal explanation of the small metric differences and limit the comparison to a descriptive reference. 5.4 Static level 3 XAI versus UBFC-rPPG Beyond Intuition achieved the highest median skin coverage in both evaluations, with 0.789 on Static level 3 and 0.826 on UBFC-rPPG (Table 8) [22]. It also achieved the highest median SaCo in both evaluations, with 0.837 on Static level 3 and 0.917 on UBFC-rPPG. The secondary rankings nevertheless differed: attention rollout, raw attention, and attention flow yielded median SaCo values of 0.703, 0.701, and 0.641 on Static level 3, compared with 0.851, 0.846, and 0.846 on UBFC-rPPG (Table 8). The repeated top rank for Beyond Intuition is descriptive evidence from these two evaluations rather than proof that the full XAI ranking transfers across datasets. The split violins also show substantial within-dataset heterogeneity that the medians conceal (Figure 5). For Static level 3, the SaCo interquartile ranges were 0.228–0.280 across methods, and individual values extended from 0.017 to 1.000. The corresponding UBFC-rPPG interquartile ranges were 0.135–0.201, with a Beyond Intuition outlier at −0.534-0.534 and upper values of 1.000. Chen and Nordling [22] traced that outlier to the participant’s hand entering the field of view in six of that clip’s 160 frames, 3.75 % of it. Replacing those six frames with stable neighbouring frames from the same clip lifts the Beyond Intuition coefficient from −0.53-0.53 to 0.760.76 and moves all four attribution methods towards F≈0.8F≈ 0.8, so the clip rather than the method produced the value. Refined skin coverage also varied widely among clips, particularly for Beyond Intuition and rollout. Part of this spread is consistent with how SaCo itself is computed: partitioning each 4×44× 4 salience map into K=8K=8 rank-ordered groups of two patches each leaves only 28 pairwise comparisons, so a small change in the salience map that moves one patch across a group boundary can reverse several comparisons at once and push F towards the extremes reported above. The groups are also large, and they are formed by salience rank alone rather than by anatomy: K=8K=8 divides the map into eight groups whatever its resolution, so each group covers an eighth of the 128×128128× 128 crop, as two 32×3232× 32 patches for the attention-only methods and eight 16×1616× 16 patches for the 8×88× 8 map of Beyond Intuition. A region that size cannot follow the boundary of the skin mask, so several groups hold skin and non-skin together in one perturbation, and a low coefficient need not mean that any part of the map is anatomically implausible. The comparison also rests on one trained checkpoint per condition from a single random seed with no validation-based selection, as noted above; the resulting attribution behaviour reflects that run’s particular idiosyncrasies as well as genuine clip-level variation, a source of noise that the more controlled Static level 3 recording protocol does not by itself reduce. Consequently, a higher median does not imply uniformly better explanations, and pooled correlations may be influenced by scenario imbalance and extreme clips. The relation between pre-attention and refined attention also differed. On UBFC-rPPG, refinement produced a large increase in raw-attention skin coverage and decreases in rollout and attention-flow coverage [22]. On Static level 3, the corresponding pooled mean differences were only +0.0179+0.0179, +0.0004+0.0004, and −0.0048-0.0048 (Table 5). The pronounced UBFC-rPPG stage effect therefore did not recur at a comparable magnitude. The Chen and Nordling UBFC-rPPG study directly quantified multi-hop leakage, whereas the Static level 3 analysis did not quantify leakage mass. The small stage differences on Static level 3 consequently neither reproduce nor refute that mechanism. The clip-level correlation structures provide a further contrast. On UBFC-rPPG, the skin coverage of attention rollout and of attention flow was positively related to waveform Pearson correlation within participants, at Spearman ρ=0.32ρ=0.32 (p=.047p=.047) and 0.210.21 (p=.041p=.041) respectively (Table 6), and these are two of only twenty-five relationships resolved out of 252. On Static level 3, the corresponding coefficients were close to zero at ρ=−0.00ρ=-0.00 and −0.09-0.09; the raw-attention and attention-flow coefficients were likewise small at 0.090.09 and 0.010.01. Heart-rate MAE and skin coverage were positively related for all four Static level 3 methods (ρ=0.19ρ=0.19–0.260.26), whereas the UBFC-rPPG coefficients were negative or close to zero (ρ=−0.22ρ=-0.22–−0.06-0.06), both in Table 6. The MAE–SaCo coefficients remained close to zero on both datasets: ρ=−0.06ρ=-0.06–0.060.06 on Static level 3 and 0.000.00–0.040.04 on UBFC-rPPG. For SaCo–skin coverage, Beyond Intuition was resolved on UBFC-rPPG at ρ=0.28ρ=0.28 (p=.007p=.007) and unresolved on Static level 3. The wider lesson is not that the two evaluations disagree but that almost nothing survives in either: holding the participant fixed, neither skin coverage nor SaCo predicts how well the model does on a given clip. That the coefficients vanish once the participant is held fixed says the measures tell different stories at the clip level: among the clips of one person in one condition, skin coverage, SaCo, heart-rate error, waveform correlation and the signal-to-noise ratio move independently of one another. They stop being independent at the levels above the clip. Between the participants of a condition the three relationships pairing skin coverage with a performance measure take a consistent sign (Table 10), and across the eight scenarios the performance measures and the skin coverage of Beyond Intuition move together (Table 11), which is what illumination and movement acting on all of them at once would produce. SaCo is the exception at the scenario level, moving opposite to the reliability measures rather than with them, so the common driver reaches where an attribution lands but not how faithful its ordering is. The correlation p values are one-sample t tests over participants, which are independent, so they are inferential rather than descriptive. The clips are nested within subjects and therefore are not independent samples for confirmatory inference. 5.5 Do skin coverage and SaCo track reliability across scenarios? The two measures answer this differently, and the difference is the clearest separation between them that this work produces. Neither is resolved at the scenario level, where eight points put p below 0.05 only at |ρ|=0.74|ρ|=0.74, so what follows is a reading of directions and magnitudes rather than of established relationships (Table 11). Skin coverage tracks reliability for the method that attributes most tightly, and for that method only. The coverage of Beyond Intuition falls as the heart-rate error rises, at ρ=−0.43ρ=-0.43, and rises with the waveform correlation and SNR, at +0.57+0.57 and +0.43+0.43: the three directions the reliability hypothesis predicts, and the three largest coverage coefficients in the table. The three attention-only methods give nothing in either direction, staying within |ρ|=0.21|ρ|=0.21 of zero on all three measures, so the property belongs to Beyond Intuition’s maps rather than to skin coverage as a measure. SaCo does the opposite of what a reliability indicator should do. For raw attention, attention rollout, and attention flow, median SaCo rises with the heart-rate error, at +0.31+0.31 to +0.33+0.33, and falls with the waveform correlation and SNR, at −0.17-0.17 to −0.24-0.24 and −0.40-0.40 to −0.50-0.50, each the reverse of the predicted direction. Beyond Intuition alone points the predicted way on the error and the waveform correlation, and even there its three coefficients sit within 0.100.10 of zero, which is weaker than any of its coverage coefficients. A method can therefore order its own perturbation responses consistently in a condition on which the model performs badly, which is what the attention-only rows say. This does not invalidate SaCo as a within-clip perturbation-faithfulness measure. SaCo asks whether the salience ordering agrees with the relative effects of perturbing spatial groups for a particular model output; it is not defined as a surrogate for prediction accuracy or waveform quality across independently trained models, and the scenario-level analysis shows only that its median does not serve as one. The contrast with coverage is what makes the negative result informative: where an attribution lands carries scenario-level information about how well the model does, at least for the method whose maps are concentrated enough for the location to mean something, and how faithfully that attribution is ordered does not. The two measures nevertheless fail together in one place. At 40 lux the median skin coverage and the median SaCo are both at their lowest, and 28 % of the SaCo values there fall below zero against 0.6 % at 200 lux. Coverage and faithfulness falling together points at the attribution’s location rather than at its ordering alone: in dim light the model does not concentrate on the skin, so the regions a map ranks highest are no longer the ones the prediction depends on. The two also agree with each other more than either agrees with performance, at +0.48+0.48 to +0.50+0.50 across the scenarios for every method but raw attention. With eight heterogeneous scenarios and one checkpoint each, all of this is a directional diagnostic and not significance, causal, or generalisation evidence. 5.6 The evaluation window, and what the literature compares The reported error depends on the evaluation window more strongly than on any other choice in Section 3.6, and lengthening the window always lowers it (Supplementary Table 19). Stability cannot decide the window on its own, because the steadiest window is the whole recording, which returns one number for a quantity that moved. Judged instead against a target that needs no window, the beat-to-beat heart rate taken from the systolic-peak intervals of the reference photoplethysmogram, the error splits into a part that falls with length, the spectral estimate against the mean reference over the same window, and a part that grows with it, that window mean against the instantaneous reference at the window centre. On Static level 3 the first falls from 2.62 to 0.84 beats per minute between 5.12 and 61.44 s while the second rises from 3.42 to 4.27, and the sum is least at 7.68 s (Supplementary Table 21). The curve is shallow, 3.34 beats per minute at that minimum against 3.60 at 5.12 s, so the clip we use costs 7 % of a quantity that already carries a standard error of 0.34. We keep 5.12 s because it is what makes the UBFC-rPPG row of Table 4 comparable at all, its clip being 5.33 s. Under cycling the minimum moves to 8.96 and 10.24 s and sits three to five times higher, on curves that vary by no more than 17 % of their own minimum across a twelvefold range of length: no window is good there, and the limitation is the reference rather than the window. Whether a published error is comparable with ours depends on the same choice, and the field does not make it consistently. Across the 58 articles published from 2020 onwards in our collection that apply a remote photoplethysmography method, the window is the least standardised of the four settings we recorded (Supplementary Section S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020). Those that state one use between 5 and 60 s, one computes a single heart rate for a whole video, and the two commonest choices differ by a factor of three; in our own data, moving between those two commonest choices changes the mean absolute error by 0.23 to 1.63 beats per minute across the eight conditions, which is the order of the differences that separate the methods being compared. Only nine of the 58 state the transform length or the window function behind the spectrum they take a peak from, although at 30 frames per second an N-point transform resolves 1800/N1800/N beats per minute, so an unpadded 160-frame clip resolves no better than 11. The frame rate is the one setting the field has settled, with 50 of the 58 at 30 Hz, which is why a window given in frames is shorter in seconds here, at 50 Hz, than in most of that literature and why we give both. A heart-rate error is therefore not a number that can be compared across articles without the window, the padding, and the frame rate that produced it. 5.7 Reproducibility, limitations, and future work The participant-independent split, the face-region conversion, the retention of the source video’s 50 Hz frame cadence, and the quality gates applied during conversion together define the adaptation. No participant and no session appeared in both the training and the test set of any scenario, and Supplementary Sections S4. Low-variance label checks and S5. Training configuration state the inclusion thresholds and the training settings that were in force. These results rest on two datasets, and each was chosen for what the other is not. UBFC-rPPG is a public benchmark on which the heart rate is recognised as easy to recover: its subjects are stationary and evenly lit, and the median clip error here is 0.67 beats per minute against 1.14 on the most favourable of our own conditions. NCKU-rPPG was recorded in our laboratory with the illumination set to 40, 200, and 700 lux, with speaking, head rotation, and cycling, and with an exercise bike that widens the reference heart rate from 54.5–127.2 beats per minute when the participant is stationary to 47.5–150.0 when cycling. Two datasets, one easy and one deliberately varied, are still two. The training protocol is a deliberate limitation rather than an oversight. RhythmFormer was trained for 30 epochs, once, from a single random seed, as a separate model per condition so that each is specialised to its own data distribution and the nine differ in how well they perform; the final-epoch checkpoint was evaluated with no validation loader and no checkpoint selection (Supplementary Table 18). Nothing about the test data therefore reached the model at any point, so every figure in Table 4 is an estimate of generalisation error rather than a value selected against the set it is reported on. A better model for each condition could almost certainly be obtained by training longer, selecting a checkpoint on a validation split, and searching over seeds. We did not, because an observation made only on highly optimised models may hold only for highly optimised models. The span was the point: conditions in which the error approaches the reference’s own beat-to-beat noise, conditions in which it reaches 15 to 19 beats per minute, and models at both ends of that range. The main limit on how far these conclusions generalise is that one model structure produced all of them. Every statement here is about RhythmFormer, and whether the same attribution behaviour appears in another architecture is untested. Training the nine-condition protocol for further architectures is beyond what we can afford at present, and we judge the observations worth reporting now rather than holding until it is. The reference bounds what any of these numbers can mean. One heart rate per 5.12 s clip, taken from the reference photoplethysmogram by the same spectral estimator, changes by more than 10 beats per minute between consecutive clips of one recording in 601 of 3905 such pairs, from 2.5 % on Static level 5 to 23.8 % on Bike level 5, so a share of the reported error is reference movement inside the window rather than model error. Before any training or testing, 365 of the 25,848 complete raw clips were excluded, 80 of them from test sets, because the reference trace was non-finite or flat to within 10−610^-6 of its own standard deviation and therefore carried no measurable pulse (Supplementary Table 15); no model output entered that decision. A further 30 of the 2559 cycling test clips are censored at the top of the 45–150 beats-per-minute search band. The 5.12 s window is at the short end of the literature, so the errors reported here are larger than a 30 s or whole-recording evaluation of the same predictions would give: lengthening our own window to 61.44 s lowers the mean absolute error in every condition, by 1.4 beats per minute on Static level 3 and 6.6 on Bike level 1 (Supplementary Table 19). Section 5.6 sets out what that costs against the beat-to-beat reference and why the window was kept nonetheless. A condition’s mean absolute error also conceals a spread of two orders of magnitude (Supplementary Figure 8). On Static level 3 the middle half of clips lies between 0.51 and 2.18 beats per minute, the ninetieth percentile is 3.49, and the worst clip reaches 25.0; on Bike level 1 the middle half spans 1.97 to 29.86, the ninetieth percentile is 53.90, and the worst reaches 98.8, with 42.9 % of clips above 10 beats per minute against 2.8 % on Static level 3. Every condition is right-skewed and none is bimodal, so each mean sits above the bulk of its own clips and a comparison of means understates how often a clip fails outright. The model comparison remains limited by one seed, independently trained condition-specific models, final-epoch checkpoints, no validation-based selection, and 12 test participants per scenario. The XAI comparison adds differences in attribution resolution and layer aggregation: Beyond Intuition supplied a refined 8×88× 8 map to SaCo, while the attention-only methods supplied 4×44× 4 maps, and the methods handled temporal and cross-layer aggregation differently. Skin coverage is also sensitive to the face parser and mask definition. Finally, the UBFC-rPPG and Static level 3 summaries used different clip counts and different checkpoints, so neither method rankings nor correlation magnitudes should be treated as controlled dataset effects. Future work should repeat training across seeds, compare joint-condition and condition-specific models, and recover matched participant identities for paired condition analyses. It should also carry the same protocol to model structures beyond RhythmFormer, which is the single change that would most extend how far these observations reach. XAI evaluation should compare SaCo at a common spatial resolution, directly quantify leakage on Static level 3, and test the stability of skin masks and attribution rankings across preprocessing choices. Training and evaluation across NCKU-rPPG and UBFC-rPPG under matched preprocessing would then permit a direct test of cross-dataset transfer. The multi-hop path mechanism should be evaluated in additional experiments so that its consistency with the mechanism observed on UBFC-rPPG can be tested directly. The skin-mask and face-tracking pipeline should be strengthened by adopting FaceXFormer for face localisation and region estimation. Analyses of additional scenarios should establish whether apparently unreliable XAI results arise from differences in model training quality rather than from a lack of interpretive value in the XAI methods themselves. Finally, integrating results across multiple scenarios should help identify clear relationships among model-performance, waveform-quality, and XAI metrics. 6 Conclusion Held within one participant of one condition, neither skin coverage nor SaCo is related to the heart-rate error, the waveform correlation, or the signal-to-noise ratio of a clip: 186 of the 252 coefficients fall below |ρ|=0.10|ρ|=0.10 and 28 reach p<0.05p<0.05 against the 13 chance would give. The attribution measures therefore carry information complementary to the performance measures rather than a proxy for them, and attributing to the skin does not by itself guarantee a good estimate. Pooling the same clips yields 146 coefficients at p<0.05p<0.05 and reaches 0.710.71, which measures the participants of a condition and not the clips, so a clip-pooled analysis of this kind answers a different question from the one it appears to answer. Of the four attribution methods, Beyond Intuition is the most useful here: it supplies 14 of the 25 directional relationships, holds the highest median refined skin coverage in seven of the nine conditions, and is the only method whose coverage follows the heart-rate error, the waveform correlation, and the signal-to-noise ratio across the eight scenarios. The three attention-only methods warrant caution, their SaCo running opposite to each of those three measures at that level, and the four are neither interchangeable nor independent: two of them can order the clips almost identically while agreeing on where they attribute at ρ=0.18ρ=0.18. Beyond Intuition fails in one condition only, the dimmest, where its median coverage falls to 0.1800.180 and its median SaCo to −0.178-0.178; motion degrades the estimates far more and produces no such drop. What an attribution reveals about a condition is therefore where the model looks, not how faithfully its map is ordered, and not how well the model performs. On NCKU-rPPG the static ordering is non-monotonic in illumination while the cycling one is monotonic on four of the six measures, and no measure separates the three cycling conditions at the clip level. The coexistence on Static level 3 of a heart-rate correlation of 0.968±0.0790.968± 0.079 with a waveform correlation of 0.626±0.0370.626± 0.037 shows that neither illumination labels nor heart-rate error alone characterise robustness, and every condition’s mean absolute error hides a clip-level spread of two orders of magnitude. Measured on clips of comparable length, the UBFC-rPPG reproduction predicts about twice as accurately as Static level 3, and the magnitudes and secondary rankings of the attribution measures differ between the two. These findings are descriptive. Each condition used one independently trained final-epoch checkpoint from a single seed, which is what makes the reported errors estimates of generalisation error but also leaves each model unoptimised; one architecture produced every result; the 5.12 s window sits at the short end of a literature whose windows range from 5 to 60 s, so no heart-rate error here is comparable with a published one without the window, padding, and frame rate that produced it. Repeated seeds, further architectures, common-resolution attribution evaluation, and controlled cross-dataset experiments are needed before any causal claim about illumination, generalisation, or clinical validity. Declarations Ethical approval. The National Cheng Kung University Human Research Ethics Committee approved the experimental procedures, as stated in Section 3.1.1, and every participant gave written informed consent. Generative artificial intelligence. During the preparation of this work, the authors used Claude (Anthropic) for grammar and spelling checking, code generation for the analysis pipeline and figure-plotting scripts, and LaTeX formatting assistance. After using these tools and services, the authors reviewed and edited the content as needed and take full responsibility for the content of this work. Data availability. UBFC-rPPG is a public dataset and is described by its authors [4]. NCKU-rPPG is not publicly available at the time of writing. The recordings of the participants who consented to their release are planned for publication in a later article, and until then the data can be made available by the corresponding author on reasonable request and subject to the approvals in Section 3.1.1. Code availability. The conversion, training, evaluation, and attribution code is not publicly released. It can be made available by the corresponding author on reasonable request. Funding. This work was supported by the Taiwan National Science and Technology Council (NSTC) under Grant 111-2221-E-006-186 and Grant 114-2221-E-006-089. Competing interests. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Author contributions. Louis Chen: Software, Data curation, Investigation, Formal analysis, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing. Torbjörn E. M. Nordling: Conceptualization, Methodology, Formal analysis, Validation, Visualization, Writing – original draft, Writing – review & editing, Supervision, Project administration, Resources, Funding acquisition. References [1] R. Stricker, S. Müller, and H. Groß (2014) Non-contact video-based pulse rate measurement on a mobile service robot. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, Scotland, UK, p. 1056–1062. Note: PURE dataset Cited by: §1, §2. [2] G. Heusch, A. Anjos, and S. Marcel (2017) A reproducible study on remote heart rate measurement. Note: arXiv preprint arXiv:1709.00962 [cs.CV] Cited by: §1, §2. [3] X. Niu, H. Han, S. Shan, and X. Chen (2018) VIPL-hr: a multi-modal database for pulse estimation from less-constrained face video. In Asian Conference on Computer Vision, p. 562–576. Cited by: §1, §2. [4] S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, and J. Dubois (2019) Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters 124, p. 82–90. Cited by: §1, §2, Data availability.. [5] J. Tang, K. Chen, Y. Wang, Y. Shi, S. Patel, D. McDuff, and X. Liu (2023) Mmpd: multi-domain mobile video physiology dataset. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), p. 1–5. Cited by: §1, §2. [6] X. Liu, D. McDuff, G. Narayanswamy, A. Paruchuri, S. Patel, R. Sengupta, J. Tang, and Y. Wang (2023) RPPG-toolbox: deep remote ppg toolbox. In Advances in Neural Information Processing Systems 36, p. 68485–68510. Cited by: §1, §3.6, §3.6, §3.6, Heart rates at the upper edge of the search band., Table 25. [7] U. Debnath and S. Kim (2025) A comprehensive review of heart rate measurement using remote photoplethysmography and deep learning. BioMedical Engineering OnLine 24 (1), p. 73. Cited by: §1. [8] V. Hassija, V. Chamola, A. Mahapatra, A. Singal, D. Goel, K. Huang, S. Scardapane, I. Spinelli, M. Mahmud, and A. Hussain (2024) Interpreting black-box models: a review on explainable artificial intelligence. Cognitive Computation 16, p. 45–74. Cited by: §1. [9] B. Zou, Z. Guo, J. Chen, J. Zhuo, W. Huang, and H. Ma (2025) RhythmFormer: extracting patterned rPPG signals based on periodic sparse attention. Pattern Recognition 164, p. 111511. Cited by: §1, §2, §3.4, §3.5, §3.6, §3.6. [10] M. Soleymani, J. Lichtenauer, T. Pun, and M. Pantic (2012) A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing 3 (1), p. 42–55. Cited by: §2. [11] R. Špetlík, V. Franc, and J. Matas (2018) Visual heart rate estimation with convolutional neural network. In Proceedings of the British Machine Vision Conference, Newcastle, UK, p. 3–6. Cited by: §2. [12] J. A. M. Correa, M. K. Abadi, N. Sebe, and I. Patras (2021) Amigos: a dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Computing 12 (2), p. 479–493. Cited by: §2. [13] C. S. Pilz, S. Zaunseder, J. Krajewski, and V. Blazek (2018) Local group invariance for heart rate estimation from face videos in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, p. 1254–1262. Cited by: §2. [14] A. Dasari, S. K. A. Prakash, L. s. A. Jeni, and C. S. Tucker (2021) Evaluation of biases in remote photoplethysmography methods. NPJ digital medicine 4 (1), p. 91. Cited by: §2, Table 25. [15] D. McDuff, M. Wander, X. Liu, B. Hill, J. Hernandez, J. Lester, and T. Baltrusaitis (2022) Scamps: synthetics for camera measurement of physiological signals. Advances in Neural Information Processing Systems 35, p. 3744–3757. Cited by: §2. [16] X. Li, I. Alikhani, J. Shi, T. Seppanen, J. Junttila, K. Majamaa-Voltti, M. Tulppo, and G. Zhao (2018) The obf database: a large face video database for remote physiological signal measurement and atrial fibrillation detection. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), p. 242–249. Cited by: §2. [17] W. Qian, D. Guo, K. Li, X. Zhang, X. Tian, X. Yang, and M. Wang (2024) Dual-path tokenlearner for remote photoplethysmography-based physiological measurement with facial videos. IEEE Transactions on Computational Social Systems 11 (3), p. 4465–4477. Cited by: §2, Table 25. [18] T. Liu, H. Xiao, Y. Sun, K. Zuo, Z. Li, Z. Yang, and S. Liu (2025) PhysKANNet: a kan-based model for multiscale feature extraction and contextual fusion in remote physiological measurement. Biomedical Signal Processing and Control 100, p. 107111. Cited by: §2, Table 25. [19] P. Huang, T. Chen, Y. Chan, K. Chen, and C. Hsu (2025) D-rPPGNet: de-interfering and descriptive feature learning for unsupervised rPPG estimation. IEEE Transactions on Information Forensics and Security 20, p. 4956–4970. Cited by: §2, Table 25. [20] Y. Li, J. Huang, J. Zhao, D. Wu, and M. Zheng (2025) TS-can+: a improved ts-can architecture for non-contact heart rate measurement. IEEE Transactions on Consumer Electronics 71 (1), p. 1393–1401. Cited by: §2, Table 25. [21] Q. Li, D. Guo, W. Qian, X. Tian, X. Sun, H. Zhao, and M. Wang (2024) Channel-wise interactive learning for remote heart rate estimation from facial video. IEEE Transactions on Circuits and Systems for Video Technology 34 (6), p. 4542–4555. Cited by: §2, Table 25. [22] L. Chen and T. E. M. Nordling (2026) Explaining RhythmFormer: a systematic XAI analysis of periodic sparse attention for remote photoplethysmography. In Proceedings of the Third Workshop on Explainable Artificial Intelligence for the Medical Domain (EXPLIMED 2026), G. Casalino, G. Castellano, K. Kaczmarek-Majer, A. G. Valerio, and G. Zaza (Eds.), CEUR Workshop Proceedings. Note: In press Cited by: §2, §3.6, §3.6, §3.6, §3.9, §3.9, §4.1, Table 4, Table 4, Table 5, §5.3, §5.3, §5.4, §5.4, §5.4. [23] C. Wang (2020) Non-contact heart rate measurement based on facial videos. Master’s Thesis, National Cheng Kung University, No. 1, Dasyue Rd, East District, Tainan City, 701. Cited by: §3.1, Video-to-photoplethysmogram synchronisation., Table 25. [24] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23 (10), p. 1499–1503. Cited by: item 4, §3.2. [25] K. Narayan, V. VS, R. Chellappa, and V. M. Patel (2025) FaceXFormer: a unified transformer for facial analysis. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 11369–11382. Cited by: item 5, §3.2. [26] X. Niu, S. Shan, H. Han, and X. Chen (2020) RhythmNet: end-to-end heart rate estimation from face via spatial-temporal representation. IEEE Transactions on Image Processing 29, p. 2409–2423. Cited by: §3.6. [27] Z. Yu, Y. Shen, J. Shi, H. Zhao, P. H. Torr, and G. Zhao (2022) PhysFormer: facial video-based physiological measurement with temporal difference transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4176–4186. Cited by: §3.6, Table 25. [28] G. De Haan and A. Van Leest (2014) Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiological measurement 35 (9), p. 1913–1926. Cited by: §3.6, Heart rates at the upper edge of the search band.. [29] S. Abnar and W. Zuidema (2020) Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p. 4190–4197. Cited by: §3.7, §3.7. [30] J. Chen, X. Li, L. Yu, D. Dou, and H. Xiong (2023) Beyond intuition: rethinking token attributions inside transformers. Transactions on Machine Learning Research. Cited by: §3.7. [31] C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. In Computer Vision – ECCV 2018, p. 334–349. Cited by: §3.8. [32] J. Wu, W. Kang, H. Tang, Y. Hong, and Y. Yan (2024) On the faithfulness of vision transformer explanations. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10936–10945. Cited by: §3.8. [33] R. Song, H. Chen, J. Cheng, C. Li, Y. Liu, and X. Chen (2021) PulseGAN: learning to generate realistic pulse waveforms in remote photoplethysmography. IEEE Journal of Biomedical and Health Informatics 25 (5), p. 1373–1384. Cited by: S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020, Table 25. [34] P. Huang, B. Wu, and B. Wu (2021) A heart rate monitoring framework for real-world drivers using remote photoplethysmography. IEEE Journal of Biomedical and Health Informatics 25 (5), p. 1397–1408. Cited by: S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020, Table 25. [35] K. Wang, J. Tang, Y. Wei, M. Liu, X. Liu, and Y. Wang (2024) A plug-and-play temporal normalization module for robust remote photoplethysmography. Note: arXiv preprint arXiv:2411.15283 [eess.IV] Cited by: S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020, Table 25. [36] J. Speth, N. Vance, P. Flynn, and A. Czajka (2023) Non-contrastive unsupervised learning of physiological signals from video. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14464–14474. Cited by: S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020, Table 25. [37] E. Lee, E. Chen, and C. Lee (2020) Meta-rPPG: remote heart rate estimation using a transductive meta-learner. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12372 LNCS, p. 392–409. Cited by: Table 25. [38] S. Liu and P. C. Yuen (2020) A general remote photoplethysmography estimator with spatiotemporal convolutional network. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), p. 481–488. Cited by: Table 25. [39] X. Liu, J. Fromm, S. Patel, and D. McDuff (2020) Multi-task temporal shift attention networks for on-device contactless vitals measurement. Advances in Neural Information Processing Systems 33, p. 19400–19411. Cited by: Table 25. [40] O. Perepelkina, M. Artemyev, M. Churikova, and M. Grinenko (2020) HeartTrack: convolutional neural network for remote video-based heart rate monitoring. Conference paper In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Vol. 2020-June, p. 1163–1171. Cited by: Table 25. [41] Y. Tsou, Y. Lee, C. Hsu, and S. Chang (2020) Siamese-rPPG network: remote photoplethysmography signal estimation from face videos. In Proceedings of the 35th Annual ACM Symposium on Applied Computing, p. 2066–2073. Cited by: Table 25. [42] Y. Tsou, Y. Lee, and C. Hsu (2020) Multi-task learning for simultaneous video generation and remote photoplethysmography estimation. In Proceedings of the Asian Conference on Computer Vision, p. 392–407. Cited by: Table 25. [43] Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao (2020) AutoHR: a strong end-to-end baseline for remote heart rate measurement with neural searching. IEEE Signal Processing Letters 27, p. 1245–1249. Cited by: Table 25. [44] M. Hu, F. Qian, D. Guo, X. Wang, L. He, and F. Ren (2021) ETA-rPPGNet: effective time-domain attention network for remote heart rate measurement. IEEE Transactions on Instrumentation and Measurement 70, p. 1–12. Cited by: Table 25. [45] H. Lu, H. Han, and S. K. Zhou (2021) Dual-GAN: joint BVP and noise modeling for remote physiological measurement. Conference paper In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 12399–12408. Cited by: Table 25. [46] Z. Yang, H. Wang, and F. Lu (2022) Assessment of deep learning-based heart rate estimation using remote photoplethysmography under different illuminations. IEEE Transactions on Human-Machine Systems 52 (6), p. 1236–1246. Cited by: Table 25. [47] L. Birla and P. Gupta (2022) AND-rPPG: a novel denoising-rPPG network for improving remote heart rate estimation. Computers in Biology and Medicine 141, p. 105146. Cited by: Table 25. [48] H. Lin, Y. Dong, B. Wu, and B. Wu (2022) Measurement of blood oxygen based on remote-photoplethysmography. In 2022 International Conference on System Science and Engineering (ICSSE), p. 121–126. Cited by: Table 25. [49] X. Liu, W. Wei, H. Kuang, and X. Ma (2022) Heart rate measurement based on 3D central difference convolution with attention mechanism. Sensors 22 (2), p. 688. Cited by: Table 25. [50] Z. Sun and X. Li (2022) Contrast-phys: unsupervised video-based remote physiological measurement via spatiotemporal contrast. In Computer Vision – ECCV 2022, Lecture Notes in Computer Science, Vol. 13672, p. 492–510. Cited by: Table 25. [51] Y. Sun, Y. Yang, B. Wu, P. Huang, S. Cheng, B. Wu, and C. Chen (2022) Contactless facial video recording with deep learning models for the detection of atrial fibrillation. Scientific Reports 12 (1), p. 281. Cited by: Table 25. [52] B. Wu, Y. Wu, and Y. Chou (2022) A compensation network with error mapping for robust remote photoplethysmography in noise-heavy conditions. IEEE Transactions on Instrumentation and Measurement 71, p. 1–11. Cited by: Table 25. [53] X. Liu, B. Hill, Z. Jiang, S. Patel, and D. McDuff (2023) EfficientPhys: enabling simple, fast and accurate camera-based cardiac measurement. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 4997–5006. Cited by: Table 25. [54] J. S. Lee, G. Hwang, M. Ryu, and S. J. Lee (2023) LSTC-rPPG: long short-term convolutional network for remote photoplethysmography. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), p. 6015–6023. Cited by: Table 25. [55] J. Li, Z. Yu, and J. Shi (2023) Learning motion-robust remote photoplethysmography through arbitrary resolution videos. Proceedings of the AAAI Conference on Artificial Intelligence 37 (1), p. 1334–1342. Cited by: Table 25. [56] Z. Li and L. Yin (2023) Contactless pulse estimation leveraging pseudo labels and self-supervision. Conference paper In Proceedings of the IEEE International Conference on Computer Vision, p. 20531–20540. Cited by: Table 25. [57] N. Nguyen, L. Nguyen, C. Álvarez Casado, O. Silvén, and M. Bordallo López (2023) Non-contact heart rate measurement from deteriorated videos. In 2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA), p. 1–8. Cited by: Table 25. [58] L. Chiu, Y. Chou, Y. Wu, and B. Wu (2023) Deep-learning-based remote photoplethysmography measurement in driving scenarios with color and near-infrared images. IEEE Transactions on Instrumentation and Measurement 72, p. 1–12. Cited by: Table 25. [59] Y. Wu, L. Chiu, B. Wu, L. L. Lin, T. Ho, M. Chung, and S. Wu (2023) Motion robust remote photoplethysmography measurement during exercise for contactless physical activity intensity detection. IEEE Transactions on Instrumentation and Measurement 72, p. 1–14. Cited by: Table 25. [60] B. Wu, B. Wu, S. Cheng, Y. Sun, and M. Chung (2023) Motion-robust atrial fibrillation detection based on remote-photoplethysmography. IEEE Journal of Biomedical and Health Informatics 27 (6), p. 2705–2716. Cited by: Table 25. [61] Z. Yu, Y. Shen, J. Shi, H. Zhao, Y. Cui, J. Zhang, P. Torr, and G. Zhao (2023) PhysFormer++: facial video-based physiological measurement with slowfast temporal difference transformer. Note: arXiv preprint arXiv:2302.03548 [cs.CV] Cited by: Table 25. [62] S. Chen, K. Wong, J. Chin, T. Chan, and R. H. Y. So (2024) DiffPhys: enhancing signal-to-noise ratio in remote photoplethysmography signal using a diffusion model approach. Bioengineering 11 (8), p. 743. Cited by: Table 25. [63] R. Castellano Ontiveros, M. Elgendi, and C. Menon (2024) A machine learning-based approach for constructing remote photoplethysmogram signals from video cameras. Communications Medicine 4 (1), p. 109. Cited by: Table 25. [64] N. Qia, K. Li, D. Guo, B. Hu, and M. Wang (2024) Cluster-phys: facial clues clustering towards efficient remote physiological measurement. Conference paper In M 2024 - Proceedings of the 32nd ACM International Conference on Multimedia, p. 330–339. Cited by: Table 25. [65] J. Xiong, W. Ou, Y. Yao, Y. Liu, Z. Gao, Z. Liu, and J. Gou (2024) STGNet: spatio-temporal graph neural networks considering inherent properties of physiological signals for camera-based remote photoplethysmography. Biomedical Signal Processing and Control 98, p. 106690. Cited by: Table 25. [66] Y. Wu, C. Lin, L. Chiu, B. Wu, M. Chung, S. Tang, and Y. Sun (2024) Contact-free atrial fibrillation screening with attention network. IEEE Journal of Biomedical and Health Informatics 28 (9), p. 5124–5135. Cited by: Table 25. [67] Z. Yang, H. Wang, B. Liu, and F. Lu (2024) CbPPGGAN: a generic enhancement framework for unpaired pulse waveforms in camera-based photoplethysmography. IEEE Journal of Biomedical and Health Informatics 28 (2), p. 598–608. Cited by: Table 25. [68] B. Zou, Y. Zhao, X. Hu, C. He, and T. Yang (2024) Remote physiological signal recovery with efficient spatio-temporal modeling. Frontiers in Physiology 15, p. 1428351. Cited by: Table 25. [69] I. Liu, F. Liu, Q. Zhong, F. Ma, and S. Ni (2024) Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging. Cited by: Table 25. [70] L. D. van Putten, K. E. Bamford, I. Veleslavov, and S. Wegerif (2024) From video to vital signs: using personal device cameras to measure pulse rate and predict blood pressure using explainable ai. Discover Applied Sciences 6 (4), p. 184. Cited by: Table 25. [71] L. Chiu, Y. Chou, Y. Wu, M. Chung, B. Wu, and K. Chou (2024) Video-based contactless detection of sleep apnea with deep-learning model. IEEE Transactions on Instrumentation and Measurement 73, p. 1–13. Cited by: Table 25. [72] K. Cen, C. Fu, and H. Hong (2025) Robust and generalizable heart rate estimation via deep learning for remote photoplethysmography in complex scenarios. Note: arXiv preprint arXiv:2507.07795 [cs.CV] Cited by: Table 25. [73] J. Choi and S. J. Lee (2025) Periodic-mae: periodic video masked autoencoder for rppg estimation. Cited by: Table 25. [74] S. Chu, M. Xia, M. Yuan, X. Liu, T. Seppanen, G. Zhao, and J. Shi (2025) CodePhys: robust video-based remote physiological measurement through latent codebook querying. Note: arXiv preprint arXiv:2502.07526 [cs.CV] Cited by: Table 25. [75] C. Tseng, B. Wu, and Y. Sun (2025) A real-time contact-free atrial fibrillation detection system for mobile devices. IEEE Journal of Biomedical and Health Informatics 29 (1), p. 17–29. Cited by: Table 25. [76] Y. Wang (2025) Comparative analysis of non-end-to-end and end-to-end deep learning models with 2D and 3D face alignment for remote heart rate estimation. Master’s Thesis, National Cheng Kung University, No. 1, Dasyue Rd, East District, Tainan City, 701. Cited by: Table 25. [77] J. Zhang, H. Sun, Y. Hu, G. Zhu, F. Liu, B. Yan, J. Pu, X. Du, J. Liu, L. Liu, et al. (2025) Channel attention pyramid network for remote physiological measurement. Scientific Reports 15 (1), p. 22495. Cited by: Table 25. [78] B. Zou, Z. Guo, J. Chen, J. Zhuo, W. Huang, and H. Ma (2024) RhythmFormer: extracting patterned rppg signals based on periodic sparse attention. Note: arXiv preprint arXiv:2402.12788 [cs.CV] Cited by: Table 25. [79] T. Chang, Y. Wu, M. Chung, B. Wu, H. Cheng, W. Lee, and S. Chang (2026) Novel application of contactless remote photoplethysmography in identifying heart failure with left ventricular systolic dysfunction. Journal of Cardiology 87 (6), p. 489–495. Cited by: Table 25. [80] W. Qian, D. Guo, J. Zhou, B. Zou, Z. Yu, and M. Wang (2026) FreqPhys: repurposing implicit physiological frequency prior for robust remote photoplethysmography. Note: arXiv preprint arXiv:2604.00534 [cs.CV] Cited by: Table 25. [81] N. N. Sahoo, V. Sachidanand, M. N. Gayathri, B. Murugesan, K. Ram, J. Joseph, and M. Sivaprakasam (2026) KDPhys: an attention guided 3d to 2d knowledge distillation for real-time video-based physiological measurement. Note: arXiv preprint arXiv:2601.00714 [eess.IV] Cited by: Table 25. Supplementary Material S1. Conversion contract and quality control The production conversion contract fixed the converter source revision, the preprocessing implementation, the FaceXFormer version (commit), model weight checkpoint digest, output codec, quality thresholds, expected session/subject identifiers, and expected skips before we ran the conversion. Table 12 summarises the contract. Table 12: Production conversion contract. Contract item Recorded requirement or result Source sessions 618 Successful or resumed outputs 607 Skipped sessions 11 Conversion failures 0 Output video 128×128128× 128 pixels; source frame rate; FFV1 Per-session detection rate At least 0.80 before interpolation Per-session temporal RGB correlation Prespecified minimum 0.99 after encode–decode; measured accepted-session values not retained Aggregate detection rate At least 0.90 Aggregate interpolation rate At most 0.10 Session-level count checks Reported video frames = decoded video frames = PPG samples Nine sessions were skipped because their alignment CSV files were absent. One session failed the prespecified raw detection-rate gate, and one source video was corrupt. Table 13 lists the complete skip set. Table 13: Sessions skipped by the production conversion contract. Session identifier Recorded reason 113–120, 284 Alignment CSV missing (nine sessions). 381 Raw face-detection rate 0.5995, below the 0.80 gate. 402 Corrupt H.264 source; only 7,552 of 7,560 source frames were decodable. S2. Participant and session split The fixed manifest represented 77 original participants and 607 converted sessions. One of the 78 participants of the 618 source sessions had every session skipped, which is why the converted cohort has 77. The global split assigned 65 participants and 511 sessions to training and 12 participants and 96 sessions to testing. Scenario filtering followed the global participant split. Because each scenario contributed at most one converted session per participant, the participant and session counts are equal within a scenario. Table 14: Participants and sessions per scenario. The overlap columns compare training with testing. Scenario Train participants Test participants Train sessions Test sessions Participant overlap Session overlap Static level 1 64 12 64 12 0 0 Static level 3 64 12 64 12 0 0 Static level 5 63 12 63 12 0 0 Speak 64 12 64 12 0 0 Rotate 64 12 64 12 0 0 Bike level 1 65 12 65 12 0 0 Bike level 3 64 12 64 12 0 0 Bike level 5 63 12 63 12 0 0 S3. Clip retention Table 15 gives the training and test clip counts per scenario together with the clips the low-variance gate excluded. “Complete raw clips” denotes the non-overlapping 256-frame clips available after discarding each session’s incomplete terminal remainder but before we applied the label gate. The eight scenario rows sum to 25,848 complete raw clips, of which 25,483 were retained and 365 were excluded. Table 15: Clip counts by scenario. Scenario Train clips Test clips Excluded train / test Complete raw clips Static level 1 1,553 289 43 / 13 1,898 Static level 3 1,562 289 28 / 10 1,889 Static level 5 1,529 289 35 / 7 1,860 Speak 1,554 288 35 / 23 1,900 Rotate 1,552 287 27 / 6 1,872 Bike level 1 4,651 853 40 / 5 5,549 Bike level 3 4,577 853 30 / 6 5,466 Bike level 5 4,504 853 47 / 10 5,414 Total 21,482 4,001 285 / 80 25,848 A clip was excluded when its ground-truth PPG signal failed one of two checks that the preprocessing applied before any training or testing ran. The first check rejected a label containing a non-finite 32-bit floating-point value. The second rejected a label whose 32-bit floating-point standard deviation was no greater than 10−610^-6, that is, a flat or near-constant trace carrying no measurable pulse. Either condition alone was sufficient, and no other property of the clip entered the decision. No model error, predicted heart rate, Pearson correlation, or SNR value determined clip inclusion. Table 16 identifies every session in which the gate excluded at least one clip, with the number excluded. These counts were obtained as ⌊Ns/256⌋−rs N_s/256 -r_s, where NsN_s is the aligned frame count in the session mapping and rsr_s is the retained file-list count for session s. The preprocessing implementation numbered retained outputs consecutively after skipping an invalid temporal chunk, so the filenames preserve the affected session and exclusion count but not the original within-session temporal index. Table 16: Sessions containing clips that the low-variance gate excluded. Parentheses give the number of excluded 256-frame clips in that session. Scenario Training sessions (excluded clips) Test sessions (excluded clips) Static level 1 006 (1), 054 (1), 102 (1), 126 (4), 134 (1), 200 (3), 216 (2), 232 (1), 272 (5), 280 (1), 352 (2), 392 (3), 424 (5), 432 (6), 480 (1), 520 (1), 536 (1), 544 (1), 592 (3) 168 (1), 192 (2), 224 (1), 472 (4), 512 (3), 576 (2) Static level 3 055 (1), 095 (1), 111 (2), 127 (1), 135 (1), 201 (5), 217 (1), 281 (1), 353 (1), 393 (4), 425 (1), 433 (1), 481 (1), 489 (1), 521 (1), 537 (1), 593 (3), 601 (1) 225 (1), 409 (1), 449 (1), 473 (4), 577 (1), 609 (2) Static level 5 136 (1), 202 (3), 218 (1), 242 (2), 354 (5), 394 (4), 426 (4), 434 (3), 482 (1), 490 (1), 498 (1), 522 (3), 538 (1), 546 (1), 594 (3), 618 (1) 144 (1), 450 (2), 474 (3), 578 (1) Speak 037 (1), 053 (1), 101 (1), 109 (1), 125 (1), 133 (1), 199 (4), 215 (1), 239 (1), 255 (1), 271 (1), 279 (1), 327 (1), 351 (2), 391 (7), 423 (1), 431 (1), 455 (1), 519 (1), 535 (2), 591 (4) 167 (1), 175 (1), 191 (5), 407 (1), 447 (5), 471 (3), 511 (3), 575 (1), 607 (3) Rotate 068 (1), 084 (1), 124 (1), 198 (3), 214 (1), 270 (1), 278 (2), 350 (1), 358 (2), 390 (4), 422 (1), 430 (1), 518 (1), 534 (2), 542 (2), 590 (3) 446 (1), 470 (4), 574 (1) Bike level 1 081 (1), 121 (2), 195 (12), 203 (3), 211 (2), 227 (4), 235 (1), 267 (1), 387 (3), 419 (1), 427 (2), 475 (1), 491 (1), 515 (2), 531 (2), 587 (2) 467 (5) Bike level 3 002 (1), 098 (1), 122 (5), 196 (4), 212 (2), 388 (5), 420 (2), 428 (2), 492 (1), 516 (3), 532 (2), 588 (2) 468 (5), 572 (1) Bike level 5 051 (1), 083 (14), 123 (2), 131 (1), 197 (5), 213 (1), 269 (1), 277 (1), 389 (5), 421 (4), 429 (2), 477 (3), 517 (3), 533 (2), 589 (2) 469 (5), 573 (5) Heart rates at the upper edge of the search band. A second kind of loss affects retained clips rather than excluded ones, and we report it here because a reader counting what the analysis could not see should find both in the same place. The heart rate is searched over 45–150 beats per minute, that is 0.75–2.5 Hz, which is the rPPG-Toolbox default and the band behind the published values we compare with [6, 28]. Thirty of the 2559 cycling test clips, 1.2 %, have a reference heart rate at the upper edge of that band and are therefore censored: 17 clips in Bike level 1, 12 in Bike level 5, and 1 in Bike level 3. No clip of any stationary condition reaches the edge, and no clip of any condition reaches the 45 beats-per-minute lower edge. Every censored clip belongs to one held-out participant. The session manifest resolves the three cycling sessions 467, 468 and 469 of Table 16 to a single person, and each is the eighth held-out session of its condition. That participant has the highest median reference heart rate of the twelve in all eight conditions, from 110.0 beats per minute in Static level 1 to 135.9 in Bike level 5, against condition medians of 77.2 to 90.9. The censored clips fall between 250.9 and 353.3 s, which is the third heart-rate level of the protocol and the rest minute that follows it, so they are the clips at the highest cycling load. We kept the 150 beats-per-minute limit rather than widening it, and applied it to the reference and to the prediction alike. Widening it is not a post-processing choice. The archived waveforms are not band-limited, so the reference heart rate can be re-estimated over a wider band without retraining, and doing so changes no clip of any stationary condition. The prediction cannot follow it. RhythmFormer’s frequency-domain training target lies in the same 45–150 band, so the model does not predict above 150 whatever band the estimator searches, and raising the ceiling on the reference alone enlarges the error by an amount the estimator has no way to avoid, which measures the band rather than the model. Keeping the band flatters nothing: at these 30 clips the model predicts between 51.2 and 116.4 beats per minute against a reference at the ceiling, so they are already among the largest errors in the evaluation. Recovering these heart rates requires a model trained with a wider target band. S4. Low-variance label checks Two low-variance checks acted on the reported experiment, at different stages and on different objects. Table 17 states which threshold defines the reported sample. Table 17: Low-variance label checks applied in the experiment. Stage Threshold Interpretation Preprocessing 10−610^-6 Clips with a non-finite FP32 label or FP32 standard deviation no greater than the threshold were excluded. Training augmentation fallback 10−810^-8 An invalid augmented label restored the original paired video and label after random draws; it did not exclude any clip. The 10−610^-6 gate excluded 285 training clips and 80 test clips before training began. The augmentation fallback acted only on a label produced by a random temporal draw and never removed a clip from the sample. S5. Training configuration Every scenario used the same training code revision and the same participant manifest, and differed only in the scenario, the run identifier, and the cache, file-list, and output paths. Table 18 gives the settings they shared. During training, a non-finite or low-variance augmented label restored the original video–label pair at a threshold of 10−810^-8. Table 18: Settings shared by the eight training runs. Setting Value Training code revision 3b1b3dbe4c9ace25637ce669495c07ed7a59b521, identical for all eight runs Model and precision RhythmFormer; FP32 Random seed 100 Runtime PyTorch 2.0.1+cu118; CUDA 11.8 Hardware Four NVIDIA RTX A5000 graphics processing units Epochs 30 Training batch size 4 Test batch size 2 Initial maximum learning rate 0.009 Scheduler OneCycleLR with 30 epochs and the training-loader length as steps per epoch Optimiser AdamW with weight decay 0 and ϵ=10−8ε=10^-8 Drop path rate 0 Data-loader workers 16 Temporal input 256 frames at 50 Hz Validation and checkpoint selection No validation loader; final epoch used for testing S6. Explainable artificial intelligence results Figure 6: Predicted and reference photoplethysmography waveforms for one held-out clip of Static level 3 and one of Bike level 1. Each clip is the one whose predicted-to-reference correlation is the median over that scenario’s held-out clips, so neither panel is selected for its appearance. Both signals are standardised, and both clips are 256 frames at 50 Hz, that is 5.12 s. Figure 6 contrasts the two ends of the range in Table 4. Static level 3 recovers the heart rate almost exactly, at rHR=0.968±0.079r_HR=0.968± 0.079, yet its median clip still follows the reference waveform only loosely, while the Bike level 1 clip tracks it hardly at all. Because each panel is the clip at its condition’s median correlation rather than a chosen example, the pair shows directly why the two correlation columns of Table 4 must be read together. Figure 7 shows the same contrast in the frequency domain, where both the heart rate and the signal-to-noise ratio are actually read, for the clip of each condition whose absolute heart-rate error is the median of that condition. On Static level 3 the Welch spectrum carries a single lobe, the rate is estimated at 77.9 against a reference of 79.0, an error of 1.14 beats per minute, and the largest bin of the periodogram, at 82 beats per minute, is the one the six-beat template covers, no other bin holding more than two fifths of its power; the clip’s signal-to-noise ratio is +3.06+3.06 dB. On Bike level 1 the rate is estimated at 65.4 against 71.0, an error of 5.52 beats per minute and still inside the template, but the largest periodogram bin lies at 59 beats per minute, below the template altogether, and the bins at 94 and 106 hold four fifths and two thirds of that power, which is what −10.29-10.29 dB records. A rate recovered to within six beats per minute therefore coexists with a spectrum that is mostly noise, which is the separation between the two measures that Section 3.6 sets out. The two spectra also show why the two measures are read from different ones: at 256 frames the periodogram resolves 11.7 beats per minute and its bins are wider than the template, while the Welch spectrum, zero-padded to 200,000 points, locates the peak to a fraction of a beat. Figure 7: The two spectra of one held-out clip of Static level 3 and one of Bike level 1, each the clip whose absolute heart-rate error is the median over that condition’s clips, so neither panel is chosen for its appearance. The Welch spectrum of Equation (7), in which the heart rate is located, is drawn against the periodogram of Equation (9), which the signal-to-noise ratio partitions and which is zero-padded only to the next power of two above the clip. Both are normalised to their own maximum, since only the location of the power and its distribution matter here. The shaded bands are that clip’s template, ±6± 6 beats per minute of the reference rate and of its second harmonic; the second harmonic of Static level 3 lies above 150 beats per minute and is therefore outside the plotted range and outside the band the denominator runs over. Figure 8 gives the clip-level distributions behind the same table, one kernel density per condition for each of the three measures. Every condition is right-skewed in the heart-rate error and none is bimodal, so the mean of Table 4 sits above the bulk of its own clips: the median error of Bike level 3 is 8.20 beats per minute against a mean of 16.1±2.316.1± 2.3, and that of Static level 3 is 1.14 against 1.96±0.341.96± 0.34. The conditions separate far less cleanly than those means suggest, with the motion densities overlapping the static ones over most of their range and differing mainly in how much weight sits in the tail. The waveform correlation separates them better than the heart-rate error does, and it is the only one of the three on which UBFC-rPPG stands clear of every NCKU-rPPG condition. Figure 8: Kernel densities of the heart-rate error, the waveform Pearson correlation, and the signal-to-noise ratio, with one value per held-out clip and one curve per condition. The colours are those of Figure 5, and the sample size of each condition is given in the legend, 4142 clips in total. The densities are Gaussian with the Silverman bandwidth used for the reference heart-rate distributions, and each is estimated from all of its clips; the axes are clipped at 60 beats per minute, at ±1± 1, and at −30-30 to 2020 dB, which leaves out the 3.1 % of clips whose error exceeds 60 beats per minute and the 10.0 % whose signal-to-noise ratio falls outside the plotted range. A density is bounded by neither axis limit, so the curves carry the usual edge bias where a measure is bounded, as the waveform correlation is at +1+1. Table 19 varies the evaluation window, which is the one methodological choice in Section 3.6 that the reported errors depend on strongly. Lengthening the window from 5.12 to 61.44 s lowers the mean absolute error in every condition, by 1.4 beats per minute on Static level 3 and 6.6 on Bike level 1, so part of the error even at 5.12 s is an averaging gain rather than pure model error. The reduction is nonetheless far smaller than the spread between conditions, and it never brings a motion condition near a static one: at 61.44 s the weakest static condition still reads 4.06 beats per minute against 6.63 for the strongest motion condition. The separation of the two groups therefore holds at every window, and the ordering of Table 4 holds within each of them. The order among the motion conditions does not: between 5.12 and 61.44 s Bike level 3 overtakes Rotate, and Bike levels 1 and 5 exchange places. The last column bounds what any window can achieve: on a single clip the reference photoplethysmogram already disagrees with itself by a median of 0.65–1.09 beats per minute when the participant is stationary and by a mean of 5.49–8.54 when cycling. Table 19: Mean absolute error and root mean square error in beats per minute, written as MAE/RMSE, against the number of consecutive clips concatenated into one evaluation window. One clip is 256 frames at 50 Hz, that is 5.12 s, and the manuscript reports the one-clip column. Windows do not overlap and never cross a participant boundary, so the number of windows falls as the width grows. The last column gives the reference photoplethysmogram’s agreement with itself on a single clip, as median/mean absolute difference between its spectral heart rate and the heart rate from the median interval between its detected systolic peaks. The sweep uses the same spectral estimator as the evaluation, whose Welch segment is capped at 256 samples, so a longer window buys segment averaging rather than frequency resolution. It recomputes every heart rate from the archived waveforms, whereas Table 4 uses the per-clip values of the explainability run, so the one-clip column differs from that table’s mean absolute error by up to 0.7 beats per minute. Window (clips, seconds) Reference Scenario 1, 5.12 2, 10.24 4, 20.48 6, 30.72 12, 61.44 self-agreement Static level 1 5.69/12.05.69/12.0 5.18/9.85.18/9.8 4.80/8.74.80/8.7 4.95/8.74.95/8.7 4.06/7.04.06/7.0 0.77/1.810.77/1.81 Static level 3 2.00/3.72.00/3.7 0.97/1.40.97/1.4 0.68/1.00.68/1.0 0.62/0.90.62/0.9 0.63/1.00.63/1.0 0.78/1.690.78/1.69 Static level 5 3.71/8.83.71/8.8 1.98/5.21.98/5.2 1.43/4.01.43/4.0 1.03/1.91.03/1.9 1.11/2.11.11/2.1 0.78/1.110.78/1.11 Speak 9.58/16.39.58/16.3 8.24/15.38.24/15.3 6.58/13.46.58/13.4 7.04/14.67.04/14.6 6.63/14.16.63/14.1 1.09/1.921.09/1.92 Rotate 12.77/18.212.77/18.2 11.30/16.311.30/16.3 11.48/16.211.48/16.2 10.85/15.410.85/15.4 11.59/16.211.59/16.2 0.65/1.740.65/1.74 Bike level 1 18.68/30.718.68/30.7 15.73/27.115.73/27.1 14.83/25.814.83/25.8 14.10/25.314.10/25.3 12.08/23.312.08/23.3 1.81/7.611.81/7.61 Bike level 3 15.36/23.215.36/23.2 13.17/20.213.17/20.2 11.41/18.011.41/18.0 12.26/19.512.26/19.5 10.55/16.710.55/16.7 1.81/5.491.81/5.49 Bike level 5 15.10/23.915.10/23.9 14.19/22.914.19/22.9 13.58/22.513.58/22.5 13.83/22.913.83/22.9 13.21/22.413.21/22.4 2.47/8.542.47/8.54 Video-to-photoplethysmogram synchronisation. Each recording is synchronised on its own light-off trigger [23], so a synchronisation error is a property of one participant in one recording and is constant within it. Table 20 fits that constant by maximising the participant’s mean clip waveform correlation and reports the correlation before and after. The correction is large on NCKU-rPPG and absent on UBFC-rPPG. Its median magnitude is 290–700 milliseconds across the eight NCKU-rPPG conditions and 0 milliseconds on UBFC-rPPG, where no fitted lag exceeds two samples on eleven of the twelve subjects. Fitting one lag per participant and recording spends 108 free parameters on the data the correlation is then read from, so we report the gain against a null. The null repeats the identical fit with each participant’s predictions paired against another participant’s reference, where no true lag exists and any gain measures what the fitting itself buys. On NCKU-rPPG the gain exceeds that floor by two to seven times, from +0.343+0.343 against +0.052+0.052 on Static level 3 to +0.171+0.171 against +0.083+0.083 on Rotate. On UBFC-rPPG the gain of +0.043+0.043 is below its own null of +0.110+0.110, which is what a correctly synchronised recording looks like. That the same architecture and the same code need a large correction on one dataset and none on the other places the cause in the recording rather than in the model. Every waveform correlation reported in this manuscript uses the fitted lag of Table 20. The waveform column of Table 4 is not computed from the same predicted signal as the five columns beside it. The mean absolute error, the root mean square error, the mean absolute percentage error, the heart-rate correlation, and the signal-to-noise ratio are the per-clip values of the explainability run, while the waveform correlation is computed from the predicted waveforms the evaluation run wrote, which are the ones a per-recording lag can be fitted to. The two runs used the same checkpoint on the same clips, and their reference traces agree exactly, clip for clip, in every condition; their predictions do not. Recomputing the heart rate from the evaluation run’s waveforms with the estimator of Section 3.6 moves it from the value Table 4 reports by a median of 0.23 beats per minute on Static level 3 and 0.44 on Bike level 1, with single clips moving by up to 17 and 67, and moves the signal-to-noise ratio by a median of 1.2 and 1.7 decibels. Anyone recomputing one column from one of the two sets of predictions will therefore not reproduce the other’s value exactly, and the discrepancy is larger under cycling than under the stationary conditions. Table 20: Waveform correlation before and after fitting one lag per participant and recording, with the fitted lag in milliseconds. A positive lag means the prediction is delayed relative to the reference. Participants are numbered within a condition and the numbering is consistent across conditions. The search spans ±1± 1 second, which covers half a beat period down to 30 beats per minute, and the maximiser is interior for every participant except two of Bike level 5, whose fitted lag sits on the +1+1 s edge and is therefore censored; a lag is recoverable only modulo the beat period and the smallest-magnitude maximiser is taken. The null is the same fit applied to mismatched participant pairs, where no true lag exists. Condition Participant rwaver_wave at lag 0 rwaver_wave at fitted lag Lag (ms) Static level 1 1 −0.408-0.408 0.6630.663 −480-480 2 0.8400.840 0.8590.859 +20+20 3 −0.464-0.464 0.6960.696 −460-460 4 0.6700.670 0.7670.767 −40-40 5 −0.091-0.091 0.5050.505 −260-260 6 0.5990.599 0.6110.611 −20-20 7 0.0510.051 0.1320.132 −780-780 8 0.0710.071 0.1310.131 −540-540 9 −0.017-0.017 0.2380.238 +480+480 10 0.8180.818 0.8350.835 −700-700 11 0.2030.203 0.4290.429 +500+500 12 −0.143-0.143 0.3260.326 −420-420 mean 0.1770.177 0.5160.516 gain +0.338+0.338, null +0.070+0.070 Static level 3 1 −0.004-0.004 0.6350.635 −640-640 2 0.5050.505 0.6600.660 +700+700 3 −0.668-0.668 0.7330.733 −440-440 4 0.4090.409 0.6430.643 +580+580 5 0.5850.585 0.6060.606 −40-40 6 0.2290.229 0.7660.766 +700+700 7 0.1750.175 0.4950.495 +540+540 8 −0.153-0.153 0.3580.358 −920-920 9 0.4300.430 0.5980.598 +840+840 10 0.7290.729 0.8560.856 +660+660 11 0.6140.614 0.6140.614 +0+0 12 0.5470.547 0.5480.548 +20+20 mean 0.2830.283 0.6260.626 gain +0.343+0.343, null +0.052+0.052 Static level 5 1 0.2330.233 0.5970.597 −660-660 2 0.5070.507 0.7440.744 +620+620 3 −0.130-0.130 0.4570.457 −460-460 4 0.5540.554 0.6190.619 +600+600 5 0.1280.128 0.4960.496 +620+620 6 0.2870.287 0.6980.698 +700+700 7 0.7810.781 0.7960.796 −660-660 8 0.1400.140 0.3070.307 −700-700 9 0.5440.544 0.7210.721 +780+780 10 0.8250.825 0.8420.842 +680+680 11 0.4460.446 0.5410.541 +580+580 12 0.0120.012 0.2330.233 +540+540 mean 0.3610.361 0.5880.588 gain +0.227+0.227, null +0.070+0.070 Speak 1 0.2090.209 0.5250.525 −620-620 2 0.2460.246 0.4650.465 +600+600 3 −0.352-0.352 0.4350.435 −500-500 4 0.1440.144 0.2000.200 +680+680 5 0.2140.214 0.2950.295 +720+720 6 0.3580.358 0.4020.402 −60-60 7 0.3370.337 0.3790.379 −620-620 8 0.1560.156 0.2220.222 −560-560 9 0.0600.060 0.1020.102 +680+680 10 0.4520.452 0.5490.549 +640+640 11 0.1720.172 0.1920.192 +540+540 12 −0.078-0.078 0.1210.121 −440-440 mean 0.1600.160 0.3240.324 gain +0.164+0.164, null +0.054+0.054 Rotate 1 −0.291-0.291 0.3290.329 −380-380 2 0.2840.284 0.3560.356 −820-820 3 −0.387-0.387 0.4970.497 −380-380 4 0.2890.289 0.4200.420 −720-720 5 0.3100.310 0.3270.327 −860-860 6 0.2730.273 0.2850.285 +40+40 7 0.3120.312 0.3680.368 −680-680 8 −0.029-0.029 0.0720.072 −900-900 9 0.1840.184 0.2340.234 +720+720 10 0.4580.458 0.5070.507 −40-40 11 0.0410.041 0.0460.046 +540+540 12 0.1050.105 0.1600.160 +960+960 mean 0.1290.129 0.3000.300 gain +0.171+0.171, null +0.083+0.083 Bike level 1 1 −0.022-0.022 0.2720.272 +960+960 2 0.4330.433 0.4790.479 +40+40 3 −0.018-0.018 0.0820.082 −520-520 4 0.3510.351 0.4030.403 +60+60 5 0.0670.067 0.0750.075 +680+680 6 0.1610.161 0.2680.268 +640+640 7 −0.020-0.020 0.2550.255 −640-640 8 0.0470.047 0.0990.099 −900-900 9 0.0790.079 0.0810.081 −20-20 10 −0.127-0.127 0.2540.254 −400-400 11 −0.006-0.006 0.0860.086 −720-720 12 0.1960.196 0.4630.463 +140+140 mean 0.0950.095 0.2350.235 gain +0.140+0.140, null +0.046+0.046 Bike level 3 1 −0.166-0.166 0.1770.177 +280+280 2 0.4370.437 0.4430.443 −20-20 3 −0.018-0.018 0.1660.166 −580-580 4 0.0730.073 0.1930.193 +560+560 5 −0.035-0.035 0.1070.107 −240-240 6 0.0860.086 0.1200.120 −760-760 7 0.2050.205 0.2730.273 −700-700 8 −0.005-0.005 0.1340.134 −240-240 9 −0.023-0.023 0.0950.095 −900-900 10 0.4360.436 0.4850.485 +520+520 11 0.0300.030 0.0660.066 −420-420 12 0.3970.397 0.5280.528 +80+80 mean 0.1180.118 0.2320.232 gain +0.114+0.114, null +0.040+0.040 Bike level 5 1 −0.085-0.085 0.1510.151 −400-400 2 0.3980.398 0.4350.435 +700+700 3 0.4280.428 0.4850.485 +60+60 4 0.3670.367 0.3990.399 +40+40 5 0.1010.101 0.1280.128 −80-80 6 0.4330.433 0.4620.462 +40+40 7 0.3000.300 0.3940.394 −700-700 8 0.0450.045 0.1020.102 +1000+1000 9 0.5700.570 0.5860.586 +20+20 10 0.0990.099 0.1650.165 +1000+1000 11 0.0500.050 0.1870.187 +440+440 12 0.0250.025 0.3390.339 +180+180 mean 0.2280.228 0.3190.319 gain +0.092+0.092, null +0.039+0.039 UBFC-rPPG 1 0.6790.679 0.6880.688 +33+33 2 0.8720.872 0.9080.908 −33-33 3 0.8740.874 0.8740.874 +0+0 4 0.8960.896 0.8960.896 +0+0 5 0.4970.497 0.7140.714 +67+67 6 0.8020.802 0.8020.802 +0+0 7 0.8890.889 0.8890.889 +0+0 8 0.4820.482 0.7340.734 +733+733 9 0.8500.850 0.8500.850 +0+0 10 0.8340.834 0.8340.834 +0+0 11 0.9090.909 0.9090.909 +0+0 12 0.8380.838 0.8380.838 +0+0 mean 0.7850.785 0.8280.828 gain +0.043+0.043, null +0.110+0.110 Why the window was not lengthened. Table 19 shows the error falling as the window grows, which invites the question why 5.12 s was kept. Stability alone cannot answer it, because a longer window always steadies a spectral estimate and the steadiest window is the whole recording, which returns one number for a quantity that moved. The window has to be judged against a target that needs no window, and the beat-to-beat heart rate of the reference photoplethysmogram, taken from its systolic-peak intervals, is one. Against that target the error splits in two, as Table 21 reports. The estimation part, the spectral estimate minus the mean reference over the same window, falls as more cycles are averaged. The representation part, that window mean minus the instantaneous reference at the window centre, grows as the heart rate moves inside the window. Their sum has a minimum, and that minimum is the optimal window. On Static level 3 the estimation error falls from 2.62 to 0.84 beats per minute between 5.12 and 61.44 s while the representation error rises from 3.42 to 4.27, and the total is least at 7.68 s. The curve is shallow: 3.34 beats per minute there against 3.60 at 5.12 s, a penalty of 7 % for the clip the manuscript uses. We keep 5.12 s because it is what makes the UBFC-rPPG row of Table 4 comparable at all, its clip being 5.33 s, and 7 % of a quantity that already carries a standard error of 0.34 beats per minute is not worth losing that comparison for. Two limits of this decomposition are worth stating. The cycling minima, 10.24 s on Bike level 3 and 8.96 on Bike level 1, sit on curves that vary by 17 and 13 % of their own minimum across a twelvefold range of window length, and at 3.7 and 4.8 times the Static level 3 minimum of 3.34 beats per minute, because there the beat-to-beat reference is itself motion-corrupted; no window is good under cycling, and the limitation is the reference rather than the window. The instantaneous target also carries respiratory sinus arrhythmia and detection jitter, which inflate the representation term and bias the optimum short, so 7.68 s should be read as a lower bound on the best window rather than a precise value. Table 21: Root mean square error of the spectral heart rate against the beat-to-beat heart rate of the reference photoplethysmogram, in beats per minute, decomposed by window length. Estimation is the spectral estimate minus the mean reference over the same window and falls with length; representation is that window mean minus the instantaneous reference at the window centre and grows with it; total is the error against the instantaneous reference and is least at the row in bold. Windows advance one second at a time and every value is the mean over the 12 held-out participants. Scenario Window (s) Estimation Representation Total Static Level 3 5.125.12 2.622.62 3.423.42 3.603.60 6.406.40 2.612.61 3.333.33 3.603.60 7.687.68 1.961.96 3.563.56 3.343.34 8.968.96 2.112.11 3.693.69 3.383.38 10.2410.24 1.781.78 4.244.24 4.284.28 11.5211.52 2.032.03 4.244.24 4.264.26 12.8012.80 1.551.55 4.104.10 4.304.30 15.3615.36 1.461.46 4.364.36 4.434.43 20.4820.48 1.311.31 4.514.51 4.504.50 30.7230.72 0.990.99 4.564.56 4.684.68 40.9640.96 0.890.89 5.015.01 5.125.12 61.4461.44 0.840.84 4.274.27 4.254.25 Bike Level 3 5.125.12 9.489.48 9.399.39 12.9312.93 6.406.40 9.909.90 9.729.72 13.4113.41 7.687.68 9.589.58 10.0610.06 13.2713.27 8.968.96 9.869.86 9.429.42 12.8612.86 10.2410.24 8.818.81 9.389.38 12.2312.23 11.5211.52 8.918.91 10.3910.39 14.3214.32 12.8012.80 8.428.42 10.2510.25 13.1913.19 15.3615.36 8.158.15 9.789.78 12.9112.91 20.4820.48 7.637.63 9.979.97 12.6612.66 30.7230.72 6.936.93 10.7510.75 13.2013.20 40.9640.96 6.756.75 10.8310.83 12.7312.73 61.4461.44 6.366.36 11.5911.59 13.1213.12 Bike Level 1 5.125.12 13.3813.38 10.2610.26 16.2016.20 6.406.40 13.6413.64 10.6610.66 16.6716.67 7.687.68 12.8912.89 10.7510.75 16.6316.63 8.968.96 13.1613.16 10.4510.45 16.0616.06 10.2410.24 12.9312.93 11.7811.78 17.5617.56 11.5211.52 12.9912.99 10.8810.88 17.0917.09 12.8012.80 12.5712.57 11.5811.58 17.4917.49 15.3615.36 12.4912.49 11.6211.62 17.0617.06 20.4820.48 12.3612.36 11.9211.92 17.3117.31 30.7230.72 12.0912.09 12.0812.08 17.6717.67 40.9640.96 12.3412.34 12.3712.37 18.1818.18 61.4461.44 11.9611.96 12.4112.41 18.1018.10 Table 22 gives all 252 participant-averaged Spearman estimates from the arrays plotted in Figure 9. Table 22: Two-sided Spearman summaries for all eight NCKU-rPPG scenarios and UBFC-rPPG, taken within each participant and averaged over participants. Each cell gives ρ;pρ;p; coefficients are displayed to two decimal places, and p values to three. The seven relationships pair each of the three clip-level performance measures, heart-rate error, waveform correlation and signal-to-noise ratio, with each of the two attribution summaries, and add SaCo against skin coverage. Cells in bold satisfy both p<0.05p<0.05 and |ρ|≥0.10|ρ|≥ 0.10, the condition Table 9 counts as directional. The first sample-size pair in each condition label gives the number of skin-only clips followed by the number of clips in relationships involving SaCo; the second gives the participants contributing to each. Those differ because the SaCo run does not cover every held-out participant: it reaches nine of twelve in the five stationary conditions and only three of twelve in the three cycling conditions, so a SaCo relationship there is averaged over three people and should be read accordingly. Condition Method MAE–skin MAE–SaCo Pearson–skin Pearson–SaCo SNR–skin SNR–SaCo SaCo–skin Static level 1 (n=289/200n=289/200) (P=12/9P=12/9) raw −0.02;.709-0.02;.709 −0.19;.013-0.19;.013 0.14;.0460.14;.046 0.06;.5220.06;.522 0.06;.2940.06;.294 0.13;.1140.13;.114 0.06;.5960.06;.596 rollout −0.07;.295-0.07;.295 −0.11;.063-0.11;.063 0.03;.5670.03;.567 0.00;.9850.00;.985 0.06;.2960.06;.296 0.03;.6580.03;.658 0.04;.5060.04;.506 flow 0.03;.4140.03;.414 −0.03;.829-0.03;.829 0.11;.1660.11;.166 0.04;.5630.04;.563 0.08;.0720.08;.072 0.00;.9970.00;.997 0.03;.7850.03;.785 BI 0.14;.1480.14;.148 0.14;.0800.14;.080 −0.09;.443-0.09;.443 −0.22;.019-0.22;.019 −0.17;.118-0.17;.118 −0.01;.922-0.01;.922 0.11;.1450.11;.145 Static level 3 (n=289/200n=289/200) (P=12/9P=12/9) raw −0.14;.020-0.14;.020 0.01;.9530.01;.953 0.13;.0510.13;.051 0.11;.2930.11;.293 −0.06;.390-0.06;.390 0.02;.7940.02;.794 0.09;.4270.09;.427 rollout 0.04;.5080.04;.508 −0.02;.821-0.02;.821 0.02;.7500.02;.750 0.12;.2840.12;.284 0.00;.9700.00;.970 0.01;.8740.01;.874 0.03;.8530.03;.853 flow 0.09;.1110.09;.111 −0.00;.944-0.00;.944 0.05;.4220.05;.422 0.02;.8640.02;.864 −0.05;.612-0.05;.612 0.04;.7640.04;.764 −0.13;.125-0.13;.125 BI 0.06;.5130.06;.513 −0.19;.344-0.19;.344 −0.16;.086-0.16;.086 0.02;.8090.02;.809 −0.08;.218-0.08;.218 0.19;.0440.19;.044 −0.13;.264-0.13;.264 Static level 5 (n=289/200n=289/200) (P=12/9P=12/9) raw −0.02;.725-0.02;.725 −0.02;.694-0.02;.694 0.11;.0610.11;.061 0.00;.9760.00;.976 0.02;.7980.02;.798 0.13;.2130.13;.213 0.04;.4640.04;.464 rollout −0.08;.213-0.08;.213 0.06;.4920.06;.492 0.07;.2080.07;.208 −0.02;.834-0.02;.834 0.06;.2910.06;.291 0.06;.5530.06;.553 −0.07;.482-0.07;.482 flow −0.00;.974-0.00;.974 0.05;.4340.05;.434 −0.00;.961-0.00;.961 −0.14;.147-0.14;.147 −0.07;.247-0.07;.247 −0.00;.996-0.00;.996 −0.12;.179-0.12;.179 BI −0.11;.091-0.11;.091 −0.07;.165-0.07;.165 0.02;.7930.02;.793 0.14;.1560.14;.156 0.06;.3900.06;.390 0.18;.0350.18;.035 0.17;.0310.17;.031 Speak (n=288/200n=288/200) (P=12/9P=12/9) raw 0.05;.5520.05;.552 −0.11;.450-0.11;.450 −0.02;.690-0.02;.690 0.05;.5350.05;.535 0.07;.1060.07;.106 0.02;.8190.02;.819 −0.18;.092-0.18;.092 rollout 0.04;.4540.04;.454 −0.11;.398-0.11;.398 0.15;.0420.15;.042 0.07;.3020.07;.302 −0.03;.599-0.03;.599 0.03;.7760.03;.776 −0.01;.845-0.01;.845 flow 0.02;.7890.02;.789 0.03;.6760.03;.676 0.19;.0500.19;.050 0.07;.4250.07;.425 −0.04;.638-0.04;.638 −0.02;.841-0.02;.841 −0.04;.556-0.04;.556 BI 0.24;<−0.24;<10^-3 0.04;.5750.04;.575 −0.19;.038-0.19;.038 −0.02;.832-0.02;.832 −0.25;<−-0.25;<10^-3 −0.13;.329-0.13;.329 0.17;.1860.17;.186 Rotate (n=287/200n=287/200) (P=12/9P=12/9) raw −0.05;.456-0.05;.456 0.12;.1420.12;.142 −0.00;.973-0.00;.973 −0.16;.063-0.16;.063 0.03;.7070.03;.707 −0.02;.794-0.02;.794 0.03;.6170.03;.617 rollout −0.22;.023-0.22;.023 0.06;.4570.06;.457 0.16;.0820.16;.082 −0.10;.204-0.10;.204 0.13;.1430.13;.143 0.04;.6250.04;.625 −0.15;.132-0.15;.132 flow −0.05;.498-0.05;.498 −0.05;.517-0.05;.517 0.00;.9740.00;.974 −0.02;.731-0.02;.731 0.03;.7770.03;.777 0.05;.4440.05;.444 0.09;.3390.09;.339 BI 0.07;.4210.07;.421 0.08;.2850.08;.285 −0.09;.250-0.09;.250 −0.22;.005-0.22;.005 0.02;.7810.02;.781 −0.18;.041-0.18;.041 0.22;.0200.22;.020 Bike level 1 (n=853/200n=853/200) (P=12/3P=12/3) raw 0.01;.9100.01;.910 0.02;.6710.02;.671 −0.01;.824-0.01;.824 −0.04;.789-0.04;.789 0.02;.6700.02;.670 0.10;.3240.10;.324 −0.01;.899-0.01;.899 rollout −0.00;.941-0.00;.941 0.03;.6020.03;.602 −0.04;.235-0.04;.235 −0.02;.894-0.02;.894 0.01;.9130.01;.913 0.08;.3550.08;.355 −0.00;.939-0.00;.939 flow −0.05;.160-0.05;.160 0.05;.7310.05;.731 −0.00;.938-0.00;.938 −0.07;.717-0.07;.717 0.02;.5130.02;.513 −0.02;.906-0.02;.906 0.03;.5980.03;.598 BI 0.24;<−0.24;<10^-3 0.04;.7880.04;.788 −0.10;.027-0.10;.027 −0.07;.451-0.07;.451 −0.12;.011-0.12;.011 0.05;.7150.05;.715 0.29;.2060.29;.206 Bike level 3 (n=853/200n=853/200) (P=12/3P=12/3) raw −0.01;.762-0.01;.762 −0.03;.857-0.03;.857 0.00;.9930.00;.993 0.08;.5620.08;.562 0.08;.0820.08;.082 0.08;.2790.08;.279 −0.06;.703-0.06;.703 rollout −0.07;.163-0.07;.163 −0.01;.931-0.01;.931 0.06;.1130.06;.113 0.09;.4230.09;.423 0.07;.1340.07;.134 0.11;.1180.11;.118 −0.07;.064-0.07;.064 flow −0.04;.599-0.04;.599 0.13;.0230.13;.023 0.04;.2610.04;.261 −0.05;.797-0.05;.797 0.02;.7150.02;.715 −0.06;.394-0.06;.394 0.11;.5860.11;.586 BI 0.06;.2530.06;.253 −0.00;.936-0.00;.936 −0.05;.207-0.05;.207 −0.05;.591-0.05;.591 −0.01;.736-0.01;.736 −0.03;.475-0.03;.475 −0.05;.300-0.05;.300 Bike level 5 (n=853/200n=853/200) (P=12/3P=12/3) raw −0.01;.727-0.01;.727 −0.05;.015-0.05;.015 0.02;.6010.02;.601 0.02;.8870.02;.887 −0.03;.424-0.03;.424 0.03;.4050.03;.405 0.03;.5970.03;.597 rollout 0.12;.0120.12;.012 −0.05;.102-0.05;.102 −0.03;.617-0.03;.617 0.07;.4390.07;.439 −0.10;.029-0.10;.029 0.01;.7900.01;.790 0.15;.5550.15;.555 flow −0.02;.651-0.02;.651 −0.06;.592-0.06;.592 0.02;.7010.02;.701 0.13;.0210.13;.021 −0.01;.788-0.01;.788 0.05;.5120.05;.512 0.09;.4450.09;.445 BI 0.11;.0090.11;.009 −0.02;.603-0.02;.603 0.02;.5700.02;.570 −0.02;.851-0.02;.851 −0.09;.101-0.09;.101 −0.04;.143-0.04;.143 0.17;.4170.17;.417 UBFC-rPPG (n=141/141n=141/141) (P=12/12P=12/12) raw −0.05;.714-0.05;.714 −0.01;.853-0.01;.853 −0.12;.372-0.12;.372 0.01;.9140.01;.914 −0.08;.471-0.08;.471 −0.03;.821-0.03;.821 −0.08;.656-0.08;.656 rollout −0.09;.405-0.09;.405 −0.05;.490-0.05;.490 0.32;.0470.32;.047 0.06;.4950.06;.495 0.15;.1130.15;.113 0.01;.9310.01;.931 0.01;.8690.01;.869 flow −0.07;.326-0.07;.326 0.02;.8480.02;.848 0.21;.0410.21;.041 0.06;.5330.06;.533 −0.01;.932-0.01;.932 −0.03;.796-0.03;.796 0.14;.0660.14;.066 BI −0.20;.073-0.20;.073 −0.02;.800-0.02;.800 0.25;.0620.25;.062 0.05;.6590.05;.659 0.13;.1450.13;.145 0.04;.7730.04;.773 0.28;.0070.28;.007 Table 22 is mostly empty of resolved relationships, and that is its result. Twenty-five of the 252 coefficients are directional, between one and five in each of the nine conditions, and the largest of them reaches ρ=+0.32ρ=+0.32. No condition resolves more than five, no relationship resolves in more than six of its 36 method–condition pairs, and Beyond Intuition supplies 14 of the 25, more than the three attention-only methods together, which give 3, 4 and 4. Within a participant, then, neither skin coverage nor SaCo predicts how well the model does on a given clip in any condition, which supports the conclusion drawn from the scenario-level comparison in Section 4.4 on evidence of a different kind. (a) MAE–skin coverage. Figure 9: Combined descriptive scatter plots for raw attention, attention rollout, attention flow, and Beyond Intuition across all eight NCKU-rPPG scenarios and UBFC-rPPG. NCKU-rPPG clips use circular markers and UBFC-rPPG clips use crosses; no observations are sampled, and all panels use linear axes that retain the complete data ranges. Each method panel contains one ordinary least-squares line fitted to all displayed clips, which is a visual aid only. Each panel is annotated with the same estimator the tables report, the Spearman correlation taken inside a participant and averaged through the Fisher transform, with a one-sample t test over participants. The panel pools all nine conditions, treating each participant of each condition as its own group, so its coefficient summarises the whole display rather than reproducing any single row of Table 6 or Supplementary Table 22; restricted to one condition the identical estimator reproduces that table’s entry exactly. Colours denote Static level 1 (blue, #0173B2), Static level 3 (orange, #DE8F05), Static level 5 (green, #029E73), Speak (red-orange, #D55E00), Rotate (purple, #C78BC), Bike level 1 (brown, #CA9161), Bike level 3 (pink, #FBAFE4), Bike level 5 (light blue, #56B4E9), and UBFC-rPPG (dark grey, #4D4D4D). Table 6 reports the Static level 3 and UBFC-rPPG Spearman summaries, and Supplementary Table 22 reports all nine groups. (a) MAE–SaCo. Figure 10: Combined NCKU-rPPG and UBFC-rPPG descriptive scatter plots (continued). (a) Waveform Pearson–skin coverage. Figure 11: Combined NCKU-rPPG and UBFC-rPPG descriptive scatter plots (continued). (a) Waveform Pearson–SaCo. Figure 12: Combined NCKU-rPPG and UBFC-rPPG descriptive scatter plots (continued). (a) SNR–skin coverage. Figure 13: Combined NCKU-rPPG and UBFC-rPPG scatter plots (continued). (a) SNR–SaCo. Figure 14: Combined NCKU-rPPG and UBFC-rPPG scatter plots (continued). (a) SaCo–skin coverage. Figure 15: Combined NCKU-rPPG and UBFC-rPPG scatter plots (continued). Table 23 gives the complete cross-method skin-coverage summary. Each row reports the pooled pre-attention and refined-attention means for one scenario and attribution method, together with their difference. Table 23: Mean skin coverage pooled over all held-out clips and TPT1–TPT3 for the three attention-only attribution methods, as estimate ± its standard error across the 12 test participants of the scenario. The difference is between the two means of the same clips and therefore carries its own, smaller standard error. Scenario Method Pre Refined Difference Static level 1 Raw Attention 0.345±0.0220.345± 0.022 0.350±0.0120.350± 0.012 0.004±0.0190.004± 0.019 Static level 1 Attention Rollout 0.374±0.0270.374± 0.027 0.359±0.0250.359± 0.025 −0.015±0.011-0.015± 0.011 Static level 1 Attention Flow 0.343±0.0270.343± 0.027 0.331±0.0180.331± 0.018 −0.012±0.011-0.012± 0.011 Static level 3 Raw Attention 0.398±0.0310.398± 0.031 0.416±0.0330.416± 0.033 0.018±0.0180.018± 0.018 Static level 3 Attention Rollout 0.465±0.0390.465± 0.039 0.465±0.0380.465± 0.038 0.0004±0.00680.0004± 0.0068 Static level 3 Attention Flow 0.474±0.0380.474± 0.038 0.470±0.0290.470± 0.029 −0.005±0.014-0.005± 0.014 Static level 5 Raw Attention 0.359±0.0300.359± 0.030 0.500±0.0200.500± 0.020 0.141±0.0190.141± 0.019 Static level 5 Attention Rollout 0.487±0.0320.487± 0.032 0.461±0.0320.461± 0.032 −0.0256±0.0080-0.0256± 0.0080 Static level 5 Attention Flow 0.478±0.0270.478± 0.027 0.431±0.0220.431± 0.022 −0.047±0.010-0.047± 0.010 Speak Raw Attention 0.397±0.0150.397± 0.015 0.483±0.0330.483± 0.033 0.086±0.0260.086± 0.026 Speak Attention Rollout 0.576±0.0350.576± 0.035 0.557±0.0330.557± 0.033 −0.019±0.019-0.019± 0.019 Speak Attention Flow 0.587±0.0330.587± 0.033 0.521±0.0290.521± 0.029 −0.066±0.015-0.066± 0.015 Rotate Raw Attention 0.402±0.0180.402± 0.018 0.469±0.0130.469± 0.013 0.067±0.0110.067± 0.011 Rotate Attention Rollout 0.473±0.0240.473± 0.024 0.391±0.0160.391± 0.016 −0.0818±0.0094-0.0818± 0.0094 Rotate Attention Flow 0.410±0.0230.410± 0.023 0.348±0.0120.348± 0.012 −0.062±0.013-0.062± 0.013 Bike level 1 Raw Attention 0.391±0.0160.391± 0.016 0.496±0.0230.496± 0.023 0.105±0.0150.105± 0.015 Bike level 1 Attention Rollout 0.443±0.0300.443± 0.030 0.324±0.0210.324± 0.021 −0.118±0.019-0.118± 0.019 Bike level 1 Attention Flow 0.358±0.0260.358± 0.026 0.295±0.0200.295± 0.020 −0.062±0.014-0.062± 0.014 Bike level 3 Raw Attention 0.375±0.0150.375± 0.015 0.411±0.0230.411± 0.023 0.036±0.0120.036± 0.012 Bike level 3 Attention Rollout 0.622±0.0240.622± 0.024 0.619±0.0270.619± 0.027 −0.003±0.019-0.003± 0.019 Bike level 3 Attention Flow 0.602±0.0280.602± 0.028 0.563±0.0280.563± 0.028 −0.039±0.017-0.039± 0.017 Bike level 5 Raw Attention 0.409±0.0230.409± 0.023 0.529±0.0210.529± 0.021 0.1195±0.00870.1195± 0.0087 Bike level 5 Attention Rollout 0.467±0.0270.467± 0.027 0.339±0.0270.339± 0.027 −0.128±0.013-0.128± 0.013 Bike level 5 Attention Flow 0.421±0.0290.421± 0.029 0.351±0.0210.351± 0.021 −0.070±0.012-0.070± 0.012 Read across all nine conditions, Table 23 qualifies the contrast drawn in Section 4.2. Refinement raises the skin coverage of raw attention in every one of the eight NCKU-rPPG scenarios, from +0.004+0.004 on Static level 1 to +0.141+0.141 on Static level 5, and lowers that of attention flow in every one, from −0.005-0.005 to −0.070-0.070; attention rollout falls in seven of the eight, and the three directions are the same on UBFC-rPPG, where the changes are +0.224+0.224, −0.117-0.117 and −0.167-0.167. What separates the two evaluations is therefore magnitude and not direction: every one of the three changes is smaller than twice its standard error only in Static levels 1 and 3, and in Bike level 5 they reach about 14 and 10 times it for raw attention and attention rollout, so the refinement step tracks how difficult the condition is rather than which dataset it belongs to. Table 24 gives the TPT-level mean and standard deviation values. Each row covers one scenario, attribution method, and attention type, with one column per Temporal Periodic Transformer level. Table 24: Clip-level skin coverage as mean ± standard deviation by scenario, attribution method, attention type, and Temporal Periodic Transformer (TPT) level. Scenario Method Type TPT1 TPT2 TPT3 Static level 1 Raw Attention Pre-Attention 0.385±0.0760.385± 0.076 0.294±0.0890.294± 0.089 0.356±0.0950.356± 0.095 Static level 1 Raw Attention Refined Attention 0.338±0.0670.338± 0.067 0.163±0.0450.163± 0.045 0.549±0.0830.549± 0.083 Static level 1 Attention Rollout Pre-Attention 0.314±0.0860.314± 0.086 0.296±0.0910.296± 0.091 0.51±0.120.51± 0.12 Static level 1 Attention Rollout Refined Attention 0.348±0.0940.348± 0.094 0.363±0.0860.363± 0.086 0.367±0.0810.367± 0.081 Static level 1 Attention Flow Pre-Attention 0.34±0.110.34± 0.11 0.336±0.0830.336± 0.083 0.352±0.0830.352± 0.083 Static level 1 Attention Flow Refined Attention 0.345±0.0880.345± 0.088 0.330±0.0510.330± 0.051 0.317±0.0570.317± 0.057 Static level 3 Raw Attention Pre-Attention 0.37±0.120.37± 0.12 0.368±0.0950.368± 0.095 0.46±0.120.46± 0.12 Static level 3 Raw Attention Refined Attention 0.40±0.150.40± 0.15 0.29±0.100.29± 0.10 0.57±0.120.57± 0.12 Static level 3 Attention Rollout Pre-Attention 0.39±0.140.39± 0.14 0.43±0.140.43± 0.14 0.58±0.130.58± 0.13 Static level 3 Attention Rollout Refined Attention 0.44±0.140.44± 0.14 0.47±0.130.47± 0.13 0.49±0.120.49± 0.12 Static level 3 Attention Flow Pre-Attention 0.46±0.140.46± 0.14 0.46±0.130.46± 0.13 0.50±0.130.50± 0.13 Static level 3 Attention Flow Refined Attention 0.48±0.130.48± 0.13 0.464±0.0970.464± 0.097 0.463±0.0710.463± 0.071 Static level 5 Raw Attention Pre-Attention 0.37±0.140.37± 0.14 0.31±0.100.31± 0.10 0.398±0.0730.398± 0.073 Static level 5 Raw Attention Refined Attention 0.55±0.100.55± 0.10 0.46±0.100.46± 0.10 0.485±0.0990.485± 0.099 Static level 5 Attention Rollout Pre-Attention 0.41±0.100.41± 0.10 0.48±0.110.48± 0.11 0.57±0.130.57± 0.13 Static level 5 Attention Rollout Refined Attention 0.446±0.0950.446± 0.095 0.47±0.110.47± 0.11 0.47±0.120.47± 0.12 Static level 5 Attention Flow Pre-Attention 0.487±0.0840.487± 0.084 0.470±0.0980.470± 0.098 0.48±0.100.48± 0.10 Static level 5 Attention Flow Refined Attention 0.493±0.0860.493± 0.086 0.415±0.0840.415± 0.084 0.384±0.0610.384± 0.061 Speak Raw Attention Pre-Attention 0.474±0.0880.474± 0.088 0.364±0.0640.364± 0.064 0.354±0.0670.354± 0.067 Speak Raw Attention Refined Attention 0.40±0.130.40± 0.13 0.50±0.120.50± 0.12 0.55±0.130.55± 0.13 Speak Attention Rollout Pre-Attention 0.56±0.140.56± 0.14 0.58±0.130.58± 0.13 0.59±0.120.59± 0.12 Speak Attention Rollout Refined Attention 0.54±0.120.54± 0.12 0.56±0.120.56± 0.12 0.56±0.120.56± 0.12 Speak Attention Flow Pre-Attention 0.61±0.130.61± 0.13 0.59±0.120.59± 0.12 0.57±0.110.57± 0.11 Speak Attention Flow Refined Attention 0.57±0.120.57± 0.12 0.52±0.110.52± 0.11 0.468±0.0900.468± 0.090 Rotate Raw Attention Pre-Attention 0.440±0.0780.440± 0.078 0.40±0.150.40± 0.15 0.366±0.0700.366± 0.070 Rotate Raw Attention Refined Attention 0.320±0.0730.320± 0.073 0.57±0.110.57± 0.11 0.518±0.0910.518± 0.091 Rotate Attention Rollout Pre-Attention 0.35±0.100.35± 0.10 0.46±0.130.46± 0.13 0.60±0.120.60± 0.12 Rotate Attention Rollout Refined Attention 0.321±0.0710.321± 0.071 0.42±0.100.42± 0.10 0.44±0.100.44± 0.10 Rotate Attention Flow Pre-Attention 0.326±0.0990.326± 0.099 0.41±0.110.41± 0.11 0.49±0.110.49± 0.11 Rotate Attention Flow Refined Attention 0.336±0.0720.336± 0.072 0.373±0.0790.373± 0.079 0.337±0.0700.337± 0.070 Bike level 1 Raw Attention Pre-Attention 0.346±0.0690.346± 0.069 0.467±0.0890.467± 0.089 0.360±0.0490.360± 0.049 Bike level 1 Raw Attention Refined Attention 0.37±0.130.37± 0.13 0.51±0.110.51± 0.11 0.609±0.0830.609± 0.083 Bike level 1 Attention Rollout Pre-Attention 0.29±0.140.29± 0.14 0.45±0.110.45± 0.11 0.593±0.0950.593± 0.095 Bike level 1 Attention Rollout Refined Attention 0.265±0.0820.265± 0.082 0.347±0.0750.347± 0.075 0.361±0.0740.361± 0.074 Bike level 1 Attention Flow Pre-Attention 0.24±0.110.24± 0.11 0.364±0.0960.364± 0.096 0.468±0.0930.468± 0.093 Bike level 1 Attention Flow Refined Attention 0.225±0.0970.225± 0.097 0.316±0.0790.316± 0.079 0.345±0.0610.345± 0.061 Bike level 3 Raw Attention Pre-Attention 0.496±0.0740.496± 0.074 0.297±0.0460.297± 0.046 0.331±0.0650.331± 0.065 Bike level 3 Raw Attention Refined Attention 0.41±0.140.41± 0.14 0.39±0.110.39± 0.11 0.44±0.120.44± 0.12 Bike level 3 Attention Rollout Pre-Attention 0.66±0.100.66± 0.10 0.588±0.0630.588± 0.063 0.62±0.110.62± 0.11 Bike level 3 Attention Rollout Refined Attention 0.63±0.110.63± 0.11 0.615±0.0920.615± 0.092 0.616±0.0900.616± 0.090 Bike level 3 Attention Flow Pre-Attention 0.63±0.130.63± 0.13 0.590±0.0880.590± 0.088 0.582±0.0790.582± 0.079 Bike level 3 Attention Flow Refined Attention 0.60±0.130.60± 0.13 0.588±0.0910.588± 0.091 0.501±0.0920.501± 0.092 Bike level 5 Raw Attention Pre-Attention 0.345±0.0610.345± 0.061 0.417±0.0870.417± 0.087 0.47±0.120.47± 0.12 Bike level 5 Raw Attention Refined Attention 0.487±0.0890.487± 0.089 0.514±0.0880.514± 0.088 0.586±0.0740.586± 0.074 Bike level 5 Attention Rollout Pre-Attention 0.31±0.130.31± 0.13 0.48±0.110.48± 0.11 0.611±0.0970.611± 0.097 Bike level 5 Attention Rollout Refined Attention 0.28±0.110.28± 0.11 0.36±0.110.36± 0.11 0.38±0.110.38± 0.11 Bike level 5 Attention Flow Pre-Attention 0.32±0.140.32± 0.14 0.42±0.110.42± 0.11 0.52±0.110.52± 0.11 Bike level 5 Attention Flow Refined Attention 0.32±0.100.32± 0.10 0.360±0.0760.360± 0.076 0.373±0.0720.373± 0.072 Table 24 shows that the two families of method acquire their depth dependence at opposite stages. Pre-attention skin coverage rises with Temporal Periodic Transformer level for attention rollout in seven of the eight scenarios and for attention flow in five, by as much as 0.30 from TPT1 to TPT3 on Bike level 1, and refinement then compresses that gradient to at most 0.12. Raw attention behaves the other way: its pre-attention coverage carries no consistent depth trend, rising in four scenarios and falling in four, and a rising one appears after refinement in seven of the eight, reaching +0.24+0.24 from TPT1 to TPT3 on Bike level 1 against +0.01+0.01 before it. Pooling the three levels, as Table 5 and Table 23 do, therefore averages over a gradient that runs in opposite directions for raw attention and for the two propagation-based methods. Static level 1 is the condition where the attributions are least consistent with the rest and refinement changes them least. Figure 16 shows the attribution maps before and after refinement, which move the pooled skin coverage of raw attention by +0.004±0.019+0.004± 0.019, of attention rollout by −0.015±0.011-0.015± 0.011 and of attention flow by −0.012±0.011-0.012± 0.011. All three changes are smaller than twice their standard error, so this and Static level 3 are the only two NCKU-rPPG conditions in which refinement is not resolved for any attention-only method. Figure 17 nonetheless separates the four methods sharply: Beyond Intuition is the only one with a median SaCo below zero, −0.178-0.178, and its median refined skin coverage of 0.1800.180 is the lowest of any condition, against 0.34–0.38 for the three attention-only methods. Raw attention gives the heart-rate error against SaCo at ρ=−0.19ρ=-0.19 (p=.013p=.013) and the waveform correlation against skin coverage at ρ=+0.14ρ=+0.14 (p=.046p=.046), both in the direction a reliability indicator would take, while Beyond Intuition gives the waveform correlation against SaCo at ρ=−0.22ρ=-0.22 (p=.019p=.019), which runs opposite to it. Figure 16: Static level 1 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Static level 1 test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 17: Static level 1 clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Static level 5 is where refinement changes the attention-only methods most among the static conditions. Figure 18 shows the maps that produce that change: raw attention gains 0.141±0.0190.141± 0.019 of skin coverage between the two stages, more than seven times its standard error, while attention rollout loses 0.026±0.0080.026± 0.008 and attention flow 0.047±0.0100.047± 0.010. Figure 19 places the median SaCo of the four methods between 0.416 for attention flow and 0.708 for Beyond Intuition, and their median refined skin coverage between 0.435 and 0.779 in the same order. Figure 18: Static level 5 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Static level 5 test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 19: Static level 5 clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Figure 20 shows the attribution maps for speaking, where refinement raises the skin coverage of raw attention by 0.086±0.0260.086± 0.026 and lowers that of attention flow by 0.066±0.0150.066± 0.015, while its effect on attention rollout, −0.019±0.019-0.019± 0.019, is not resolved. Figure 21 gives the highest median SaCo of any NCKU-rPPG condition, 0.824–0.863 across the four methods, even though the heart-rate error of 8.9±2.68.9± 2.6 beats per minute is the fifth worst of the eight in Table 4. That pairing is the clearest single illustration of the conclusion in Section 4.4, because a faithfulness score near its maximum accompanies a model that is not working well. Figure 20: Speak attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Speak test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 21: Speak clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Figure 22 shows attribution maps that remain concentrated on the face despite the changing pose, with refinement raising the skin coverage of raw attention by 0.067±0.0110.067± 0.011 and lowering that of attention rollout by 0.082±0.0090.082± 0.009 and attention flow by 0.062±0.0130.062± 0.013. Figure 23 gives median SaCo values of 0.652–0.819, close to those of the static conditions, while Table 4 places rotation second worst of the eight on SNR and fourth worst on heart-rate error. Figure 22: Rotate attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Rotate test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 23: Rotate clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Bike level 1 has the largest heart-rate error of the eight conditions, 18.6±5.518.6± 5.5 beats per minute. Figure 24 shows refinement pulling the attention-only methods apart, raising the skin coverage of raw attention by 0.105±0.0150.105± 0.015 and lowering that of attention rollout by 0.118±0.0190.118± 0.019 and attention flow by 0.062±0.0140.062± 0.014. Figure 25 gives the lowest median SaCo of the three cycling conditions, 0.201 for Beyond Intuition and 0.345 for attention flow, the first being the lowest of any condition except Static level 1. Figure 24: Bike level 1 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Bike level 1 test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 25: Bike level 1 clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Figure 26 shows the maps of the condition in which refinement is nearly inert for attention rollout, at −0.003±0.019-0.003± 0.019, while it moves raw attention by +0.036±0.012+0.036± 0.012 and attention flow by −0.039±0.017-0.039± 0.017. Figure 27 pairs a high median SaCo, 0.665–0.830 across the four methods, with the worst SNR of the eight conditions, −11.9±2.0-11.9± 2.0 dB in Table 4. Figure 26: Bike level 3 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Bike level 3 test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 27: Bike level 3 clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. Figure 28 shows the condition in which refinement moves skin coverage furthest, by +0.119±0.009+0.119± 0.009 for raw attention and −0.128±0.013-0.128± 0.013 for attention rollout, about 14 and 10 times their standard errors, which is visible as a change of map between the two stages. Figure 29 gives median SaCo values of 0.680–0.767 for a condition whose heart-rate error is 15.1±3.515.1± 3.5 beats per minute. Figure 28: Bike level 5 attribution visualisation. The published representative face appears only in the top panels labelled Face, Skin Mask, and Skin Region. The attribution maps below those panels, the average face, and the displayed summary statistics use the complete Bike level 5 test set rather than that representative alone. (a) SaCo. (b) Skin coverage. Figure 29: Bike level 5 clip-level distribution summaries for raw attention, attention rollout, attention flow, and Beyond Intuition. Each SaCo distribution contains clip-level faithfulness values, and each skin-coverage distribution contains clip-level values averaged across TPT1–TPT3 before plotting. S7. Heart-rate estimation settings in the remote photoplethysmography literature since 2020 The evaluation in Section 3.6 fixes a 5.12 s window, a Welch segment capped at 256 samples, and a 45–150 beats-per-minute search band, and Table 19 shows how strongly the reported error depends on the first of those. A reader comparing our numbers with published ones therefore needs to know what the published ones used. We read every article in our literature collection that was published from 2020 onwards and that applies a remote photoplethysmography method, and recorded four things from each: the video frame rate, the length of the window over which one heart rate is computed, how far consecutive windows overlap, and the spectral settings and the estimator that turn the waveform into a heart rate. Table 25 gives one row for each of the 58 articles, and a last row for the present work. Where an article does not state a setting we write that it does not, rather than filling in the value its toolbox would have used. Four things follow from the table. The window is the least standardised of the four settings. It runs from 5 s to 60 s among the articles that state one, with a further article computing a single heart rate over the whole video, and the two commonest choices, a 30 s clip and a 10 s clip, differ by a factor of three; Table 19 measures that factor in our own data: moving from 10.24 to 30.72 s changes the mean absolute error by 0.23 to 1.63 beats per minute across the eight conditions, which is of the same order as the difference between many of the methods being compared. The 5.12 s we use is at the short end of the range but is not alone there: four articles evaluate on 160-frame clips, that is 5.33 s at 30 frames per second. Second, consecutive windows do not overlap in most of the articles, so a longer window buys segment averaging at the cost of fewer independent estimates, exactly as in our sweep. Ten articles overlap their evaluation windows, by between one frame and 97 % of the window, and three more overlap only their training clips. Third, almost every article estimates the heart rate as the largest spectral peak inside a physiological band, most often 0.75–2.5 Hz, and yet only nine of the 58 state the transform length or the window function that produces that spectrum. Since the spectral resolution of an N-point transform at 30 frames per second is 1800/N1800/N beats per minute, a 160-frame clip transformed without zero-padding resolves no better than 11 beats per minute, which is of the same order as the errors being reported. The articles that do state these settings differ widely: a 1024-point transform [33], zero-padding to 2048 points [34], a 256-sample Hanning window zero-padded to 3.3 k [35], and zero-padding chosen to give a resolution of 0.33 beats per minute [36]. This is the setting a reader is least able to recover from a published method section, and it is the one we state explicitly in Section 3.6. Fourth, the frame rate is the one setting on which the field has converged. Fifty of the 58 articles process video at 30 Hz, either as recorded or after resampling to it, and the remaining eight work at 20, 25, 28, 50 or 84 Hz, or state no rate at all. Our recordings are at 50 Hz, so a window of a given number of frames is shorter in seconds here than in most of this literature, which is why we give both throughout. Taken together, the table shows that the settings a heart rate depends on most are the settings least often reported. Seventeen of the 58 articles state no window over which one heart rate is computed, 23 state no overlap between consecutive windows, and four do not identify the spectral method at all. Forty-three read the heart rate from the largest peak of a spectrum, eight derive it from detected beats or regress it directly from the video, three estimate no heart rate at all because they classify atrial fibrillation from the waveform, and four do not say which they do. A published mean absolute error is therefore not comparable with ours, or with another study’s, without the four settings this table records, and for close to a third of the literature they cannot be recovered from the article. That is the reason we state ours in Section 3.6 and vary the one that matters most in Table 19. Table 25: Frame rate, evaluation window, window overlap, spectral settings, and heart-rate estimator of every article in our literature collection published from 2020 onwards that applies a remote photoplethysmography method, ordered by year and then by first author. “Window” is the span of signal from which one heart rate is computed, which is not always the model’s input clip; where the two differ, both are given. “Not stated” means the article does not report the setting, and we did not substitute the default of the toolbox it used. Frame rates written as a→ba→ b mean the recording rate a was resampled to b before processing. Study Frame rate (Hz) Window Overlap Spectral settings Heart-rate estimator Meta-rPPG [37] 30; 61→3061→ 30 60-frame (2 s) model input; heart rate reported over a 30 s clip (frames 306–2135) Not stated Power spectral density; settings not stated Ordinal regression to the waveform, then the spectral peak DeeprPPG [38] 30; 20; 61 30 s clip (frames 306–2135); region-of-interest clips of length T processed with stride T/2T/2 50 % between region-of-interest clips Welch power spectral density with a Hann window Spectral peak MTTS-CAN [39] 120→30120→ 30; 25 30 s; 10-frame temporal window inside the model None Second-order Butterworth, 0.75–2.5 Hz; transform not stated Spectral peak HeartTrack [40] 25; 50 200 frames (8 s) Not stated None Direct convolutional regression of the median heart rate Siamese-rPPG [41] 30; 20 600 frames, that is 30 s at 30 Hz and 20 s at 20 Hz None Fast Fourier transform; settings not stated Spectral peak Multi-task generation [42] 30; 20 600 frames, following Siamese-rPPG None Fast Fourier transform; settings not stated Spectral peak Wang [23] 50 30 s 25 s, that is a 5 s shift Overlap-add with a Hann window of 80 frames for CHROM and POS and 320 frames for ICA Spectral peak; peak intervals rejected as unreliable at low signal-to-noise ratio AutoHR [43] 61→3061→ 30 30 s clip split into three 10 s clips None Power spectral density; settings not stated Spectral peak, averaged over the three clips Dasari et al. [14] 25 10 s Not stated Not stated CHROM, POS, ICA and HR-CNN; estimator not stated ETA-rPPGNet [44] 30; 20; 25 10 s, with 4, 6 and 8 s also reported None Power spectral density; settings not stated Spectral peak Huang et al. [34] 30 512 samples (17.06 s) 497 samples (16.56 s), a hop of 15 samples (0.5 s); zero for one Bland–Altman figure Zero-padded to 2048 points; third-order Butterworth, 30–200 beats per minute Spectral peak tracking Dual-GAN [45] 30 256 frames (8.53 s) Not stated Power spectral density; settings not stated Spectral peak PulseGAN [33] 30; 25–30; 61 10 s sliding window Not stated 1024-point fast Fourier transform in the spectral loss 60 divided by the mean inter-beat interval of the detected peaks Yang et al. [46] 30; about 20 128-frame and 64-frame model clips sampled from a 30 s video; heart-rate window not stated Not stated Fast Fourier transform; settings not stated Spectral peak AND-rPPG [47] 30; 20 Fixed-size clips; length not stated None Fast Fourier transform whose length follows the clip frame rate Spectral peak Lin et al. [48] 60; 30 10 s 5 s, a 5 s stride, with clips summed over the overlap Hamming window; eleventh-order Chebyshev I band-pass at the heart rate ±0.15± 0.15 Hz CHROM spectral peak CDCA-rPPGNet [49] 30 Not stated Not stated Power spectral density after a Butterworth filter, 0.7–2.5 Hz Spectral peak PhysFormer [27] 61→3061→ 30; 60→3060→ 30 30 s video split into three 10 s clips None Power spectral density, also the training target Spectral peak, averaged over the three clips Contrast-Phys [50] 30 30 s None Power spectral density; settings not stated Spectral peak Sun et al. [51] 84 30 s segments, voted over the whole recording None Not applicable Not estimated; the network classifies atrial fibrillation from the waveform Wu et al. [52] 30 Not stated Not stated Magnitude spectrum over the 0.5–3 Hz band, its sorted bin indices used as features CHROM spectral peak plus a learned error-compensation term EfficientPhys [53] 30 The whole video, one heart rate per video None Band-pass 0.75–2.5 Hz (45–150 beats per minute) Peak detection and fast Fourier transform rPPG-Toolbox [6] 30; 25 Non-overlapping chunks of N frames, 180 in the worked example None Second-order Butterworth, 0.75–2.5 Hz (45–150 beats per minute) Fast Fourier transform peak or peak detection LSTC-rPPG [54] 30 160 frames (5.33 s) 130 frames in training, a 30-frame shift; none in testing Power spectral density in the frequency loss Spectral peak PFE and TFA [55] →30→ 30 160 frames (5.33 s) None Not stated Not stated Li and Yin [56] →30→ 30 10 s None Power spectral density; settings not stated Spectral peak, one average pulse rate per clip Nguyen et al. [57] 28–35→2028--35→ 20 Not stated Not stated Fast Fourier transform over 0.75–4 Hz for POS; Butterworth 0.75–2.5 Hz for the learned model Spectral peak Speth et al. [36] 30 120-frame (4 s) training clip; 10 s evaluation window Sliding; step not stated Fast Fourier transform, input zero-padded to a resolution of 0.33 beats per minute Highest spectral peak in 0.66–3 Hz (40–180 beats per minute) Chiu et al. [58] 30 RGB; 60 near-infrared; 15 at night 1024 samples for the Res-MF estimator, that is 34.1 s at 30 Hz One frame Fast Fourier transform in the training loss Res-MF regression network, benchmarked against the spectral peak Wu et al. [59] 30 16 s in the fatigue analysis None, the step equals the window Fast Fourier transform loss during training Ordinal-regression network output, smoothed by a conditional moving average Wu et al. [60] 84; 84→2884→ 28 30 s, with 60 s also reported None Not applicable Not estimated; peak localisation feeds atrial fibrillation classification PhysFormer++ [61] 61→3061→ 30; 60→3060→ 30 30 s video split into three 10 s clips None Power spectral density; settings not stated Spectral peak averaged over the three clips; MATLAB findpeaks for the inter-beat intervals Dual-path TokenLearner [17] 30; 25–30 300 frames (10 s) per spatial-temporal map 9.5 s, one segment every 0.5 s Fast Fourier transform; settings not stated Peak detection and fast Fourier transform per segment, averaged to one heart rate per video DiffPhys [62] 30 Not stated Not stated Not used for the heart rate 60N60N divided by the sum of the detected inter-beat intervals Castellano Ontiveros et al. [63] 25; 30 10 s None Welch power spectral density; segment settings not stated Highest spectral peak CIN-rPPG [21] 30 180 frames in training; 300 frames (10 s) in testing 15 frames in training; none in testing Welch power spectral density via the fast Fourier transform, after a second-order Butterworth filter Spectral peak Cluster-Phys [64] 30 Segment length not stated; one heart rate per video by averaging segments Not stated Power spectral density after a first-order Butterworth filter, 0.75–2.5 Hz (45–150 beats per minute) Spectral peak Temporal normalisation [35] 30; 20 Not stated Not stated Welch with a 256-sample Hanning window, zero-padded to 3.3 k Spectral peak STGNet [65] 30 Not stated Not stated Not stated The network maps the waveform to beats per minute; estimator not stated Wu et al. [66] 84; 30 30 s segment; a sliding window of length w over the inter-beat-interval series Not stated Frequency-domain heart-rate variability indices Mean inter-beat interval from peak detection CbPPGGAN [67] 30 128-frame (4.27 s) model clips, merged before the heart rate is computed 118 frames in training, a 10-frame step; none in testing Fast Fourier transform; settings not stated Most prominent frequency of the merged waveform; SciPy peak detection for the inter-beat intervals Zou et al. [68] 30; 20 Not stated Not stated Power spectrum after a Butterworth filter Highest spectral peak pyMMER [69] Not stated 6 s Overlapping; step not stated Fourier power spectrum; settings not stated Most significant frequency Lifelight [70] Not stated Not stated Not stated Butterworth band-pass, 0.125–4 Hz; no spectral heart-rate step Fiducial points counted on the second derivative of the waveform Chiu et al. [71] 30 Not stated Not stated Spectral peak of the waveform, used to reject implausible R–R intervals Spectral peak, with peak detection for the intervals Cen et al. [72] 30 Not stated Not stated Welch power spectral density after a second-order Butterworth filter Frequency of maximum spectral power Periodic-MAE [73] 30; 25 160 frames (5.33 s) Not stated Power spectral density via the fast Fourier transform Spectral peak CodePhys [74] →30→ 30 30 s video split into three 10 s clips None Power spectral density; settings not stated Spectral peak averaged over the three clips D-rPPGNet [19] 30 30 s None Power spectral density; settings not stated Spectral peak TS-CAN+ [20] 30 Not stated Not stated Not stated The network outputs the heart rate; estimator not stated PhysKANNet [18] 30; 20; 35 Not stated; the sampling interval is half the temporal dimension of the spatial-temporal block Half the block length between samples Power spectral density after a second-order Butterworth filter Spectral peak Tseng et al. [75] 84; 84→3084→ 30 20, 30 and 60 s compared None Not applicable Not estimated; a one-stage network classifies atrial fibrillation Wang [76] 50; 30 500-frame model clip; 5 s short-time Fourier transform window for the heart rate None between clips; 4 s between transform windows, a 1 s step Short-time Fourier transform over 0.65–4 Hz (39–240 beats per minute); Hamming window in preprocessing Dominant short-time Fourier transform frequency CAP-rPPG [77] 30; 60 180 frames (6 s) in training; 900 frames (30 s) in testing None Fast Fourier transform; a power spectral density loss over a defined band Spectral peak RhythmFormer [78] 30; 20 160 frames (5.33 s) None Welch power spectral density after a second-order Butterworth filter, 0.75–2.5 Hz Spectral peak Chang et al. [79] 30 Not stated Not stated Frequency-domain heart-rate variability indices, reported as sensitive to the observation window Mean heart rate from a proprietary software development kit; algorithm not disclosed FreqPhys [80] 30 Not stated Not stated Discrete Fourier transform with an ideal band-pass over 0.66–3.0 Hz Dominant spectral peak, multiplied by 60 KDPhys [81] 30; 20 10 s Not stated Fast Fourier transform on Hanning-windowed signals Peak of the periodogram This work 50 5.12 s (256 frames), swept to 61.44 s in Table 19 None Welch power spectral density, segment capped at 256 samples, 0.75–2.5 Hz (45–150 beats per minute) Spectral peak