Paper deep dive
Subject-Relative Micro-Motion and Sleep Dynamics for Near-Infrared Video Sleep Staging
Kunmin Jang, You Rim Choi, Hun Heo, Heonjun Lee, Suahn Bae, Dongik Park, Hyun-Woo Shin, Hyung-Sin Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/9/2026, 4:20:05 AM
Summary
The paper introduces ViNUSS (Video-Native Unmediated Sleep Staging), a framework for contactless sleep staging using only Near-Infrared (NIR) video without reconstructing physiological proxies. ViNUSS employs subject-relative micro-motion learning via a Pixel-Wise Temporal Encoder (PTE) and within-subject stage contrast, combined with two-scale sleep dynamics modeling using Mamba architectures to capture within-epoch and full-night temporal patterns. Evaluated on the Korea Video Sleep Study (KVSS) dataset with 475 recordings, ViNUSS achieves 0.80 accuracy and 0.78 macro-F1 for four-class sleep staging.
Entities (7)
Relation Signals (5)
ViNUSS → evaluatedon → KVSS
confidence 95% · Evaluation is conducted on the Korea Video Sleep Study (KVSS)... ViNUSS achieves 0.80 accuracy
NIR Video → usedfor → Sleep Staging
confidence 93% · Near-infrared (NIR) video is a promising modality for contactless sleep monitoring
ViNUSS → usescomponent → Pixel-Wise Temporal Encoder
confidence 92% · we introduce a Pixel-Wise Temporal Encoder (PTE)... The enriched clip is then encoded by a compact MoViNet-A2 backbone
Polysomnography → provideslabelsfor → Sleep Staging
confidence 91% · labels defined by polysomnography (PSG), where the model infers sleep stages from NIR video alone
ViNUSS → usescomponent → Mamba
confidence 90% · We instantiate both temporal scales with Mamba... ShortMamba then summarizes the temporal evolution... LongMamba module
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Near-infrared (NIR) video is a promising modality for contactless sleep monitoring, but recent video-based sleep staging methods often use it as a route to reconstructed respiratory/cardiac proxies or cross-modal physiological representations. We study video-only sleep staging under labels defined by polysomnography (PSG), where the model infers sleep stages from NIR video alone without explicit physiological proxy reconstruction or auxiliary physiological signal supervision. This tests whether NIR video itself can provide informative sleep-stage evidence, rather than only serving as an input for recovering physiological proxies. We propose ViNUSS (Video-Native Unmediated Sleep Staging), a framework that combines subject-relative micro-motion learning with full-night sleep dynamics modeling. Spatially anchored pre-spatial micro-motion encoding preserves localized temporal variation together with its spatial context. Within-subject stage contrast learns stage cues with respect to each subject's night-specific baseline. Two-scale sleep dynamics modeling captures within-epoch motion evolution and organizes epoch-level evidence into a coherent full-night sleep-stage trajectory. On 475 overnight NIR recordings (~3,250 hours), ViNUSS achieves 0.80 accuracy and 0.78 macro-F1 for four-class sleep staging. Interpretability analysis suggests attention to thoraco-abdominal periodic motion and gross body movements associated with arousals and position changes. These results support NIR video as an independently informative and complementary modality for PSG-defined sleep-stage estimation
Tags
Links
- Source: https://arxiv.org/abs/2609.05550v1
- Canonical: https://arxiv.org/abs/2609.05550v1
Trouble viewing inline? Open PDF directly →
Full Text
79,095 characters extracted from source content.
Expand or collapse full text
Subject-Relative Micro-Motion and Sleep Dynamics for Near-Infrared Video Sleep Staging Kunmin Jang ∗ Seoul National University jangk10@snu.ac.kr You Rim Choi ∗ Seoul National University yrchoi@snu.ac.kr Hun Heo Seoul National University greathunh@snu.ac.kr Heonjun Lee Seoul National University johnlee0321@snu.ac.kr Suahn Bae Seoul National University snubbb@snu.ac.kr Dongik Park Seoul National University breadsora@snu.ac.kr Hyun-Woo Shin † Seoul National University Hospital charlie@snu.ac.kr Hyung-Sin Kim † Seoul National University hyungkim@snu.ac.kr Abstract Near-infrared (NIR) video is a promising modality for contactless sleep monitoring, but recent video-based sleep staging methods often use it as a route to reconstructed respiratory/cardiac proxies or cross-modal physiological representations. We study video-only sleep staging under labels defined by polysomnography (PSG), where the model infers sleep stages from NIR video alone without explicit physiological proxy reconstruction or auxiliary physiological signal supervision. This tests whether NIR video itself can provide informative sleep-stage evidence, rather than only serving as an input for recovering physiological proxies. We propose ViNUSS (Video-Native Unmediated Sleep Staging), a framework that combines subject-relative micro-motion learning with full-night sleep dynamics modeling. Spatially anchored pre-spatial micro-motion encoding preserves localized temporal variation together with its spatial context. Within-subject stage contrast learns stage cues with respect to each subject’s night-specific baseline. Two-scale sleep dynamics modeling captures within-epoch motion evolution and organizes epoch- level evidence into a coherent full-night sleep-stage trajectory. On 475 overnight NIR recordings (∼3,250 hours), ViNUSS achieves 0.80 accuracy and 0.78 macro-F1 for four-class sleep staging. Interpretability analysis suggests attention to thoraco- abdominal periodic motion and gross body movements associated with arousals and position changes. These results support NIR video as an independently informative and complementary modality for PSG-defined sleep-stage estimation. 1 Introduction Sleep staging is a central procedure in sleep medicine, assigning consecutive 30-second epochs of an overnight recording to Wake, Light (N1/N2), Deep (N3), or REM [30,2]. The clinical reference is polysomnography (PSG), where trained technicians score multimodal physiological signals such as EEG, EOG, and EMG [34]. Although PSG provides the standard definition of sleep stages, it requires contact sensors, overnight instrumentation, and expert scoring, limiting scalability for longitudinal ∗ These authors contributed equally. † Corresponding authors. Preprint. arXiv:2609.05550v1 [cs.CV] 3 Sep 2026 and home-based monitoring [45,26]. These limitations have motivated contactless approaches that estimate sleep stages under PSG-defined labels while reducing sensing burden. Near-infrared (NIR) video is a promising modality for contactless sleep monitoring because it operates in darkness and captures spatially localized body motion, including respiration-related motion, limb movement, position changes, and brief arousals. Recent video-based sleep staging methods often use video as a route to another representation, such as reconstructed respiratory/cardiac signals, optical- flow-derived motion features, or EEG-guided cross-modal features [29,38,6,43,46,16]. Proxy-first approaches are valuable and interpretable, but blankets, position changes, and face anonymization can make respiratory or rPPG extraction unreliable, while one-dimensional proxies may discard regional motion patterns and transient movements. Cross-modal physiological approaches are complementary, but they answer a different question: how video can align with auxiliary physiological representations. In this work, we study video-only sleep staging under labels defined by PSG. The model infers sleep stages from NIR video alone, without explicit physiological proxy reconstruction or auxiliary physiological representation learning. This formulation does not replace PSG as the clinical reference. Instead, it tests whether NIR video itself can provide informative sleep-stage evidence, rather than only serving as an input for recovering physiological proxies or aligning to representations derived from other physiological modalities. This setting differs from generic video understanding. Standard video models are typically designed for actions, objects, and scene changes, where discriminative cues often appear as large motion or semantic interaction [20,11,24,3,36]. Action-oriented temporal modules further enhance motion representation through temporal shifts, motion excitation, or temporal differences [23,22,42]. In NIR sleep video, however, the scene is mostly static, useful cues may appear as weak local micro- motion, and subject-specific factors such as body shape, bedding, camera geometry, and baseline respiration-related motion can dominate the representation. The interpretation of motion also depends on its location, since weak thoraco-abdominal periodicity, fast limb or body movements, and broad position changes provide different sleep-stage evidence. Effective video-only sleep staging therefore requires preserving localized temporal variation with its spatial context, learning stage evidence relative to each subject’s night-specific baseline, and modeling sleep dynamics across the full night. We propose ViNUSS, a framework for subject-relative micro-motion learning and full-night sleep dynamics modeling. First, spatially anchored pre-spatial micro-motion encoding densely scans the full 30-second epoch with short temporal filters before spatial aggregation, retaining the spatial context of each temporal change. This design targets weak, spatially localized traces of sleep-relevant motion over the clinically defined epoch before they are diluted by static appearance cues. Second, within-subject stage contrast learns stage evidence relative to the subject’s own visual and motion baseline rather than as an absolute appearance template. Third, two-scale sleep dynamics modeling captures motion evolution within each 30-second epoch and organizes epoch-level evidence into a coherent full-night sleep-stage trajectory. Evaluation is conducted on the Korea Video Sleep Study (KVSS) [9], which contains 475 overnight NIR recordings, approximately 3,250 hours of video, synchronized with PSG labels. The recordings include far-field viewpoints, bedding occlusion, weak motion, and face anonymization, making them challenging for both proxy extraction and raw-video modeling. ViNUSS achieves 0.80 accuracy and 0.78 macro-F1 for four-class staging. Ablations show complementary benefits from micro-motion representation, within-subject stage contrast, and full-night sleep dynamics. Sleep Video EigenCAM (SVECAM) further suggests that ViNUSS uses physiologically meaningful motion, including thoraco- abdominal periodicity and larger movements associated with arousals and position changes. Our contributions are summarized as follows: • Video-only formulation under PSG-defined labels. We study NIR video sleep staging where PSG provides the clinical labels but the model uses NIR video alone, without explicit physiological proxy reconstruction or EEG-guided representation learning. •Subject-relative micro-motion learning. We introduce pre-spatial micro-motion encoding and within-subject stage contrast to preserve sleep-relevant temporal traces over the 30-second epoch while learning stage evidence relative to each subject’s night-specific baseline. • Two-scale sleep dynamics. We model within-epoch motion evolution and full-night temporal context, organizing video-derived epoch evidence into coherent overnight sleep-stage trajectories. 2 •Large-scale evaluation and visual evidence analysis. We evaluate ViNUSS on 475 overnight NIR recordings and analyze physiologically meaningful visual evidence. 2 Related Work 2.1 Sleep Staging and Contactless Monitoring The clinical reference for sleep staging is polysomnography (PSG), where trained technicians score multi-channel physiological signals such as EEG, EOG, and EMG according to the American Academy of Sleep Medicine (AASM) guidelines [2]. Although PSG provides high-fidelity labels, it requires contact sensors, overnight instrumentation, and expert scoring, which limits scalability for longitudinal monitoring [45]. Wearable devices based on actigraphy, PPG, or simplified EEG reduce this burden [12,7], but still require physical contact and can be affected by comfort, adherence, and motion artifacts. Fully contactless sensing, including audio [19], under-mattress pressure [40], and radio-frequency methods [18], avoids body-worn sensors but often captures only limited aspects of sleep behavior. Among contactless modalities, near-infrared (NIR) video is particularly relevant because it captures spatially localized body motion in darkness. 2.2 Video-Based and Cross-Modal Sleep Staging Video-based sleep staging has been studied through handcrafted motion features and proxy-first pipelines. Early camera-based methods used frame-difference features with SVMs [29], while recent methods estimate physiological proxies before classification. For example, van Meulen et al. [38] used camera-based PPG and heart-rate variability, Video-PSG [43] extracted respiratory and cardiac signals, and SleepVST [6] modeled respiratory/cardiac waveforms and optical-flow features with pretrained transformers. These proxy-first methods are valuable because they provide physiologically interpretable inputs. However, explicit proxy reconstruction can be unreliable in realistic NIR recordings: bedding and position changes can corrupt camera-based respiratory motion [44], while far-field views, occlusion, and face anonymization can weaken rPPG-based cardiac estimation [25,4,15]. Moreover, proxy compression can discard regional motion patterns and transient movements that are not captured by predefined physiological signals. Recent work has also connected infrared video with EEG-derived knowledge through EEG-teacher distillation [46] or cross-modal retrieval between infrared videos and EEG representations [16]. These studies show that IR video contains sleep-relevant visual evidence when guided by physiological supervision. Our work is complementary: PSG defines the labels, but our model does not use EEG-teacher features, EEG retrieval databases, or explicit respiratory/cardiac proxy reconstruction. We instead study NIR video as an independently informative modality through subject-relative micro-motion learning and full-night video-conditioned dynamics. 2.3 Video Understanding and Full-Night Temporal Modeling Modern video understanding has developed strong spatiotemporal representations for action recog- nition and semantic video learning, including efficient 3D CNNs, spatiotemporal transformers, and self-supervised video encoders [20,11,24,3,36]. Temporal modules further improve action recognition by exchanging information across frames, exciting motion-sensitive channels, or mod- eling temporal differences [23,22,42]. These methods motivate motion-aware video processing, but they primarily target salient action cues, whereas NIR sleep video requires preserving dense, low-amplitude temporal traces over a clinically defined 30-second epoch before spatial abstraction. Sleep staging also requires temporal context beyond isolated epochs, and prior PSG/EEG-based neural models commonly exploit neighboring-epoch structure [35,31,32]. For NIR video, the evidence is indirect and subject-dependent, making longer temporal context important for interpreting local motion. ViNUSS therefore summarizes motion within each 30-second epoch and then models the resulting representations across the subject’s full-night trajectory using state-space sequence modeling [14]. This couples full-night temporal context with pre-spatial micro-motion encoding and within-subject stage contrast. 3 REM 0.170.200.230.270.30 Fre quency (H z) 0.17 0.20 0.230.270.30 Fre quency (H z) Ma gnitu de Ma gnitu de Amp litud e Amp litud e 0 Time (s) 102030 0 Time (s) 102030 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 Ground Tru th Reconstructed 322 nd epoch322 nd epoch 682 nd epoch682 nd epoch Deep 104 th epoch104 th epoch 682 nd epoch682 nd epoch 345 th epoch345 th epoch 322 nd epoch322 nd epoch 345 th epoch345 th epoch 104 th epoch104 th epoch (a)(b)(c) Figure 1: Frequency analysis and respiratory reconstruction from NIR sleep video. (a) Video frequency maps from pixel-wise FFT over 0.17–0.30 Hz; brighter regions indicate stronger periodic motion. (b) Contact thoracic spectra from chest-belt signals. (c) Video-reconstructed respiratory waveforms. Sex: Male BMI: 34.7 Age: 60 OSA:Severe Sex: Male BMI: 42 Age: 38 OSA: Severe Sex: Male BMI: 21 Age: 43 OSA: Mild Sex: Female BMI: 28 Age: 67 OSA: Severe Sex: Male BMI: 24 Age: 40 OSA: Moderate Sex: Female BMI: 22 Age: 64 OSA: Normal AllStagesDeep WakeLight REM Subject1Subject2Subject3Subject4Subject5Subject6 (a)(b) Figure 2: Baseline 3D CNN behavior on raw NIR sleep videos. (a) Frame-difference and spatial at- tention maps highlight static high-contrast regions rather than temporally coherent thoraco-abdominal motion. (b) t-SNE [37] projection of epoch-level embeddings from six subjects shows subject- dominant clustering and within-stage subject dispersion. 3 Method Each overnight recording is divided into 30-second epochs, following clinical sleep-staging prac- tice. Given a full-night NIR video sequenceV = X 1 ,...,X N withNepochs, our goal is to predict a stage sequence ˆ Y = ˆy 1 ,..., ˆy N for ground-truth labelsY = y 1 ,...,y N , where y t ∈Wake, Light, Deep, REM. The labels are defined by PSG scoring, but the model input is NIR video only. We do not reconstruct respiratory or cardiac waveforms, nor use auxiliary physiological signals such as EEG as model inputs or for cross-modal supervision. 3.1 Design Motivation and ViNUSS Overview Figure 1 illustrates why explicit proxy reconstruction is informative but incomplete in realistic NIR sleep videos. We reconstruct respiratory waveforms using video motion magnification and camera-based respiratory extraction methods [41,44]. Figure 1(a) shows localized periodic responses around the thoraco-abdominal region, with different dominant frequencies across epochs, indicating respiration-related video evidence. However, panel (b) shows that video-dominant frequencies can deviate from contact thoracic spectra under position changes, occlusion, weak motion, or non- respiratory body movement, and panel (c) shows attenuation or frequency mismatch in waveforms. For example, the Deep682 nd and REM322 nd epochs show frequency mismatch, while the REM 345 th epoch shows non-respiratory movement contamination. Thus, representing video with a single proxy may discard spatially localized evidence or introduce proxy-specific errors. 4 30 seconds Clip Subject-Relative Micro-Motion Representation Params: 4.04M | GFLOPs: 17.9 Vi deo Feature Embedding PTE Sl eep St age Cl assification Loss (Per Epoch) Wake Light Deep REM [11, 50, 224, 224][640, 50, 7, 7] [N_Epochs, 640] REM REM Wake Light Light MoVi Net-A2 Temporal Convolution Kernels Time(5 fps) Temporally Enriched Feature Map Pixels in short time [1, 150, 224, 224] Pi xel- wi se Tempo ral Encoder (PTE) Intra- subj ect Di scriminability Loss Spatial Pooling Within-epoch Dynamics Full-Night Dynamics [640, 50, 7, 7] Wake REM Light Deep Sleep Stage Classifier Sleep Stage Classifier Params: 2.76M | GFLOPs: 43.8 Full-Night Sleep Dynamics Modeling Hierarchical Temporal Sleep Mode ling Full-Night Predictions W H T Short-MambaShort-Mamba Bidirectional Long-Mamba Bidirectional Long-Mamba [640] 30sec Full Night Extr act Fine-Grained Information from RAW Video before latent representation Figure 3: Overview of ViNUSS. Naively training a generic video encoder on raw NIR clips is also insufficient. As shown in Figure 2(a), a baseline 3D CNN (MoViNet-A2 [20]) attends to static high-contrast structures such as bedding folds, sensor cables, or limb boundaries, even when frame differences indicate motion around the thoraco-abdominal region. Figure 2(b) shows that epoch-level embeddings cluster primarily by subject rather than by sleep stage, with substantial dispersion across subjects even within the same stage. This subject-dominant structure reflects differences in body shape, bedding, camera geometry, OSA severity, and baseline respiration-related motion. Moreover, motion patterns can vary across the night even within the same subject and stage, reinforcing the need to interpret local video evidence within a full-night trajectory. Taken together, these analyses motivate learning directly from spatially localized video evidence rather than reducing each epoch to a reconstructed proxy or a generic raw-video embedding. Since single-epoch visual evidence remains indirect and sleep stages evolve across the night, ViNUSS preserves weak local micro-motion before spatial aggregation, learns stage evidence relative to each subject’s overnight baseline, and models sleep dynamics over the full night. Figure 3 summarizes how ViNUSS implements these design goals. 3.2 Subject-Relative Micro-Motion Representation We first describe how ViNUSS forms epoch-level video representations before full-night modeling. The representation module addresses the first two design goals by enriching each 30-second clip with pixel-wise temporal responses and using within-subject stage contrast to reduce subject-dominant structure during training. Spatially anchored pre-spatial micro-motion encoding.LetX t ∈ R C×T×H×W denote the video clip for epocht, with channel, frame, height, and width dimensions. Standard video backbones may mix and downsample local regions before weak temporal evidence becomes separable. This is problematic for NIR sleep video because sleep-relevant motion can be highly localized and may be diluted unless temporal variation is first encoded at each pixel. To preserve dense pixel-level temporal traces over the epoch before spatial reduction, we introduce a Pixel-Wise Temporal Encoder (PTE): ̃ X t = Proj (Concat [X t ,M 3 (X t ),M 5 (X t )]),(1) whereM 3 (·)andM 5 (·)are learnable temporal encoding branches with kernel sizes 3 and 5, applied along the frame axis independently at each spatial location.Concat(·)denotes channel-wise concate- nation with the original clip, andProj(·)is a shallow temporal projection layer that reduces the frame dimension via temporal striding before the video backbone. PTE does not introduce a separate spatial stream; spatial mixing is deferred to the backbone. Detailed architecture is provided in Appendix B. PTE applies short temporal filters over the full 30-second epoch while preserving the spatial location of each temporal response. Although each convolution uses a small kernel, stacking expands the effective receptive field to local sub-epoch intervals of roughly one to three seconds at 5 fps. Thus, PTE targets local motion traces rather than full-epoch or longer-range respiratory dynamics. It exposes displacement-like variation, short oscillatory changes, and transient movement onsets before 5 they are mixed with static appearance. Unlike hand-crafted frame differences or late-fusion motion branches, PTE produces learnable, spatially localized micro-motion traces before backbone-level spatial mixing. The enriched clip is then encoded by a compact MoViNet-A2 backbone, producing a spatiotemporal feature map F t . Within-subject stage contrast. Even after motion-sensitive encoding, epoch features may reflect subject- and acquisition-specific factors rather than stage-related motion. We therefore contrast sleep stages within the same overnight recording. For subjects, an anchor and a positive epoch are sampled from the same stage, while a negative epoch is sampled from a different stage of the same subject. This within-subject sampling makes the comparison subject-relative, encouraging the representation to emphasize stage-related motion differences with respect to the subject’s night-specific baseline. We implement this objective as an Intra-Subject Discriminability Loss (ISDL). Let(x a s,i ,x p s,i ,x n s,i ) denote thei-th anchor-positive-negative triplet of epoch features for subjects, and letf (·)be a projection head. Given N tri sampled triplets and margin α, ISDL is L ISDL = 1 N tri X i ∥f (x a s,i )− f (x p s,i )∥ 2 2 −∥f (x a s,i )− f (x n s,i )∥ 2 2 + α + .(2) This follows a triplet-style margin objective [17], but with within-subject sampling. The representation objective combines the sleep-stage classification cross-entropy lossL CE with ISDL: L rep =L CE + λL ISDL ,(3) where λ controls the weight of the contrastive regularizer. The key role of ISDL is not the margin loss itself, but the sampling semantics. Unlike dataset-level contrastive objectives, ISDL forms comparisons within the same overnight recording, reducing variation from cross-subject and acquisition differences. 3.3 Full-Night Sleep Dynamics Modeling Although the representation module produces motion-sensitive features for each epoch, sleep staging is inherently sequential. We introduce Sleep Dynamics Modeling (SDM) as a two-scale sequence module that first summarizes within-epoch motion evolution and then models the resulting epoch representations across the full night. This design allows temporal modeling to operate on compact video-derived motion summaries rather than frame-level features or post-hoc label sequences. We instantiate both temporal scales with Mamba [14]; implementation details are provided in Appendix B. Within-epoch motion dynamics.The backbone outputF t retains spatial and temporal dimensions. We first pool the spatial dimensions while preserving the temporal axis: R t = SpatialPool(F t ).(4) ShortMamba then summarizes the temporal evolution within the 30-second epoch into an epoch-level motion representation: z t = ShortMamba(R t ).(5) This step condenses within-epoch temporal dynamics, such as respiration-related periodicity and transient body movement, before full-night sequence modeling. Full-night sleep dynamics. The epoch-level motion representations are arranged chronologically asZ =z 1 ,...,z N and processed by a bidirectional LongMamba module: (s 1 ,...,s N ) = LongMamba(z 1 ,...,z N ).(6) The sleep-stage probability for epoch t is then p(y t |V) = softmax(Classifier(s t )).(7) SDM operates on video-derived epoch representations before classification, rather than smoothing predicted labels after classification. By relating local motion summaries to the subject’s full-night trajectory, SDM helps calibrate subject-specific motion baselines, maintain coherent sleep progression, and reduce abrupt epoch-wise fluctuations. 6 Table 1: Performance comparison of video-based sleep staging methods. Results for non-KVSS datasets are taken from their respective papers, while KVSS results are from our own implementation. Oracle indicates experiments using ground-truth physiological signals from contact sensors. MethodInput / ProxyDataset# of Data Accκ MF1 van Meulen et al. [38]HRVHealthBed460.680.490.63 Video-PSG [43]HR+HRV+BR+MShenzhen200.730.620.74 SleepVST [6]BW+CW+MOxford500.790.710.78 SleepVST [6] BW*+CW*+M KVSS475 0.780.630.73 BW+CW*+M0.610.230.37 MoViNet-A2 [20]Video onlyKVSS4750.600.350.50 MoViNet-A2 [20] + PTEVideo onlyKVSS4750.620.400.55 MoViNet-A5 [20] + PTEVideo onlyKVSS4750.600.380.53 Video Swin-Tiny [24] + PTEVideo onlyKVSS4750.510.270.46 ViNUSSVideo onlyKVSS4750.800.680.78 Acc: accuracy; κ: Cohen’s kappa; MF1: macro F1-score. BR: breathing rate; HR: heart rate; HRV: heart-rate variability; BW: breathing waveform; CW: cardiac waveform; M: motion feature. Asterisks (*) indicate waveforms from PSG sensors (oracle setting). Rows marked “+PTE” include only the Pixel-Wise Temporal Encoder, excluding ISDL and SDM. 4 Experiments 4.1 Experimental Setup Dataset.We evaluate on the Korea Video Sleep Study (KVSS) dataset [1,9], using 475 overnight NIR sleep recordings (∼3,250 hours) paired with synchronized PSG. Sleep stages were annotated by certified technicians according to AASM guidelines [2]. The cohort includes healthy and sleep- disordered individuals with diverse age, BMI, sex, and OSA severity. Videos were captured at 5 fps using a wall-mounted NIR camera under routine overnight sleep-study conditions, including far-field viewpoints, bedding occlusion, weak motion, and face anonymization. Subjects were split into disjoint train/validation/test sets of 335/70/70 using stratified sampling over sex, age, and OSA severity. Additional acquisition details and cohort characteristics are provided in Appendix A. Implementation.We implement ViNUSS in PyTorch and train on four NVIDIA RTX A6000 GPUs. The model is trained with five AASM sleep-stage labels (Wake, N1, N2, N3, REM), and N1/N2 are merged into Light sleep for the reported four-class evaluation. We report accuracy, Cohen’s kappa (κ), and macro-F1 (MF1). MF1 is emphasized because sleep stages are imbalanced and minority stages such as Deep and REM are clinically important. Architecture, preprocessing, optimization, and hyperparameter details are provided in Appendix B. The submitted anonymized implementation will be made publicly available upon acceptance. 4.2 Overall Performance Table 1 compares ViNUSS with prior video-based sleep staging methods. Results from non-KVSS datasets are included for reference, as datasets differ in acquisition protocols, cohort size, camera setup, and label distribution. For controlled comparisons on KVSS, we include SleepVST [6], a representative proxy-first method, and video-only backbone variants. Reproducing the fully video-based SleepVST pipeline on KVSS is not feasible because its video-to- waveform extraction module is not publicly released, and face anonymization and bedding occlusion make rPPG and respiratory reconstruction unreliable. We therefore evaluate two oracle-input variants: a partial-oracle setting with contact-sensor cardiac waveforms and a full-oracle setting with both contact-sensor respiratory and cardiac waveforms. These variants are oracle-input versions of one representative proxy-first architecture, not universal upper bounds for physiological-signal-based sleep staging; details are provided in Appendix C. Video-only backbone variants perform substantially below ViNUSS. Although PTE improves the vanilla MoViNet-A2 baseline, the gap to the full ViNUSS remains large, indicating that pre-spatial micro-motion encoding alone is insufficient without within-subject stage contrast and full-night dynamics. MoViNet-A5+PTE does not improve over MoViNet-A2+PTE despite its larger size, and Video Swin-Tiny+PTE performs worse, suggesting that simply scaling or replacing the backbone 7 Wake Light DeepREM Predicted Wake Light Deep REM True 0.73 (10412) 0.20 (2813) 0.03 (440) 0.04 (507) 0.16 (5131) 0.53 (16637) 0.23 (7295) 0.07 (2139) 0.15 (817) 0.24 (1321) 0.57 (3093) 0.04 (219) 0.12 (820) 0.29 (2080) 0.05 (340) 0.54 (3854) 0.610.730.280.57 0.73 0.53 0.57 0.54 Per-class Precision Per-class Recall Wake Light DeepREM Predicted Wake Light Deep REM True 0.73 (10322) 0.25 (3485) 0.00 (18) 0.02 (347) 0.06 (1817) 0.85 (26664) 0.06 (1857) 0.03 (864) 0.01 (68) 0.29 (1577) 0.68 (3723) 0.01 (82) 0.03 (218) 0.14 (1007) 0.00 (0) 0.83 (5869) 0.830.810.670.82 0.73 0.85 0.68 0.83 Per-class Precision Per-class Recall Wake Light DeepREM Predicted Wake Light Deep REM True 0.74 (10552) 0.22 (3075) 0.00 (12) 0.04 (533) 0.08 (2427) 0.82 (25459) 0.06 (1739) 0.05 (1577) 0.02 (112) 0.40 (2186) 0.56 (3029) 0.02 (123) 0.02 (173) 0.10 (718) 0.00 (16) 0.87 (6187) 0.800.810.630.73 0.74 0.82 0.56 0.87 Per-class Precision Per-class Recall (b)(a)(c) Figure 4: Confusion matrices on the KVSS test set for four-class sleep staging. (a) Without SDM. (b) SDM with LongMamba only. (c) Full SDM with ShortMamba and LongMamba. is insufficient for NIR sleep video. MoViNet-A2 is used in ViNUSS because it provides the best accuracy-efficiency tradeoff among the evaluated variants. On KVSS, ViNUSS achieves 0.80 accuracy, 0.68κ, and 0.78 MF1. Compared with the full-oracle SleepVST variant, ViNUSS improves MF1 from 0.73 to 0.78. This result supports NIR video as an independently informative and complementary modality for PSG-defined sleep-stage estimation, while PSG remains the clinical reference for defining the labels. Appendix D.3 further reports stratified per-subject analyses across recording conditions and subject characteristics, including occlusion, camera angle, illumination, sex, age, BMI, and AHI groups. 4.3 Ablation Study Table 2: Ablation study of key components on KVSS. Exp.1 corresponds to the vanilla MoViNet- A2.✓ indicates that each component is applied. Exp.# Method Acc.MF1 PTEISDLSDM 1–0.600.50 2✓–0.620.55 3✓–0.600.56 4–✓0.750.67 5✓–✓0.770.72 6✓0.800.78 Effect of PTE and ISDL. Table 2 shows the contribution of each component, starting with the representation modules without SDM. The baseline (Exp. 1) achieves 0.60 accuracy and 0.50 MF1. Adding PTE (Exp. 2) improves ac- curacy to 0.62 and MF1 to 0.55, indicating that pre-spatial micro-motion encoding helps pre- serve weak local temporal cues before spatial processing. Adding ISDL (Exp. 3) further im- proves MF1 to 0.56, although accuracy remains comparable, suggesting that within-subject stage contrast improves stage-wise separability under subject-specific variation. Overall, PTE and ISDL improve representation quality from complementary perspectives: PTE enhances motion sensitivity, while ISDL regularizes the embedding space to reduce subject-dependent bias. Effect of SDM. Adding SDM to the vanilla backbone (Exp. 4) improves MF1 from 0.50 to 0.67, showing the importance of full-night temporal context. However, the benefit of SDM depends strongly on the quality of the epoch-level representation. With the same SDM module, adding PTE (Exp. 5) increases MF1 from 0.67 to 0.72, and adding both PTE and ISDL (Exp. 6) further increases MF1 to 0.78. Thus, the representation and dynamics modules are complementary: PTE and ISDL make the epoch-level video evidence more informative and subject-relative, while SDM organizes this evidence into a coherent full-night sleep-stage trajectory. Figure 4 shows how SDM changes class-wise prediction behavior beyond the aggregate scores in Table 2. Without SDM (a), predictions are broadly less balanced, with substantial confusion between Light and neighboring stages and low precision for Deep sleep. Adding LongMamba only (b) improves the overall prediction structure by incorporating full-night temporal context, increasing recall for Light and REM and improving precision for Wake and Deep. This indicates that full-night context helps organize epoch-level predictions into a more consistent sleep-stage trajectory. The full SDM (c) further incorporates within-epoch motion summarization. Compared with LongMamba only, it improves Deep precision and recall from 0.63/0.56 to 0.67/0.68 and increases REM precision 8 (a)(b) Sample Subject 1 Sample Subject 2 w/o SDM Pred. with SDM Pred. GT Label 00 198 198396396594594793793 0 233466699933 Epoch Epoch LightWakeDeepREM 1.01.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 Wake Wake Light Light Deep DeepREMREM w/o SDM Pred. with SDM Pred. GT Label Figure 5: Full-night prediction behavior and temporal coherence. (a) Full-night predictions. Red boxes highlight representative segments. (b) Temporal coherence. Time Normaliz ed V alue Temporal Saliency GT Breathing Wave Time Normalized V alue Temporal Saliency GT Breathing Wave (b)(a) Figure 6: SVECAM visualizations for representative NIR sleep-video epochs. (a) Stable Deep sleep, showing thoraco-abdominal saliency and temporal saliency aligned with breathing. (b) Brief arousals and position change, showing both respiration-related motion and larger body movement. from 0.73 to 0.82, although REM recall slightly decreases from 0.87 to 0.83. Together, these results support the two-scale design, where LongMamba provides trajectory-level context and ShortMamba refines the epoch-level motion evidence used by the full-night model. Additional experiments of SDM design choices are provided in Appendix D.4, including temporal context-length sweeps and comparisons with alternative sequence models. Full-night prediction behavior. Figure 5(a) illustrates how SDM affects complete overnight predictions. For Subject 1, predictions without SDM frequently oscillate between adjacent stages in the highlighted regions, whereas SDM suppresses abrupt fluctuations and preserves smoother stage trajectories. For Subject 2, predictions without SDM fail to recover sustained Deep sleep segments, while SDM accumulates temporally consistent evidence and restores continuous Deep predictions. These examples show that full-night dynamics not only smooths noisy local predictions, but also helps recover sustained sleep structure when single-epoch visual evidence is weak. Figure 5(b) further shows how ISDL interacts with full-night dynamics by improving the temporal coherence of epoch-level representations. We measure coherence as the cosine similarity between consecutive epoch embeddings,s t = cos(f t , f t+1 ). ISDL increases this coherence across stages, indicating that within-subject stage contrast produces smoother feature trajectories within each subject. These more stable epoch representations provide better inputs for SDM, explaining why combining PTE, ISDL, and SDM yields the strongest performance. Together with Table 2 and Figure 4, these results show that micro-motion representation, subject-relative contrast, and full-night temporal modeling provide complementary benefits for video-only sleep staging. 4.4 Interpretation Analysis To inspect the visual evidence used by ViNUSS, we apply Sleep Video EigenCAM (SVECAM), a visualization method adapted from EigenCAM [27] for 3D sleep-video features. SVECAM produces class-agnostic spatial saliency maps and temporal saliency curves, allowing us to inspect where and when the model attends within each epoch. Implementation details are provided in Appendix E. Figure 6 shows two representative cases. During stable Deep sleep (a), saliency concentrates around the thoraco-abdominal region, and temporal saliency is aligned with respiratory rhythm despite the absence of explicit respiratory reconstruction. During arousals and position changes (b), the model still captures respiration-related motion but also attends to larger body movements. These visualizations suggest that ViNUSS uses physiologically meaningful motion evidence encoded in NIR video, without explicitly reconstructing predefined physiological proxies. 9 The observed saliency patterns also help explain the difficulty of single-epoch video-only staging. Stable Deep sleep often depends on subtle periodic motion with little gross movement, whereas arousal-related epochs can produce broader and more transient saliency. Light sleep and REM can re- main visually heterogeneous, often sharing low-motion or transition-heavy patterns with neighboring stages. This ambiguity supports the need for both subject-relative micro-motion representation and full-night sleep dynamics modeling. 5 Conclusion We presented ViNUSS, a framework for PSG-defined sleep staging from near-infrared (NIR) video that combines subject-relative micro-motion learning with full-night sleep dynamics modeling. ViNUSS preserves localized temporal motion before spatial reduction, learns stage evidence relative to each subject’s overnight baseline, and organizes video-derived epoch evidence into coherent full-night sleep-stage trajectories. On 475 overnight NIR recordings, ViNUSS achieves strong four-class staging performance, and ablations show that micro-motion representation, within-subject stage contrast, and sleep dynamics provide complementary benefits. Sleep Video EigenCAM further suggests that the model uses physiologically meaningful visual motion, including thoraco-abdominal periodicity and broader body movements associated with arousals and position changes. These results support NIR video as an independently informative and complementary modality for contactless sleep-stage estimation under PSG-defined labels. Broader validation across additional clinical sites and home environments remains important, and future work may explore integration with physiological signals or wearable modalities for more comprehensive sleep monitoring. References [1]AI Hub. Korea video-based sleep study dataset (kvss).https://aihub.or.kr/aihubdata/ data/view.do?currMenu=115&topMenu=&aihubDataSe=data&dataSetSn=638, 2021. [2]Richard B Berry, Rita Brooks, Charlene E Gamaldo, Susan M Harding, Carole Marcus, Bradley V Vaughn, et al. The aasm manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine, 176(2012):7, 2012. [3]Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Proceedings of the International Conference on Machine Learning, 2021. [4] Saksham Bhutani, Mohamed Elgendi, and Carlo Menon. Preserving privacy and video quality through remote physiological signal removal. Communications Engineering, 4(1):66, 2025. [5] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017. [6]Jonathan F Carter, João Jorge, Oliver Gibson, and Lionel Tarassenko. Sleepvst: Sleep staging from near-infrared video signals using pre-trained transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479–12489, 2024. [7]Amelia A. Casciola, Sebastiano K. Carlucci, Brianne A. Kent, Amanda M. Punch, Michael A. Muszynski, Daniel Zhou, Alireza Kazemi, Maryam S. Mirian, Jason Valerio, Martin J. McKe- own, and Haakon B. Nygaard. A deep learning strategy for automatic sleep staging based on two-channel eeg headband data. Sensors, 21(10):3316, 2021. [8]Xiaoli Chen, Rui Wang, Phyllis Zee, Pamela L. Lutsey, Sogol Javaheri, Carmela Alcántara, Chandra L. Jackson, Michelle A. Williams, and Susan Redline. Racial/ethnic differences in sleep disturbances: The multi-ethnic study of atherosclerosis (mesa). Sleep, 38(6):877–888, 06 2015. [9]You Rim Choi, Dongik Park, Hyun Kyung Lee, Suahn Bae, Sehyun Park, Dong-Kyu Kim, Yong Min Kim, Chan-Soon Park, Hyung-Sin Kim, and Hyun-Woo Shin. Non-contact on-device detection of obstructive sleep apnea from infrared video. Nature Communications, 2026. 10 [10]Luo donghao and wang xue. ModernTCN: A modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations, 2024. [11]Christoph Feichtenhofer. X3d: Expanding architectures for efficient video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. [12]Pedro Fonseca, Marco Ross, Andreas Cerny, Peter Anderer, Fokke van Meulen, Hennie Janssen, Angelique Pijpers, Sylvie Dujardin, Pauline van Hirtum, Merel van Gilst, and Sebastiaan Overeem. A computationally efficient algorithm for wearable sleep staging in clinical popula- tions. Scientific Reports, 13(1):9182, 2023. [13]Alex Graves and Jürgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5-6):602–610, 2005. [14]Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. [15]Divij Gupta and Ali Etemad. Privacy-preserving remote heart rate estimation from facial videos. In 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 706–712, 2023. [16]Jianan Han, Aidong Men, Yang Liu, Ziming Yao, Shaoxing Zhang, Yan Yan, and Qingchao Chen. Iot-v2e: An uncertainty-aware cross-modal hashing retrieval between infrared-videos and eegs for automated sleep state analysis. IEEE Internet of Things Journal, 11(3):4551–4569, 2024. [17]Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017. [18]Hong Hong, Li Zhang, Chen Gu, Yusheng Li, Guangxin Zhou, and Xiaohua Zhu. Noncontact sleep stage estimation using a cw doppler radar. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 8(2):260–270, 2018. [19] Joonki Hong, Hai Hong Tran, Jinhwan Jung, Hyeryung Jang, Dongheon Lee, In-Young Yoon, Jung Kyung Hong, and Jeong-Whun Kim. End-to-end sleep staging using nocturnal sounds from microphone chips for mobile devices. Nature and Science of Sleep, pages 1187–1201, 2022. [20]Dan Kondratyuk, Liangzhe Yuan, Yandong Li, Li Zhang, Mingxing Tan, Matthew Brown, and Boqing Gong. Movinets: Mobile video networks for efficient video recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16020–16030, 2021. [21] Till Kroeger, Radu Timofte, Dengxin Dai, and Luc Van Gool. Fast optical flow using dense inverse search. In European conference on computer vision, pages 471–488. Springer, 2016. [22] Yanyi Li, Bin Ji, Xintian Shi, Jianguo Zhang, Bin Kang, and Limin Wang. Tea: Temporal excitation and aggregation for action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [23]Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understand- ing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. [24]Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202–3211, 2022. [25]Daniel McDuff. Camera measurement of physiological vital signs. ACM Computing Surveys, 55(9):1–40, 2023. 11 [26]Tomi Miettinen, Katja Myllymaa, Taina Hukkanen, Juha Töyräs, Kirsi Sipilä, and Sami Mylly- maa. Home polysomnography reveals a first-night effect in patients with low sleep bruxism activity. Journal of Clinical Sleep Medicine, 14(8):1377–1386, 2018. [27] Mohammed Bany Muhammad and Mohammed Yeasin. Eigen-cam: Class activation map using principal components. In 2020 international joint conference on neural networks (IJCNN), pages 1–7. IEEE, 2020. [28]Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023. [29]Teruaki Nochino, Yuko Ohno, Takafumi Kato, Masako Taniike, and Shima Okada. Sleep stage estimation method using a camera for home use. Biomedical engineering letters, 9(2):257–265, 2019. [30]Aakash K Patel, Vamsi Reddy, Karlie R Shumway, and John F Araujo. Physiology, sleep stages. In StatPearls [Internet]. StatPearls Publishing, 2024. [31] Huy Phan, Oliver Y Chén, Minh C Tran, Philipp Koch, Alfred Mertins, and Maarten De Vos. Xsleepnet: Multi-view sequential model for automatic sleep staging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5903–5915, 2021. [32] Huy Phan, Kaare Mikkelsen, Oliver Y Chén, Philipp Koch, Alfred Mertins, and Maarten De Vos. Sleeptransformer: Automatic sleep staging with interpretability and uncertainty quantification. IEEE Transactions on Biomedical Engineering, 69(8):2456–2467, 2022. [33] Stuart F. Quan, Barbara V. Howard, Conrad Iber, James P. Kiley, F. Javier Nieto, George T. O’Connor, David M. Rapoport, Susan Redline, John Robbins, Jonathan M. Samet, and Patri- cia W. Wahl. The sleep heart health study: Design, rationale, and methods. Sleep, 20(12):1077– 1085, 12 1997. [34]Jessica Vensel Rundo and Ralph Downey I. Polysomnography. Handbook of clinical neurology, 160:381–392, 2019. [35]Akara Supratak, Hao Dong, Chao Wu, and Yike Guo. Deepsleepnet: A model for automatic sleep stage scoring based on raw single-channel eeg. IEEE transactions on neural systems and rehabilitation engineering, 25(11):1998–2008, 2017. [36]Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022. [37]Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579–2605, 2008. [38] Fokke B van Meulen, Angela Grassi, Leonie Van den Heuvel, Sebastiaan Overeem, Merel M van Gilst, Johannes P van Dijk, Henning Maass, Mark JH Van Gastel, and Pedro Fonseca. Contactless camera-based sleep staging: The healthbed study. Bioengineering, 10(1):109, 2023. [39]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [40]Nikhil Vyas, Kelly Ryoo, Hosanna Tesfaye, Ruhan Yi, and Marjorie Skubic. Sleep stage classification using non-invasive bed sensing and deep learning. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 3232–3238. IEEE, 2021. [41]Neal Wadhwa, Michael Rubinstein, Frédo Durand, and William T Freeman. Phase-based video motion processing. ACM Transactions on Graphics (ToG), 32(4):1–10, 2013. [42]Limin Wang, Zhan Tong, Bin Ji, and Gangshan Wu. Tdn: Temporal difference networks for efficient action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1895–1904, 2021. 12 [43]Qiongyan Wang, Hanrong Cheng, and Wenjin Wang. Video-psg: An intelligent contactless monitoring system for sleep staging. IEEE Transactions on Biomedical Engineering, 72(3):965– 977, 2024. [44]W. Wang and A. C. den Brinker. Algorithmic insights of camera-based respiratory motion extraction. Physiol Meas, 43(7), 2022. [45]Magdy Younes, Jill Raneri, and Patrick Hanly. Staging sleep in polysomnograms: analysis of inter-scorer variability. Journal of Clinical Sleep Medicine, 12(6):885–894, 2016. [46]Zhen Yu, Shaoxing Zhang, Yang Liu, and Qingchao Chen. Ir-based sleep monitoring: Move- ment, respiration, and sleep staging with a structure-aware cross-modal eeg knowledge distilla- tion. IEEE Internet of Things Journal, 12(18):37482–37499, 2025. 13 Appendix A KVSS Dataset A.1 Dataset Composition (a)(b) EEG EEG Chin EMG ECG Flow Thermistor Thoracic Motion Oxygen Saturation AbdominalMotion ⋮ qCase Info qReport qVideo Info qEvent Info -Hypopnea -Apnea Obstructive -Respiratory Arousal ⋮ JSON MP4 EDF EEG EOG Chin EMG ECG Flow Thermistor Abdominal Movement 푆푝푂 ! •640 x 480, 5fps •Anonymization: identifiable regions (e.g.face, tatoo) blurred Thoracic Movement Figure A.1: Composition of the KVSS dataset. (a) Overnight PSG acquisition with synchronized multimodal signals, anonymized NIR video, and expert sleep-stage annotations. (b) Representative NIR frames showing variation in subjects, body positions, bedding conditions, and room layouts. Figure A.1 summarizes the data composition and acquisition pipeline of the Korea Video Sleep Study (KVSS) [1,9]. As shown in Figure A.1(a), the Hospital A subset contains 475 overnight NIR sleep recordings (∼3,250 hours) paired with synchronized PSG and structured clinical metadata. PSG signals include EEG, EOG, EMG, ECG, airflow, thoracic and abdominal respiratory effort, and blood oxygen saturation (SpO 2 ), from which sleep stages were annotated by certified technicians following AASM guidelines [2]. NIR videos were captured at 640×480 resolution and 5 fps using a wall- mounted camera positioned approximately 3.5 m from the subject with an oblique view. Identifiable regions, including faces and tattoos, were blurred to protect privacy while preserving motion cues relevant to sleep analysis. The recordings were collected during routine overnight sleep studies with a fixed far-field viewpoint, where subjects were typically covered by bedding. Figure A.1(b) illustrates substantial variation in body position, bedding, occlusion, and room layout. Compared with closer-range or more controlled camera setups, these conditions attenuate visible motion, increase occlusion, and introduce inter- subject variability, making KVSS a challenging testbed for video-based sleep staging. For all experiments, we split the 475 subjects into disjoint training, validation, and test sets of 335/70/70 using stratified sampling over sex, age, and OSA severity. A.2 Demographic and Clinical Characteristics Age Group Counts 80-89 70-79 60-69 50-59 40-49 30-39 20-29 0 100 200 300 Sex Male Female 22 37 91 80 59 51 316 4 9 41 43 17 122 (25.7%) 353 (74.3%) BMI Sex MaleFemale 40 35 30 25 20 26.20 25.55 10 30 50 70 90 110 130 200 100 0 AHI Counts Severity MildModerateSevereNormal 39 (8.2%) 103 (21.7%) 125 (26.3%) 208 (43.8%) (a)(b)(c) Figure A.2: Demographic and clinical characteristics of the Hospital A cohort. (a) Sex and age. (b) BMI by sex. (c) OSA severity and AHI. 14 [640] Video Input (150 Frames) Pixelwise Temporal Encoder (PTE) Sleep Stage Channel Concatenation Feature for ISDL Conv3D (11 →11, k=3×1×1, s=3) [C, T, H, W] = [1, 150, 224,224] Projection [11, 150, 224,224] Conv3D (1 →3, k=3×1×1, s=1) Temporal Conv3D (3 →5, k=3×1×1, s=1) Temporal Conv3D (5 →5, k=3×1×1, s=1) Temporal Convolution with Batch Normalization and Swish Activation (퐶 !" →퐶 #$% , k: kernel, s: stride) Conv3D (1 →3, k=5×1×1, s=1) Temporal Conv3D (3 →5, k=5×1×1, s=1) Temporal Conv3D (5 →5, k=3×1×1, s=1) Classifier MoViNet-A2 Same configuration as in the original paper, except for the input channels [640, 50, 7, 7] [5][256] [11, 50, 224,224] Global Avg Pool MLP Layer Feature for SDM [640, 50, 7, 7] Figure B.1: Architecture of the subject-relative micro-motion representation module. PTE enriches each 30-second epoch with pixel-wise temporal responses, which are concatenated with the original video before strided temporal projection and MoViNet-A2 encoding. GAP features are used for epoch-level classification and ISDL, while pre-GAP feature maps are passed to SDM. Figure A.2 summarizes the demographic and clinical characteristics of the 475 subjects from Hospi- tal A. The cohort spans a broad age range and includes 353 males (74.3%) and 122 females (25.7%), reflecting the composition of patients undergoing clinical sleep testing. BMI distributions show substantial inter-individual variability, with median BMI values of 26.20 for males and 25.55 for females. The cohort also covers a wide OSA severity spectrum, with 39 normal (8.2%), 103 mild (21.7%), 125 moderate (26.3%), and 208 severe cases (43.8%) according to AHI. Together, Figures A.1 and A.2 show that the KVSS Hospital A cohort is diverse in demographics, body type, clinical severity, and video acquisition conditions. This diversity supports evaluation under heterogeneous sleep populations and realistic clinical recording conditions. B ViNUSS Architecture and Experimental Details B.1 ViNUSS Architecture B.1.1 Subject-Relative Micro-Motion Representation. Figure B.1 illustrates the detailed architecture of the subject-relative micro-motion representation module. Each 30-second epoch, comprising 150 frames at 5 fps, is first processed by the Pixel-Wise Temporal Encoder (PTE). PTE consists of temporal-only convolution branches applied independently at each spatial location, without spatial mixing. The branches use stackedk × 1× 1temporal convolutions with short kernels, producing local temporal responses that are concatenated with the original video along the channel dimension. A strided3× 1× 1temporal projection then reduces the temporal resolution before MoViNet-A2 encoding. Although the individual kernels are short, stacking expands the effective temporal receptive field. Including the temporal projection, the two PTE branches cover local neighborhoods of up to 9 and 13 frames, corresponding to about 1.8 s and 2.6 s at 5 fps. Thus, PTE is designed to expose local sub-epoch motion traces before spatial mixing, while longer-range within-epoch dynamics are handled by the sleep dynamics modeling module. All MoViNet-A2 settings follow the original 15 [E, 128] Spatial Pooling Pre-GAP Spatiotemporal Feature [T, C, H, W] = [50, 640, 7, 7] [51, 256] Configs n_layer : 2 d_state : 64 d_conv : 4 expand : 2 ShortMamba Blocks Short-term Summari ze r Append Learnable Summary Token [50, 640] [51, 256] Projection [256] Projection [50, 256] [E, 128] Bidirectional LongMamba Blocks Add Learnable Positional embedding [E, 128] Classifier Epoch-wise Sleep-stage Predictions [E, 5] Configs n_layer : 4 d_state : 32 d_conv : 4 expand : 2 Extracted Summary Token Long-range Sl eep Dynamics Model Full-night Summary Tokens [E, C] = [E, 256] E: Number of epochs Stack Summary Tokens Figure B.2: Architecture of the sleep dynamics modeling module. ShortMamba summarizes each epoch by spatially pooling pre-GAP feature maps, appending a learnable summary token, and extracting the summary-token output. Bidirectional LongMamba models the stacked epoch tokens over the full night with positional embeddings and outputs stage predictions for all epochs. specification [20], except that the first convolutional layer is modified to accommodate the increased number of input channels after concatenation. The MoViNet-A2 backbone produces spatiotemporal feature maps. Global average pooling (GAP) yields epoch-level features used for sleep-stage classification and the Intra-Subject Discriminability Loss (ISDL). The pre-GAP feature maps are retained as input to the sleep dynamics modeling module. B.1.2 Full-Night Sleep Dynamics Modeling. Figure B.2 illustrates the architecture of Sleep Dynamics Modeling (SDM), which operates on the pre-GAP spatiotemporal feature maps from the representation module. SDM has two temporal scales: ShortMamba summarizes motion evolution within each 30-second epoch, and bidirectional LongMamba models the resulting epoch tokens across the full night. For each epoch, pre-GAP feature maps are spatially pooled into a temporal sequence with 50 temporal positions. After projection, a learnable summary token is appended to the sequence. ShortMamba models within-epoch temporal dynamics, and the output corresponding to the summary token is used as the compact epoch-level motion representation. The epoch-level tokens are stacked chronologically across the full night. Each token is augmented with a learnable positional embedding, and the resulting sequence is processed by bidirectional LongMamba. The updated tokens are fed into a shared classifier to produce sleep-stage predictions for all epochs. This two-scale design captures within-epoch motion evolution and full-night sleep progression while avoiding post-hoc smoothing of predicted labels. The Mamba implementation enables linear-time modeling of the full-night epoch sequence. B.2 ViNUSS Training Details Video Preprocessing. All NIR sleep videos in KVSS were stored as MP4 files at 640×480 resolution and 5 fps. For each night, frames were cropped to a 400×400 region centered on the bed to retain body motion while removing irrelevant background, and then resized to 224×224 to reduce computational cost. The continuous video stream was segmented into 30 s epochs aligned with sleep-stage annotations, so that each epoch contained 150 frames. For epochs with fewer than 150 frames, such as recording boundaries, we applied last-frame padding to enforce a fixed temporal length. For efficient I/O during training, all epochs from each subject were stored as fixed-size tensors in a single HDF5 file and accessed directly by the data loader. 16 Subject-Relative Micro-Motion Representation Training. The MoViNet-A2 backbone in the subject-relative micro-motion representation module is initialized from a publicly available Kinetics- 600-pretrained checkpoint [5,20] and fine-tuned on KVSS. Training is performed on four NVIDIA A6000 GPUs using weighted random sampling. Each optimizer epoch samples approximately 9% of the full training set. The effective batch size is 8 triplet samples, corresponding to 24 sleep epochs per optimization step because each sample contains an anchor, positive, and negative epoch for within-subject contrast. We use AdamW with a learning rate of10 −3 . The module is optimized with the sum of the epoch-level cross-entropy loss and ISDL. For ISDL triplet construction, we use class-balanced anchor sampling to reduce the effect of sleep-stage imbalance. For each sampled subject, an anchor stage is first sampled uniformly from the five AASM stages with valid candidate epochs for that subject. An anchor epoch is then sampled from the selected subject and stage. The positive epoch is sampled from the same subject and stage, and the negative epoch is sampled from the same subject but a different stage. Positive and negative epochs are selected uniformly from their respective candidate pools. We apply the following on-the-fly augmentations: normalization with dataset mean and standard deviation, random brightness jitter (±0.2), random contrast jitter (±0.2), horizontal flip with probability 0.5, random rotation up to±30 ◦ , and random scaling uniformly sampled from [0.8, 1.1]. The model is trained with five AASM stage labels, Wake, N1, N2, N3, and REM. For four-class evaluation, N1 and N2 are merged into Light sleep. Full-Night Sleep Dynamics Modeling Training.After training the subject-relative micro-motion representation module, we extract and store the pre-GAP spatiotemporal feature maps for all epochs. These saved features are used as input to the sleep dynamics modeling module. SDM is trained on four NVIDIA A6000 GPUs using subject-level sequences, where each data point corresponds to one full-night recording. To handle variable-length nights without padding, sequences are processed individually and gradient accumulation is used to obtain an effective batch size of 64 subject-level sequences. We use AdamW with a learning rate of 10 −4 . B.3 Evaluation Metrics We evaluate four-class sleep staging performance using accuracy, macro-F1, and Cohen’sκ. For completeness, we summarize the definitions used in the main paper. Accuracy. Accuracy measures the proportion of correctly classified epochs: Acc = 1 N N X i=1 I(ˆy i = y i ), where N is the total number of evaluated epochs. Macro-F1 (MF1). For each class c, precision, recall, and F1-score are computed as Precision c = TP c TP c + FP c ,Recall c = TP c TP c + FN c , F1 c = 2· Precision c · Recall c Precision c + Recall c . Macro-F1 is the unweighted average across classes: MF1 = 1 C C X c=1 F1 c , whereCis the number of sleep-stage classes. This metric assigns equal weight to each stage and is therefore useful under class imbalance. Cohen’s Kappa (κ).κmeasures agreement between predictions and ground truth while accounting for chance agreement: κ = p o − p e 1− p e , 17 where p o is the observed agreement and p e is the expected agreement under random labeling: p e = C X c=1 n c,pred · n c,true N 2 . Here, n c,pred and n c,true denote the number of predicted and true labels for class c. Per-subject accuracy for stratified analysis.For robustness analyses across recording conditions and subject characteristics, we compute accuracy separately for each subject: Acc s = 1 N s N s X i=1 I(ˆy s,i = y s,i ), whereN s is the number of epochs for subjects. For a subgroupG, we report the mean per-subject accuracy: Acc G = 1 |G| X s∈G Acc s . This assigns equal weight to each subject within a subgroup, regardless of recording length. C SleepVST Re-implementation Following the original SleepVST setup [6], we pre-train the model on the SHHS [33] and MESA [8] contact-sensor datasets for four-class sleep staging (Wake, N1/N2, N3, REM). C.1 Datasets for SleepVST Training Sleep Heart Health Study (SHHS) [33]. SHHS is a large multicenter cohort designed to study the cardiovascular and respiratory consequences of sleep-disordered breathing in adults. It includes overnight home polysomnography from more than 6,400 participants, together with demographic, anthropometric, and clinical questionnaires. PSG recordings include EEG, EOG, EMG, airflow, respiratory effort, and SpO 2 , with sleep stages and respiratory events manually scored. SHHS is widely used for sleep staging and sleep-disordered breathing research, and access requires approval through the National Sleep Research Resource (NSRR). Multi-Ethnic Study of Atherosclerosis (MESA) [8]. MESA is a longitudinal NHLBI cohort originally consisting of 6,814 Black, White, Hispanic, and Chinese-American adults aged 45–84. Between 2010 and 2012, 2,237 participants completed the MESA Sleep Exam, which included unattended overnight PSG, 7-day wrist actigraphy, and sleep questionnaires. The dataset provides PSG recordings with rich multi-ethnic demographic and cardiovascular phenotyping, and is available through approved data access requests. C.2 Architecture and Pre-training Electrocardiogram (ECG) and thoracic respiratory effort signals are extracted from each dataset and converted into cardiac waveforms (CW) and breathing waveforms (BW), respectively. After filtering, normalization, and 30-second patchification, the waveforms are encoded through convolutional layers and a Transformer encoder to produce per-epoch representations. The model is pre-trained on 240-epoch, 2-hour windows using four-class cross-entropy loss with the AdamW optimizer. C.3 Transfer to KVSS The KVSS dataset provides NIR video recordings with time-aligned contact-sensor signals. SleepVST requires reliable cardiac and respiratory waveforms, but direct recovery of these waveforms from KVSS videos is challenging. Facial regions are blurred for privacy, which limits recovery of subtle facial color changes needed for rPPG-based cardiac waveform estimation. We also attempted to extract respiratory waveforms from NIR video using two representative video-based respiratory estimation methods, phase-based motion magnification [41] and a camera-based respiratory extraction 18 method [44]. On KVSS, the reconstructed waveforms showed substantial deviations in dominant frequency and amplitude compared with contact thoracic-belt signals, especially under bedding, position changes, weak motion, and non-respiratory body movement. Consequently, a faithful video-only implementation of the original SleepVST pipeline could not be realized under KVSS conditions. To enable a controlled comparison on KVSS, we evaluated two oracle-input variants of SleepVST. In the partial-oracle variant, the cardiac waveform was replaced by the corresponding contact-sensor signal from PSG, while the breathing waveform was video-extracted using phase-based motion magnification [41]. In the full-oracle variant, both cardiac and breathing waveforms were obtained directly from contact sensors. These variants should be interpreted as oracle-input versions of one representative proxy-first architecture, not as universal upper bounds for physiological-signal-based sleep staging or practical video-only baselines. For both variants, the preprocessed waveforms are passed through the frozen SleepVST encoder and Transformer backbone to obtain per-epoch signal features. To incorporate motion information from video, optical flow is estimated using the DIS (Dense Inverse Search) algorithm [21], and the resulting motion feature vector is concatenated with the SleepVST signal feature. A Random Forest classifier is then trained on the combined features for four-class sleep staging on KVSS. D Additional Experiments and Analyses D.1 ViNUSS Results Table D.1: Detailed class-wise performance of ViNUSS. (a) Five-class evaluation. (b) Four-class evaluation after merging N1/N2 into Light and treating N3 as Deep. (a) 5-Class Classification ClassPrec.Rec.F1 Wake0.830.730.78 N10.580.570.58 N20.690.750.72 N30.670.680.67 REM0.820.830.82 Macro Avg.0.720.710.72 Weighted Avg.0.720.710.72 Acc.0.71 κ0.62 (b) 4-Class Classification ClassPrec.Rec.F1 Wake0.830.730.78 Light0.820.860.83 Deep0.670.680.67 REM0.820.830.82 Macro Avg.0.780.770.78 Weighted Avg.0.810.800.80 Acc.0.80 κ0.68 In the four-class setting, Light denotes N1+N2 and Deep denotes N3. Table D.1 reports the detailed class-wise performance of ViNUSS. The model is trained with five AASM sleep-stage labels (Wake, N1, N2, N3, REM). For the four-class evaluation used in the main paper, both predictions and ground-truth labels are mapped by merging N1 and N2 into Light sleep and treating N3 as Deep sleep. Thus, the 5-class and 4-class results are obtained from the same trained model, with only the evaluation label mapping changed. D.2 Backbone Selection for Subject-Relative Micro-Motion Representation Table D.2: Backbone selection for the subject-relative micro-motion representation module. All models are trained with the same pre-spatial micro-motion encoding and within-subject stage contrast. ModelParam.Acc.MF1 Video Swin-Tiny [24]28.3M0.510.46 MoViNet-A5 [20]17.4M0.600.53 MoViNet-A2 [20]4.0M0.600.56 To select the video backbone for the subject-relative micro-motion representation module, we evaluated three candidate architectures: Video Swin-Tiny [24], MoViNet-A5, and MoViNet-A2 [20]. 19 All models in this comparison are trained with the same pre-spatial micro-motion encoding and within-subject stage contrast, so the results isolate the effect of the backbone architecture within the representation module. Table D.2 summarizes model size and classification performance. Video Swin-Tiny performs worse than the MoViNet variants, suggesting that generic attention-based video representations do not directly solve NIR sleep staging. This may reflect the mismatch between action-oriented video representations and NIR sleep video, where discriminative evidence is weak, localized, and easily dominated by static subject or bedding cues. The convolutional inductive bias of MoViNet provides a more suitable local spatiotemporal prior for this setting. Among the MoViNet variants, MoViNet-A2 provides the best accuracy-efficiency tradeoff. Although MoViNet-A5 has more than four times the parameters of MoViNet-A2, it does not improve accuracy and yields lower MF1. This suggests that increasing generic backbone capacity alone is insufficient for NIR sleep video, where the key challenge is preserving weak, localized motion evidence under strong subject- and acquisition-specific variability. We therefore select MoViNet-A2 as the backbone of ViNUSS, balancing performance, model size, and practical deployment constraints. D.3 Stratified Performance across Recording Conditions and Subject Characteristics None Partial Full Diagonal Low Medium High No Yes Low Med High 0.0 0.2 0.4 0.6 0.8 1.0 Acc Setup Blanket Occlusion Camera AngleRoom Re-entryLight Intensity Female Male <40 40-60 >60 Under Normal Over Obese <5 5-15 15-30 >30 0.0 0.2 0.4 0.6 0.8 1.0 Acc Demographics Sex Age Group BMI ClassAHI Score Subgroup Mean Overall Mean (0.804) Figure D.1: Per-subject accuracy stratified by recording conditions and subject characteristics on the held-out test set. Each dot represents one subject. The upper row shows recording-condition groups, including blanket occlusion, camera angle, room re-entry, and light intensity. The lower row shows subject-characteristic groups, including sex, age, BMI class, and AHI group. To assess performance consistency under deployment-relevant variation, we conduct a stratified analysis on the held-out test set. Subjects are grouped by recording condition, including blanket occlusion, camera angle, room re-entry, and light intensity, and by subject characteristic, including sex, age group, BMI class, and AHI group. Figure D.1 reports per-subject accuracy for each subgroup. ViNUSS maintains comparable performance across the observed subgroups without clear systematic degradation, suggesting stable behavior across the heterogeneous recording environments and patient characteristics represented in KVSS. D.4 Additional Experiments on Full-Night Sleep Dynamics Modeling D.4.1 Comparison with alternative sequence models. We compare the full-night sequence modeling component of ViNUSS with temporal backbones from four families: recurrent neural networks (RNNs), Transformer-based models, convolutional neural networks (CNNs), and state-space models (SSMs). In this experiment, each 30-second epoch is treated as one timestamp in the full-night sequence. All inter-epoch-only sequence models receive the 20 Table D.3: Comparison of sequence models for full-night sleep dynamics modeling. All models operate on epoch-level representations, where each 30-second epoch is treated as one timestamp. Best results are shown in bold, and second-best results are shown in italics. ModelFamily OverallPer-class F1 Acc. MF1κWake Light Deep REM BiLSTM [13]RNN0.770.720.620.760.800.530.80 PatchTST [28]Transformer 0.740.700.580.710.780.530.76 ModernTCN [10]CNN0.730.680.560.700.770.500.75 Vanilla Transformer [39]Transformer 0.710.620.520.720.770.330.66 Mamba (Long only) [14]SSM0.780.740.650.770.810.590.80 SDM (Short+Long) [14] SSM0.800.780.680.780.830.670.82 RNN: recurrent neural network; CNN: convolutional temporal model; SSM: state-space model. Light denotes N1+N2 and Deep denotes N3. same epoch-level representations, so their comparison isolates the effect of the inter-epoch temporal model. The “Mamba (Long only)” row corresponds to the bidirectional LongMamba path without the within-epoch ShortMamba summarizer, whereas “SDM (Short+Long)” reports our full two-scale design. Table D.3 reports overall and per-class results. Mamba (Long only) achieves the strongest performance among inter-epoch-only sequence models, with 0.78 accuracy, 0.74 MF1, and 0.65κ. BiLSTM is the closest alternative, while vanilla Transformer, PatchTST, and ModernTCN perform worse under the same epoch-level input setting. This suggests that full-night sleep staging benefits from sequence models that maintain and update compact temporal states over the overnight trajectory, and supports Mamba as an efficient state-space backbone for long-sequence modeling. Per-class results show that Mamba (Long only) outperforms the other inter-epoch-only sequence models across all classes, with the clearest advantage on Deep sleep. The full SDM further improves every class over LongMamba alone, especially Deep sleep, where F1 increases from 0.59 to 0.67. These results support our design choice: LongMamba is effective for inter-epoch full-night modeling, while ShortMamba improves the epoch-level motion summaries passed to the full-night model. Implementation details of compared sequence models. All compared models are trained to produce per-epoch five-class logits (Wake, N1, N2, N3, REM); reported four-class results are obtained by merging N1 and N2 into Light and treating N3 as Deep. BiLSTM. We use a two-layer bidirectional LSTM [13] with hidden size 256. Epoch representations are projected to 256 dimensions using LayerNorm and a linear layer, processed over the full-night sequence, and decoded by a two-layer MLP head at each epoch. Vanilla Transformer. We use a vanilla Transformer [39] with three encoder layers,d model = 256, 8 attention heads,d f = 1024, pre-normalization, GELU activations, and dropout 0.1. This base- line models the full-night epoch sequence with bidirectional self-attention and a per-epoch MLP classification head. PatchTST. We adapt PatchTST [28] to epoch-level sequence classification. Patches are formed over consecutive epoch-level timestamps, not raw video frames, usingpatch_len = 16andstride = 8. We use the channel-independent TSTiEncoder withd model = 128, 16 heads, 3 layers, andd f = 256. Because PatchTST is originally designed for forecasting, we replace the forecasting head with a classification head that maps patch-level outputs back to epoch-level predictions by overlap averaging. ModernTCN. We use a ModernTCN [10] encoder with ETTh1-style hyperparameters and attach a per-epoch classification head. The encoder uses RevIN, a stem layer, and one ModernTCN stage with large-kernel depthwise temporal convolution and grouped ConvFFN blocks. The output is compressed and decoded by a two-layer MLP head at each epoch. Mamba (Long only). The Long-only Mamba baseline uses the full-night LongMamba path without the within-epoch ShortMamba summarizer. It consists of four bidirectional LongMamba blocks with d model = 128 and d state = 32, using epoch-index embeddings and a shared per-epoch classifier. SDM (Short+Long). The full SDM uses both ShortMamba and LongMamba. ShortMamba first summarizes within-epoch motion evolution into compact epoch tokens, and LongMamba then models the stacked epoch tokens across the full night. 21 D.4.2 Effect of temporal context length. To examine how temporal context length affects full-night sleep dynamics modeling, we train the Mamba (Long only) backbone with context windows ofW ∈10, 30, 50, 100, 200, 500, full, while keeping all other hyperparameters fixed. Each 30-second epoch is treated as one timestamp, and each training iteration samplesK = 5random length-Wcrops per subject. When a recording contains fewer thanWepochs, zero padding is used withignore_indexmasking. At inference, we use overlapping sliding windows with strideW/10and aggregate predictions by averaging logits for epochs covered by multiple windows. This setup isolates the effect of temporal context length while keeping the sequence model and training recipe fixed. Table D.4: Effect of temporal context length on full-night sleep dynamics modeling. Each 30-second epoch is treated as one timestamp. Window sizeAcc.MF1 κ 100.720.640.53 300.740.670.58 500.760.710.61 1000.770.720.63 2000.770.730.63 5000.780.740.64 full0.780.740.65 Table D.4 shows that longer context consistently improves sleep dynamics modeling. Accuracy increases from 0.72 atW = 10to 0.78 with full-night context, while MF1 improves from 0.64 to 0.74 andκfrom 0.53 to 0.65. The largest gains occur as the window grows up toW = 100, after which accuracy begins to plateau. However, MF1 andκcontinue to improve with longer context, suggesting that full-night information is especially helpful for less frequent or temporally structured stages such as Deep and REM. D.5 Example Full-Night Test Predictions Subj ect -21 Epoc h Subj ect -30 Epoc h Subj ect -39 Epoc h Subj ect -53 Epoc h WakeLightDeepREM GT Label Pred GT Label Pred GT Label Pred GT Label Pred Figure D.2: Example full-night predictions on heterogeneous test subjects. Ground-truth (GT) labels and model predictions are shown for four representative subjects with different sleep-stage trajectories. 22 Figure D.2 provides additional full-night prediction examples complementing Figure 5. The selected subjects exhibit heterogeneous sleep architectures, ranging from multi-cycle sleep patterns to irregular profiles dominated by Wake and Light sleep. Subject 21 shows multiple stage transitions across the night, and the model follows the changing proportions of Light, Deep, and REM stages without collapsing to a fixed template. Subjects 30, 39, and 53 illustrate more atypical patterns, including recordings with little or no Deep/REM sleep, prolonged wakefulness, and frequent Wake–Light alternations. Across these examples, most prediction errors occur near stage boundaries or short transient segments, while longer sleep-stage structures are largely preserved. These cases further illustrate that full-night modeling helps ViNUSS track subject-specific overnight trajectories under diverse sleep patterns. E Sleep Video EigenCAM (SVECAM) Saliency-overla id frames Spatial SaliencyTemporal Saliency (a) Subject-64 Epoch-394(b) Subject-53 Epoch-540 Spatial SaliencyTemporal Saliency Time Normalized Value Temporal Saliency GT Breathing Wave Time Normalized Value Temporal Saliency GT Breathing Wave Figure E.1: SVECAM visualizations for two representative NIR sleep-video epochs. For each epoch, we show spatial saliency maps, temporal saliency curves, and saliency-overlaid frames. Frames are sampled at 1 Hz over the 30 s epoch and arranged sequentially from left to right, wrapping every five seconds. NIR sleep videos provide weak and indirect visual evidence for sleep-stage discrimination, mak- ing it important to inspect which spatiotemporal regions the model uses. We therefore extend EigenCAM [27] to 3D activation tensors from sleep-video models. Given an activation tensorA∈ R C×T×H×W , a direct EigenCAM application would flatten temporal and spatial dimensions before singular value decomposition (SVD), which can mix motion-related and appearance-related structure. We instead use Sleep Video EigenCAM (SVECAM), which derives temporal and spatial channel weights separately and then combines them. 23 •Temporal weighting. For each spatial location(h,w), we apply SVD toA h,w ∈ R C×T and use the leading left singular vectoru (h,w) T ∈ R C as a channel-weight vector capturing temporal co-variation at that location. •Spatial weighting. For each framet, we apply SVD toA t ∈ R C×(H·W ) and use the leading left singular vectoru (t) S ∈ R C as a channel-weight vector capturing spatially coherent activation at that time. We form a spatiotemporal weight tensorW ∈ R C×T×H×W by broadcasting and summing the normalized temporal and spatial channel weights: W(c,t,h,w) = ̃ u (h,w) T (c) + ̃ u (t) S (c), where ̃ udenotes anℓ 2 -normalized vector. The final SVECAM saliency is computed as a channel- weighted activation map followed by ReLU: SVECAM(t,h,w) = ReLU X c A c,t,h,w W c,t,h,w ! . This formulation highlights activation channels that are temporally informative at each spatial location and spatially coherent within each frame. Because NIR sleep videos are captured from a fixed camera and the body layout is relatively stable within many 30-second epochs, preserving this spatiotemporal structure is useful for interpreting localized sleep-related motion. SVECAM is applied to the subject-relative micro-motion representation module. Figure E.1 presents visualizations for representative 30 s epochs from different subjects. In example (a), saliency is con- centrated around the thoraco-abdominal region, and the temporal saliency curve follows respiration- related periodic motion. This suggests that the model uses breathing-related visual motion without explicitly reconstructing a respiratory waveform. In example (b), the epoch contains transient body movements in addition to respiration-related motion. SVECAM highlights larger movements when they occur, while saliency returns to the thoraco-abdominal region during quieter intervals. These examples suggest that ViNUSS uses both localized respiration-related motion and transient behavioral motion as visual evidence for sleep staging. 24