Paper deep dive
PPG-LM: A Photoplethysmography-Language Model with Multi-Level Clinical Alignment
Xiaoda Wang, Minxiao Wang, Maxwell A Xu, Patrick Langer, Kaiqiao Han, Defu Cao, Xiao Luo, Yuzhe Yang, Yan Liu, Xiao Hu, Yizhou Sun, Wei Wang, Carl Yang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Photoplethysmography (PPG) is widely recorded by clinical monitors and consumer wearables, providing a scalable source of continuous physiological information. These recordings offer an opportunity for physiological assessment at scale, but realizing this potential requires models to learn from both signal-derived physiological supervision and broader clinical context captured in electronic health records (EHRs). This involves aligning information spanning local observations, care events, and entire visits with PPG representations at corresponding temporal scales. However, existing PPG foundation models primarily rely on task-specific prediction heads, while the medical knowledge of large language models does not necessarily translate into waveform understanding. To bridge this gap, we introduce PPG-LM, the first PPG-language model family to learn physiological representations from both signal-derived supervision and broader clinical context captured in EHRs. To construct clinically grounded captions, we develop an automatic captioning pipeline that generates segment-, event-, and visit-level descriptions from signal measurements and structured EHR records. We then learn from these pairs through a two-stage framework that first establishes segment-language correspondence through contrastive learning and waveform-conditioned captioning, then extends alignment to events and visits through time-aware aggregation and temporal statement matching. Pretrained on approximately 73k hours of PPG, PPG-LM supports language-based recognition, cross-modal retrieval, and segment captioning. Experiments on MC-MED, MIMIC-III, and VitalDB show improved retrieval and caption factuality over language-model baselines and gains over PPG and time-series foundation models on multiple clinical prediction tasks.
Tags
Links
- Source: https://arxiv.org/abs/2609.33516v1
- Canonical: https://arxiv.org/abs/2609.33516v1
Trouble viewing inline? Open PDF directly ā
Full Text
131,303 characters extracted from source content.
Expand or collapse full text
PPG-LM: A PhotoplethysmographyāLanguage Model with Multi-Level Clinical Alignment Xiaoda Wang, Minxiao Wang, Maxwell A Xu, Patrick Langer, Kaiqiao Han, Defu Cao Xiao Luo, Yuzhe Yang, Yan Liu, Xiao Hu, Yizhou Sun, Wei Wang, Carl Yang Emory University University of California, Los Angeles Google Stanford University University of Southern California University of WisconsināMadison Abstract Photoplethysmography (PPG) is widely recorded by clinical monitors and consumer wearables, providing a scalable source of continuous physiological information. These recordings offer an opportunity for physiological assessment at scale, but realizing this potential requires models to learn from both signal-derived physiological supervision and broader clinical context captured in electronic health records (EHRs). This involves aligning information spanning local observations, care events, and entire visits with PPG representations at corresponding temporal scales. However, existing PPG foundation models primarily rely on task-specific prediction heads, while the medical knowledge of large language models does not necessarily translate into waveform understanding. To bridge this gap, we introduce PPG-LM, the first PPGālanguage model family to learn physiological representations from both signal-derived supervision and broader clinical context captured in EHRs. To construct clinically grounded captions, we develop an automatic captioning pipeline that generates segment-, event-, and visit-level descriptions from signal measurements and structured EHR records. We then learn from these pairs through a two-stage framework that first establishes segmentālanguage correspondence through contrastive learning and waveform-conditioned captioning, then extends alignment to events and visits through time-aware aggregation and temporal statement matching. Pretrained on approximately 73k hours of PPG, PPG-LM supports language-based recognition, cross-modal retrieval, and segment captioning. Experiments on MC-MED, MIMIC-I, and VitalDB show improved retrieval and caption factuality over language-model baselines and gains over PPG and time-series foundation models on multiple clinical prediction tasks. 1 Introduction Photoplethysmography (PPG) is a low-cost optical measure of peripheral blood-volume changes, widely recorded by clinical monitors and wearables (Allen, 2007; Charlton et al., 2022). These recordings provide cardiac and vascular information at scale, motivating foundation models that learn generalizable waveform representations (Pillai et al., 2025; Saha et al., 2025; Nie et al., 2026). However, existing PPG foundation models primarily support task-specific prediction, with limited support for recognizing, retrieving, or describing physiological observations through clinical language. Recent efforts to adapt large language models (LLMs) to PPG question answering (Pham et al., 2026) address a complementary gap: textual medical knowledge alone does not ensure an understanding of pulse morphology and timing (Wang et al., 2026). PPG and Language alignment offers a shared interface for these capabilities, with demonstrated benefits in medical imaging (Huang et al., 2021), wearable sensing (Zhang et al., 2026a), and sleep analysis (Xu et al., 2026). For PPG, electronic health records (EHRs) provide clinical context, including patient presentation, medical history, and care events, that complements signal-derived physiological measurements (He et al., 2026; Kansal et al., 2025). Expressing these complementary sources as language supervision offers an opportunity to learn both waveform characteristics and broader clinical associations. Training a PPGālanguage model to learn both fine-grained waveform characteristics and broader clinical information poses two coupled challenges: (i) Clinically grounded caption generation. Clinical records contain heterogeneous measurements, histories, and care events but do not directly provide descriptions paired with PPG observations. Constructing useful supervision requires selecting and verbalizing information from both signal measurements and EHRs, distinguishing measured waveform properties from documented clinical context, and organizing the resulting descriptions at segment, event, and visit levels. (i) Multi-level temporal alignment. These descriptions correspond to individual windows, observations around care events, and variable-length visits. Event- and visit-level alignment therefore requires aggregating multiple windows, but pooling without temporal information can obscure physiological changes. A model must capture these temporal relationships while maintaining segmentālanguage correspondence as broader clinical supervision is introduced. Figure 1: Motivation and overview of PPG-LM. PPG FMs learn waveform representations with limited integration of language and EHR context. General LLMs capture medical knowledge from text but require adaptation to encode raw PPG. PPG-LM bridges this gap by pairing PPG with captions derived from physiological measurements and structured EHRs. The radar plot highlights PPG-LMās superior performance over baselines on selected segment-, event-, and visit-level tasks. Table 1: Comparison of PPGālanguage studies. Study Signal Text Source Text scope Raw Context Segment Event Visit SensorLM (Zhang et al., 2026a) ā 1 day Sensor ā ā ā NormWear (Luo et al., 2024) ā 6 s Labels ā ā ā CSFM (Gu et al., 2026) ā 10 s Labels ā ā ā PulseLM (Pham et al., 2026) ā 10 s Labels ā ā ā CAP (He et al., 2026) ā 5 min EHR ā ā ā PPG-LM ā Variable Signal+EHR ā ā ā To address these challenges, we introduce PPG-LM, a family of PPGālanguage models trained with clinically grounded, multi-level supervision. To construct clinically grounded captions, we develop an automatic captioning pipeline combines signal measurements and structured EHR records to generate segment-, event-, and visit-level descriptions, each paired with PPG observations over the corresponding temporal scope. We then learn these correspondences through a two-stage alignment framework with waveform and text encoders shared across levels. Stage 1 learns segmentālanguage correspondence from captions derived from signal measurements and time-matched vital signs, combining contrastive learning and waveform-conditioned captioning with auxiliary clinical-attribute supervision. Stage 2 extends alignment to medication events and variable-length visits: signed time offsets guide event aggregation, while global and time-anchored visit latents support clinical-caption alignment and temporal statement matching. Our main contributions are as follows: ā¶PPG-LM, a family of PPGālanguage models. We introduce the first PPGālanguage model family to learn physiological representations from multi-level clinical language supervision, training on approximately 73,700 hours of PPG to support recognition, retrieval, and captioning. ā·Clinically grounded multi-level caption generation. We develop an automatic pipeline that combines signal measurements and structured EHR records to generate segment-, event-, and visit-level captions, preserving the evidence source and temporal scope of each description. āøTwo-stage multi-level alignment. We propose a two-stage framework that first learns segmentālanguage correspondence through contrastive learning and captioning, then extends alignment to event and visit levels through time-aware aggregation and temporal statement matching. ā¹Comprehensive evaluation. Experiments on MC-MED, MIMIC-I, and VitalDB assess recognition, retrieval, captioning, temporal understanding, and clinical prediction against PPG FMs, and LLMs, demonstrating language capabilities and transferable representations across care settings. 2 Related Work Foundation models for PPG and time series. PPG foundation models learn transferable representations through contrastive and non-contrastive learning with participant and temporal cues (Abbaspourazad et al., 2024; Ding et al., 2024), morphology-aware objectives, and uncurated wearable data (Pillai et al., 2025; Saha et al., 2025). Generative approaches use autoregressive prediction and masked reconstruction (Chen et al., 2025; Guo et al., 2026; Zhang et al., 2026b), while cross-signal methods use synchronized ECG and respiratory supervision (Nie et al., 2026; Geenjaar et al., 2026). However, their signal-centered objectives leave waveformālanguage correspondence unmodeled, and clinical applications typically require task-specific predictors. LLMs for PPG modeling. LLM-based methods use numerical prompting and parameter-efficient adaptation for physiological estimation (Liu et al., 2024; Ali et al., 2025). PulseLM (Pham et al., 2026) pairs pretrained PPG encoders with instruction-tuned LLMs for question answering, while arrhythmia systems combine signalātext alignment with retrieval-augmented report generation (Huh et al., 2026). Video-based remote PPG methods use text prototypes and physiological prompts for hemodynamic modeling (Xie et al., 2026). These methods center on prescribed estimates, answers, or reports, leaving joint waveformālanguage learning across clinical temporal levels unaddressed. Time series and language alignment. Time seriesālanguage models pair sensor signals with activity, trend, behavioral, and affective descriptions (Moon et al., 2023; Li et al., 2025; Zhang et al., 2026a; Singh et al., 2024; Chen et al., 2026), and clinical signals with text across ECG, EEG, sleep, and multivariate recordings (Wang et al., 2025; Jiang et al., 2025; Xu et al., 2026; Langer et al., 2026). PPGāEHR alignment uses medical histories as patient-level anchors (He et al., 2026). Multi-level objectives capture local and global semantics in medical imaging and video (Huang et al., 2021; Wang et al., 2022; Ashutosh et al., 2023; Yuan et al., 2024; Bannur et al., 2023), and finer physiological structure in ECG and sleep (Wang et al., 2025; Xu et al., 2026). However, these approaches do not directly support PPGālanguage alignment across multi-level clinical timescales: none jointly grounds local physiological descriptions, care-event context, and visit-level clinical information. 3 Clinically Grounded Caption Generation Figure 2: Clinically Grounded Caption Generation: (1) Segment-level captions describe physiological features and signal quality within 30-second PPG windows. (2) Event-level captions provide medication context for fixed windows before and after administration. (3) Visit-level captions associate variable-length recordings with demographics, and medical history from clinical EHRs. To train PPG-LM, we construct a corpus from MC-MED (Kansal et al., 2025), which pairs PPG recordings with structured electronic health records (EHRs) from emergency-department visits. We generate segment-, event-, and visit-level captions from signal measurements and EHRs, pairing them with PPG observations at the corresponding temporal scales (Figure 2). Captions use fixed language templates with reproducible template selection and sentence shuffling. Corpus details, captioning rules, and template examples are provided in Appendices D.1, B, and E, respectively. Segment-level captions. Using pyPPG (Goda et al., 2024), we describe up to eight attributes per PPG window: pulse rate, rate trend, rhythm regularity, pulsatile amplitude, signal quality, augmentation-related morphology, a vascular-response index, and beat-to-beat amplitude variability. Available fields yield numerical or qualitative descriptions. Event-level captions. We extract administration timestamps, drug names, and routes from EHR medication records. Route filtering and drug-name rules map eligible records to ten classes; near-duplicates are removed within each visit and class. For an administration at time a, we select up to three latest windows in [aā30,a)[a-30,a) and up to five evenly spaced windows in [a,a+60][a,a+60] (times in minutes). The caption names the administered class and is paired with these windows. Controls follow the same sampling rule within the visit, excluding anchors near recorded administrations. Visit-level captions. We combine encounter records and demographics to summarize age, sex, arrival mode, triage acuity, normalized chief complaints, and first recorded vital signs. Prior-diagnosis codes are mapped to clinical descriptions, deduplicated, and ranked by relevance and recency; active home medications are summarized by class. These fields form one caption paired with observations across the encounter. Ordered segment rates and rhythm labels additionally yield temporal statements of state occurrence, proportion, contiguous duration, and rate ordering. 4 Method Figure 3: Two-stage multilevel alignment. (a) Stage 1 learns PPG and text encoders through contrastive segmentālanguage alignment and PPG-conditioned caption generation. (b) Stage 2 extends the shared encoders to event- and visit-level alignment. The captions in Section 3 pair local physiological measurements and broader clinical context with PPG observations at different temporal scopes. Learning from these pairs requires both a window-level language interface and representations that combine observations across time. PPG-LM therefore uses shared waveform and text encoders with scope-specific read-outs, trained in two stages (Figure 3). Stage 1 establishes segmentālanguage correspondence through contrastive learning and captioning. Stage 2 learns to aggregate window features for event and visit descriptions, adding temporal statement supervision while retaining the segment objective. Problem definition. A visit is a time-ordered sequence V=((xi,ti))i=1nVV=((x_i,t_i))_i=1^n_V, where xiāā3750x_i ^3750 is a standardized 30-second PPG window and tit_i is its start time in minutes since the first window. A caption cāc , āāseg,evt,vis ā\seg,evt,vis\, is paired with a single window, windows around a medication or control anchor, or observations distributed across a visit, respectively. In addition to its clinical caption, a visit may have temporal statements TV=dVārr=1mVT_V=\d_Vr\_r=1^m_V describing state occurrence, proportion, contiguous duration, and rate ordering. PPG and text encoders. We share both encoders across levels to reuse waveform and language features, while adapting their read-outs to each descriptionās scope. The PPG encoder fĪøf_Īø divides a window into non-overlapping patches, adds learned positional embeddings, and applies Transformer blocks (Vaswani et al., 2017). Attention pooling summarizes the resulting patch states HiH_i into a window feature gig_i. The text encoder ĻĻ _Ļ is a causal Transformer using CLIPās byte-pair encoding (BPE) vocabulary (Radford et al., 2021). A learned prefix eāe_ identifies the description level, and a level-specific projection PāP_ maps the end-of-sequence (EOS) state to a normalized text embedding: gi=fĪøā(xi)g_i=f_Īø(x_i), uāā(c)=normā”(PāāĻĻā([eā;c])EOS),u (c)=norm\! (P_ \, _Ļ([e_ ;c])_EOS ), where normā”(v)=v/āvā2norm(v)=v/\|v\|_2. Waveform read-outs operate on gig_i for segment alignment and on collections of window features for event and visit alignment. Encoder details are given in Appendix C.1. 4.1 Stage 1: SegmentāLanguage Pretraining Following CoCa (Yu et al., 2022), Stage 1 combines contrastive alignment of windows and descriptions with token-level caption supervision. For a batch of B windowācaption pairs (xi,ci)(x_i,c_i) from Section 3, we form normalized waveform embeddings ziseg=normā”(Wsegāgi)z_i^seg=norm(W_segg_i) and logits Qiāj=ssegā(ziseg)ā¤āusegā(cj)Q_ij=s_seg(z_i^seg) u^seg(c_j), with learned scale sseg>0s_seg>0. To avoid treating captions with identical binned measurements as negatives, we assign content keys Īŗi _i and use iseg=iāŖj:Īŗjā ĪŗiC_i^seg=\i\āŖ\j: _jā _i\ within the batch. For candidate sets C, InfoNCE (van den Oord et al., 2018) in CLIPās symmetric form (Radford et al., 2021) is āNCE(Q;)=ā12āBāi=1B[logexpā”(Qiāi)ājāiexpā”(Qiāj)+logexpā”(Qiāi)ājāiexpā”(Qjāi)].L_NCE(Q;C)=- 12B _i=1^B [ (Q_i) _j _i (Q_ij)+ (Q_i) _j _i (Q_ji) ]. (1) The two terms align signals and text in both directions; later levels reuse this loss with their respective logits and candidate sets. For complementary token-level supervision, a separate attention pooler forms waveform memory MiM_i from the shared patch states HiH_i. A multimodal decoder receives causal token states from the shared text encoder and cross-attends to MiM_i to predict caption tokens. With teacher forcing, we minimize ācap=āNtokā1āi,rlogpĻ,Ī·(ci,rā£ci,<r,Mi)L_cap=-N_tok^-1 _i,r p_Ļ,Ī·(c_i,r c_i,<r,M_i) over the NtokN_tok non-padding targets, including EOS; Ī· denotes the additional captioning parameters. The combined objective āseg=āNCEā(Q,seg)+ācapL_seg=L_NCE(Q;C^seg)+L_cap trains both shared encoders from scratch and is retained in Stage 2. 4.2 Stage 2: Multi-Level Alignment Event and visit descriptions concern multiple observations. Stage 2 therefore adds temporal aggregation over the shared window features and aligns the resulting representations with descriptions at the corresponding scope. Event-level alignment. Event captions pair a medication class or no-administration control with nearby PPG windows (Section 3). For view j, we preserve timing relative to its anchor aja_j by adding signed offset embeddings Ļā”(tiāaj) (t_i-a_j) and pooling features over selected window indices EjE_j with a learned query: zjevt=normā”(WevtāAttnā”(qevt;gi+Ļā”(tiāaj)iāEj)).z_j^evt=norm\! (W_evtAttn (q_evt;\g_i+ (t_i-a_j)\_iā E_j ) ). (2) Offsets are measured in minutes on the visit clock, distinguishing observations before and after the anchor and their temporal distance. Using Equation 1, we minimize āevt=āNCEā(Qevt,evt)L_evt=L_NCE(Q^evt;C^evt), with logits Qjākevt=sevtā(zjevt)ā¤āuevtā(ckevt)Q_jk^evt=s_evt(z_j^evt) u^evt(c_k^evt) and learned scale sevt>0s_evt>0. Candidate sets retain the matched pair and exclude other views of the same class to avoid contrasting identical treatment categories; controls form one class. Attention details appear in Appendix C.3. Visit-level alignment. A clinical visit caption summarizes the encounter, while temporal statements may require evidence from different parts of its trajectory. To bound computation, we select at most 256 windows indexed by āVI_V across the monitoring span. We encode monitoring time tit_i and time since ED arrival Ī“i _i as bi=gi+Ī³monā(ti)+Ī³arrā(Ī“i)b_i=g_i+ _mon(t_i)+ _arr( _i). The input BV=[(bi)iāāV;GV;eV]B_V=[(b_i)_i _V;G_V;e_V] also includes gap tokens GVG_V for monitoring interruptions and an evidence token eVe_V for coverage and quality. Two attention blocks aggregate BVB_V through learned queries (Jaegle et al., 2021). We fix their number at eight: one global query and seven with reference times rkr_k relative to the first window (Shukla and Marlin, 2021). We apply RoFormerās rotary position embeddings (RoPE) (Su et al., 2024) to actual time offsets in minutes. For an anchored query qkq_k and a window key kik_i, q~kā¤k~i=(qkrot)ā¤RĪ(tiārk)kirot+(qkfree)ā¤kifree,k=1,ā¦,7. q_k k_i=(q_k^rot) R_ (t_i-r_k)k_i^rot+(q_k^free) k_i^free, k=1,ā¦,7. (3) Here RĪR_ rotates the rotrot channels and freefree channels remain unchanged. All queries attend to every valid input token; anchors do not restrict attention to fixed intervals. The output latents āVāk _Vk share a projection hVāk=normā”(WlatāāVāk)h_Vk=norm(W_lat _Vk), k=0,ā¦,7k=0,ā¦,7. We align the global embedding zVvis=hVā0z_V^vis=h_V0 with uvisā(cVvis)u^vis(c_V^vis) through in-batch InfoNCE, āvisL_vis, and use all eight hVākh_Vk for temporal statement matching. Encoding and anchor details appear in Appendix C.3. Temporal statement matching. We adapt ColBERTās late interaction (Khattab and Zaharia, 2020) to match each statement to its most compatible visit latent. For trajectory i and a visit j with mj>0m_j>0 available statements, let vjār=uvisā(djār)v_jr=u^vis(d_jr): Riāj=1mjāār=1mjmaxk=0,ā¦,7ā”hiākā¤āvjār.R_ij= 1m_j _r=1^m_j _k=0,ā¦,7h_ik v_jr. The mean normalizes for statement count, and each statement can select a different latent without a prescribed statement-to-anchor assignment. Alongside in-batch alignment, we rank the true set above a version iāi^- with one incompatible statement: ātmp=āNCEā(svisāR,full)+12āiā[ālogā”Ļā”(svisā(RiāiāRi,iā))],L_tmp=L_NCE(s_visR;C^full)+ 12\,E_i\! [- Ļ\! (s_vis(R_i-R_i,i^-) ) ], (4) where Ļ is the logistic function and fullC^full admits all in-batch pairs. Changing one statement preserves most of the description while making its temporal content the discriminating target. Missing statements and unavailable corruptions are masked. Stage 2 initializes from Stage 1 and alternates segment, visit, and event batches, retaining segment supervision to preserve local alignment (Ashutosh et al., 2023). For batch type bāseg,evt,visbā\seg,evt,vis\ and windows Q re-encoded with gradients, the alignment objective is ā(b)=āsegā()+Ī»dstāādstā()+b=evtāāevt+b=visā(āvis+ātmp).L^(b)=L_seg(Q)+ _dstL_dst(Q)+I_b=evtL_evt+I_b=vis(L_vis+L_tmp). (5) Here I is an indicator, and ādstL_dst is the mean cosine distance to frozen Stage-1 window embeddings. Training schedules, hyperparameter settings, and inference procedures are given in Appendix C.5. 5 Experiments Datasets. We use MC-MED (Kansal et al., 2025) for pretraining and in-domain evaluation, with patient-disjoint training, validation, and test splits. MC-MED pairs PPG recordings from Stanfordās adult emergency department (ED) with structured EHRs. For external evaluation, we use MIMIC-I (Johnson et al., 2016), which links intensive-care bedside waveforms from Beth Israel Deaconess Medical Center to clinical records, and VitalDB (Lee et al., 2022), which pairs intraoperative biosignals from Seoul National University Hospital with perioperative clinical information. Baselines. For language-based tasks, we compare with four general-purpose LLMsāGPT-5.6-luna (OpenAI, 2026a), GPT-6-luna (OpenAI, 2026b), GPT-6-sol (OpenAI, 2026c), and Claude Opus 5.5 (Anthropic, 2026). We also include the Qwen2.5-7B-Instruct variant of PulseLM (Pham et al., 2026), trained with a PaPaGei PPG encoder (Pillai et al., 2025). Separately, we train a Qwen3-4B (SFT) baseline (Qwen Team, 2025) using a frozen PaPaGei-S encoder and supervised fine-tuning on PPG signals paired with captions generated by our captioning pipeline. For clinical prediction, we include PPG foundation models PaPaGei-S (Pillai et al., 2025), Pulse-PPG (Saha et al., 2025), AnyPPG (Nie et al., 2026), and SIGMA-PPG (Guo et al., 2026), together with general time-series foundation models MOMENT-L (Goswami et al., 2024) and Chronos-2 (Ansari et al., 2025). 5.1 Zero-Shot Physiological Understanding Table 2: Zero-shot physiological recognition. Macro-F1 (ā ) for rhythm, quality (Qual.), perfusion (Perf.), morphology (Morph.), and respiratory-rate class (R). Bold marks the best in each column. Model MC-MED MIMIC-I VitalDB Rhythmā Qual.ā Perf.ā Morph.ā Rā Rhythmā Qual.ā Rā Rhythmā Qual.ā Rā GPT-5.6-luna 0.322 0.314 0.174 0.259 0.332 0.376 0.291 0.266 0.317 0.162 0.313 GPT-6-luna 0.319 0.361 0.193 0.263 0.259 0.357 0.315 0.253 0.326 0.175 0.307 GPT-6-sol 0.560 0.332 0.250 0.354 0.390 0.572 0.295 0.380 0.375 0.128 0.326 Opus 5.5 0.508 0.436 0.521 0.284 0.380 0.598 0.343 0.397 0.330 0.228 0.330 PulseLM 0.319 0.199 0.188 0.195 0.405 0.242 0.297 0.354 0.181 0.326 0.165 Qwen3-4B (SFT) 0.399 0.578 0.464 0.826 0.158 0.272 0.564 0.171 0.231 0.515 0.015 PPG-LM 0.651 0.673 0.531 0.886 0.407 0.670 0.786 0.428 0.389 0.583 0.329 Zero-shot physiological recognition. We test recognition of physiological categories from PPG in Table 2. Macro-F1 is reported for rhythm regularity, signal quality (Qual.), waveform-derived perfusion strength (Perf.) and augmentation-related morphology (Morph.), and respiratory-rate class (R). PPG-LM achieves the highest scores on 10 of 11 datasetātask pairs, including rhythm and quality on all three datasets and both perfusion and morphology on MC-MED. The exception is VitalDB R, where it closely matches Opus 5.5 (0.329 vs. 0.330). These results support recognition of multiple waveform attributes through a shared language interface, with transfer across emergency, intensive-care, and surgical settings. Zero-shot physiological estimation. We assess whether generated descriptions preserve physiological facts and numerical measurements in Table 3. Fact measures the accuracy of stated facts across six shared attributes; HR, R, SBP, and MAP report MAE for heart rate, respiratory rate, systolic blood pressure, and mean arterial pressure, respectively. PPG-LM achieves the highest fact accuracy on MC-MED, MIMIC-I, and VitalDB (0.923, 0.920, and 0.820) and the lowest reported errors for all displayed measurements. HR MAE is 3.90ā4.27 bpm against an independent ECG reference, supporting physiological agreement beyond the PPG-derived training captions. R errors are computed on valid outputs, so differences in output coverage limit their direct comparison. Table 3: Zero-shot physiological estimation. Fact is the accuracy of stated facts across six shared attributes (ā ). Numerical columns report MAE (ā ): HR in bpm, R in breaths/min, and SBP/MAP in mmHg. MAE uses valid numerical outputs; bold marks the best reported value in each column. Model MC-MED MIMIC-I VitalDB HRā Rā SBPā MAPā Factā HRā Rā Factā HRā Rā Factā GPT-5.6-luna 10.06 3.95 21.73 15.12 0.499 9.42 5.15 0.456 8.49 2.99 0.486 GPT-6-luna 11.24 3.93 22.24 14.51 0.559 11.52 4.71 0.533 8.86 2.34 0.579 GPT-6-sol 4.58 3.85 21.56 14.48 0.659 5.21 5.10 0.660 5.51 2.26 0.619 Opus 5.5 4.67 3.32 19.80 14.26 0.696 4.37 3.99 0.647 4.88 2.44 0.642 PulseLM 29.20 4.21 20.44 15.47 0.356 29.44 4.59 0.440 37.12 7.03 0.481 Qwen3-4B (SFT) 7.36 5.10 20.14 13.93 0.743 8.54 5.20 0.665 8.98 7.80 0.674 PPG-LM 3.90 3.10 12.50 6.17 0.923 4.27 3.83 0.920 3.98 2.23 0.820 Table 4: Zero-shot cross-modal retrieval. R@1 with 100 candidates. Bold: best. Model MC-MED MIMIC-I VitalDB500 S2Tā T2Sā S2Tā T2Sā S2Tā T2Sā GPT-5.6-luna 0.105 ā 0.090 ā 0.075 ā GPT-6-luna 0.110 ā 0.085 ā 0.085 ā GPT-6-sol 0.240 ā 0.260 ā 0.200 ā Opus 5.5 0.190 ā 0.275 ā 0.230 ā Qwen3-4B (SFT) 0.385 0.600 0.160 0.185 0.125 0.180 PPG-LM 0.975 0.965 0.805 0.755 0.555 0.550 Zero-shot cross-modal retrieval. We test whether a PPG signal retrieves its paired description (signal-to-text, S2T) and vice versa (text-to-signal, T2S). Table 4 reports R@1, the fraction of queries whose paired item ranks first, averaged over two 100-candidate pools. Following SensorLM and SleepLM (Zhang et al., 2026a; Xu et al., 2026), we evaluate general-purpose LLMs only in S2T to avoid the long numerical contexts required to include all 100 candidate waveforms in a T2S prompt. PPG-LM leads both directions across all three datasets, with S2T R@1 of 0.975, 0.805, and 0.555 on MC-MED, MIMIC-I, and VitalDB, respectively; the corresponding T2S scores are 0.965, 0.755, and 0.550. These results support bidirectional signalātext alignment that transfers to external datasets, with lower retrieval accuracy outside MC-MED. 5.2 Zero-Shot Event- and Visit-Level Physiological Recognition Table 5: Zero-shot event-level physiological recognition. Class-versus-control AUROC (ā ) for the five highest-scoring MC-MED classes for PPG-LM and all available VitalDB classes. Bold marks the best score in each column. Model MC-MED VitalDB Rate ctrl.ā Vasopress.ā Vasodil.ā IV fluidā Diureticā Opioidā Sedativeā Vasopress.ā Vasodil.ā GPT-5.6-luna 0.674 0.707 0.553 0.587 0.499 0.567 0.641 0.578 0.614 GPT-6-luna 0.626 0.715 0.648 0.530 0.478 0.600 0.594 0.480 0.693 GPT-6-sol 0.651 0.699 0.648 0.536 0.467 0.663 0.679 0.480 0.585 Opus 5.5 0.595 0.707 0.636 0.633 0.460 0.722 0.766 0.571 0.634 PulseLM 0.591 0.589 0.458 0.493 0.553 0.591 0.613 0.602 0.480 Qwen3-4B (SFT) 0.324 0.464 0.507 0.531 0.453 0.532 0.386 0.503 0.481 PPG-LM 0.860 0.830 0.772 0.759 0.754 0.731 0.807 0.612 0.741 Zero-shot event-level physiological recognition. We distinguish observations around medication administrations from no-administration controls (Table 5). Each column reports class-versus-control AUROC for a medication class: rate-control agents (Rate ctrl.), vasopressors (Vasopress.), vasodilators (Vasodil.), intravenous fluids (IV fluid), diuretics, opioids, or sedatives. PPG-LM outperforms the reported baselines on all displayed classes, with AUROC of 0.754ā0.860 for the five selected MC-MED classes and 0.612ā0.807 for the four VitalDB classes. These results support recognition of documented treatment context across emergency and surgical settings. Table 6: Zero-shot visit-level recognition. Balanced accuracy (ā ) for home-medication use (MC-MED) and clinical status (VitalDB). PPG-LM uses eight-window fact embeddings with held-out threshold calibration. Bold marks the best score in each column. Model MC-MED VitalDB CCBā AntiHTNā AntiPLā AntiDMā HTNā DMā Anemiaā eGFRā GPT-5.6-luna 0.468 0.509 0.477 0.520 0.500 0.465 0.487 0.500 GPT-6-luna 0.497 0.484 0.421 0.464 0.475 0.507 0.495 0.498 GPT-6-sol 0.500 0.500 0.505 0.507 0.523 0.502 0.523 0.500 Opus 5.5 0.500 0.443 0.468 0.507 0.519 0.512 0.484 0.500 PulseLM 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 Qwen3-4B (SFT) 0.535 0.540 0.438 0.444 0.500 0.500 0.500 0.500 PPG-LM 0.558 0.570 0.544 0.532 0.520 0.562 0.531 0.559 Zero-shot visit-level recognition. We assess whether eight PPG windows identify recorded home-medication use and clinical status, reporting balanced accuracy in Table 6. The MC-MED columns denote calcium-channel blockers (CCB), antihypertensives (AntiHTN), antiplatelets (AntiPL), and antidiabetic drugs (AntiDM). VitalDB columns cover hypertension (HTN), diabetes (DM), anemia, and reduced kidney function (eGFR below 60 mL/min/1.73 m2). With held-out threshold calibration, PPG-LM leads on seven of eight tasks; the exception is HTN (0.520 vs. 0.523 for GPT-6-sol). Balanced accuracy of 0.520ā0.570 indicates modest discrimination of these clinical facts. Additional MC-MED evaluations cover eight triage attributes and eleven medical-history conditions in Appendix D.5, Table 21(a) and (b), respectively. 5.3 Clinical Prediction with Frozen Representations Table 7: Clinical prediction with linear probes. Three classification and three regression tasks per dataset (mean Ā± SD). Bold: best mean; ā : window-level target. Task Foundation models PPG-LM PaPaGei-S PulsePPG AnyPPG SIGMA-PPG MOMENT-L Chronos-2 Stage 1 Stage 2 MC-MED (ED) ā AUROC ā Sex 0.692Ā± 0.006 0.719Ā± 0.007 0.783Ā± 0.008 0.712Ā± 0.008 0.724Ā± 0.005 0.739Ā± 0.008 0.813Ā± 0.006 0.856Ā± 0.005 ED admission 0.673Ā± 0.011 0.681Ā± 0.009 0.750Ā± 0.013 0.679Ā± 0.011 0.700Ā± 0.011 0.728Ā± 0.013 0.765Ā± 0.012 0.787Ā± 0.010 Atrial fibrillation 0.742Ā± 0.012 0.752Ā± 0.013 0.845Ā± 0.010 0.755Ā± 0.013 0.768Ā± 0.014 0.815Ā± 0.012 0.857Ā± 0.010 0.870Ā± 0.011 MC-MED (ED) ā R2āR^2 Age 0.395Ā± 0.016 0.428Ā± 0.018 0.626Ā± 0.020 0.389Ā± 0.017 0.475Ā± 0.017 0.521Ā± 0.016 0.691Ā± 0.016 0.782Ā± 0.014 Hemoglobin 0.083Ā± 0.011 0.098Ā± 0.013 0.196Ā± 0.013 0.102Ā± 0.012 0.123Ā± 0.014 0.164Ā± 0.014 0.238Ā± 0.014 0.274Ā± 0.012 Glucose 0.032Ā± 0.007 0.039Ā± 0.007 0.065Ā± 0.008 0.041Ā± 0.009 0.028Ā± 0.007 0.049Ā± 0.011 0.071Ā± 0.007 0.083Ā± 0.011 MIMIC-I (ICU) ā AUROC ā Sex 0.651Ā± 0.024 0.686Ā± 0.014 0.713Ā± 0.018 0.672Ā± 0.016 0.682Ā± 0.019 0.664Ā± 0.020 0.730Ā± 0.019 0.733Ā± 0.018 In-hospital mortality 0.697Ā± 0.029 0.741Ā± 0.034 0.756Ā± 0.031 0.732Ā± 0.025 0.751Ā± 0.025 0.772Ā± 0.020 0.786Ā± 0.022 0.793Ā± 0.021 Circulatory diagnosis 0.666Ā± 0.015 0.690Ā± 0.022 0.725Ā± 0.017 0.658Ā± 0.018 0.672Ā± 0.012 0.677Ā± 0.017 0.720Ā± 0.022 0.726Ā± 0.023 MIMIC-I (ICU) ā R2āR^2 Age 0.295Ā± 0.039 0.308Ā± 0.031 0.468Ā± 0.029 0.288Ā± 0.036 0.347Ā± 0.036 0.367Ā± 0.028 0.502Ā± 0.030 0.507Ā± 0.028 BUN 0.029Ā± 0.011 0.029Ā± 0.009 0.065Ā± 0.019 0.034Ā± 0.015 0.055Ā± 0.015 0.074Ā± 0.012 0.081Ā± 0.012 0.086Ā± 0.013 Hemoglobin 0.035Ā± 0.010 0.048Ā± 0.010 0.079Ā± 0.023 0.034Ā± 0.009 0.046Ā± 0.012 0.076Ā± 0.018 0.092Ā± 0.024 0.095Ā± 0.020 VitalDB (OR) ā AUROC ā ASA ā„3ā„ 3 0.606Ā± 0.021 0.642Ā± 0.020 0.690Ā± 0.026 0.605Ā± 0.035 0.625Ā± 0.035 0.655Ā± 0.040 0.699Ā± 0.034 0.704Ā± 0.025 Hypertension 0.598Ā± 0.016 0.607Ā± 0.020 0.635Ā± 0.019 0.595Ā± 0.025 0.610Ā± 0.016 0.617Ā± 0.023 0.656Ā± 0.016 0.655Ā± 0.017 ICU admission 0.690Ā± 0.009 0.700Ā± 0.004 0.747Ā± 0.002 0.683Ā± 0.009 0.662Ā± 0.011 0.714Ā± 0.009 0.733Ā± 0.004 0.751Ā± 0.006 VitalDB (OR) ā R2āR^2 Age 0.240Ā± 0.026 0.263Ā± 0.036 0.403Ā± 0.024 0.259Ā± 0.026 0.302Ā± 0.025 0.337Ā± 0.032 0.418Ā± 0.030 0.435Ā± 0.029 EtCO2ā 0.098Ā± 0.023 0.121Ā± 0.034 0.105Ā± 0.028 0.113Ā± 0.024 0.133Ā± 0.029 0.150Ā± 0.031 0.153Ā± 0.038 0.152Ā± 0.037 BISā 0.380Ā± 0.019 0.398Ā± 0.022 0.393Ā± 0.021 0.387Ā± 0.020 0.402Ā± 0.021 0.404Ā± 0.024 0.407Ā± 0.022 0.408Ā± 0.022 We evaluate frozen PPG representations on 32 datasetātask pairs across MC-MED, MIMIC-I, and VitalDB using logistic and ridge regression with patient-disjoint splits. All models use the same four windows per visit: baselines and Stage 1 mean-pool features, while Stage 2 uses its learned aggregator. Stage 2 PPG-LM exceeds all six foundation-model baselines in mean performance on all 32 tasks and Stage 1 on 30, except VitalDB hypertension and EtCO2, although glucose prediction remains weak (R2=0.083R^2=0.083). Table 7 presents selected results; Appendix D.7, Table 22 provides complete results and protocols. 5.4 Model Analysis Table 8: Ablation studies. Variant Temporal acc.ā Event AUROCā Segment R@1ā w/o temporal loss 0.511 0.698 0.446 w/o signed offsets 0.923 0.617 0.445 w/o segment loss 0.922 0.697 0.202 w/o distillation 0.920 0.700 0.445 w/o seg. loss + distill. 0.919 0.699 0.015 PPG-LM 0.925 0.707 0.447 Ablation studies. Table 8 shows that temporal matching and event offsets support temporal and event discrimination, respectively. Continued segment supervision preserves retrieval, while distillation helps mainly when it is removed. Comparing CLIP and SigLIP, CoCa gives the best Stage-2 rhythm, quality recognition and retrieval (Appendix D.9). Few-shot adaptation. Figure 4(a) shows HR-class prediction with linear probes on frozen PPG-LM representations. Mean macro-F1 increases consistently as labeled examples per class grow from 5 to 50 on MC-MED, MIMIC-I, and VitalDB. Appendix D.8 reports complete results across tasks and label budgets, along with the separate LLM in-context protocol, whose label budget and adaptation procedure differ from those of the probes. Training dynamics. Figure 4(b) tracks an extended Stage-2 CoCa run. Segment retrieval improves with training, while medication-event AUROC largely plateaus by 50ā60k steps, indicating different benefits from additional optimization across the two tasks. Visit context. With the Stage-2 CoCa model fixed, increasing context from four to sixteen windows improves visit retrieval, with little further gain from longer contexts (Figure 4(c)). This suggests diminishing returns from additional observations in this retrieval setting. Figure 4: Model analysis. (a) HR-class macro-F1 with labeled examples per class; shading denotes Ā±1Ā± 1 SD over five support draws. VitalDB uses the 40 Hz waveform view. (b) Segment retrieval and medication-event AUROC during training. (c) Visit retrieval versus context length in windows. 6 Conclusion We presented PPG-LM, a PPGālanguage model family that connects waveforms with clinical context. Our framework combines signal- and EHR-derived captions with two-stage alignment across segment, event, and visit levels. Experiments on MC-MED, MIMIC-I, and VitalDB support its effectiveness in language-based recognition, retrieval, captioning, and clinical prediction. AI use statement During manuscript preparation, we used generative AI tools solely for language polishing, including grammar correction, sentence refinement, and improvements to clarity and readability. The authors reviewed all suggested edits to ensure that the revised text preserved the intended technical meaning and accurately reflected the methods and results. The authors retained final editorial control and take full responsibility for the manuscriptās content, claims, and conclusions. Ethics statement This study involves secondary analysis of de-identified data from MC-MED, MIMIC-I, and VitalDB (Kansal et al., 2025; Johnson et al., 2016; Lee et al., 2022), in accordance with their respective access requirements and data-use agreements, with no new patient recruitment. PPG-LM is intended for research and has not been validated for clinical decision-making. Medication-event predictions reflect associations and do not establish causal treatment effects. Prospective validation and assessment of subgroup performance and generalizability are needed before clinical deployment, as discussed in Appendix A. Reproducibility statement Sections 3ā5 describe caption construction, the model framework, and the experiments. Appendices B and E provide captioning rules, language templates, and evaluation prompts, while Appendix C details the architecture, training objectives, hyperparameters, and computational setup. Dataset preprocessing, patient-disjoint splits, baseline configurations, task definitions, and evaluation protocols are documented in Appendix D. References Abbaspourazad et al. (2024) S. Abbaspourazad, O. Elachqar, A. C. Miller, S. Emrani, U. Nallasamy, and I. Shapiro Large-scale training of foundation models for wearable biosignals. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: Ā§2. Ali et al. (2025) S. A. Ali, M. W. Nawaz, J. Rashid, A. H. Mahmood, J. Kim, M. M. U. Rahman, and Q. H. Abbasi PEFT QLORA-based fine-tuning of foundation models for vitals estimation using PPG and ECG-based medical IoT data: a feasibility study. In 2025 IEEE International Conference on Data Mining Workshops (ICDMW), p. 468ā477. External Links: Document Cited by: Ā§2. Allen (2007) J. Allen Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement 28 (3), p. R1āR39. External Links: Document Cited by: Ā§1. Ansari et al. (2025) A. F. Ansari, O. Shchur, J. KĆ¼ken, A. Auer, B. Han, P. Mercado, S. S. Rangapuram, H. Shen, L. Stella, X. Zhang, M. Goswami, S. Kapoor, D. C. Maddix, P. Guerron, T. Hu, J. Yin, N. Erickson, P. M. Desai, H. Wang, H. Rangwala, G. Karypis, Y. Wang, and M. Bohlke-Schneider Chronos-2: from univariate to universal forecasting. arXiv preprint arXiv:2510.15821. External Links: Document Cited by: Ā§5. Anthropic (2026) Anthropic Introducing Claude Opus 5.5. Note: https://w.anthropic.com/claude-opus-5-5Released September 22, 2026. Accessed September 25, 2026 Cited by: Ā§5. Ashutosh et al. (2023) K. Ashutosh, R. Girdhar, L. Torresani, and K. Grauman HierVL: learning hierarchical video-language embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 23066ā23078. Cited by: Ā§C.4, Ā§2, Ā§4.2. Bannur et al. (2023) S. Bannur, S. Hyland, Q. Liu, F. PĆ©rez-GarcĆa, M. Ilse, D. C. Castro, B. Boecking, H. Sharma, K. Bouzid, A. Thieme, A. Schwaighofer, M. Wetscherek, M. P. Lungren, A. Nori, J. Alvarez-Valle, and O. Oktay Learning to exploit temporal structure for biomedical vision-language processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 15016ā15027. Cited by: Ā§2. Charlton et al. (2022) P. H. Charlton, P. A. Kyriacou, J. Mant, V. Marozas, P. Chowienczyk, and J. Alastruey Wearable photoplethysmography for cardiovascular monitoring. Proceedings of the IEEE 110 (3), p. 355ā381. External Links: Document Cited by: Ā§1. Chen et al. (2026) Y. Chen, A. Pillai, Y. Y. Wu, T. Z. Griffin, L. Marsch, M. V. Heinz, N. C. Jacobson, and A. Campbell Learning transferable sensor models via language-informed pretraining. arXiv preprint arXiv:2603.11950. External Links: Document, Link Cited by: Ā§2. Chen et al. (2025) Z. Chen, C. Ding, S. Kataria, R. Yan, M. Wang, R. J. Lee, and X. Hu GPT-PPG: a GPT-based foundation model for photoplethysmography signals. Physiological Measurement 46 (5), p. 055004. External Links: Document Cited by: Ā§2. Ding et al. (2024) C. Ding, Z. Guo, Z. Chen, R. J. Lee, C. Rudin, and X. Hu SiamQuality: a ConvNet-based foundation model for photoplethysmography signals. Physiological Measurement 45 (8), p. 085004. External Links: Document Cited by: Ā§2. Geenjaar et al. (2026) E. Geenjaar, V. D. Calhoun, S. Daly, G. KV, L. Lu, T. Mittal, and D. P. Darcy A robust PPG foundation model using multimodal physiological supervision. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: Ā§2. Goda et al. (2024) M. Ć. Goda, P. H. Charlton, and J. A. Behar pyPPG: a Python toolbox for comprehensive photoplethysmography signal analysis. Physiological Measurement 45 (4), p. 045001. External Links: Document Cited by: Ā§B.1, Ā§3. Goswami et al. (2024) M. Goswami, K. Szafer, A. Choudhry, Y. Cai, S. Li, and A. Dubrawski MOMENT: a family of open time-series foundation models. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 16115ā16152. Cited by: Ā§5. Gu et al. (2026) X. Gu, W. Tang, J. Han, V. Sangha, F. Liu, S. N. Gowda, A. H. Ribeiro, P. Schwab, K. Branson, L. Clifton, A. L. P. Ribeiro, Z. Liu, and D. A. Clifton Cardiac health assessment across scenarios and devices using a multimodal foundation model pretrained on data from 1.7 million individuals. Nature Machine Intelligence 8 (2), p. 220ā233. External Links: Document, Link Cited by: Table 1. Guo et al. (2026) Z. Guo, T. Chen, Y. Jiao, Y. Pan, X. Hu, and M. Ferrario SIGMA-PPG: statistical-prior informed generative masking architecture for PPG foundation model. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: Ā§2, Ā§5. He et al. (2026) C. He, X. Shao, S. Huang, B. Huang, D. Zhang, M. Jin, and C. Ding CAP: towards PPG universal representation learning with patient-level supervision. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, p. 11050ā11060. External Links: Document Cited by: Table 1, Ā§1, Ā§2. Huang et al. (2021) S. Huang, L. Shen, M. P. Lungren, and S. Yeung GLoRIA: a multimodal global-local representation learning framework for label-efficient medical image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 3942ā3951. Cited by: Ā§1, Ā§2. Huh et al. (2026) Y. Huh, M. Noh, D. Ji, Y. Oh, and S. Sun Multimodal PPG-based arrhythmia detection using a CLIP-initialized multi-task U-Net and LLM-assisted reporting. Sensors 26 (8), p. 2316. External Links: Document Cited by: Ā§2. Jaegle et al. (2021) A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira Perceiver: general perception with iterative attention. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 4651ā4664. Cited by: Ā§C.3, Ā§4.2. Jiang et al. (2025) W. Jiang, Y. Wang, B. Lu, and D. Li NeuroLM: a universal multi-task foundation model for bridging the gap between language and eeg signals. In International Conference on Learning Representations, Vol. 2025, p. 55436ā55457. Cited by: Ā§2. Johnson et al. (2016) A. E. W. Johnson, T. J. Pollard, L. Shen, L. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark MIMIC-I, a freely accessible critical care database. Scientific Data 3 (1), p. 160035. External Links: Document Cited by: 2nd item, Ā§5, Ā§6. Kansal et al. (2025) A. Kansal, E. Chen, B. T. Jin, P. Rajpurkar, and D. A. Kim MC-MED, multimodal clinical monitoring in the emergency department. Scientific Data 12 (1), p. 1094. External Links: Document Cited by: 1st item, Ā§1, Ā§3, Ā§5, Ā§6. Khattab and Zaharia (2020) O. Khattab and M. Zaharia ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 39ā48. External Links: Document Cited by: Ā§C.3, Ā§4.2. Langer et al. (2026) P. Langer, T. Kaar, M. Rosenblattl, M. A. Xu, W. Chow, M. Maritsch, R. Jakob, N. Wang, J. Liu, A. Verma, B. Han, D. Kim, H. Chubb, S. Ceresnak, A. Zahedivash, A. Sandhu, F. Rodriguez, D. McDuff, E. Fleisch, O. Aalami, F. Barata, and P. Schmiedmayer OpenTSLM: time-series language models for reasoning over multivariate medical text- and time-series data. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: Ā§2. Lee et al. (2022) H. Lee, Y. Park, S. B. Yoon, S. M. Yang, D. Park, and C. Jung VitalDB, a high-fidelity multi-parameter vital signs database in surgical patients. Scientific Data 9 (1), p. 279. External Links: Document Cited by: 3rd item, Ā§5, Ā§6. Li et al. (2025) Z. Li, S. Deldari, L. Chen, H. Xue, and F. D. Salim SensorLLM: aligning large language models with motion sensors for human activity recognition. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 354ā379. External Links: Document Cited by: Ā§2. Liu et al. (2024) Z. Liu, C. Chen, J. Cao, M. Pan, J. Liu, N. Li, F. Miao, and Y. Li Large language models for cuffless blood pressure measurement from wearable biosignals. In Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, p. 1ā11. External Links: Document Cited by: Ā§2. Luo et al. (2024) Y. Luo, Y. Chen, A. Salekin, and T. Rahman Toward foundation model for multivariate wearable sensing of physiological signals. arXiv preprint arXiv:2412.09758. External Links: Document, Link Cited by: Table 1. Makowski et al. (2021) D. Makowski, T. Pham, Z. J. Lau, J. C. Brammer, F. Lespinasse, H. Pham, C. Schƶlzel, and S. H. A. Chen NeuroKit2: a Python toolbox for neurophysiological signal processing. Behavior Research Methods 53 (4), p. 1689ā1696. External Links: Document Cited by: Ā§B.1. Moon et al. (2023) S. Moon, A. Madotto, Z. Lin, A. Saraf, A. Bearman, and B. Damavandi IMU2CLIP: language-grounded motion sensor translation with multimodal contrastive learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, p. 13246ā13253. External Links: Document Cited by: Ā§2. Nie et al. (2026) G. Nie, X. Fang, G. Tang, Y. Xiao, J. Li, B. Liu, H. Li, and S. Hong Anyppg: an ecg-guided ppg foundation model trained on over 100,000 hours of recordings for holistic health profiling. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 11717ā11727. Cited by: Ā§1, Ā§2, Ā§5. OpenAI (2026a) OpenAI GPT-5.6 Luna model. Note: https://developers.openai.com/api/docs/models/gpt-5.6-lunaOpenAI API documentation. Accessed September 25, 2026 Cited by: Ā§5. OpenAI (2026b) OpenAI GPT-6 Luna model. Note: https://developers.openai.com/api/docs/models/gpt-6-lunaOpenAI API documentation. Accessed September 25, 2026 Cited by: Ā§5. OpenAI (2026c) OpenAI GPT-6 Sol model. Note: https://developers.openai.com/api/docs/models/gpt-6-solOpenAI API documentation. Accessed September 25, 2026 Cited by: Ā§5. Pham et al. (2026) H. M. Pham, J. Wu, X. Ma, Y. Zhang, Y. Xu, A. Saeed, B. Zhu, Z. Pan, and D. Ma PulseLM: a foundation dataset and benchmark for PPG-text learning. arXiv preprint arXiv:2603.03331. External Links: Document Cited by: Table 1, Ā§1, Ā§2, Ā§5. Pillai et al. (2025) A. Pillai, D. Spathis, F. Kawsar, and M. Malekzadeh PaPaGei: open foundation models for optical physiological signals. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: Ā§1, Ā§2, Ā§5. Qwen Team (2025) Qwen Team Qwen3 technical report. External Links: 2505.09388, Link Cited by: Ā§5. Radford et al. (2021) A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 8748ā8763. Cited by: Ā§C.1, Ā§C.2, Ā§D.9, Ā§4.1, Ā§4. Saha et al. (2025) M. Saha, M. A. Xu, W. Mao, S. Neupane, J. M. Rehg, and S. Kumar Pulse-PPG: an open-source field-trained PPG foundation model for wearable applications across lab and field settings. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 9 (3), p. 1ā35. External Links: Document Cited by: Ā§1, Ā§2, Ā§5. Shukla and Marlin (2021) S. N. Shukla and B. M. Marlin Multi-time attention networks for irregularly sampled time series. In International Conference on Learning Representations, Cited by: Ā§C.3, Ā§4.2. Singh et al. (2024) P. Singh, R. Budhiraja, A. Gupta, A. Goswami, M. Kumar, and P. Singh EEVR: a dataset of paired physiological signals and textual descriptions for joint emotion representation learning. In Advances in Neural Information Processing Systems, Vol. 37, p. 15765ā15778. External Links: Document Cited by: Ā§2. Su et al. (2024) J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu RoFormer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. External Links: Document Cited by: Ā§C.3, Ā§4.2. van den Oord et al. (2018) A. van den Oord, Y. Li, and O. Vinyals Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. External Links: Link Cited by: Ā§4.1. Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Å. Kaiser, and I. Polosukhin Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. External Links: Link Cited by: Ā§C.1, Ā§4. Wang et al. (2025) F. Wang, J. Xu, and L. Yu From token to rhythm: a multi-scale approach for ECG-language pretraining. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 65059ā65074. Cited by: Ā§2. Wang et al. (2022) F. Wang, Y. Zhou, S. Wang, V. Vardhanabhuti, and L. Yu Multi-granularity cross-modal alignment for generalized medical visual representation learning. In Advances in Neural Information Processing Systems, Vol. 35, p. 33536ā33549. External Links: Document Cited by: Ā§2. Wang et al. (2026) X. Wang, C. Chang, D. Cao, K. Han, F. Sun, Y. Huang, M. Wang, C. Xu, X. Luo, R. Yan, X. Zhang, X. Hu, Y. Liu, Y. Sun, W. Wang, and C. Yang Position: beyond prediction: toward verifiable physiological waveform reasoning with foundation models and agentic LLMs. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: Ā§1. Xie et al. (2026) Y. Xie, B. Zhao, M. Dai, J. Zhou, Y. Sun, T. Tan, W. Xie, L. Shen, and Z. Yu PhysLLM: harnessing large language models for cross-modal remote physiological sensing. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Ā§2. Xu et al. (2026) Z. Xu, Z. Shuai, E. Mozaffari, R. S. Aysola, R. Kumar, and Y. Yang SleepLM: natural-language intelligence for human sleep. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: Ā§1, Ā§2, Ā§5.1. Yu et al. (2022) J. Yu, Z. Wang, V. Vasudevan, L. Yeung, M. Seyedhosseini, and Y. Wu CoCa: contrastive captioners are image-text foundation models. Transactions on Machine Learning Research. External Links: Link Cited by: Ā§C.2, Ā§C.2, Ā§4.1. Yuan et al. (2024) K. Yuan, V. Srivastav, N. Navab, and N. Padoy HecVL: hierarchical video-language pretraining for zero-shot surgical phase recognition. In Medical Image Computing and Computer Assisted Intervention ā MICCAI 2024, Lecture Notes in Computer Science, Vol. 15006, p. 306ā316. External Links: Document Cited by: Ā§2. Zhai et al. (2023) X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 11975ā11986. Cited by: Ā§D.9. Zhang et al. (2026a) Y. Zhang, K. Ayush, S. Qiao, A. A. Heydari, G. Narayanswamy, M. Xu, A. Metwally, J. Xu, J. Garrison, X. Xu, et al. SensorLM: learning the language of wearable sensors. Advances in Neural Information Processing Systems 38, p. 46813ā46849. Cited by: Table 1, Ā§1, Ā§2, Ā§5.1. Zhang et al. (2026b) Z. Zhang, H. Lu, and Q. Zhao PPGPT: transferring next-token modeling from language to PPG signals. Proceedings of the AAAI Conference on Artificial Intelligence 40 (34), p. 28573ā28581. External Links: Document Cited by: Ā§2. Appendix A Limitations We pretrain PPG-LM on single-center adult emergency-department data. Although MIMIC-I and VitalDB provide external validation, broader multi-center data would strengthen generalization across clinical settings. Clinical supervision depends on available EHR information, whose completeness and temporal accuracy affect caption quality and may limit model understanding and generalization. Richer longitudinal records and clinician-verified descriptions could improve this supervision. Medication-event recognition captures associations rather than causal drug effects, since treatment indication and concurrent interventions may confound observed signals. The current model is intended for research and is not ready for direct clinical decision-making; prospective validation, clinician assessment, subgroup analyses, and uncertainty calibration are needed before clinical deployment. Appendix B Caption Construction Details We summarize how waveform measurements and clinical records produce segment, visit, event, and temporal captions. The tables describe their content; Appendix E provides language templates and rendered examples. B.1 Segment captions and signal quality Measurements. Each segment contains 30 seconds of PPG sampled at 125 Hz. NeuroKit2 provides pulse timing, amplitude, and quality measurements; pyPPG provides augmentation and signal-quality indices (Makowski et al., 2021; Goda et al., 2024). Synchronized ECG supplies rhythm annotations and supports quality checks; the model input remains PPG. Table 9: Segment-caption content. Perfusion and morphology are waveform surrogates rather than independent clinical diagnoses. Attribute Evidence and description Pulse rate Median pulse-interval rate, expressed as a value and a category. Rate trend Instantaneous rate slope: falling, stable, or rising. Rhythm ECG RāR interval variability, expressed as a regularity category. Perfusion Pulsatile amplitude relative to baseline: weak, moderate, or strong. Signal quality NeuroKit2 quality score: clean, fair, or noisy but usable. Augmentation The pyPPG augmentation index: low, moderate, or high. SVRI Post-peak/pre-peak mean-amplitude ratio in normalized beats: low, mid-range, or high. Amplitude variability Beat-amplitude variation: steady to highly variable. Quality filtering. Usability combines PPG quality with available signal-quality, flatline, and clipping checks; missing optional checks do not reject a window. Good windows meet stricter criteria, including PPGāECG rate agreement when reliable ECG exists; other usable windows are fair. Selection tiers differ from caption quality labels. Unavailable or unreliable ECG rhythm annotations are omitted. Rendering. Missing features are omitted. Poor windows receive quality-only descriptions and are excluded from training but retained for validation and testing. Each feature has ten templates; template selection and sentence shuffling use reproducible example-specific seeds. Signal-plus-vitals view. For the signal-plus-vitals configuration reported in Section 5, available charted HR, SpO2, R, and BP measurements augment the eight signal attributes described in Section 3. Each value is matched to the nearest charted observation within 300 seconds of the window timestamp; unavailable values are omitted. These measurements enter the text target, while the waveform input remains PPG. B.2 Visit captions and clinical context Clinical records. Visit captions summarize encounter context (Table B.2). History entries are deduplicated and ranked by clinical relevance, then recency. Table 10: Visit-caption content. Component Evidence and description Demographics Recorded age and sex. Presentation Arrival mode, Emergency Severity Index, and normalized chief complaints. Triage measurements Available heart rate, respiratory rate, oxygen saturation, blood pressure, and temperature. Clinical history Selected prior ICD-10 diagnoses, expressed as clinical descriptions. Home medications Recorded active home-medication classes. Field selection. Known post-arrival diagnoses are excluded; history with missing diagnosis or arrival dates is retained. Home medications use patient-level active records without encounter-specific temporal verification. These fields therefore have incomplete evidence of availability time. Disposition, discharge diagnoses, and length of stay are excluded. B.3 Event captions and within-visit controls Administration records. Eligible records require a parseable first-administration timestamp and a normalized drug name. Route and class filters exclude topical, ophthalmic, and other ineligible administrations. Name rules and class-keyword fallback map the remaining records to opioids, intravenous fluids, rate-control agents, vasodilators, bronchodilators, sedatives, non-opioid analgesics/antipyretics, vasopressors, diuretics, or antiemetics. Unmapped records do not become targets; near-duplicate administrations are removed within each visit and class. Table 11: Event-caption construction. Component Evidence and description Event anchor A recorded administration timestamp for an eligible medication class. Pre-event context The latest available PPG windows preceding the administration. Post-event context PPG windows sampled evenly across the period following administration. Caption target The medication class, with individual drug names, doses, and exact latencies omitted. Control view The same sampling rule around a within-visit anchor free of nearby eligible administrations. Sampling and controls. Event views use up to three windows in the preceding 30 minutes and up to five in the following 60 minutes, with at least one pre-administration and two post-administration windows. Control anchors are sampled uniformly over the same visitās monitored span; their exclusion interval also covers eligible but unmapped drugs. Controls are capped at the retained event count, subject to availability. This construction avoids a fixed window-start cue but does not establish causal effects or remove all temporal confounding. Antiemetics serve as an additional comparison class. B.4 Temporal descriptions and corrupted statements Trajectory evidence. Ordered segment-caption rates and rhythm labels produce the temporal descriptions in Table B.4. They summarize observed tachycardia, irregularity, and changes in pulse rate across a visit. Table 12: Temporal-caption content. Operator Evidence and description Occurrence Whether tachycardia or irregularity appears among known observations. Proportion The affected share of known windows, described as brief, part, or most. Contiguous duration Consecutive affected windows, grouped into short, intermediate, or long runs. Rate ordering Consecutive groups of observed rates, describing rising, falling, stable, rise-then-fall, or fall-then-rise patterns. Missing observations. Proportion and duration descriptions require the state to occur. Unknown states and monitoring gaps interrupt runs; unsupported operators and ambiguous rate profiles are omitted. Corrupted descriptions. Each supported operator value has three templates. A negative substitutes an incompatible value in one sentence and re-renders that sentence, retaining the remaining facts. Padded sentences and unavailable negatives are masked. Appendix C Method and Training Details This appendix expands the architecture and alignment objectives in Section 4, using the same notation throughout. The waveform feature is gig_i, the level-conditioned text embedding is uāā(c)u (c), and normnorm denotes ā2 _2 normalization. We first specify the two encoders and their CoCa training interface, then describe event and visit aggregation, preservation of segment alignment, and the two-stage optimization schedule. C.1 Waveform and text encoders Waveform encoder. The encoder fĪøf_Īø operates on a single-channel, 30-second PPG window xiāā3750x_i ^3750 sampled at 125 Hz. Each window is standardized using its own mean and standard deviation, with 10ā610^-6 added to the denominator. A one-dimensional convolution with kernel and stride 25 produces 150 non-overlapping patches, each spanning 0.2 seconds. Learned absolute positional embeddings are added to the width-384 patch embeddings before six Transformer blocks (Vaswani et al., 2017), each with six attention heads and an MLP hidden width of 4Ć3844Ć 384. Denoting the final normalized patch states by Hiāā150Ć384H_i ^150Ć 384, the two waveform representations are gi=Pool1ā”(Hi)āā384,ziseg=normā”(Wsegāgi)āā512.g_i=Pool_1(H_i) ^384, z_i^seg=norm(W_segg_i) ^512. (6) Here Pool1Pool_1 is a single learned-query attention pooler with a residual MLP, and Wsegāā512Ć384W_seg ^512Ć 384 is bias-free. The waveform encoder, including this projection, has approximately 12.7M parameters. As in the clinical prediction experiments, the 768-dimensional window probe feature concatenates gig_i with the mean of the 150 patch states, before the alignment projection. Text encoder. The shared text tower ĻĻ _Ļ is an eight-layer causal Transformer with width 512, eight attention heads, and an MLP hidden width of 4Ć5124Ć 512. It uses CLIPās byte-pair encoding vocabulary of 49,408 tokens (Radford et al., 2021), learned token and absolute positional embeddings, and a learned prefix eāe_ for āāseg,evt,vis ā\seg,evt,vis\. The prefix is prepended to the text sequence and participates in causal self-attention. The level-conditioned read-out in Section 4 is uāā(c)=normā”(PāāĻĻā([eā;c])EOS),Pāāā512Ć512.u (c)=norm\! (P_ _Ļ([e_ ;c])_EOS ), P_ ^512Ć 512. (7) The end-of-sequence state summarizes the available text, and the level-specific projection selects its alignment space. The same tower encodes full captions and individual temporal statements; the latter use the visit prefix and projection. Both waveform and text towers are trained from random initialization. CLIP supplies the tokenizer and contrastive formulation, rather than pretrained encoder weights. C.2 CoCa alignment and captioning objectives Contrastive branch. CoCa combines paired representation learning with conditional caption generation (Yu et al., 2022). In our waveform adaptation, the contrastive branch connects the single-query waveform read-out in Equation 6 to the text-only end-of-sequence read-out in Equation 7. For a batch of B matched windowācaption pairs, the normalized embeddings give Qiāj Q_ij =sseg(ziseg)ā¤useg(cj),iseg=iāŖj:Īŗjā Īŗi, =s_seg(z_i^seg) u^seg(c_j), _i^seg=\i\āŖ\j: _jā _i\, (8) piājsegātext p^seg _ij =expā”Qiājākāisegexpā”Qiāk,ptextāsegiāj=expā”Qjāiākāisegexpā”Qkāi,jāiseg. = Q_ij _k _i^seg Q_ik, p^text _ij= Q_ji _k _i^seg Q_ki, j _i^seg. The scale sseg>0s_seg>0 is learned. The content key Īŗi _i represents the binned measurements underlying a caption, independently of template wording and sentence order. Off-diagonal pairs with equal keys are masked in both directions, while each matched pair is retained. Symmetric InfoNCE (Radford et al., 2021) minimizes the negative log-probability of the diagonal match in both directions: āNCE(Q;seg)=ā12āBāi=1B(logpiāisegātext+logpiāitextāseg).L_NCE(Q;C^seg)=- 12B _i=1^B ( p^seg _i+ p^text _i ). (9) This is Equation 1 with the segment candidate set from Section 4.1. Excluded pairs do not contribute to either denominator; they are not treated as additional positives. Captioning branch. A separate attention pooler with 32 learned queries reads the same patch states HiH_i and produces waveform memory Mi=Pool32ā”(Hi)āā32Ć384M_i=Pool_32(H_i) ^32Ć 384. A learned projection maps the memory to width 512. The causal token states from the eight-layer text tower, with the level-prefix position removed, then pass through four additional multimodal decoder layers of width 512, eight heads, and MLP ratio four. Each layer combines causal text self-attention with cross-attention to the projected waveform memory. This implements CoCaās separation between text-only contrastive features and waveform-conditioned caption features (Yu et al., 2022). Let di,rāā512d_i,r ^512 be the final layer-normalized decoder state that predicts token ci,rc_i,r from the preceding caption tokens, and let Etokāā49408Ć512E_tok ^49408Ć 512 be the shared token embedding matrix. The tied output head and training objective are pĻ,Ī·ā(ci,rā£ci,<r,Mi) p_Ļ,Ī·(c_i,r c_i,<r,M_i) =[softmaxā”(Etokādi,r)]ci,r, = [softmax(E_tokd_i,r) ]_c_i,r, (10) ācap _cap =ā1NtokāiārāilogpĻ,Ī·(ci,rā£ci,<r,Mi), =- 1N_tok _i _r _i p_Ļ,Ī·(c_i,r c_i,<r,M_i), āseg _seg =āNCEā(Q,seg)+ācap. =L_NCE(Q;C^seg)+L_cap. Here iT_i contains non-padding target positions, including EOS, Ntok=āi|i|N_tok= _i|T_i|, and Ī· denotes the additional parameters of the waveform-memory and multimodal decoding branch, as in Section 4.1. Training uses teacher forcing and unit weights for both losses. The two branches share the waveform patch encoder and causal text tower, so caption supervision also updates the representations used for alignment. Caption loss applies to segment descriptions in every batch type; event and visit descriptions are alignment targets. The CLIP objective compared in the experiments retains the contrastive branch without caption loss. C.3 Event and visit aggregation Event read-out. For a medication or control anchor aja_j, EjE_j contains up to three latest windows in [ajā30,aj)[a_j-30,a_j) and up to five evenly spaced windows in [aj,aj+60][a_j,a_j+60], following Appendix B.3. Both tit_i and aja_j use the visit clock, so tiāajt_i-a_j is a signed offset in minutes. Each view has at least one pre-anchor and two post-anchor windows. The time embedding Ļ maps 64 sinusoidal features, with periods from two minutes to two weeks, through an MLP to width 384. The attention operator in Equation 2 includes layer normalization of the waveform content and is implemented by Uj(0) U_j^(0) =[qevt;(LNā”(gi)+Ļā”(tiāaj))iāEj], =[q_evt;(LN(g_i)+ (t_i-a_j))_iā E_j], (11) Uj(b) U_j^(b) =ā¬bevt(Uj(bā1)),b=1,2, =B_b^evt(U_j^(b-1)), b=1,2, zjevt z_j^evt =normā”(WevtāLNā”([Uj(2)]0)). =norm\! (W_evtLN([U_j^(2)]_0) ). Each ā¬bevtB_b^evt is a self-attention block of width 384, six heads, and MLP ratio four; key-padding masks exclude missing windows. Reading the learned-query position yields one 512-dimensional event embedding. Signed offsets preserve the distinction between observations before and after the anchor, including for control views. Visit read-out and reference times. For a visit V, we use the input construction in Section 4.2: bi=gi+Ī³monā(ti)+Ī³arrā(Ī“i),BV=[(bi)iāāV;GV;eV].b_i=g_i+ _mon(t_i)+ _arr( _i), B_V=[(b_i)_i _V;G_V;e_V]. (12) Here āVI_V indexes at most 256 selected windows, GVG_V contains gap tokens, and eVe_V is the evidence token. We prepend eight learned latent queries and update the complete sequence through two self-attention blocks of width 384, six heads, and MLP ratio four: UV(0) U_V^(0) =[q0;ā¦;q7;BV], =[q_0;ā¦;q_7;B_V], (13) UV(b) U_V^(b) =ā¬bvis(UV(bā1);ĻV),b=1,2, =B_b^vis(U_V^(b-1); _V), b=1,2, āVāk _Vk =LN([UV(2)]k),hVāk=norm(WlatāVāk),k=0,ā¦,7. =LN([U_V^(2)]_k), h_Vk=norm(W_lat _Vk), k=0,ā¦,7. The blocks use the token times ĻV _V and a validity mask; both the latent and observation positions are updated. Inspired by Perceiverās learned latent representations (Jaegle et al., 2021), only the eight latent positions are read out. Attention operates on the full concatenated sequence, with at most 8+256+16+1=2818+256+16+1=281 tokens. The global latent has reference time r0=|āV|ā1āāiāāVtir_0=|I_V|^-1 _i _Vt_i and supplies zVvis=hVā0z_V^vis=h_V0 for clinical-caption alignment. The remaining reference times are (r1,ā¦,r7)=(0,15,30,60,120,240,480)(r_1,ā¦,r_7)=(0,15,30,60,120,240,480) minutes after the first window. Following the reference-time idea in mTAN (Shukla and Marlin, 2021), these anchors give the queries temporal identities while allowing attention to all valid observations. Continuous-time rotary encoding. We adapt rotary position embeddings from RoFormer (Su et al., 2024) to elapsed minutes. In each 64-dimensional attention head, 16 channel pairs are rotated and the other 32 channels are unchanged. With Tp=2ā(20160/2)(pā1)/15T_p=2(20160/2)^(p-1)/15 minutes and Īøp=2āĻ/Tp _p=2Ļ/T_p, define RĪā(t)=diagp=116ā(cosā”(Īøpāt)āsinā”(Īøpāt)sinā”(Īøpāt)cosā”(Īøpāt)).R_ (t)=diag_p=1^16 pmatrix ( _pt)&- ( _pt)\\ ( _pt)& ( _pt) pmatrix. (14) For query and key times Ļ and Ļā²Ļ , respectively, q~ā(Ļ) q(Ļ) =[RĪā(Ļ)āqrot;qfree], =[R_ (Ļ)q^rot;q^free], k~ā(Ļā²) k(Ļ ) =[RĪā(Ļā²)ākrot;kfree], =[R_ (Ļ )k^rot;k^free], (15) q~ā(Ļ)ā¤āk~ā(Ļā²) q(Ļ) k(Ļ ) =(qrot)ā¤āRĪā(Ļā²āĻ)ākrot+(qfree)ā¤ākfree. =(q^rot) R_ (Ļ -Ļ)k^rot+(q^free) k^free. Setting Ļ=rkĻ=r_k and Ļā²=tiĻ =t_i recovers Equation 3. Each head divides these scores by 64 64, applies the validity mask, and normalizes with softmax. Using actual minutes distinguishes irregular intervals that share the same window-index separation. The relative-rotation identity holds for fixed query/key content; additive clock features also affect that content. Time content and monitoring coverage. As in Equation 12, learned MLPs map 64-dimensional sinusoidal features of monitoring time tit_i and time since ED arrival Ī“i _i into additive width-384 features. A learned vector represents unavailable arrival time. These features expose time through the values as well as the attention scores. Gaps longer than ten minutes are detected over the full visit before context subsampling; the longest 16 are retained. Each gap token uses its midpoint as its time and encodes the logarithmically transformed duration, logā”(1+duration) (1+duration). The evidence token maps six coverage descriptors through a 6ā384ā3846ā 384ā 384 MLP: window count, coverage fraction, monitoring span, fraction of good-quality windows, arrival-to-monitoring delay, and an arrival-time availability indicator. Window count, monitoring span, and available arrival delay use log(1+ā ) (1+Ā·) transforms, with times measured in minutes. Coverage is minā”(1,0.5ān/maxā”(span,0.5)) (1,0.5n/ (span,0.5)) for n observed 30-second windows, with a half-minute lower bound on the denominator for a single-window visit. These inputs describe the amount and timing of observed evidence without imputing physiology within monitoring gaps. Alignment across scopes. Event alignment uses Equation 1 with Qiāj=sevtā(zievt)ā¤āuevtā(cj)Q_ij=s_evt(z_i^evt) u^evt(c_j) and ievt=iāŖj:classā”(j)ā classā”(i)C_i^evt=\i\āŖ\j:class(j) (i)\; all control views share one category. Global visit alignment uses Qiāj=svisā(zivis)ā¤āuvisā(cj)Q_ij=s_vis(z_i^vis) u^vis(c_j) with the full in-batch candidate set. Temporal matching uses all eight hVākh_Vk and the mean-over-statements, max-over-latents score RiājR_ij in Equation 4.2, adapting ColBERTās late interaction (Khattab and Zaharia, 2020) to whole statements. Each visit supplies at most eight statements, with no fixed statement-to-latent assignment. Equation 4 combines in-batch alignment of svisāRs_visR with a corruption ranking term of weight 0.5. For the temporal contrastive term, we retain the matched sub-batch of visits with mV>0m_V>0 and use all candidates within that sub-batch; the term is zero if no visit has available statements. Padded statements do not enter the mean defining RiājR_ij. The ranking term averages only over visits with an available corruption; if none are available, this term is zero. A visit without temporal statements still participates in clinical-caption alignment. Appendix B.4 specifies the corruptions. No cross-batch negative queue is used at any scope. C.4 Preserving segment alignment Stage 2 introduces event and visit targets whose temporal scope differs from that of a segment caption. Updating the shared encoder solely for these broader targets can change the local waveformālanguage geometry learned in Stage 1. We therefore retain the segment objective on every batch and use a frozen Stage-1 waveform encoder as a reference, alongside the multilevel training strategy motivated by HierVL (Ashutosh et al., 2023). The ongoing segment objective continues to learn from paired captions; the reference penalty limits changes to the normalized waveform embeddings. Let Q be the windows re-encoded in the current batch. A frozen copy of the Stage-1 waveform encoder and segment projection supplies zi(1)=normā”(Wseg(1)āgi(1))z_i^(1)=norm(W_seg^(1)g_i^(1)). With no gradients through this reference branch, we use ādstā() _dst(Q) =1||āāiā(1ā(ziseg)ā¤āzi(1)) = 1|Q| _i (1-(z_i^seg) z_i^(1) ) (16) =12ā||āāiāāzisegāzi(1)ā22. = 12|Q| _i \|z_i^seg-z_i^(1) \|_2^2. The equality follows from unit normalization. The penalty constrains embedding directions while allowing the pre-projection waveform features to adapt. The teacher remains in evaluation mode with dropout disabled; the current waveform encoder and projection receive gradients after the initial freezing period in Appendix C.5. The alignment terms in Equation 5 can then be written as ā(seg) ^(seg) =āsegā()+Ī»dstāādstā(), =L_seg(Q)+ _dstL_dst(Q), (17) ā(evt) ^(evt) =āsegā()+Ī»dstāādstā()+āevt, =L_seg(Q)+ _dstL_dst(Q)+L_evt, ā(vis) ^(vis) =āsegā()+Ī»dstāādstā()+āvis+ātmp, =L_seg(Q)+ _dstL_dst(Q)+L_vis+L_tmp, where Ī»dst=0.1 _dst=0.1. For segment and event batches, Q contains their valid input windows, each paired with its own segment caption. For visit batches, it contains up to eight live windows per visit; detached context features participate in aggregation but incur no segment or distillation loss. The same content-key mask is applied whenever the segment objective is evaluated, including on live windows from event and visit batches. C.5 Optimization, feature caching, and inference Two-stage schedule. Stage 1 uses the segment objective in Section 4.1. Both towers are trained from scratch for six epochs over 8,842,271 training windows, using batches of 256 and dropping the final incomplete batch. This gives 207,240 updates. Stage 2 initializes from Stage 1 and trains the shared encoders and scope-specific read-outs under the freezing schedule below for 30,000 global updates, cycling through segment, visit, and event batches in a 4:2:14:2:1 ratio. Batch sizes are 256 windows, 64 visits, and 64 event views, respectively. The extended Stage-2 run used for the training-dynamics analysis in Figure 4(b) is separate from this standard schedule. Optimization. Both stages use AdamW with Ī²=(0.9,0.98)Ī²=(0.9,0.98), Ļµ=10ā6Īµ=10^-6, peak learning rate 5Ć10ā45Ć 10^-4, and weight decay 0.2. The learning rate follows a 2,000-update linear warmup and cosine decay to zero over each stageās own schedule. Weight decay excludes biases, normalization parameters, token and positional embeddings, learned queries and prefixes, missing-time embeddings, and logit scales. During the first 2,000 Stage-2 updates, the waveform and text towers and the segment logit scale have zero learning rate; subsequently they use one tenth of the scheduled rate. Aggregators and other read-out parameters use the full scheduled rate from the start. Global gradient norms are clipped at 1.0. Dropout is 0.1 in waveform Transformer blocks and the event and visit aggregators, and zero in the text tower, caption decoder, and attention poolers. Each level has a learned scale sā=minā”(expā”(Ī±ā),100)s_ = ( ( _ ),100); newly introduced scales are initialized to 1/0.071/0.07, while Stage 2 retains the learned segment scale. Token budgets are 256 for segment and visit captions, 64 for event captions, and 48 for temporal statements. Loss weights. All contrastive alignment terms and the segment captioning loss use unit weights. Stage 2 uses a distillation weight of 0.10.1 and a temporal corruption-ranking weight of 0.50.5. Live windows and cached context. The Stage-1 waveform encoder initializes an fp16 cache of the 384-dimensional window features gig_i. Each visit batch selects a context of at most 256 windows per visit. Up to eight distinct windows per visit are sampled for live re-encoding, using all windows when fewer than eight are available; their features replace the corresponding cached entries within this context, so the live windows count toward the context limit. For the selected set āVI_V and live subset V=ā©āVQ_V=Q _V, aggregation uses g~i=fĪøā(xi),iāV,stopgradā”(gicache),iāāVāV. g_i= casesf_Īø(x_i),&i _V,\\ stopgrad(g_i^cache),&i _V _V. cases (18) This lets visit-level losses update the aggregators over a longer context while limiting waveform-encoder activations to the live subset. The context cap at global Stage-2 update s is Cmaxā(s)=ā8+248āminā”(1,s10,000)ā.C_ (s)= 8+248 \! (1, s10,000 ) . (19) Half of the visit batches use this cap; the other half draw a context budget log-uniformly between 8 and the current cap, subject to the available windows. Longer visits are subsampled uniformly over time while retaining the live windows. Detached live features are written back to the cache after visit batches, and segment batches also refresh their encoded windows. Event views are encoded directly from raw windows and do not update the cache. Each training run uses its own cache. Inference. All model parameters are frozen. We encode the windows selected by each benchmark using the evaluation checkpoint; the training feature cache is not required. Segment and event tasks use zsegz^seg and zevtz^evt, respectively, while clinical-caption visit retrieval uses the global latent zVvis=hVā0z_V^vis=h_V0. Retrieval and class-prompt scoring compare normalized waveform and corresponding level-conditioned text embeddings by cosine similarity; task-specific prompt sets and read-outs are specified in Appendix D. Temporal descriptions use all eight latents and the score in Equation 4.2. Segment captioning starts with a beginning-of-sequence token and uses greedy autoregressive decoding conditioned on MiM_i, stopping at EOS or the segment token limit. Compute. The main CoCa model is trained with seed 0 and bfloat16 autocast on one NVIDIA H200 GPU. Recorded training times are approximately 26.3 hours for Stage 1 and 6.0 hours for Stage 2, excluding cache construction and evaluation. Caption cross-entropy is computed in chunks of 8,192 target tokens with gradient checkpointing to reduce memory use, without changing the token-averaged objective in Equation 10. Appendix D Additional Evaluation Details D.1 Experimental setup Datasets and patient splits. The three datasets cover emergency, intensive-care, and intraoperative monitoring. Only MC-MED is used to pretrain PPG-LM; MIMIC-I and VitalDB evaluate transfer to external clinical settings. ā MC-MED (Kansal et al., 2025) pairs PPG from Stanfordās adult emergency department (ED) with structured electronic health records. The processed cohort contains 36,430 patients and 53,473 visits. The patient-random 80/10/10 split yields the retained counts in Table 13. PPG is divided into non-overlapping 30-second windows at 125 Hz; the training set provides approximately 73,700 hours. Poor-quality windows are excluded from pretraining, while the validation and test corpus splits retain them. ā MIMIC-I (Johnson et al., 2016) links intensive-care bedside waveforms from Beth Israel Deaconess Medical Center to clinical records. The external benchmark uses PPG paired with synchronized ECG where available, charted vital signs, demographics, laboratory measurements, and hospital outcomes. Language evaluation uses fixed patient-level samples. Clinical probes are fitted on the target datasetās training patients and evaluated on patient-disjoint test patients. ā VitalDB (Lee et al., 2022) pairs intraoperative biosignals from Seoul National University Hospital with perioperative clinical information. It provides preoperative conditions and laboratory measurements, intraoperative monitoring, medication records, and postoperative ICU outcomes. The language benchmark has official 40 Hz and original 500 Hz waveform views of the same evaluation items. The main PPG-LM language results, retrieval analysis, and prompt-sensitivity analysis use the 500 Hz source; supplementary caption/read-out and few-shot analyses use the 40 Hz view. Baselines retain their recorded input views. Clinical probes use target-dataset training patients and patient-disjoint testing. Table 13: Processed MC-MED splits. Patients are disjoint across splits; window counts refer to the retained corpus. Split Patients Visits 30-second windows Training 29,177 42,856 8,842,271 Validation 3,641 5,421 1,183,278 Test 3,612 5,196 1,099,395 Total 36,430 53,473 11,124,944 Waveform preparation and references. PPG-LM inputs are resampled to 125 Hz and standardized within each window as (xāĪ¼)/(Ļ+10ā6)(x-Ī¼)/(Ļ+10^-6). The captioning pipeline supplies reference descriptions on all three datasets; Appendix B describes its signal measurements, quality gates, and clinical-record matching. ECG supports rhythm annotation and the independent HR reference, but is not a model input. Baseline configurations. For language tasks, GPT-5.6-luna, GPT-6-luna, GPT-6-sol, and Claude Opus 5.5 receive each 30-second PPG window mean-pooled to 25 Hz and standardized, giving 750 numerical values, and return JSON answers. Prompts include the annotation class definitions without reference measurements. Signal-to-text retrieval shuffles candidate-caption order independently for each query and requests five preferred indices. PulseLM uses its Qwen2.5-7B-Instruct variant. Our Qwen3-4B (SFT) baseline uses a frozen PaPaGei-S encoder and full language-model fine-tuning on 100,000 window and 40,000 visit examples from our captioning corpus; evaluation uses prompt scoring or generated answers as appropriate to the task. Clinical prediction compares frozen PaPaGei-S, Pulse-PPG, AnyPPG, SIGMA-PPG, MOMENT-L, and Chronos-2 features with the same logistic- or ridge-regression probe protocol (Appendix D.7). For visit-level clinical targets, all baselines and both PPG-LM stages receive the same four windows; baselines and Stage 1 use mean pooling, while Stage 2 uses learned visit aggregation. MOMENT-L is the largest evaluated MOMENT variant; PaPaGei-S is the morphology-aware variant. Unless otherwise stated, PPG-LM uses the caption-style template prompt bank. Benchmark sampling and scoring. Language evaluations use fixed items and cached model responses. Window recognition and HR evaluation use 300 windows per dataset, with one window per patient, stratified by rhythm and HR class and excluding the poor-quality tier. Captioning uses the first 200 of these windows. Clinical-fact recognition in Table 6 uses 400 MC-MED visits and 300 VitalDB cases, with eight windows per case and enriched sampling of rare labels. Its thresholds are calibrated on held-out MC-MED validation visits or separate VitalDB patients, without using test labels. The broader visit-inference benchmark instead samples 200 encounters per dataset, requires at least eight windows, and stratifies by sex, age group, and outcome. Visit benchmarks retain one encounter per patient. The medication-class results in Table 5 use 393 administration views and 393 controls on MC-MED and 139 of each on VitalDB; Appendix D.6 additionally reports the smaller event cohort. Off-label or missing classification answers count as errors. Numerical MAE uses items with both a numerical answer and an available reference; the HR comparisons have 300 valid ECG-referenced items per system and dataset. Dashes denote unavailable or unreported evaluations. Evaluation protocol. Unless otherwise stated, PPG-LM denotes the Stage-2 model. All model backbones remain frozen during evaluation. Within each benchmark, comparisons use PPG inputs at the same temporal level and with the same number of windows across models. Language benchmarks use the same evaluation examples and class definitions across models; PPG-LM performs prompt-based recognition, cross-modal retrieval using cosine similarity, and caption generation with greedy decoding. External zero-shot segment recognition, retrieval, and captioning use no target-dataset fitting. Clinical-fact recognition uses read-out thresholds calibrated on held-out MC-MED visits or separate VitalDB patients. For supervised clinical prediction, we fit logistic regression probes for classification and ridge regression probes for continuous targets on frozen features from training patients, and evaluate on patient-disjoint test sets. D.2 Clinical task definitions We define the physiological attributes, medication and visit facts, and clinical prediction targets used in the main experiments. Tables D.2 and D.2 explain the language-based tasks; Tables D.2 and D.2 cover all 32 clinical prediction targets (17 MC-MED, six MIMIC-I, and nine VitalDB). In these tables, M denotes MC-MED, I denotes MIMIC-I, and V denotes VitalDB. Definitions distinguish the recorded target from its clinical interpretation. Physiological understanding. Window-level tasks describe rate, regularity, signal properties, and charted physiology (Table D.2). Their annotation rules are given in Appendix B.1. Rhythm regularity is a window attribute, whereas atrial fibrillation in the clinical probe is a recorded prior diagnosis. Cross-modal retrieval tests whether a signal retrieves its paired description and vice versa; R@k measures whether that pair appears in the top k candidates (Appendix D.3). It is a correspondence task rather than a separate disease label. Table 14: Physiological task definitions. Tasks in Tables 2 and 3, including the HR-class target used in few-shot evaluation. CV denotes coefficient of variation. Task Reference and clinical interpretation Rhythm Regular, slightly irregular, or markedly irregular according to synchronized ECG RāR interval CV (<0.05<0.05, [0.05,0.12)[0.05,0.12), or ā„0.12ā„ 0.12). Measures beat-timing regularity, without assigning a specific arrhythmia. Quality (Qual.) Clean, fair, or noisy according to the annotation quality score and usability gate. Describes waveform reliability and artifact burden. Perfusion (Perf.) Weak, moderate, or strong pulsatile amplitude relative to the signal baseline, using the waveform-derived AC/DC index. A peripheral perfusion surrogate, not a direct blood-flow measurement. Morphology (Morph.) Low, moderate, or high augmentation from the pyPPG augmentation index. Describes pulse-wave shape; it does not establish a vascular disease diagnosis. HR class Five pulse-rate bins: <50<50, 50ā59, 60ā100, 101ā120, and >120>120 bpm, spanning markedly slow to fast rates. These are benchmark annotation categories. HR estimation Heart rate in beats/min, evaluated against synchronized ECG. This independent reference differs from the PPG-derived pulse rate used to construct training captions. R class / estimation Low, normal, or elevated respiratory rate, or its numerical value in breaths/min. Both use charted respiratory rate as the reference and describe breathing frequency. SBP / MAP Systolic blood pressure (peak arterial pressure during a beat) and mean arterial pressure (pressure averaged over the cardiac cycle), in mmHg, evaluated against the corresponding charted measurements. HR trend Falling, stable, or rising rate within a window, from the slope of instantaneous pulse rate. Included in caption factuality. Amplitude variability Steady, moderately variable, or highly variable beat amplitudes, from beat-amplitude CV. Included in caption factuality. Caption factuality (Fact) Pooled accuracy of stated facts across six shared fields: HR, HR class, rhythm, HR trend, amplitude variability, and quality. Numerical HR is correct within Ā±5Ā± 5 bpm of the caption reference; categorical fields require agreement. Medication events and visit facts. Table D.2 defines every medication class and binary visit fact displayed in the main results. Event AUROC compares each medication class with no-administration controls. The class describes the recorded administration, without asserting a causal physiological response. Home-medication facts instead use patient-level active medication records, whose timing is not verified for each encounter (Appendix B.2); their classes may overlap. VitalDB facts describe preoperative status. In particular, the binary eGFR target identifies reduced estimated filtration, without establishing the duration required for a chronic kidney disease diagnosis. Table 15: Medication and visit-fact definitions. Dataset abbreviations refer to the tasks displayed in Tables 5 and 6. Task Data Recorded target and clinical meaning Administration events: medication class versus control Rate control M Administration of a rate-controlling medication, used to slow heart rate. Vasopressor M, V Administration of an agent used to increase arterial blood pressure. Vasodilator M, V Administration of a medication that dilates blood vessels. IV fluid M Intravenous fluid administration, representing fluid delivery to the circulation. Diuretic M Administration of medication that promotes renal salt and water excretion. Opioid V Administration of an opioid analgesic, used for pain control. Sedative V Administration of medication used to induce sedation or reduce arousal. Table D.2 (continued) Task Data Recorded target and clinical meaning Recorded home-medication use: binary visit facts CCB M Recorded use of calcium-channel blockers, which act on vascular and/or cardiac muscle. AntiHTN M Recorded use of antihypertensive medication, used to lower blood pressure. AntiPL M Recorded use of antiplatelet medication, which inhibits platelet activation or aggregation. AntiDM M Recorded use of glucose-lowering medication for diabetes management. Preoperative status: binary visit facts HTN V Preoperative hypertension, a recorded condition of persistently elevated blood pressure. DM V Preoperative diabetes mellitus, a disorder of glucose regulation. Anemia V The benchmarkās preoperative anemia label, indicating reduced hemoglobin or red-cell mass. eGFR <60<60 V Estimated glomerular filtration rate below 60 mL/min/1.73 m2, indicating reduced kidney filtration. Clinical prediction targets. The 15 classification and 17 regression targets in Table 22 use recorded categories and numerical measurements, respectively (Tables D.2āD.2). All are visit-level except VitalDB EtCO2 and BIS, which are window-level targets from capnography and EEG monitoring. MC-MED disease and device labels use prior ICD-10 records. Its laboratory targets use the first available ED-visit value; eGFR is calculated with the race-free CKD-EPI 2021 equation from creatinine, age, and sex. For MIMIC-I, laboratory matching takes the earliest available value from the 24 hours preceding the recording and the recording itself; a measurement during the recording can therefore be retained when no prior value is available. VitalDB albumin is preoperative. These tasks evaluate agreement with clinical records and measurements at their specified times, rather than uniformly prospective outcome prediction. Table 16: Clinical classification targets. These rows cover all 15 datasetātask pairs in Table 22; sex occurs in all three datasets. Positive disease labels refer to recorded conditions. Task Data Definition and clinical interpretation Shared demographic target Sex M, I, V Recorded patient sex. MC-MED: ED disposition and prior clinical history ED admission M Hospital admission following the ED encounter, versus no admission. Atrial fibrillation M Prior atrial fibrillation: disorganized atrial electrical activity associated with an irregular rhythm. The label records history, not necessarily AF in the observed window. Heart failure M Prior heart failure, a clinical syndrome involving impaired cardiac pumping and/or filling. Cardiac device M Prior recorded cardiac-device status. Chronic kidney disease M Prior chronic kidney disease, indicating persistent impairment of kidney structure or function. Diabetes M Prior diabetes mellitus, a disorder of glucose regulation. Anemia M Prior anemia, characterized by reduced hemoglobin or red-cell mass; the target is a diagnosis record, not a threshold applied to current Hb. MIMIC-I: hospital outcome and diagnostic category In-hospital mortality I Death during the indexed hospitalization. Circulatory diagnosis I The primary coded hospital diagnosis belongs to the circulatory-system chapter of ICD-9. VitalDB: preoperative status and postoperative disposition ASA ā„3ā„ 3 V Preoperative American Society of Anesthesiologists physical status I or higher; class I denotes severe systemic disease. Hypertension V Recorded preoperative hypertension, indicating persistently elevated blood pressure. ICU admission V Subsequent postoperative admission to intensive care. Table 17: Clinical regression targets. These rows cover all 17 datasetātask pairs in Table 22. Shared measurements are defined once; sampling times and target levels are specified in the accompanying text. Task Data Measurement and clinical interpretation Demographics and body size Age M, I, V Patient age in years. MIMIC-I values above 120 are treated as missing because of de-identification shifts. BMI V Body mass index: weight divided by squared height (kg/m2), a measure of body size relative to height. Kidney function, hematology, and blood chemistry eGFR M Estimated glomerular filtration rate (mL/min/1.73 m2), reflecting kidney filtration function. This target is continuous. Hemoglobin (Hb) M, I Blood concentration of the oxygen-carrying protein in red blood cells. Albumin M, V Serum concentration of albumin, a major protein synthesized by the liver. BUN M, I Blood urea nitrogen: the concentration of nitrogen in circulating urea, related to nitrogen metabolism and renal handling. Potassium M Blood potassium concentration, relevant to electrical activity of nerve and muscle cells, including the heart. Calcium M Blood calcium concentration, relevant to neuromuscular function and multiple cellular processes. Sodium M Blood sodium concentration, relevant to water balance and extracellular fluid regulation. Glucose M Blood glucose concentration, reflecting glycemic status at the sampled time. Independent intraoperative monitoring channels EtCO2 V End-tidal carbon dioxide partial pressure measured by capnography (mmHg), describing expired CO2 at the end of a breath. BIS V EEG-derived, unitless bispectral index, reflecting the hypnotic effect of anesthesia. D.3 Cross-modal retrieval We extend the 100-candidate evaluation in Table 4 to pools of 2,000 candidates, testing how well PPG-LM identifies paired signals and descriptions as the search space grows. Table D.3 reports both signal-to-text (S2T) and text-to-signal (T2S) retrieval on MC-MED, MIMIC-I, and VitalDB. Candidates are ranked by cosine similarity in the aligned embedding space. We report R@1, R@5, and R@10: the fraction of queries whose paired target appears among the top one, five, or ten candidates. The 100-candidate results average two disjoint pools, each with one window per patient and distinct reference captions. Effect of candidate-pool size. Expanding the pool from 100 to 2,000 candidates reduces recall in both directions, with larger R@1 declines on the external datasets. On MC-MED, S2T/T2S R@1 decreases from 0.975/0.965 to 0.780/0.744, while R@10 remains 0.998/0.995. At the same pool size, MIMIC-I reaches R@10 of 0.853/0.762 and VitalDB reaches 0.510/0.483. Thus, matching the exact pair becomes more difficult as the candidate pool grows, particularly after transfer to the external datasets, while MC-MED retains nearly all paired targets within the top ten results. Table 18: Bidirectional retrieval with varying candidate pools. PPG-LM recall (ā ) with N=100N=100 or 2,000 candidates. S2T: signal to text; T2S: text to signal. Direction Candidates (N) R@1ā R@5ā R@10ā [0pt][0pt] MC-MED (ED) S2T 100 0.975 1.000 1.000 2,000 0.780 0.988 0.998 T2S 100 0.965 1.000 1.000 2,000 0.744 0.980 0.995 [0pt][0pt] MIMIC-I (ICU) S2T 100 0.805 1.000 1.000 2,000 0.258 0.671 0.853 T2S 100 0.755 0.990 1.000 2,000 0.207 0.560 0.762 [0pt][0pt] VitalDB (OR) S2T 100 0.555 0.885 0.965 2,000 0.108 0.344 0.510 T2S 100 0.550 0.905 0.955 2,000 0.097 0.324 0.483 D.4 Caption evaluation and numerical read-out We assess the factual accuracy of PPG descriptions and the accuracy of heart rates read from those descriptions. We compare two PPG-LM interfaces: direct caption generation, which produces a new description from the waveform using the caption decoder with greedy decoding; and nearest-caption retrieval, which returns the highest-scoring existing caption by embedding similarity. Retrieval uses a fixed bank of 20,000 MC-MED captions from patients disjoint from the evaluation queries, including for external evaluation on MIMIC-I and VitalDB. The VitalDB analyses here use the official 40 Hz waveform view of the same evaluation items. The main PPG-LM results in Table 3 use the original 500 Hz source, accounting for the different VitalDB values in Tables D.4 and D.4. Caption evaluation. The deterministic parser checks stated fields against reference measurements; a numerical HR statement is correct within Ā±5Ā± 5 bpm. Fact accuracy is the number of correct stated fields divided by all stated fields, and coverage is the fraction of gradable fields that are stated. Table D.4 uses the six common fields across datasets for fact accuracy. The original MC-MED evaluation additionally grades perfusion and morphology, yielding an eight-field accuracy of 0.932 for direct generation and 0.889 for nearest-caption retrieval, versus 0.923 and 0.877 for the common six fields. The LLM judge, GPT-5.6-luna, receives the reference fact sheet and candidate caption. It classifies fields as correct, incorrect, or absent, counts unsupported claims, and scores factuality and completeness from 1 to 5. Optional charted vitals serve as supporting evidence but do not count toward completeness. Pairwise evaluation uses both presentation orders (400 judgments for 200 windows); generated PPG-LM captions are preferred to GPT-5.6-luna captions at rates of 0.922, 0.855, and 0.777 on MC-MED, MIMIC-I, and VitalDB. As a calibration check, reference captions receive factuality scores of 4.40, 4.80, and 4.85, with 0.86, 0.23, and 0.17 unsupported claims per caption, respectively, indicating that the judge is an imperfect evaluator. Direct generation has higher parser fact accuracy on all three datasets, whereas nearest-caption retrieval receives higher judge factuality scores and fewer unsupported claims (Table D.4). Table 19: Caption factuality by output method. Direct generation produces a new caption; nearest-caption retrieval selects an existing caption from the fixed bank. Fact accuracy covers six common fields; coverage includes all gradable fields. Judge factuality is on a 1ā5 scale; unsupported claims are counted per caption. VitalDB uses the 40 Hz view. Bold marks the best value within each dataset. Rule-based evaluation LLM-judge evaluation Model / caption source Fact acc.ā Coverageā Factualityā Unsupportedā [0pt][0pt] MC-MED (ED) GPT-5.6-luna 0.499 0.99 2.63 6.54 PPG-LM: direct generation 0.923 1.00 3.64 2.94 PPG-LM: nearest-caption retrieval 0.877 1.00 3.83 1.51 [0pt][0pt] MIMIC-I (ICU) GPT-5.6-luna 0.456 0.95 2.83 3.31 PPG-LM: direct generation 0.920 1.00 3.59 2.88 PPG-LM: nearest-caption retrieval 0.859 1.00 3.67 1.47 [0pt][0pt] VitalDB (OR) GPT-5.6-luna 0.486 0.98 2.88 3.21 PPG-LM: direct generation 0.837 0.98 3.30 3.17 PPG-LM: nearest-caption retrieval 0.778 1.00 3.43 1.52 Heart-rate read-out. For each PPG-LM interface, we extract the numerical HR stated in its output caption and compare it with the synchronized ECG reference. Table D.4 also includes two digital signal processing (DSP) baselines based on the dominant frequency and detected pulse peaks. HR read from directly generated captions has the lowest MAE and the highest fraction of estimates within 5 bpm on all three datasets. On MC-MED, its MAE is 0.27 bpm against the PPG-derived annotation rate but 3.90 bpm against ECG; the independent ECG reference therefore assesses agreement beyond the caption-construction target. Table 20: Heart-rate estimation from waveforms or captions. DSP estimates HR directly from the waveform; PPG-LM reads HR from a generated or retrieved caption. All entries use 300 valid ECG-referenced windows per dataset. MAE is in bpm; āWithin 5 bpmā is a fraction. VitalDB uses the 40 Hz view. Bold marks the best value within each dataset. Method / HR source MAEā Within 5 bpmā [0pt][0pt] MC-MED (ED) DSP: dominant frequency 6.07 0.857 DSP: pulse peaks 6.36 0.690 PPG-LM: HR from retrieved caption 4.70 0.847 PPG-LM: HR from generated caption 3.90 0.877 [0pt][0pt] MIMIC-I (ICU) DSP: dominant frequency 6.23 0.823 DSP: pulse peaks 6.40 0.680 PPG-LM: HR from retrieved caption 5.36 0.833 PPG-LM: HR from generated caption 4.27 0.860 [0pt][0pt] VitalDB (OR) DSP: dominant frequency 5.53 0.870 DSP: pulse peaks 7.93 0.723 PPG-LM: HR from retrieved caption 4.74 0.867 PPG-LM: HR from generated caption 4.08 0.900 D.5 Additional Visit-Level Clinical-Fact Recognition Complementing Section 5.2 and Table 6, Table 21 reports additional clinical-fact recognition on 400 MC-MED visits with eight PPG windows each. PPG-LM averages frozen pretrained fact embeddings across windows and scores them against textual fact descriptions. Read-out thresholds are calibrated on held-out MC-MED validation visits, with no calibration on test visits. Panel (a) covers eight triage-context tasks: high acuity (ESI 1ā2), arrival by ambulance versus self-arrival, three heart-rate (HR) classes, low oxygen saturation (SpO2<95%_2<95\%), high respiratory rate (R >20>20 breaths/min), fever (Temp. ā„38āā„ 38\, C), four blood-pressure (BP) classes, and ten chief-complaint (C) groups. Balanced accuracy accounts for class imbalance in both binary and multiclass tasks. Panel (b) covers eleven binary recorded conditions: hypertension (HTN), atrial fibrillation (AF), heart failure (HF), coronary artery disease (CAD), type 2 diabetes mellitus (DM), chronic kidney disease (CKD), anemia, obstructive sleep apnea (OSA), asthma, obesity, and stroke. PPG-LM achieves the highest reported balanced accuracy on every task in both panels, reaching 0.785 for HR and 0.729 for fever, with medical-history scores ranging from 0.580 to 0.687. These results measure associations with recorded visit attributes rather than independent clinical diagnoses. Table 21: Additional visit-level clinical-fact recognition. Balanced accuracy (ā ) for (a) triage context and (b) medical history on 400 MC-MED visits using eight windows each. PPG-LM read-outs use thresholds calibrated on held-out MC-MED validation visits. Bold marks column maxima. (a) Triage-context recognition Model ESIā Arrivalā HRā SpO2ā Rā Temp.ā BPā Cā GPT-5.6-luna 0.501 0.445 0.463 0.497 0.506 0.500 0.269 0.134 GPT-6-luna 0.500 0.498 0.486 0.500 0.500 0.500 0.238 0.128 GPT-6-sol 0.536 0.509 0.736 0.499 0.506 0.512 0.265 0.126 Opus 5.5 0.501 0.504 0.528 0.500 0.500 0.500 0.259 0.100 PulseLM 0.483 0.500 0.656 0.499 0.500 0.498 0.250 0.113 Qwen3-4B (SFT) 0.547 0.500 0.625 0.500 0.500 0.500 0.250 0.100 PPG-LM 0.618 0.607 0.785 0.650 0.692 0.729 0.442 0.202 (b) Medical-history recognition Model HTNā AFā HFā CADā DMā CKDā Anemiaā OSAā Asthmaā Obesityā Strokeā GPT-5.6-luna 0.500 0.502 0.495 0.460 0.483 0.481 0.496 0.531 0.500 0.478 0.512 GPT-6-luna 0.506 0.499 0.494 0.464 0.525 0.510 0.498 0.513 0.499 0.526 0.500 GPT-6-sol 0.498 0.598 0.524 0.495 0.490 0.500 0.500 0.500 0.500 0.522 0.500 Opus 5.5 0.547 0.659 0.546 0.530 0.487 0.494 0.490 0.500 0.500 0.500 0.500 PulseLM 0.504 0.534 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 Qwen3-4B (SFT) 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 0.500 PPG-LM 0.583 0.676 0.687 0.656 0.586 0.682 0.638 0.599 0.609 0.580 0.593 D.6 Medication-event evaluation Main medication-class benchmark. Table 5 uses 393 administrations and 393 controls on MC-MED and 139 administrations and 139 controls on VitalDB. Medication names are mapped to the classes used during training. Each view selects up to three nearest windows within 30 minutes before the anchor and five uniformly spaced windows within 60 minutes after it, retaining signed time offsets. Each reported AUROC compares one medication class with no-administration controls; other medication classes are excluded from that comparisonās negative group. These are class-specific ranking tasks, rather than a single measure of whether any medication was administered. Supplementary smaller-cohort evaluation. The earlier language event benchmark contains 150 administrations and 150 controls on MC-MED, with up to 15 administrations per medication class. Its selection windows are 30 minutes before and 60 minutes after the anchor, with up to three pre-event and five post-event observations. VitalDB contains 60 events and 60 controls within Ā±2Ā± 2 minutes; MIMIC-I has no event source in this benchmark. Same-visit paired accuracy uses the 56 MC-MED and 19 VitalDB pairs with both views available, assigning half credit to ties. It is not an accuracy over all event/control items. GPT-5.6-luna has tie fractions 0.70 and 0.53, respectively. Its reported AUROC comes from a Boolean answer and equals balanced accuracy, whereas PPG-LM uses a continuous difference between the highest medication-class score and the no-medication score. On MC-MED, PPG-LM balanced accuracy is 0.647, compared with 0.550 for the LLM; on VitalDB the values are 0.533 and 0.633. The separate validation diagnostic in the training report uses a different item set (AUROC 0.711, paired accuracy 0.687); those values are not directly comparable to the class-versus-control AUROCs in Table 5. D.7 Full Clinical Prediction Results This subsection provides all 32 datasetātask pairs evaluated in Section 5.3: 17 on MC-MED, six on MIMIC-I, and nine on VitalDB, comprising 15 classification and 17 regression tasks. Table 22 retains the complete results; Table 7 presents six tasks per dataset. Task definitions are given in Appendix D.2. Evaluation protocol. Logistic-regression classifiers and ridge-regression models are fitted on frozen representations using each datasetās training patients and evaluated on its test patients. Classification uses AUROC and regression uses R2R^2; results are reported as mean Ā± standard deviation. External-dataset probes therefore assess supervised transfer of the pretrained representations. For visit-level tasks on MC-MED, MIMIC-I, and VitalDB, all foundation-model baselines (including AnyPPG) and both PPG-LM stages use the same four PPG windows per visit, with evenly spaced windows on MC-MED. Baselines and Stage 1 mean-pool the four window features, while Stage 2 applies its learned visit aggregator to those same four windows. The observation budget is therefore matched across models and stages, while the visit read-out differs. VitalDB EtCO2 and BIS are evaluated on the same individual windows across models. MC-MED diagnosis targets denote recorded prior diagnoses. Its eGFR target is derived from creatinine, age, and sex and is not independent of the demographic targets. Laboratory timing differs across datasets: MC-MED uses the first ED-visit value, MIMIC-I uses the available admission/recording-associated value, and VitalDB uses preoperative measurements. MIMIC-Iās matching can retain a value measured during the recording when no earlier value is present. These probes therefore assess association with recorded clinical labels rather than a uniform prospective endpoint. Full clinical prediction results. Stage 2 exceeds all six foundation-model baselines in mean performance on all 32 tasks and improves over Stage 1 on 30. Stage 1 remains slightly stronger on VitalDB hypertension (0.656 vs. 0.655) and EtCO2 (0.153 vs. 0.152). Despite these relative gains, low R2R^2 for electrolytes, glucose, and BMI indicates limited predictive performance for these targets. Table 22: Full clinical prediction results. Test AUROC and R2R^2 (mean Ā± SD; ā ) for all 32 tasks. All models use the same four windows for visit-level targets. Bold: best mean; ā : window-level target. Foundation models PPG-LM Task PaPaGei-S PulsePPG AnyPPG SIGMA-PPG MOMENT-L Chronos-2 Stage 1 Stage 2 MC-MED (ED) ā AUROC ā Sex 0.692Ā± 0.006 0.719Ā± 0.007 0.783Ā± 0.008 0.712Ā± 0.008 0.724Ā± 0.005 0.739Ā± 0.008 0.813Ā± 0.006 0.856Ā± 0.005 ED admission 0.673Ā± 0.011 0.681Ā± 0.009 0.750Ā± 0.013 0.679Ā± 0.011 0.700Ā± 0.011 0.728Ā± 0.013 0.765Ā± 0.012 0.787Ā± 0.010 Atrial fibrillation 0.742Ā± 0.012 0.752Ā± 0.013 0.845Ā± 0.010 0.755Ā± 0.013 0.768Ā± 0.014 0.815Ā± 0.012 0.857Ā± 0.010 0.870Ā± 0.011 Heart failure 0.708Ā± 0.018 0.723Ā± 0.015 0.825Ā± 0.013 0.711Ā± 0.017 0.731Ā± 0.015 0.781Ā± 0.012 0.836Ā± 0.010 0.860Ā± 0.009 Cardiac device 0.713Ā± 0.023 0.729Ā± 0.018 0.835Ā± 0.022 0.733Ā± 0.022 0.719Ā± 0.017 0.784Ā± 0.023 0.836Ā± 0.022 0.857Ā± 0.025 Chronic kidney disease 0.647Ā± 0.017 0.666Ā± 0.016 0.761Ā± 0.016 0.666Ā± 0.014 0.673Ā± 0.013 0.735Ā± 0.013 0.774Ā± 0.013 0.792Ā± 0.014 Diabetes 0.644Ā± 0.012 0.649Ā± 0.014 0.717Ā± 0.015 0.660Ā± 0.020 0.661Ā± 0.016 0.698Ā± 0.016 0.728Ā± 0.016 0.740Ā± 0.016 Anemia 0.608Ā± 0.008 0.604Ā± 0.011 0.668Ā± 0.014 0.620Ā± 0.010 0.621Ā± 0.008 0.654Ā± 0.010 0.686Ā± 0.015 0.709Ā± 0.012 MC-MED (ED) ā R2āR^2 Age 0.395Ā± 0.016 0.428Ā± 0.018 0.626Ā± 0.020 0.389Ā± 0.017 0.475Ā± 0.017 0.521Ā± 0.016 0.691Ā± 0.016 0.782Ā± 0.014 eGFR 0.155Ā± 0.015 0.173Ā± 0.016 0.320Ā± 0.020 0.169Ā± 0.012 0.202Ā± 0.017 0.262Ā± 0.017 0.356Ā± 0.021 0.404Ā± 0.024 Hemoglobin 0.083Ā± 0.011 0.098Ā± 0.013 0.196Ā± 0.013 0.102Ā± 0.012 0.123Ā± 0.014 0.164Ā± 0.014 0.238Ā± 0.014 0.274Ā± 0.012 Albumin 0.093Ā± 0.015 0.094Ā± 0.011 0.207Ā± 0.015 0.097Ā± 0.012 0.122Ā± 0.016 0.185Ā± 0.017 0.239Ā± 0.018 0.286Ā± 0.013 BUN 0.053Ā± 0.009 0.062Ā± 0.015 0.152Ā± 0.015 0.063Ā± 0.009 0.070Ā± 0.014 0.114Ā± 0.010 0.178Ā± 0.019 0.213Ā± 0.019 Potassium 0.025Ā± 0.006 0.023Ā± 0.009 0.057Ā± 0.008 0.029Ā± 0.006 0.025Ā± 0.008 0.041Ā± 0.008 0.067Ā± 0.007 0.086Ā± 0.010 Calcium 0.002Ā± 0.006 0.011Ā± 0.010 0.041Ā± 0.008 0.010Ā± 0.005 0.001Ā± 0.008 0.024Ā± 0.009 0.051Ā± 0.009 0.076Ā± 0.012 Sodium 0.017Ā± 0.010 0.023Ā± 0.008 0.056Ā± 0.010 0.026Ā± 0.008 0.017Ā± 0.007 0.045Ā± 0.010 0.068Ā± 0.012 0.091Ā± 0.015 Glucose 0.032Ā± 0.007 0.039Ā± 0.007 0.065Ā± 0.008 0.041Ā± 0.009 0.028Ā± 0.007 0.049Ā± 0.011 0.071Ā± 0.007 0.083Ā± 0.011 MIMIC-I (ICU) ā AUROC ā Sex 0.651Ā± 0.024 0.686Ā± 0.014 0.713Ā± 0.018 0.672Ā± 0.016 0.682Ā± 0.019 0.664Ā± 0.020 0.730Ā± 0.019 0.733Ā± 0.018 In-hospital mortality 0.697Ā± 0.029 0.741Ā± 0.034 0.756Ā± 0.031 0.732Ā± 0.025 0.751Ā± 0.025 0.772Ā± 0.020 0.786Ā± 0.022 0.793Ā± 0.021 Circulatory diagnosis 0.666Ā± 0.015 0.690Ā± 0.022 0.725Ā± 0.017 0.658Ā± 0.018 0.672Ā± 0.012 0.677Ā± 0.017 0.720Ā± 0.022 0.726Ā± 0.023 MIMIC-I (ICU) ā R2āR^2 Age 0.295Ā± 0.039 0.308Ā± 0.031 0.468Ā± 0.029 0.288Ā± 0.036 0.347Ā± 0.036 0.367Ā± 0.028 0.502Ā± 0.030 0.507Ā± 0.028 BUN 0.029Ā± 0.011 0.029Ā± 0.009 0.065Ā± 0.019 0.034Ā± 0.015 0.055Ā± 0.015 0.074Ā± 0.012 0.081Ā± 0.012 0.086Ā± 0.013 Hemoglobin 0.035Ā± 0.010 0.048Ā± 0.010 0.079Ā± 0.023 0.034Ā± 0.009 0.046Ā± 0.012 0.076Ā± 0.018 0.092Ā± 0.024 0.095Ā± 0.020 VitalDB (OR) ā AUROC ā Sex 0.586Ā± 0.023 0.626Ā± 0.022 0.658Ā± 0.016 0.616Ā± 0.016 0.620Ā± 0.017 0.608Ā± 0.016 0.684Ā± 0.012 0.691Ā± 0.013 ASA ā„3ā„ 3 0.606Ā± 0.021 0.642Ā± 0.020 0.690Ā± 0.026 0.605Ā± 0.035 0.625Ā± 0.035 0.655Ā± 0.040 0.699Ā± 0.034 0.704Ā± 0.025 Hypertension 0.598Ā± 0.016 0.607Ā± 0.020 0.635Ā± 0.019 0.595Ā± 0.025 0.610Ā± 0.016 0.617Ā± 0.023 0.656Ā± 0.016 0.655Ā± 0.017 ICU admission 0.690Ā± 0.009 0.700Ā± 0.004 0.747Ā± 0.002 0.683Ā± 0.009 0.662Ā± 0.011 0.714Ā± 0.009 0.733Ā± 0.004 0.751Ā± 0.006 VitalDB (OR) ā R2āR^2 Age 0.240Ā± 0.026 0.263Ā± 0.036 0.403Ā± 0.024 0.259Ā± 0.026 0.302Ā± 0.025 0.337Ā± 0.032 0.418Ā± 0.030 0.435Ā± 0.029 BMI 0.015Ā± 0.005 0.022Ā± 0.003 0.036Ā± 0.008 0.026Ā± 0.005 0.035Ā± 0.003 0.013Ā± 0.010 0.041Ā± 0.007 0.043Ā± 0.007 Albumin 0.075Ā± 0.026 0.085Ā± 0.034 0.136Ā± 0.046 0.085Ā± 0.037 0.089Ā± 0.038 0.101Ā± 0.040 0.134Ā± 0.054 0.138Ā± 0.048 EtCO2ā 0.098Ā± 0.023 0.121Ā± 0.034 0.105Ā± 0.028 0.113Ā± 0.024 0.133Ā± 0.029 0.150Ā± 0.031 0.153Ā± 0.038 0.152Ā± 0.037 BISā 0.380Ā± 0.019 0.398Ā± 0.022 0.393Ā± 0.021 0.387Ā± 0.020 0.402Ā± 0.021 0.404Ā± 0.024 0.407Ā± 0.022 0.408Ā± 0.022 D.8 Few-Shot Adaptation We assess how labeled support affects adaptation across MC-MED, MIMIC-I, and VitalDB, complementing the clinical-transfer evaluation in Section 5.3. Few-shot adaptation of PPG-LM. We fit linear classifiers on frozen representations using kā5,10,20,50kā\5,10,20,50\ labeled examples per class from a support pool that is patient-disjoint from the query set. Table D.8 reports macro-F1 as mean Ā± standard deviation over five support draws; columns correspond to the four per-class label budgets. Tasks cover rhythm regularity, heart-rate (HR) class, sex, and dataset-specific outcomes: ED admission on MC-MED, in-hospital mortality on MIMIC-I, and ICU admission on VitalDB. HR-class performance increases consistently with more support, reaching 0.921, 0.912, and 0.873 at k=50k=50 on the three datasets, respectively. Eleven of twelve tasks attain their highest mean at k=50k=50; the exception is VitalDB ICU admission, which peaks at k=10k=10 (0.591) and falls to 0.557 at k=50k=50. Thus, additional labels generally improve adaptation, but gains remain task-dependent. For comparison, the separate LLM in-context protocol uses ten labeled windows in total, rather than k examples per class. On MC-MED, PulseLMās linear probes at k=10/50k=10/50 reach rhythm macro-F1 of 0.395/0.403 and HR-class macro-F1 of 0.750/0.811. These evaluations use distinct adaptation procedures and should be interpreted under their respective support budgets. Table 23: Few-shot adaptation of PPG-LM. Macro-F1 (mean Ā± SD; ā ) over five support draws. VitalDB uses the 40 Hz waveform view. k: examples per class. Bold: best mean per row. Task k=5k=5 k=10k=10 k=20k=20 k=50k=50 [0pt][0pt] MC-MED (ED) ā Macro-F1 ā Rhythm 0.488Ā± 0.051 0.537Ā± 0.019 0.567Ā± 0.037 0.597Ā± 0.022 HR class 0.755Ā± 0.023 0.831Ā± 0.025 0.898Ā± 0.018 0.921Ā± 0.003 Sex 0.568Ā± 0.042 0.645Ā± 0.054 0.696Ā± 0.041 0.710Ā± 0.026 ED admission 0.563Ā± 0.014 0.558Ā± 0.026 0.583Ā± 0.018 0.617Ā± 0.008 [0pt][0pt] MIMIC-I (ICU) ā Macro-F1 ā Rhythm 0.506Ā± 0.029 0.585Ā± 0.037 0.572Ā± 0.025 0.591Ā± 0.021 HR class 0.757Ā± 0.050 0.859Ā± 0.023 0.891Ā± 0.017 0.912Ā± 0.006 Sex 0.505Ā± 0.058 0.538Ā± 0.039 0.562Ā± 0.040 0.580Ā± 0.033 In-hospital mortality 0.548Ā± 0.053 0.576Ā± 0.042 0.597Ā± 0.042 0.636Ā± 0.015 [0pt][0pt] VitalDB (OR) ā Macro-F1 ā Rhythm 0.375Ā± 0.023 0.365Ā± 0.013 0.360Ā± 0.020 0.399Ā± 0.006 HR class 0.761Ā± 0.039 0.832Ā± 0.043 0.862Ā± 0.026 0.873Ā± 0.011 Sex 0.498Ā± 0.018 0.501Ā± 0.039 0.496Ā± 0.035 0.558Ā± 0.013 ICU admission 0.543Ā± 0.033 0.591Ā± 0.017 0.552Ā± 0.045 0.557Ā± 0.041 D.9 Objective ablations Table 24: Objective ablations on MC-MED. Rhythm/quality: macro-F1; retrieval: signal-to-text R@1 with 2,000 candidates. Stage-2 CLIP/CoCa report per-metric bests across configurations. Variant Rhythm ā Quality ā R@1 ā [0pt][0pt] Stage 1 CLIP 0.650 0.505 0.790 SigLIP 0.620 0.400 0.815 CoCa 0.640 0.614 0.783 [0pt][0pt] Stage 2 CLIP 0.534 0.580 0.797 SigLIP 0.600 0.470 0.820 CoCa 0.651 0.673 0.820 Table D.9 compares three segment objectives across training stages. CLIP (Radford et al., 2021) uses symmetric InfoNCE, āCLIP=āNCEā(Q,)L_CLIP=L_NCE(Q;C), while SigLIP (Zhai et al., 2023) uses the pairwise sigmoid loss āiāj=ālogā”Ļā”[yiājā(Qiāj+b)] _ij=- Ļ[y_ij(Q_ij+b)], where yiājā+1,ā1y_ijā\+1,-1\ labels matched/mismatched pairs and b is a learned bias. CoCa additionally optimizes caption likelihood. At Stage 1, CLIP leads rhythm recognition, SigLIP leads retrieval, and CoCa leads quality assessment. At Stage 2, CoCa improves all three metrics, achieving the highest rhythm and quality macro-F1 and matching SigLIPās R@1. This combination supports CoCa for attribute recognition and retrieval while retaining caption generation. The Stage-2 CoCa R@1 of 0.820 uses signal-only captions; the main modelās signal-plus-vitals configuration reaches 0.780 at the same candidate-pool size (Table D.3). Component-ablation setting. Table 8 uses the CLIP configuration on MC-MED validation, with shared Stage-1 initialization and matched Stage-2 training budgets. Temporal accuracy distinguishes true from corrupted statements; event AUROC compares any medication event against no-administration controls; segment R@1 measures segmentātext retrieval. D.10 Prompt sensitivity We examine whether expanding the class descriptions improves zero-shot recognition while keeping the model and evaluation items fixed. The Template bank contains the caption-style descriptions used in the main experiments; the Combined bank augments them with reworded variants. For each class, we average its text embeddings and score the resulting representation against the waveform embedding by cosine similarity. Table 25 reports macro-F1 for three-class rhythm, five-class heart rate (HR), and three-class signal quality on MC-MED, MIMIC-I, and VitalDB. Template prompts achieve higher rhythm and quality scores on all three datasets. Combined prompts improve HR-class macro-F1 from 0.355 to 0.393 on MC-MED and from 0.337 to 0.350 on MIMIC-I, but reduce it from 0.327 to 0.282 on VitalDB. The template bank thus performs better on seven of the nine datasetātask pairs. Expanding the prompt bank does not consistently improve recognition, indicating that the choice of class descriptions remains consequential even when the model is unchanged. Table 25: Prompt sensitivity of PPG-LM. Template uses caption-style class descriptions; Combined adds reworded variants. Macro-F1 (ā ); bold marks the best score per column. MC-MED MIMIC-I VitalDB Prompt bank Rhythm HR class Quality Rhythm HR class Quality Rhythm HR class Quality Template 0.651 0.355 0.673 0.670 0.337 0.786 0.389 0.327 0.583 Combined 0.629 0.393 0.592 0.642 0.350 0.677 0.384 0.282 0.515 Appendix E Language Templates and Examples This appendix presents the language used for training and evaluation. Training captions are rendered from structured measurements and clinical records using fixed template banks; the example identifier and random seed determine template selection and sentence order. No external LLM is used. We distinguish these training targets from zero-shot class descriptions and the modelās learned text-prefix embeddings. E.1 Representative Caption Templates Caption content. Table 26 reproduces representative template strings, with braces marking substituted fields. Segment captions describe available rate, rhythm, and quality evidence; missing features are omitted, and poor-quality segments receive a quality-only description. Event captions identify the medication class associated with the sampled windows. Visit captions combine available demographics, triage context, and prior history. Table 26: Representative training templates. Strings are reproduced verbatim; braced fields are filled from measurements or records. Target Template Segment rate The pulse rate is about val beats per minute, level. Segment rhythm The cardiac rhythm is level. Segment quality The PPG trace is level. Poor-quality segment This segmentās pulse waveform is too noisy to characterize reliably. Medication event During this period the patient was given desc. Control interval This period of monitoring contains no medication administration. Chief complaint The chief complaint at triage was c. Temporal captions and negatives. Temporal templates express occurrence, prevalence, contiguous duration, or temporal order. A negative caption replaces one eligible sentence and retains the others. For example, the implemented positive sentence āOver the stay the pulse rate trended upward.ā can be replaced by āOver the stay the pulse rate trended downward.ā This pair illustrates the corruption rule, rather than an observed trajectory. E.2 Zero-Shot Evaluation Prompts Segment scoring. Table 27 reproduces representative prompts from the language benchmark in Section 5. The main segment-recognition results use two caption-style descriptions per class, whose embeddings are averaged for cosine scoring. Appendix D.10 compares this template bank with a combined bank that adds reworded class descriptions. Quality recognition uses a separate clean/fair/noisy prompt bank. Table 27: Representative zero-shot prompts. Verbatim examples from the segment template bank and the visit and event class descriptions, grouped by task. Concept Prompt Pulse rate Marked bradycardia The measured pulse rate is markedly bradycardic. Bradycardia The measured pulse rate is bradycardic. Normal rate The measured pulse rate is within the normal range. Mild tachycardia The measured pulse rate is mildly tachycardic. Tachycardia The measured pulse rate is tachycardic. Rhythm Regular rhythm The pulse rhythm appears regular. Irregular rhythm The pulse rhythm appears slightly irregular. Marked irregularity The pulse rhythm appears markedly irregular. Signal quality Clean signal Pulse-wave quality here is clean and reliable. Fair signal Pulse-wave quality here is of fair quality. Noisy signal The signal is somewhat noisy but usable for analysis. Visit attributes Visit acuity The visit is an emergent, high-acuity presentation. Visit demographics The recording is from a male patient. Visit complaint The visit is for a patient presenting with chest pain. Medication events Medication class The patient received a bronchodilator during this period. Control No medication was given during this period. Visit and event scoring. Visit prompts describe individual attributes and are scored against the selected visit read-out. The MC-MED event bank contains ten medication classes and a no-medication description. For the supplementary any-administration diagnostic in Appendix D.6, the score is the highest medication-class score minus the no-medication score. The main medication table instead reports separate class-versus-control AUROCs. E.3 Synthetic Caption Examples and Decoder Context These captions were rendered from invented feature values and clinical fields using the captioning pipeline. They illustrate training targets; they are neither patient records nor model-generated outputs. Missing fields contribute no sentences. Segment caption. Overall the PPG signal is clean and reliable. Over the window the pulse rate stays stable. The heartbeat is regular across the window. Heart rate estimated from the pulse waveform is around 82 bpm, within the normal range. Event caption. This segment covers the time around a dose of an inhaled bronchodilator. Visit caption with history. The individual is a 67-year-old male patient (older adult). Arrival vital signs: heart rate 108 bpm (mildly tachycardic), SpO2 96% (normal). The patient got to the ED by ambulance (EMS). Emergency severity was emergent, ESI level 2. Chief complaint: shortness of breath. Past medical history includes essential hypertension. The home medication list includes statins. CoCa decoder context. CoCa tokenizes each segment caption into a beginning-of-sequence token, caption subwords, and an end-of-sequence token. The shared text encoder prepends the learned segment prefix esege_seg (denoted [WIN] in the implementation), which is a vector rather than a literal word. After removing the prefix position, the causal text states enter the separate multimodal decoder, which cross-attends to waveform memory MiM_i and predicts the next caption token; padding targets are ignored. Training uses no natural-language generation prompt. The language benchmark additionally evaluates greedy waveform-conditioned decoding alongside retrieval and class-prompt scoring.