Paper deep dive
LLMs can construct powerful representations and streamline sample-efficient supervised learning
Ilker Demirel, Larry Shi, Zeshan Hussain, David Sontag
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/22/2026, 6:26:25 AM
Summary
The paper introduces an agentic pipeline using Large Language Models (LLMs) to construct 'rubric' representations for complex, heterogeneous datasets. By synthesizing global or local rubrics from text-serialized inputs, the method standardizes data for downstream supervised learning, significantly outperforming traditional baselines on the EHRSHOT clinical benchmark while offering improved auditability, cost-efficiency, and scalability.
Entities (5)
Relation Signals (3)
Global Rubric β appliedto β EHRSHOT
confidence 99% Β· We evaluate our methods on 15 clinical prediction tasks in the EHRSHOT benchmark
LLM β constructs β Global Rubric
confidence 98% Β· LLMs can construct powerful representations... We propose an agentic pipeline to streamline this process.
Global Rubric β outperforms β CLMBR-T
confidence 95% Β· Our rubric-style representations agentically constructed by LLMs outperform... a clinical foundation model pretrained on 2.57M patients (CLMBR-T)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:As real-world datasets become increasingly complex and heterogeneous, supervised learning is often bottlenecked by input representation design. Modeling multimodal data for downstream tasks, such as time-series, free text, and structured records, often requires non-trivial domain-specific engineering. We propose an agentic pipeline to streamline this process. First, an LLM analyzes a small but diverse subset of text-serialized input examples in-context to synthesize a global rubric, which acts as a programmatic specification for extracting and organizing evidence. This rubric is then used to transform naive text-serializations of inputs into a more standardized format for downstream models. We also describe local rubrics, which are task-conditioned summaries generated by an LLM. Across 15 clinical tasks from the EHRSHOT benchmark, our rubric-based approaches significantly outperform traditional count-feature models, naive text-serialization-based LLM baselines, and a clinical foundation model, which is pretrained on orders of magnitude more data. Beyond performance, rubrics offer several advantages for operational healthcare settings such as being easy to audit, cost-effectiveness to deploy at scale, and they can be converted to tabular representations that unlock a swath of machine learning techniques.
Tags
Links
- Source: https://arxiv.org/abs/2603.11679v1
- Canonical: https://arxiv.org/abs/2603.11679v1
Trouble viewing inline? Open PDF directly β
Full Text
127,023 characters extracted from source content.
Expand or collapse full text
LLMs can construct powerful representations and streamline sample-efficient supervised learning Ilker Demirel 1 Larry Shi 1 Zeshan Hussain 1 2 David Sontag 1 Abstract As real-world datasets become increasingly complex and heterogeneous, supervised learning is often bottlenecked by input representation design. Modeling multimodal data for downstream tasks, such as time-series, free text, and structured records, often requires non-trivial domain-specific engineering. We propose an agentic pipeline to streamline this process. First, an LLM analyzes a small but diverse subset of text-serialized input examples in-context to synthesize a global rubric, which acts as a programmatic specification for extracting and organizing evidence. This rubric is then used to transform naive text-serializations of inputs into a more standardized format for downstream models. We also describe local rubrics, which are task-conditioned summaries generated by an LLM. Across 15 clinical tasks from the EHRSHOT benchmark, our rubric-based approaches significantly outperform traditional count-feature models, naive text-serialization-based LLM baselines, and a clinical foundation model, which is pretrained on orders of magnitude more data. Beyond performance, rubrics offer several advantages for operational healthcare settings such as being easy to audit, cost-effectiveness to deploy at scale, and they can be converted to tabular representations that unlock a swath of machine learning techniques. Figure 1. Performance averaged over all 15 clinical prediction tasks in the EHRSHOT benchmark with 6,739 patients (Wornow et al., 2023). Our rubric-style representations agentically constructed by LLMs outperform naive text-serialization-based LLM baseline in Hegselmann et al. (2025), as well as a clinical foundation model pretrained on 2.57M patients (CLMBR-T, (Wornow et al., 2023)), and a count feature-based gradient boosting machine (Count-GBM, (Ke et al., 2017; Wornow et al., 2023)). 1. Introduction Supervised learning underpins a wide range of applications across domains. In medicine, deep neural networks achieve specialist-level performance in pneumonia detection and diabetic retinopathy screening (Rajpurkar et al., 2017; Gulshan et al., 2016). In finance, credit risk assessment models outperform legacy scorecards (Lessmann et al., 2015). In environmental science, supervised learning enables weather forecasting from radar observations (Ravuri et al., 2021). 1 Massachusetts Institute of Technology, Cambridge, MA USA. 2 Brigham and Womenβs Hospital and Harvard Medical School, Boston, MA, USA. Correspondence to: Ilker Demirel <demirel@mit.edu>. Preprint. Project website & code: https://LRRLpaper.github.io 1 arXiv:2603.11679v1 [cs.AI] 12 Mar 2026 LLMs can construct powerful representations and streamline sample-efficient supervised learning 1.1. Representational Challenges in Complex Domains A common thread across successful applications of supervised learning is the availability of input representations that can be easily processed by off-the-shelf models. Real-world datasets, however, are increasingly more complex and heterogeneous. They combine structured fields with unstructured text, time-stamped events, and different modalities such as images. In healthcare, clinical prediction may benefit from longitudinal labs and vitals, coded events (e.g., diagnoses and procedures), free-text notes, and medical images. In finance, stock-price forecasting and risk modeling may involve time series for trading prices and volumes, unstructured text (e.g., news and filings), and structured event records such as analyst rating changes. In domains where the input comprises a mix of heterogeneous and complex components, representation design requires substantial domain expertise and bespoke feature engineering, which may be unavailable. Even when expertise is available, resulting representations are not necessarily optimal: they may discard critical signal or bury it in noise, preventing the downstream model from harnessing it effectively. We show how large language models (LLM) can build agentic supervised learning pipelines that automate the design of powerful input representations and enable sample-efficient learning. 1.2. LLMs as a Representation Layer LLMs offer a practical interface to heterogeneous data. Disparate components of the input can be serialized into text and directly processed with an LLM, bypassing domain-specific engineering. Recent work shows that LLMs can serve as effective predictors over complex inputs. Song et al. (2024) and Akhauri et al. (2025) serialize diverse system configurations and logs into unified text sequences to predict performance metrics and optimization outcomes. Hegselmann et al. (2025) serialize longitudinal electronic health records (EHR) into Markdown and train linear heads over their embeddings for clinical prediction tasks (see Figure 2, left, for an example of text-serialized EHR data). These works show LLMsβ potential to streamline supervised learning with complex datasets, but they treat text-serialization of input as fixed and leave the bulk of learning to the downstream model. In contrast, we take the text-serialized input as a starting point and show how LLMs can automate constructing better representations that dramatically improve downstream performance. A complementary line of work is related to βdata science (DS) agentsβ. Data Interpreter targets end-to-end benchmark-style problem solving, using decomposition, code generation, execution, and revision to complete data-analysis, machine-learning, and mathematical tasks (Hong et al., 2025). DS-Agent focuses on automating model-development workflows such as task understanding, model selection, and training (Guo et al., 2024). DeepAnalyze and DS-STAR push further toward autonomous data science over heterogeneous files, with an emphasis on multi-step data wrangling, open-ended querying, code execution, and report generation (Zhang et al., 2025b; Nam et al., 2025). This literature is closely related to our work in spirit, since it also uses LLMs as an interface to heterogeneous data. Our focus, however, is narrower and more controlled: rather than asking agents to plan and execute broad analyses, we study how LLMs can support representation design for supervised learning on complex data for specific downstream tasks. This lets us isolate the role of representation choice and demonstrate its effect as a first-order driver of downstream statistical performance in a controlled setting. 1.3. A Frontier: Pre-trained Knowledge for Sample-efficiency Beyond data-processing convenience, LLMs possess pretraining knowledge that can enable effective regularization, which is key to sample-efficiency. Our work aligns with literature on injecting knowledge into statistical models: LMPriors uses language descriptions as task-specific priors (Choi et al., 2022), and TabLLM demonstrates effective few-shot tabular learning (Hegselmann et al., 2023). Similarly, LLM-Select and LLM-Lasso guide feature selection and regularization (Jeong et al., 2025; Zhang et al., 2025a), and Kim et al. (2025) use LLM-encoded task metadata to construct inductive biases. While these methods yield gains, they mostly use LLMs to augment traditional models working on clean datasets. In contrast, we focus on representation design: how complex inputs should be organized prior to downstream learning. In that sense, our work also aligns with recent literature on learning in the language space, such as GEPA by Agrawal et al. (2026). 1.4. Our Contribution: Rubric Representation Learning We tackle the following question. How can LLMs help construct powerful input representations in complex domains to streamline sample-efficient supervised learning for downstream tasks? 2 LLMs can construct powerful representations and streamline sample-efficient supervised learning # Naive Text Serialization ## Patient Demographics - Patient age: 78, FEMALE [...] ## Detailed Past Medical Visits ### Inpatient Visit (14 days to pred. time, current visit) #### Conditions - Acute posthemorrhagic anemia - pH measurement, venous: 7.25, 7.31, 7.31 [...] #### Medications - furosemide 20 MG Oral Tablet - pantoprazole 20 MG Delayed Release Oral Tablet [...] #### Procedures - Chest x-ray - Electrocardiogram report [...] ### Emergency Room Visit (87 days before prediction time) #### Conditions - Benign essential hypertension - Chest pain [...] #### Medications - 2ML ondansetron 2MG/ML inject. - nitroglycerin 0.4 MG [...] #### Procedures - Ct angiography - Comprehen metabolic panel[...] # Local Rubric Representation 1. Patient Snapshot 27 yo hispanic male. Recurrent cardiology visits for congenital anomaly of coronary artery [...] 2. Main Risk Factors - Congenital coronary artery anomaly (established structural predisposition to myocardial ischemia/infarction). - Tobacco exposure (smokeless tobacco reported) [...] 3. Protective Factors - Young age (27) | lower baseline atherosclerotic burden relative to older adults. - Normal BMI (21-22). - No documented diabetes (glucose in normal range) or chronic renal impairment [...] 6. Overall Risk Impression Elevated risk of another acute myocardial infarction [...]. Rationale: although the patient is young and has favorable metabolic parameters, the combination of a congenital coronary anomaly [...] # Global Rubric Representation 3. Demographics - 55 | FEMALE | [...] 6. Recent Cardiac Symptoms (last 365 days) - Chest pain/angina: No - Dyspnea/shortness of breath: Yes (date unknown) [...] 12. Other Relevant Labs - Creatinine: 1.12 (2023-12-02) - eGFR: No data [...] 17. Known Risk Factors - Diabetes mellitus: No (A1c date unknown) - Hyperlipidemia: Yes - Family history of premature CAD: Unknown [...] 20. Non-cardiac Serious Illness That May Mimic or Alter MI Risk Interpretation - Active malignancy: No - Severe infection/sepsis in past 30 days: No - Major surgery in past 30 days: Yes | multiple inpatient procedures noted in December 2023 (e.g., CPT4/00520 on 2023-12-26 [...] Figure 2. Synthetic electronic health record (EHR) representation examples, focusing on the acute myocardial infarction (acute MI) prediction task. Left. Naive text-serialization adopted from Hegselmann et al. (2025). Middle. Local rubric representation which is a task-conditioned summary of the naive text-serialization. Right. Global rubric transformed version of the naive-text serialization. We propose rubric representation learning, where a team of LLM agents learn and implement rubrics, which are used to transform inputs into a standardized and information-rich format that can be easily and efficiently digested by downstream learners. We assume a naive text-serialization of the input is available, or it can be constructed straightforwardly (see Figure 2, left for an example). We develop two types of rubrics, which are given below and detailed in Section 2. Global rubrics. We propose global rubrics, a novel input representation for complex datasets. A global rubric is a task-level specification that defines what information should be extracted from the input and how. It is generated by prompting an LLM with a diverse set of examples and asking it to produce a useful rubric, which is then used to transform the inputs (see Figure 2, right). We use global rubrics in two ways: (i) through vector embeddings of rubric-transformed text-serializations (i) converting the rubric serialization into tabular features via an LLM (see Figure 3). Local rubrics. We prompt an LLM to produce a task-conditioned local summary with structured sections (see Figure 2, middle), similar to recent work on explainable clinical prediction models (Petridis et al., 2026). The summarization serves two key purposes: (i) it yields a less noisy representation that can be embedded more efficiently (i) it injects the pretraining knowledge into the embeddings, delegating part of the learning to the upstream representation design step. Advantages of global rubrics. Both rubrics achieve similar downstream performance and outperform the baselines. However, while all inputs share the same sections after the summarization step, local rubrics do not have the same level of standardization as global rubrics, which endows the latter with several practical desiderata lacking in the former. β’Auditable and improvable: Global rubrics are more amenable to inspections by domain experts, such as for analyzing subgroup bias risk, identifying missing or misleading evidence, and iterative refinement. β’More operationally useful: Global rubric representations can be transformed to tabular features (Figure 3, Panel (F)), immediately enabling a suite of applications, such as interpretability methods and causal analyses. For instance, intervening on or controlling for variables are far simpler to do with tabular features than with free-form summaries. β’Cheaper to deploy at scale: Global rubric transformation at inference time can be automated (see Figure 3, Panels (E) and (F)), whereas summarization requires an LLM API call for each example. This can effectively make global rubrics βfreeβ compared to local rubrics, which incur anO(N)cost in time and money. In contrast, global rubrics haveO(1) monetary cost and can be applied orders of magnitude faster using deterministic scripts on CPU at inference time. 3 LLMs can construct powerful representations and streamline sample-efficient supervised learning 1.5. Evaluation in EHRSHOT Benchmark: Clinical Outcome Prediction We evaluate our methods on 15 clinical prediction tasks in the EHRSHOT benchmark, spanning four task groups: operational outcomes (3 tasks), assignment of new diagnoses (6 tasks), anticipating lab results (5 tasks), and prediction of chest X-ray findings (1 task) (Wornow et al., 2023). We compare against a gradient boosting machine (GBM) that uses count-based features (Count-GBM, (Ke et al., 2017)), a clinical foundation model pretrained on 2.57M patients (CLMBR-T, (Wornow et al., 2023)), zero-shot chain-of-thought prompting (CoT) with Qwen3-8B and GPT5-mini 1 (Wei et al., 2022; Qwen3Team, 2025; OpenAI, 2025), and the LLM baseline in Hegselmann et al. (2025) which uses naive text-serializations of the patient EHRs. Our methods using rubric style representations outperform the baselines by a substantial margin on average. Relative to CLMBR-T, they achieve strong gains on new diagnosis and lab result tasks, and remain close for operational outcomes and chest X-ray findings tasks. 2. Rubric Representation Learning with LLMs 2.1. Global Rubrics We introduce LLM-derived global rubrics, a domain-agnostic strategy for converting heterogeneous, weakly structured inputs into task-aligned representations that are more amenable to supervised learning. While our experiments focus on electronic health records (EHR) as the input, the procedure applies whenever inputs can be rendered as text. Setup and notation.We describe the global rubric learning procedure for a single prediction task. LetD =(x i , y i ) n i=1 denote labeled training data, wherexis a raw input andy β0, 1is the task label. Lets(Β·)be some serialization procedure that maps an input to text and define x text = s(x). A rubric specifies a task-specific transformation R : x text 7β x rubric , wherex rubric is a more structured representation of the same underlying inputx, and it can be used with downstream predictors instead of x text . We describe how x text and x rubric are used for downstream training in Section 4. Global rubric synthesis. Global rubric learning has two stages, which are shown in Figure 3, Panels (A) and (B). First, we select a small, label-balanced and diverse cohort from the training split. Second, an LLM inspects this cohort in-context and synthesizes a task-specific rubric by selecting and constructing predictive signals. β’ Step 1a) Diverse cohort selection (motivation):Rubric synthesis is done through a single prompt to an LLM (GPT5-Mini) with a limited number of examples due to context length limitations (272k tokens). Since the in-context cohort drives rubric creation, cohort diversity is crucial for rubric to capture different modes of variation rather than reflecting a narrow set of training examples. Hence, we aim to build a cohort that (i) is label-balanced and (i) spans a diverse region in training distribution. β’ Step 1b) Diverse cohort selection (procedure):We first embed each text-serialized inputx text i into a vector space using a pretrained text embedding model (we use Qwen3-8B-Embedding (Zhang et al., 2025c)) and stratify by label, D + =x text i : y i = 1, D β =x text i : y i = 0. We performk-means clustering independently within each stratum in the embedding space. Here,kdenotes the number of clusters per label stratum, so the cohort contains2β kexamples in total. Due to context windows limitations, we usek = 20. From each cluster, we select the element that is closest to the center of the cluster (the medoid) into the cohort, yielding a compact cohort that covers diverse regions of the input space within both classes. β’ Step 2) Rubric synthesis:Given the selected cohort, we ask an LLM to produce a task-specific rubric that (i) identifies discriminative, task-relevant signals, (i) organizes them into coherent categories, and (i) specifies how each signal should be extracted from a new input. The rubric standardizes how the input should be reorganized and transformed, but does not encode a prediction rule. The full rubric-synthesis prompt is provided in Appendix D.1, and two full-rubric examples can be found in Appendix E.1 and E.2. 1 In compliance with the EHRSHOT license (Wornow et al., 2023), we used GPT5-Mini and GPT-5.2 via the HIPAA-compliant Microsoft Azure OpenAI Service. 4 LLMs can construct powerful representations and streamline sample-efficient supervised learning (A) Diverse Cohort Selection # Label stratified k-means in text-serialization (x text ) embedding space Y = 0 medoidY = 1 medoid Y = 0 patientY = 1 patient (B) Rubric Synthesis # Ask an LLM to synthesize a task-specific rubric. Create a rubric for predicting hypertension risk in the next year by analyzing data from 40 patients. List of EHRs (Medoids, x text format): Pt 1 : [78yo, F, HTN meds, SBP=148...] ... Pt 40 : [27yo, M, family hx, SBP=129...] Output a structured rubric. β’ Be data-driven [...] β’ Be structured and consistent [...] β’ Extract facts only [...] β’ [...] (C) Task-Specific RubricR # LLM-derived rubric R for transforming x text to x rubric Β§1. DEMOGRAPHICS β’ Age, sex, BMI Β§2. CV RISK FACTORS β’ BP readings (SBP/DBP) β’ HTN medications Β§3. COMORBIDITIES β’ Diabetes, CKD status Β§4. TEMPORAL TRENDS β’ BP trajectory (6-12mo) β’ Weight changes Β§5. ALERT FLAGS β’ Resistant HTN markers β’ End-organ damage [...] R : x text β x rubric (D) Rubric Application via LLMs # Ask an LLM to apply the rubric transformation R to each input. [...] ## Rubric R: rubricinstructions ## Patient EHR: ehr text (x text ) Fill in every field of the rubric template above using ONLY information from this patientβs EHR. Rules: β’ Follow the exact field order and section structure of the rubric. β’ If data for a field is not present, write "No data". β’ [...] (E) Rubric Application via Parser # Ask an LLM to generate a parser script to apply the learned rubric transformation R to each input. Write a Python script that reads patient EHR text serializations and fills in a structured clinical rubric template using deterministic string/regex parsing only [...] Rubric R: rubric instructions Example EHR text serializations: List of medoid pairs: (x text , x rubric ) The generated script must: β’ Use only Python standard libraries such as βreβ, βjsonβ [...] β’ No LLM API calls, no network requests, no subprocess calls to external tools [...] β’ [...] (F) Rubric Tabularization # Ask an LLM to generate a script to transform x rubric to tabular features based on R. Write a Python script to convert rubric-formatted patient EHRs into numeric feature vectors [...] Example rubric-transformed EHR serializations: List of medoids in x rubric format, obtained from x text using parser in Panel (E) Your logic must: - General: handle any value the rubric parser could plausibly produce [...] - Robust: gracefully handle missing values [...] - [...] Figure 3. Agentic global-rubric pipeline for EHRSHOT tasks. (A) Build a label-balanced and diverse patient set viak-means. (B) Patient EHRs are fed to an LLM which is prompted to synthesize a task rubric. (C) The LLM outputs a systematic rubricRthat defines how to transform any patient EHR from naive text (x text ) to textual rubric representation (x rubric ). (D) An LLM is asked to transformx text tox rubric for each patient. (E) An LLM is asked to write a script to automate the transformation step in Panel (D). (F) An LLM is asked to write a script to transform rubric representations x rubric into tabular features. Full prompts are provided in Appendix D. Global rubric application. Once a global rubric is learned, we apply it to naive text-serialization of each input,x text , to produce x rubric . We propose three different methods for the rubric transformation step, each with their own advantages. β’Rubric application via an LLM (Figure 3, Panel (D)):We prompt an LLM (GPT5-Mini) with the learned global rubric,R, and the naive text-serialization of the input,x text , asking it to returnx rubric by following the instructions inR. Full prompt is included in Appendix D.2. The method tied to this rubric representation is called Global-Rubric. β’Rubric application via a parser (Figure 3, Panel (E)):We prompt an LLM (GPT-5.2) with the learned global rubric,R, and some paired examples (40) of naive text-serialization of the input,x text , and LLM-generated rubric- transformations,x rubric . We ask the LLM to write a script that can be used to automatically convertx text tox rubric . Full prompt is included in Appendix D.3. The method tied to this rubric representation is called Global-Rubric-Auto. β’ Rubric-based tabularization (Figure 3, Panel (F)):We prompt an LLM (GPT-5.2) with the global rubric,R, parser script for applying the rubric transformation (see item above), and some examples (40) of parser-generated rubric- transformations,x rubric . We ask the LLM to write a script to convertx rubric into a set of tabular features. Full prompt is included in Appendix D.4. The method tied to this rubric representation is called Global-Rubric-Tabular. 5 LLMs can construct powerful representations and streamline sample-efficient supervised learning # Task-conditioned local rubric generation prompt GOAL: Read the patientβs text-serialized EHR and write a compact reasoning trace that characterizes the patientβs risk profile for the following clinical outcome prediction task: taskquery --- START OF EHR DATA --- NaiveText Serialization (x text ) --- END OF EHR DATA --- Your output MUST follow this exact structure: 1. Patient Snapshot 2. Main Risk Factors 3. Protective Factors 4. Whatβs Unknown / Could Swing the Risk 5. Weighing and Aggregating the Evidence 6. Overall Risk Impression # Generic local rubric generation prompt Read the patientβs EHR below and write a compact clinical summary that characterizes the patientβs overall health status, key medical conditions, risk factors, and relevant clinical context. Focus on information that would be useful for clinical decision-making. Do not make predictions about any specific outcome. Provide a general-purpose summary that captures the essential clinical information. --- START OF EHR DATA --- NaiveText Serialization (x text ) --- END OF EHR DATA --- Figure 4. Prompts used for generating local rubric representations. Left. Prompt for generating task-conditioned local rubric summaries. Right. Prompt for generating generic local rubric summaries (ablation). We also run an ablation where the global rubric is learned without showing any input examples to the LLM that synthesizes the rubricR(i.e., list of examples in Figure 3, Panel (B) is taken out). The learned rubric is then used to transform all inputs via an LLM with the approach in Figure 3, Panel (D). The method tied to this rubric representation is called Global-Rubric-Blind. 2.2. Local Rubrics Global rubric representations define a structure that is shared across all input samples (i.e.,x rubric i ) after the rubric trans- formation step. Beyond performance gains, this level of standardization in global rubrics unlocks several practical and methodological advantages. However, it is still interesting to explore how much of the gains in performance is due to the type of standardization imposed by global rubrics versus an LLMβs ability to preprocess the input prior to downstream training in the language space using its world-knowledge from pretraining. To that end, we also propose local rubrics, which are task-conditioned summaries of the inputx text with a generic section structure included in the prompt, generated by an LLM (GPT5-Mini). The method tied to this rubric representation is called Local-Rubric. We also run an ablation, where the LLM is asked to generate a generic summary of the input, as opposed to a task-conditioned one. The method tied to this rubric representation is called Local-Rubric-Generic. The full prompts used to create local rubric representations for x rubric are included in Figure 4. 3. EHRSHOT Benchmark We evaluate on EHRSHOT (Wornow et al., 2023), a longitudinal electronic health record (EHR) benchmark with train/vali- dation/test splits across tasks. EHRSHOT contains deidentified EHR data from 6,739 patients treated at Stanford Medicine, including demographics, diagnoses, procedures, medications, and laboratory measurements for each visit. The dataset captures full patient timelines with millions of coded clinical events across hospital encounters. Wornow et al. (2023) also release CLMBR-T, an autoregressive transformer pretrained on 2.57M patients on next-code prediction. More broadly, a growing body of work explores pretraining medical foundation models on large EHR or claims datasets for risk prediction and patient trajectory modeling (Steinberg et al., 2024; Renc et al., 2024; Waxler et al., 2025). Interestingly, recent work shows that general-purpose LLMs can be competitive with domain-specific pretrained models on downstream clinical tasks (Hegselmann et al., 2025), a finding we reproduce and strengthen with LLM-derived rubrics. 3.1. Clinical Prediction Tasks EHRSHOT contains 15 prediction tasks across four categories: operational outcomes (3 tasks), assignment of new diagnoses (6 tasks), anticipation of laboratory results (5 tasks), and prediction of chest X-ray findings (1 task) (Wornow et al., 2023). Each task is a patient-level classification problem evaluated at a specific prediction time along a patientβs longitudinal record. For each task, a patient can contribute multiple samples with different prediction times. 6 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 1. Number of samples (positive cases) per task and split. Lab results and Chest X-ray tasks are subsampled from the original dataset. CategoryTaskTrainValTest Operational Outcomes (3)ICU transfer2402 (113)100 (50)2037 (85) Length of stay >7 days2569 (681)100 (50)2195 (552) 30-day readmission2608 (370)100 (50)2189 (260) Assignment of New Diagnosis (6)Hypertension1259 (182)100 (50)1258 (159) Hyperlipidemia1684 (205)100 (50)1317 (172) Pancreatic cancer2576 (155)100 (50)2220 (56) Celiac disease2623 (62)22 (11)2222 (21) Lupus2570 (104)66 (33)2243 (20) Acute MI2534 (175)100 (50)2127 (144) Anticipating Lab Results (5)Thrombocytopenia2000 (1000)100 (50)2000 (1000) Hyperkalemia2000 (1000)100 (50)1896 (948) Hypoglycemia2000 (1000)100 (50)1566 (783) Hyponatremia2000 (1000)100 (50)2000 (1000) Anemia2000 (1000)100 (50)2000 (1000) Chest X-ray Findings (1)Chest X-ray abnormality2000 (1000)100 (50)2000 (1000) Operational outcome tasks predict near-term events during a hospital episode, such as long length-of-stay (LOS), intensive care unit (ICU) transfer, or 30-day readmission. Assignment of new diagnosis tasks predict whether a patient will receive a new diagnosis within the next year from the prediction time. Laboratory tasks predict whether an upcoming lab result will be abnormal based on the patient history prior to the test. The chest X-ray task predicts the presence of abnormal findings in the radiology report associated with an upcoming chest X-ray. 3.2. Subsampling As some of our methods require LLM API calls for each sample to construct representations, evaluating on the full dataset is computationally expensive. Therefore, we use a subset of the original dataset. For assignment of new diagnosis and operational outcome tasks, we subsample only the validation set, selectingmin(50, n + ) examples from each label (positive and negative), wheren + is the number of positive-labeled examples in the original split. Training and test splits are kept unchanged. For laboratory and chest X-ray tasks, which contain substantially more examples, we also subsample the training and test splits to includemin(1000, n + )examples from each label. The validation set is constructed using the same procedure as above withmin(50, n + )examples per label. Detailed sample counts for each task after subsampling are provided in Table 1. 3.3. Evaluation All models are trained on the EHRSHOT training split and evaluated on the held-out test split. Hyperparameters are tuned based on the validation split performance. We report AUROC and AUPRC as primary evaluation metrics, which are more clinically appropriate and meaningful than threshold-dependent metrics such as F1-score (Van Calster et al., 2025). We also report 95% confidence intervals (CI), which are obtained by bootstrapping the test set with replacement. 4. Methods and Training Here we describe downstream training for our methods and the baselines. 4.1. Our Rubric Methods In Sections 2.1 and 2.2, we described six ways to obtain a rubric representation,x rubric , from the naive text-serializations, x text . For five of those methods, x rubric admits a textual format. We recall them below briefly. 7 LLMs can construct powerful representations and streamline sample-efficient supervised learning # Prompt template for obtaining text input embeddings for downstream training Based on the patientβs EHR below, predict: taskquery --- Patient EHR --- x text or x rubric --- End of EHR --- Based on the above EHR, predict: task query Respond with exactly one word: Yes or No. Figure 5. Prompt for converting textual inputs to embeddings. An example task query: βWill the patient develop lupus within next year?β β’Global-Rubric: RubricRis learned in-context from some input examples (naive text-serializations) using an LLM, and then applied by another LLM to each individual x text for converting to x rubric . β’ Global-Rubric-Auto: An LLM generates a parser script to apply the rubricR to convert x text to x rubric . β’ Global-Rubric-Blind: The rubricRis generated blindly without any example data, and solely based on the LLMβs world-knowledge. Another LLM then converts x text to x rubric . β’Local-Rubric: An LLM generates a task-conditioned summary from eachx text to act asx rubric , following the prompt in Figure 4, left. β’ Local-Rubric-Generic: An LLM generates a generic summary from eachx text to act asx rubric , following the prompt in Figure 4, right. For the methods above, the resulting textualx rubric is converted into a vector embedding by passing it to a pretrained embedding model through the prompt structure in Figure 5. We use Qwen3-8B-Embedding (Zhang et al., 2025c). The resulting embedding vectors are then used to fit a logistic regression classifier with L2-penalty in the train split. The best penalty parameter is chosen based on validation split negative log likelihood. And finally, we have β’ Global-Rubric-Tabular: An LLM generates a parser script to convert x rubric into a set of tabular features. In this case,x rubric is a tabular feature vector as opposed to text, and is readily available for supervised learning. We fit an XGBoost model on top (Chen & Guestrin, 2016). We optimize the hyperparameters (number of estimators, maximum depth, learning rate, and subsampling rate) based on validation split negative log likelihood. 4.2. EHRSHOT Baselines We include the count-based gradient boosting machine, Count-GBM, baseline following EHRSHOT (Wornow et al., 2023). Each patient EHR is converted into a high-dimensional vector of code counts observed prior to the prediction time. Different time-windows are used, such as last 90 days and 90-180 days before. A LightGBM classifier is then trained (Ke et al., 2017). We also evaluate CLMBR-T, a medical foundation model release alongside the EHRSHOT benchmark dataset (Wornow et al., 2023). CLMBR-T is a transformer-based autoregressive medical foundation model, pretrained with a next-code prediction objective, using longitudinal data from 2.57M patients drawn from the same distribution as the EHRSHOT dataset. For downstream tasks, a logistic regression classifier is trained using vector embeddings extracted from CLMBR-T. 4.3. LLM-based Baselines Our rubric representations are derived from naive text-serializations of the input,x text . We adopt the serialization introduced by Hegselmann et al. (2025) (see Figure 2, left). Each record includes patient demographics, a βGeneral Medical Eventsβ section for codes that are not tied to a visit, and a βDetailed Past Medical Visitsβ section listing visits in reverse chronological order. We refer to the baseline that usesx text directly as NaiveText. Same as textual rubrics,x text is embedded using a pretrained embedding model via the prompt in Figure 5, and a logistic regression classifier is trained on the embeddings. We also evaluate zero-shot chain-of-thought (CoT) prompting (Wei et al., 2022). For each example, Qwen3-8B and GPT5-Mini are prompted to reason over the EHR step-by-step and give a finalYes/Noanswer. We sample 10 responses and estimate the probability as the fraction of Yes answers. 8 LLMs can construct powerful representations and streamline sample-efficient supervised learning 5. Quantitative Results Experimental setup.We report zero-shot chain-of-thought (CoT) baselines (n = 0) and supervised results in two regimes: n = 40andn = All. In then = 40regime, all downstream learners use the same 40 training samples that were used to synthesize the global rubrics to enable an apples-to-apples comparison. We report AUROC and AUPRC with 95% confidence intervals (CI), estimated by bootstrap on the test set using 1000 resamples with replacement. For overall and task-group metrics, averages and CIs are computed by first calculating per-task scores within each bootstrap resample and then averaging those scores across relevant tasks. Overall and task-group averages are shown in Figure 6 and Table 2. Per-task results are reported in Appendix A, Figures 8β11, and Tables 3β17. Unless otherwise stated, we use Qwen3-8B-Embedding as the text-embedding model. Tables 19 and 20 in Appendix C provide results for the 0.6B model. NaiveText is a strong baseline. NaiveText already comes close to the domain-specific pretrained model CLMBR-T, reproducing the core finding of Hegselmann et al. (2025). In the full-data regime, NaiveText reaches 0.699 AUROC and 0.391 AUPRC on average across the 15 tasks, compared to 0.727 and 0.432 for CLMBR-T. This shows that a straightforward text-serialization paired with a general-purpose embedding model already recovers a substantial fraction of the signal captured by large-scale medical pretraining. Rubric representations yield a clear second jump over NaiveText and surpass CLMBR-T on average. The main quantitative result is that rubric representations lead to a substantial improvement on top of the already strong NaiveText baseline. In then = 40regime, Local-Rubric is best overall, improving over NaiveText from 0.638 to 0.717 in AUROC and from 0.343 to 0.406 in AUPRC; Global-Rubric is close behind at 0.700 AUROC and 0.400 AUPRC, and both clearly exceed CLMBR-T at 0.657 and 0.356. In the full-data regime, Local-Rubric attains the best overall AUROC at 0.772, while Global-Rubric attains the best overall AUPRC at 0.459. Importantly, these two methods are very close overall: Global-Rubric trails Local-Rubric by only 0.009 AUROC, while exceeding it by 0.007 AUPRC. These gaps are small relative to the much larger margins by which both methods outperform NaiveText and CLMBR-T. The largest gains appear in new diagnosis and lab tasks. The task-group averages in Figure 6 show that assignment of new diagnosis and anticipating lab results tasks are where rubric learning helps most. For new diagnosis prediction in the full-data regime, Local-Rubric achieves the best average AUROC at 0.770, while Global-Rubric achieves the best average AUPRC at 0.236; both improve substantially over NaiveText (0.709 AUROC, 0.179 AUPRC) and CLMBR-T (0.697 AUROC, 0.170 AUPRC). The lab tasks show even larger gains. In then = 40regime, Global-Rubric-Tabular achieves the best average lab AUROC and AUPRC at 0.749 and 0.720, only narrowly ahead of Local-Rubric at 0.729 and 0.719. In the full-data regime, Global-Rubric-Tabular continues to be the strongest method (0.799 AUROC, 0.772 AUPRC), providing large improvements over both CLMBR-T (0.727 AUROC, 0.713 AUPRC) and NaiveText (0.657 AUROC, 0.649 AUPRC). At the same time, the comparison within the rubric family is not one-sided: Local-Rubric remains very competitive on labs, and different rubric variants lead on different metrics and data regimes. Per-task results reveal consistent gains.The per-task results in Appendix A, Figures 8β11 and Tables 3β17 show that the average improvements are not driven by a few outlier tasks. Across the new diagnosis tasks, rubric variants are consistently at or near the top, and are particularly strong in the low-data regime withn = 40. A similar pattern holds for the lab tasks, where rubric methods lead in most tasks. Among the three operational outcomes, Local-Rubric remains competitive in AUROC on two of them, but CLMBR-T is stronger overall, especially in precision-recall space. The chest X-ray task is the main exception to the broader trend. However, absolute performance is modest for all methods, and we hypothesize that the labels that were binarized from 14 categories may be noisier (Wornow et al., 2023), so we avoid over-interpreting. Operational outcome tasks remain comparatively stronger for CLMBR-T. Operational outcome tasks are the one category where the rubric methods do not overtake CLMBR-T on average. ICU transfer and long length of stay, in particular, continue to favor CLMBR-T, especially in precision-recall space. Even here, however, the comparison is not one-sided. In then = 40regime, Global-Rubric attains the best average AUROC across operational outcome tasks, and rubric methods still improve over NaiveText. One plausible explanation is that these tasks align more closely with the kind of temporal patterns which are abundant during CLMBR-T pretraining. Because CLMBR-T is trained autoregressively on millions of patient timelines, it is repeatedly exposed to visit timelines and progression, care transitions, and short-horizon hospital events. That signal may be especially relevant for outcomes such as ICU transfer and length of stay. In contrast, rubric learning appears to help most on tasks where the predictive evidence is sparser and more dispersed across the record. 9 LLMs can construct powerful representations and streamline sample-efficient supervised learning Figure 6. Average performance across different task-groups. Table 2. AUROC and AUPRC with 95% CIs. Best per column for each sample size regime is highlighted. (a) AUROC Overall (15) Operational Outcomes (3) Assignment of New Diag. (6) Anticipating Labs (5) Chest X-ray Findings (1) n = 0 Qwen3-8B-CoT.610 .601β.618 .639 .616β.660 .557 .541β.575 .667 .658β.677 .546 .519β.573 GPT5-Mini-CoT.644 .635β.653 .680 .658β.700 .613 .594β.632 .684 .675β.694 .520 .498β.541 n = 40 Count-GBM.594 .579β.609 .625 .599β.651 .630 .596β.663 .536 .523β.547 .581 .550β.612 CLMBR-T.657 .643β.670 .745 .724β.767 .662 .630β.690 .608 .597β.619 .606 .574β.638 NaiveText.638 .623β.652 .732 .710β.753 .648 .616β.680 .586 .574β.598 .559 .535β.583 Local-Rubric-Generic.688 .673β.703 .734 .711β.758 .700 .665β.733 .673 .663β.684 .561 .537β.584 Local-Rubric .717 .703β.730 .744 .721β.766 .725 .692β.757 .729 .718β.739 .533 .507β.559 Global-Rubric-Blind.682 .669β.696 .708 .683β.732 .673 .644β.701 .714 .704β.725 .505 .480β.530 Global-Rubric.700 .685β.714 .752 .731β.772 .708 .676β.742 .695 .684β.705 .526 .502β.551 Global-Rubric-Auto.694 .681β.708 .740 .720β.760 .704 .675β.732 .692 .680β.703 .510 .479β.540 Global-Rubric-Tabular.686 .672β.701 .675 .649β.697 .683 .651β.714 .749 .738β.758 .471 .441β.506 n = All Count-GBM.689 .674β.703 .741 .718β.762 .754 .722β.786 .595 .584β.606 .609 .576β.642 CLMBR-T.727 .713β.741 .818 .799β.836 .697 .666β.728 .727 .717β.737 .630 .599β.660 NaiveText.699 .684β.714 .775 .754β.793 .709 .674β.744 .657 .646β.668 .616 .592β.640 Local-Rubric-Generic.738 .725β.751 .781 .761β.800 .739 .711β.770 .736 .726β.746 .608 .583β.631 Local-Rubric .772 .758β.784 .802 .786β.818 .770 .738β.799 .789 .780β.798 .606 .583β.630 Global-Rubric-Blind.751 .738β.764 .776 .755β.794 .745 .716β.775 .777 .768β.787 .585 .559β.608 Global-Rubric.763 .748β.777 .786 .768β.805 .756 .723β.789 .791 .781β.800 .594 .567β.619 Global-Rubric-Auto.756 .743β.769 .790 .772β.807 .752 .722β.782 .776 .766β.786 .575 .543β.605 Global-Rubric-Tabular.751 .739β.764 .756 .734β.778 .742 .713β.769 .799 .790β.808 .552 .519β.582 (b) AUPRC n = 0 Qwen3-8B-CoT.316 .309β.324 .185 .172β.200 .080 .069β.093 .637 .623β.650 .526 .491β.560 GPT5-Mini-CoT .348 .337β.362 .221 .205β.240 .137 .111β.169 .647 .634β.659 .507 .476β.538 n = 40 Count-GBM.306 .295β.316 .201 .183β.221 .131 .112β.151 .531 .517β.546 .541 .502β.581 CLMBR-T.356 .344β.369 .337 .304β.373 .127 .108β.148 .593 .578β.610 .600 .558β.642 NaiveText.343 .331β.356 .271 .245β.301 .148 .123β.177 .580 .565β.595 .549 .517β.582 Local-Rubric-Generic.392 .378β.407 .296 .268β.326 .187 .158β.217 .662 .647β.676 .562 .528β.595 Local-Rubric .406 .393β.420 .299 .272β.328 .178 .152β.206 .719 .705β.732 .526 .494β.556 Global-Rubric-Blind.370 .360β.382 .263 .237β.293 .133 .113β.153 .694 .679β.708 .502 .472β.533 Global-Rubric.400 .382β.418 .298 .270β.327 .192 .154β.233 .681 .666β.696 .544 .514β.578 Global-Rubric-Auto.377 .365β.389 .285 .259β.314 .157 .133β.181 .669 .654β.685 .507 .467β.547 Global-Rubric-Tabular.380 .369β.392 .245 .221β.269 .151 .128β.178 .720 .706β.733 .462 .426β.497 n = All Count-GBM.377 .361β.392 .281 .256β.307 .220 .186β.252 .583 .569β.597 .582 .540β.624 CLMBR-T.432 .419β.446 .425 .387β.468 .170 .145β.197 .713 .699β.726 .623 .584β.666 NaiveText.391 .377β.406 .315 .283β.346 .179 .151β.208 .649 .634β.664 .609 .577β.641 Local-Rubric-Generic.434 .419β.448 .335 .307β.364 .213 .184β.243 .724 .710β.738 .601 .567β.632 Local-Rubric.452 .439β.466 .341 .310β.374 .223 .194β.251 .762 .748β.776 .605 .571β.638 Global-Rubric-Blind.428 .415β.443 .321 .292β.351 .185 .155β.217 .753 .739β.767 .587 .556β.620 Global-Rubric .459 .442β.478 .339 .309β.371 .236 .200β.276 .773 .760β.786 .582 .550β.614 Global-Rubric-Auto.437 .422β.452 .343 .312β.378 .195 .164β.226 .760 .745β.773 .557 .518β.596 Global-Rubric-Tabular.441 .426β.456 .328 .295β.368 .204 .173β.237 .772 .758β.785 .538 .501β.576 10 LLMs can construct powerful representations and streamline sample-efficient supervised learning Local and global rubrics are close overall. Local-Rubric seems to be a contender for the strongest performing method quantitatively, particularly in AUROC and especially in the low-data regime. At the same time, the comparison between local and global rubrics is much closer than a winner-take-all reading suggests. Global-Rubric achieves the best overall AUPRC, remains within 0.009 AUROC of Local-Rubric on average, and is the stronger textual rubric on several task groups and individual tasks, especially in precision-recall space. More broadly, the global-rubric family is exceptionally strong: its parser-based and tabular variants are often near the top, and on the lab tasks they are frequently the best methods overall. Given how small the performance gaps are, the practical advantages of global rubrics emphasized in the introduction become especially relevant: they are more standardized, easier to audit and refine with experts, cheaper to deploy at scale, and readily convertible into tabular features that unlock a broad toolkit of off-the-shelf machine learning methods. The ablations strengthen the case that representation choice is a first-order driver of performance. Local-Rubric improves substantially over Local-Rubric-Generic, confirming that task-conditioned representation design matters beyond generic summarization. On the global side, Global-Rubric-Blind is already strong, indicating that pretrained clinical knowledge alone can induce a useful rubric. Most importantly, Global-Rubric-Auto and Global-Rubric-Tabular remain competitive with the full LLM-applied Global-Rubric, and in several settingsβespecially the lab tasksβthe tabular variant is among the strongest methods in the paper. This is notable because these approaches learn the rubric from only 40 examples and then apply the transformation deterministically via generated parsers, rather than relying on repeated patient-specific LLM reasoning at inference time. Their strong performance therefore strengthens our central claim: the representation itself is a first-order driver of downstream performance. Full-dataset evaluation with Global-Rubric-Tabular. Because the tabularization script for a learned global rubric (Figure 3, Panel (F)) can be applied fast and at zero cost, we evaluate Global-Rubric-Tabular on the full EHRSHOT dataset without subsampling. Across the 15 benchmark tasks, the method achieves a mean AUROC of 0.770 and mean AUPRC of 0.312. Per-task results and averages across task groups are reported in Appendix B. 6. Qualitative Rubric Analyses 6.1. Case Study: Global Rubric Analysis for the Hypertension Prediction Task Here, we examine the learned global rubric instructions for the new hypertension diagnosis prediction task. Part of the rubric is given in Figure 7, and the full rubric is provided in Appendix E.1. Standardization and noise reduction. A central strength of the rubric is its preparation stage, which performs noise reduction and standardization before feature extraction. In Step A of Figure 7, it (i) defines clinically meaningful temporal windows (e.g., very recent:β€ 30days), (i) enforces unit normalization for blood pressure, height, weight, and labs, and (i) removes implausible BP values. This mirrors clinical practice, where distinguishing transient or acute BP elevations from sustained chronic hypertension is critical for diagnosis. For example, ACC/AHA guidelines stress that hypertension should be diagnosed from multiple readings over time rather than isolated measurements, particularly in inpatient or perioperative settings (Whelton et al., 2018). By encoding these temporal distinctions and filtering implausible values, the rubric reduces spurious variance and prevents downstream models from over-weighting noisy measurements, a known source of false positives in EHR-based hypertension phenotyping (Banegas et al., 2018). Extraction of higher-order features. The rubric systematically derives higher-order features well-established in the hypertension literature, but not directly available from raw EHR. In Step 2 of Figure 7, it computes summary statistics within each window, derives simple trend metrics (e.g., a recent systolic BP slope), and categorizes blood pressure using ACC/AHA clinical thresholds (Normal, Elevated, Stage 1, Stage 2) based on recent values. These transformations reflect guideline-based risk assessment: BP trajectories and progression across categories predict incident hypertension and cardiovascular risk more strongly than single-point measurements, and the ACC/AHA thresholds are designed to stratify future risk (Whelton et al., 2018). Importantly, the rubric not only computes these features but also enforces handling of discordant or variable readings, preventing ambiguous cases from collapsing into misleading averages. Bucketing features into differentially predictive domains. The synthesis-per-domain step (Figure 7, Step 9) is partic- ularly consequential for downstream prediction. It aggregates evidence across domains, assigns severity weights (major, moderate, minor), and produces explicit counts of high-, moderate-, and low-risk features, yielding composite task-specific 11 LLMs can construct powerful representations and streamline sample-efficient supervised learning # Excerpt from Global Rubric Instructions (R) for Hypertension Diagnosis Task A. Preparation (before extracting) 1. Define the prediction window: βnext yearβ relative to the EHR reference date/time. 2. Define time windows to extract: - Very recent: last 30 days - Recent: 31-180 days - Baseline/remote: >180 days 3. Standardize units and formats: - Blood pressure: mmHg (systolic/diastolic) - Weight: kg or oz β convert to kg - Height: cm or in β convert to meters [...] Step 2 - Blood pressure (BP) data extraction and normalization Extract all systolic/diastolic BP values with timestamps and context (office, inpatient, ED, home, ambulatory, perioperative). Normalize: remove implausible values (document them), ensure mmHg. For each time window (very recent, recent, baseline): compute count, mean, median, SD, min, max; identify last BP; flag highest recent BP Compute simple trend metrics (e.g., recent slope; BP variability via SD). Categorize BP per ACC/AHA categories using aggregated recent values: Normal (<120/<80), Elevated (120-129/<80), Stage 1 (130-139 or 80-89), Stage 2 (β₯140 or β₯90). [...] Step 9 - Synthesis per domain (structured fields and scoring) For each domain, record presence, supporting data, recency, and confidence (High/Moderate/Low). Domain A - BP phenotype: last BP (date/context), mean recent BP (last 30d; 31-180d), BP category, variability flag, ambulatory/home BP. Domain B - Metabolic / vascular risk: Diabetes (Y/N) - last A1c (% and date), BMI and obesity category, Hyperlipidemia (Y/N) - LDL value and date, Smoking (current/former/never) Create a simple domain scorecard: number of High/Moderate/Minor risk features. [...] Figure 7. Key excerpts from learned hypertension global rubric instructions illustrating (i) preprocessing/standardization, (i) higher-order BP feature construction (trends, variability, ACC/AHA categories), and (i) domain-wise synthesis into an aggregated scorecard. For full global rubric instructions, refer to Appendix E.1. features absent from the raw EHR. For example, a patient with Stage 1 BP plus chronic kidney disease (CKD) and diabetes will accumulate multiple high-risk domain flags even if no single raw measurement appears extreme. This structured aggregation mirrors clinical risk stratification, which emphasizes cumulative burden across BP phenotype, comorbidities, and end-organ involvement rather than isolated findings (Cuspidi et al., 2025). Encoding these aggregates in textual form provides downstream models with a rich summary of hypertension risk factors. 6.2. Case Study: Learned Tabular Features for Hyponatremia Lab Results Prediction We end with a qualitative analysis of the learned tabular features (Figure 3, Panel (F)) for prediction of hyponatremia abnormality. We find that across the 15 tasks, the auto-generated rubric feature schemas range from 147 to 450 features per task. Most tasks cluster around 200β250 features. The features are predominantly binary (72%), followed by numeric (19%) and categorical (9%). The high binary share reflects pervasive one-hot encoding of categoricals and the inclusion of a missing indicator for nearly every field. Numeric features capture lab values, vitals, and counts. We focus on learned tabular features for the hyponatremia lab task, for which the full global rubric is given in Appendix E.2. We make several observations. The feature structure closely mirrors the diagnostic decision tree used clinically when evaluating hyponatremia. A first step in clinical reasoning is determining whether apparent hyponatremia is physio- logic or artificially low due to hyperglycemia or other osmotic effects; accordingly, the rubric extracts recent glucose measurements (Glucose-Last3) and serum osmolality values, while the tabular features include indicators reflecting level of glucose in the blood, e.g.glucose-type-blood-present. If true hypotonic hyponatremia is present, clinicians next evaluate urine osmolality and urine sodium to distinguish between states of antidiuretic hormone (ADH) activity and renal sodium handling, which helps identify etiologies such as SIADH or hypovolemia (Spasovski et al., 2014). Consistent with this framework, the rubric extracts urine sodium, urine osmolality, and conditions associated with SIADH (e.g., pulmonary infections, CNS disorders, malignancy). The resulting features include indicators that reflect such conditions, e.g.,acute-cond-Pulmonary infection / pneumonia / pulmonary disease and acute-cond-any. Thus, the learned features directly operationalize the same diagnostic flow in clinical practice. 12 LLMs can construct powerful representations and streamline sample-efficient supervised learning A second group of features captures baseline risk factors and comorbidities that predispose patients to hypona- tremia. The rubric explicitly extracts conditions such as chronic kidney disease (CKD), dialysis history, malignancy, and medications known to induce hyponatremia (e.g., thiazide diuretics). These signals appear directly in the tab- ularized representation through features such asdialysis-history-Yes,procedure-Hemodialysis, and med-class-count-thiazide-diuretic. These variables correspond to well-known clinical risk factors for hy- ponatremia, including impaired renal free-water handling and medication-induced sodium loss (Verbalis et al., 2013). Finally, the most predictive signals arise from the acuity and trajectory of prior sodium measurements. The rubric explicitly extracts the three most recent sodium values and the lowest sodium in the prior 90 days, along with contextual metadata such as the setting of the measurement (e.g., inpatient vs outpatient). Correspondingly, the largest-magnitude coefficients in the tabular feature set correspond to prior sodium measurements, includingserum-na-recent-le-134 andprior-documented-hyponatremia-Yes. Clinically, this is expected, as patients with a history of chronic or recurrent hyponatremia (e.g., due to heart failure or cirrhosis) are substantially more likely to have abnormal sodium levels on subsequent laboratory testing (Upadhyay et al., 2006). This example illustrates how rubric representations surface task-relevant information that would otherwise be buried in a long, heterogeneous text-serialization of the patient record. In the naive text format, prior sodium measurements appear scattered across multiple visits and lab panels, interleaved with unrelated clinical events. The rubric reorganizes this information into a compact set of fields that explicitly capture the recent trajectory of the lab. In doing so, it converts diffuse signals in language space into structured features that a simple downstream model can use efficiently. 7. Discussion and Concluding Remarks We studied how a team of LLM agents can help construct powerful input representations for supervised learning in complex domains, where raw inputs are heterogeneous, weakly structured, and often difficult to model with off-the-shelf methods. Our central proposal was rubric representation learning: instead of treating text-serialization as a fixed preprocessing step, we use LLMs to transform naive serializations into more task-aligned representations before downstream training. Concretely, we introduced global rubrics, which define a shared and structured template for extracting task-relevant evidence, and local rubrics, which produce compact task-conditioned summaries for individual examples. We also showed that global rubrics can be operationalized beyond text through parser-based automation and tabularization, enabling deterministic application and compatibility with a broader class of downstream learners. Across 15 prediction tasks in EHRSHOT, rubric representations yielded a substantial improvement over the already strong NaiveText baseline and surpassed CLMBR-T on average, despite the latter being pretrained on 2.57M patients. The gains were especially pronounced for assignment of new diagnosis and anticipating-lab tasks, where rubric learning appears to help organize sparse, heterogeneous evidence into a form that downstream models can exploit more effectively. At the same time, the comparison within the rubric family was nuanced. Local-Rubric was strongest overall in AUROC, especially in the low-data regime, while Global-Rubric achieved the best overall AUPRC and remained very close in average performance. The parser-based and tabular global-rubric variants were also highly competitive, and were among the strongest methods on lab tasks. Taken together, these findings support the main claim of the paper: in complex domains, input representation design is a first-order driver of downstream statistical performance. Beyond predictive performance, global rubrics have several practical advantages. Because they impose a shared field structure across examples, they are easier to inspect, audit, and refine than local rubric summaries. Once a global rubric has been learned, it can be applied to naive text-serializations of the input,x text , at inference time using deterministic parser scripts. This dramatically reduces deployment cost relative to local rubrics, which require an LLM API call for each new example and therefore incur anO(N)cost in time and money. By contrast,Global-Rubric-Autoand Global-Rubric-Tabularonly incur a one-time rubric-construction cost using a fixed set of examples (40), making their monetary cost effectivelyO(1). Parser-based application also makes rubric transformation orders of magnitude faster and possible on CPUs. To summarize, global rubrics offer representation design algorithms that combine competitive statistical performance with several practical advantages making them more feasible to deploy at scale. Our results also suggest several substantive directions for future work. First, the rubric synthesis step is currently limited by context length: each global rubric is created from a diverse cohort of only 40 patients. While this already yields strong results, richer procedures for leveraging more data could improve robustness and coverage. One promising direction is iterative rubric refinement, where an initial rubric is created from a small cohort, evaluated, audited, and then updated 13 LLMs can construct powerful representations and streamline sample-efficient supervised learning using additional examples, failure cases, or expert feedback. More broadly, ourGlobal-Rubric-Blindablation shows that pretrained world knowledge alone can induce a useful rubric, but the consistent gap to data-informed rubrics suggests real value in learning the rubric from examples. Developing more principled and scalable methods for data-driven rubric construction is therefore an important direction. This work also has several limitations. Our empirical evaluation is restricted to a single benchmark in healthcare, and although EHRSHOT spans 15 tasks, it does not exhaust the range of settings where rubric representations may be useful. In particular, our experiments focus primarily on longitudinal EHR data serialized as text, and do not include richer free-text modalities such as clinical notes, nor other modalities such as imaging in a way that fully tests the generality of the approach. The chest X-ray task also remained challenging for all methods, and operational outcome tasks continued to favor CLMBR-T on average, suggesting that rubric learning may be less effective in certain settings, and characterization of such settings is crucial. Finally, some rubric variants still rely on LLM API calls during representation construction, which introduces computational cost and potential variability, even if parser-based variants mitigate this issue substantially. Overall, our findings suggest that LLMs can contribute to supervised learning as a representation layer that reorganizes complex inputs into forms that are more learnable, auditable, and operationally useful. We view rubric representation learning as one concrete instantiation of this broader idea. A wider empirical evaluation across domains, tasks, and modalities (including settings with free text, images, and non-medical data) would provide valuable insight into when rubric representations help most and how they should be designed. More generally, the results here point toward a promising direction in which LLMs help automate one of the most persistent bottlenecks in applied machine learning: the design of effective input representations from complex and heterogeneous raw data. Acknowledgements ID and DS were supported by Office of Naval Research Award No. N00014-21-1-2807. LS, ID, and DS were supported by a grant from Independence Blue Cross. The authors would also like to thank Shannon Shen from the ClinicalML group at MIT for insightful discussions. References Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., Opsahl-Ong, K., Singhvi, A., Shandilya, H., Ryan, M. J., Jiang, M., Potts, C., Sen, K., Dimakis, A., Stoica, I., Klein, D., Zaharia, M., and Khattab, O. GEPA: Reflective prompt evolution can outperform reinforcement learning. In The Fourteenth International Conference on Learning Representations, 2026. Akhauri, Y., Lewandowski, B., Lin, C.-H., Reyes, A. N., Forbes, G. C., Wongpanich, A., Yang, B., Abdelfattah, M. S., Perel, S., and Song, X. Performance prediction for large systems via text-to-text regression, 2025. Banegas, J. R., Ruilope, L. M., de la Sierra, A., Vinyoles, E., Gorostidi, M., de la Cruz, J. J., Ruiz-Hurtado, G., Segura, J., Rodr Μ Δ±guez-Artalejo, F., and Williams, B. Relationship between clinic and ambulatory blood-pressure measurements and mortality. New England Journal of Medicine, 378(16):1509β1520, 2018. Chen, T. and Guestrin, C. Xgboost: A scalable tree boosting system. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 785β794, 2016. Choi, K., Cundy, C., Srivastava, S., and Ermon, S. Lmpriors: Pre-trained language models as task-specific priors. arXiv preprint arXiv:2210.12530, 2022. Cuspidi, C., Tadic, M., and Grassi, G. Connecting cardiovascular risk scores with hypertensive mediated organ damage. The Journal of Clinical Hypertension, 27(11):e70174, 2025. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., Kim, R., Raman, R., Nelson, P. C., Mega, J. L., and Webster, D. R. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22):2402β2410, 2016. Guo, S., Deng, C., Wen, Y., Chen, H., Chang, Y., and Wang, J. DS-agent: Automated data science by empowering large language models with case-based reasoning. In International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, p. 16813β16848, 2024. 14 LLMs can construct powerful representations and streamline sample-efficient supervised learning Hegselmann, S., Buendia, A., Lang, H., Agrawal, M., Jiang, X., and Sontag, D. Tabllm: Few-shot classification of tabular data with large language models. In International conference on artificial intelligence and statistics, p. 5549β5581. PMLR, 2023. Hegselmann, S., von Arnim, G., Rheude, T., Kronenberg, N., Sontag, D., Hindricks, G., Eils, R., and Wild, B. Large language models are powerful electronic health record encoders. arXiv preprint arXiv:2502.17403, 2025. Hong, S., Lin, Y., Liu, B., Liu, B., Wu, B., Zhang, C., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhang, L., Yang, M., Zhuge, M., Guo, T., Zhou, T., Tao, W., Tang, R., Lu, X., Zheng, X., Liang, X., Fei, Y., Cheng, Y., Ni, Y., Gou, Z., Xu, Z., Luo, Y., and Wu, C. Data interpreter: An LLM agent for data science. In Association for Computational Linguistics (ACL), p. 19796β19821, 2025. Jeong, D. P., Lipton, Z. C., and Ravikumar, P. K. LLM-select: Feature selection with large language models. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. Lightgbm: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017. Kim, J., Squires, C., and Ravikumar, P. Knowledge-enriched machine learning for tabular data. In Proceedings of the International Conference on Neuro-symbolic Systems, p. 260β292, 2025. Lessmann, S., Baesens, B., Seow, H.-V., and Thomas, L. Benchmarking state-of-the-art classification algorithms for credit scoring. European Journal of Operational Research, 247(1):124β136, 2015. Nam, J., Yoon, J., Chen, J., Sinha, R., Shin, J., and Pfister, T. DS-STAR: Data science agent for solving diverse tasks across heterogeneous formats and open-ended queries. arXiv preprint arXiv:2509.21825, 2025. OpenAI. Gpt-5 system card. https://cdn.openai.com/gpt-5-system-card.pdf, August 2025. Petridis, P., Margaritis, G., Stoumpou, V., and Bertsimas, D. Holistic ai in medicine; improved performance and explainability. npj Digital Medicine, 2026. Qwen3Team. Qwen3 technical report, 2025. Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C., Shpanskaya, K., Lungren, M. P., and Ng, A. Y. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225, 2017. Ravuri, S., Lenc, K., Willson, M., Kangin, D., Lam, R., Mirowski, P., Fitzpatrick, S., Athanassiadou, M., Kashem, S., Madge, S., Prudden, R., Mandhane, A., Clark, A., Brock, A., Simonyan, K., Hadsell, R., Robinson, N., Clancy, E., Arribas, A., Mohamed, S., and Kalchbrenner, N. Skilful precipitation nowcasting using deep generative models of radar. Nature, 597:672β677, 2021. Renc, P., Jia, Y., Samir, A. E., Was, J., Li, Q., Bates, D. W., and Sitek, A. Zero shot health trajectory prediction using transformer. NPJ digital medicine, 7(1):256, 2024. Song, X., Li, O., Lee, C., Yang, B., Peng, D., Perel, S., and Chen, Y. Omnipred: Language models as universal regressors. Transactions on Machine Learning Research (TMLR), 2024. ISSN 2835-8856. Spasovski, G., Vanholder, R., Allolio, B., Annane, D., Ball, S., Bichet, D., Decaux, G., Fenske, W., Hoorn, E. J., Ichai, C., et al. Clinical practice guideline on diagnosis and treatment of hyponatraemia. Nephrology Dialysis Transplantation, 29 (suppl2):i1βi39, 2014. Steinberg, E., Fries, J. A., Xu, Y., and Shah, N. MOTOR: A time-to-event foundation model for structured medical records. In The Twelfth International Conference on Learning Representations, 2024. Upadhyay, A., Jaber, B. L., and Madias, N. E. Incidence and prevalence of hyponatremia. The American journal of medicine, 119(7):S30βS35, 2006. 15 LLMs can construct powerful representations and streamline sample-efficient supervised learning Van Calster, B., Collins, G. S., Vickers, A. J., Wynants, L., Kerr, K. F., Barre Μ nada, L., Varoquaux, G., Singh, K., Moons, K. G., Hernandez-Boussard, T., et al. Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance. The Lancet Digital Health, 2025. Verbalis, J. G., Goldsmith, S. R., Greenberg, A., Korzelius, C., Schrier, R. W., Sterns, R. H., and Thompson, C. J. Diagnosis, evaluation, and treatment of hyponatremia: expert panel recommendations. The American journal of medicine, 126(10): S1βS42, 2013. Waxler, S., Blazek, P., White, D., Sneider, D., Chung, K., Nagarathnam, M., Williams, P., Voeller, H., Wong, K., Swanhorst, M., et al. Generative medical event models improve with scale. arXiv preprint arXiv:2508.12104, 2025. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q., and Zhou, D. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, p. 24824β24837, 2022. Whelton, P. K., Carey, R. M., Aronow, W. S., Casey, D. E., Collins, K. J., Dennison Himmelfarb, C., DePalma, S. M., Gidding, S., Jamerson, K. A., Jones, D. W., et al. 2017 acc/aha/aapa/abc/acpm/ags/apha/ash/aspc/nma/pcna guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: a report of the american college of cardiology/american heart association task force on clinical practice guidelines. Journal of the American College of Cardiology, 71(19):e127βe248, 2018. Wornow, M., Thapa, R., Steinberg, E., Fries, J., and Shah, N. EHRSHOT: An ehr benchmark for few-shot evaluation of foundation models. Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks, 36: 67125β67137, 2023. Zhang, E., Goto, R., Sagan, N., Mutter, J., Phillips, N., Alizadeh, A., Lee, K., Blanchet, J., Pilanci, M., and Tibshirani, R. Llm- lasso: A robust framework for domain-informed feature selection and regularization. arXiv preprint arXiv:2502.10648, 2025a. Zhang, S., Fan, J., Fan, M., Li, G., and Du, X. Deepanalyze: Agentic large language models for autonomous data science. arXiv preprint arXiv:2510.16872, 2025b. Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., and Zhou, J. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025c. 16 LLMs can construct powerful representations and streamline sample-efficient supervised learning A. Per-task results Figure 8. AUROC per-task, n = All 17 LLMs can construct powerful representations and streamline sample-efficient supervised learning Figure 9. AUPRC per-task, n = All 18 LLMs can construct powerful representations and streamline sample-efficient supervised learning Figure 10. AUROC per-task, n = 40 19 LLMs can construct powerful representations and streamline sample-efficient supervised learning Figure 11. AUPRC per-task, n = 40 20 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 3. ICU transfer. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.707 .650β.764 GPT5-Mini-CoT.688 .631β.744 n = 40n = All Count-GBM.608 .545β.664 .730 .670β.781 CLMBR-T.739 .687β.794 .845 .797β.892 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.750 .699β.800 .722 .671β.774 .801 .751β.843 .757 .705β.803 Local-Rubric-Generic.694 .636β.752 .676 .613β.733 .779 .734β.820 .772 .720β.818 Local-Rubric.714 .659β.767 .719 .662β.772 .839 .805β.873 .819 .783β.852 Global-Rubric-Blind.701 .641β.760 .659 .600β.717 .797 .748β.841 .731 .681β.779 Global-Rubric.764 .713β.813 .672 .620β.725 .785 .738β.827 .770 .721β.820 Global-Rubric-Auto.729 .685β.773 .703 .651β.756 .811 .777β.847 .739 .694β.786 Global-Rubric-Tabular.677 .615β.739 β.783 .734β.830 β (b) AUPRC Qwen3-8B-CoT.090 .064β.120 GPT5-Mini-CoT.101 .068β.139 n = 40n = All Count-GBM.071 .047β.100 .121 .083β.168 CLMBR-T.178 .111β.262 .314 .226β.416 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.152 .100β.217 .171 .103β.250 .177 .119β.243 .166 .106β.242 Local-Rubric-Generic.127 .079β.187 .119 .071β.173 .149 .101β.209 .191 .123β.274 Local-Rubric.114 .077β.164 .119 .080β.168 .179 .124β.239 .176 .118β.246 Global-Rubric-Blind.134 .085β.198 .087 .058β.124 .173 .118β.240 .132 .085β.193 Global-Rubric.139 .093β.198 .094 .059β.142 .168 .108β.242 .167 .110β.238 Global-Rubric-Auto.103 .069β.145 .100 .067β.145 .177 .116β.252 .129 .080β.188 Global-Rubric-Tabular.102 .066β.146 β.186 .122β.267 β Table 4. Long length of stay. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.650 .629β.672 GPT5-Mini-CoT.742 .724β.763 n = 40n = All Count-GBM.642 .617β.668 .710 .685β.736 CLMBR-T.727 .704β.750 .818 .799β.837 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.685 .662β.709 .623 .597β.646 .743 .723β.765 .698 .675β.723 Local-Rubric-Generic.732 .709β.754 .674 .649β.697 .777 .756β.799 .744 .722β.766 Local-Rubric.747 .726β.769 .704 .681β.726 .783 .762β.803 .769 .749β.790 Global-Rubric-Blind.690 .665β.712 .595 .568β.622 .770 .748β.791 .728 .704β.752 Global-Rubric.713 .688β.737 .665 .642β.690 .787 .766β.809 .750 .728β.771 Global-Rubric-Auto.733 .709β.757 .671 .645β.696 .781 .761β.801 .754 .733β.774 Global-Rubric-Tabular.639 .612β.666 β.725 .703β.750 β (b) AUPRC Qwen3-8B-CoT.333 .308β.359 GPT5-Mini-CoT.416 .387β.449 n = 40n = All Count-GBM.347 .314β.381 .401 .362β.438 CLMBR-T.466 .424β.508 .589 .546β.632 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.377 .343β.414 .322 .291β.355 .455 .413β.502 .398 .362β.439 Local-Rubric-Generic.438 .397β.476 .377 .340β.414 .529 .484β.573 .464 .421β.506 Local-Rubric.453 .413β.493 .431 .390β.471 .505 .462β.550 .500 .456β.542 Global-Rubric-Blind.382 .344β.420 .319 .286β.352 .496 .450β.540 .440 .398β.483 Global-Rubric.411 .372β.452 .358 .323β.393 .526 .480β.570 .453 .411β.496 Global-Rubric-Auto.441 .398β.482 .390 .352β.430 .523 .479β.567 .479 .436β.520 Global-Rubric-Tabular.376 .336β.414 β.451 .410β.496 β 21 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 5. 30-day readmission. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.561 .534β.586 GPT5-Mini-CoT.609 .588β.627 n = 40n = All Count-GBM.623 .585β.662 .785 .754β.816 CLMBR-T.768 .735β.800 .791 .760β.819 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.760 .729β.791 .706 .669β.740 .780 .751β.808 .760 .727β.791 Local-Rubric-Generic.774 .745β.803 .759 .728β.789 .787 .757β.816 .786 .756β.816 Local-Rubric.772 .739β.799 .757 .724β.787 .783 .752β.812 .782 .749β.811 Global-Rubric-Blind.732 .698β.764 .726 .692β.758 .760 .729β.788 .744 .714β.773 Global-Rubric.779 .747β.808 .721 .686β.757 .786 .757β.814 .756 .724β.787 Global-Rubric-Auto.759 .727β.786 .688 .652β.724 .778 .748β.807 .772 .741β.801 Global-Rubric-Tabular.707 .671β.741 β.759 .725β.792 β (b) AUPRC Qwen3-8B-CoT.133 .116β.150 GPT5-Mini-CoT.147 .130β.166 n = 40n = All Count-GBM.185 .153β.220 .321 .272β.373 CLMBR-T.367 .306β.430 .373 .313β.434 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.282 .236β.334 .269 .222β.321 .312 .259β.368 .306 .253β.362 Local-Rubric-Generic.322 .271β.380 .293 .248β.350 .328 .280β.385 .359 .302β.420 Local-Rubric.330 .275β.385 .330 .273β.390 .339 .286β.393 .367 .305β.427 Global-Rubric-Blind.273 .229β.325 .300 .247β.355 .293 .246β.346 .301 .251β.352 Global-Rubric.344 .289β.404 .326 .269β.388 .324 .275β.375 .328 .275β.386 Global-Rubric-Auto.312 .258β.371 .273 .224β.329 .329 .272β.390 .353 .294β.413 Global-Rubric-Tabular.257 .213β.301 β.348 .286β.410 β Table 6. Acute MI. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.664 .629β.698 GPT5-Mini-CoT.739 .697β.782 n = 40n = All Count-GBM.604 .546β.658 .704 .654β.748 CLMBR-T.685 .643β.725 .737 .697β.775 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.734 .696β.769 .654 .606β.698 .746 .706β.785 .676 .628β.719 Local-Rubric-Generic.716 .679β.752 .646 .603β.686 .739 .701β.776 .733 .692β.770 Local-Rubric.691 .652β.726 .692 .649β.732 .756 .717β.791 .763 .725β.797 Global-Rubric-Blind.660 .616β.709 .694 .647β.739 .723 .677β.768 .706 .660β.750 Global-Rubric.706 .666β.743 .696 .654β.734 .757 .717β.793 .730 .684β.773 Global-Rubric-Auto.701 .660β.738 .734 .695β.768 .751 .713β.789 .742 .700β.783 Global-Rubric-Tabular.709 .668β.747 β.760 .723β.797 β (b) AUPRC Qwen3-8B-CoT.100 .082β.119 GPT5-Mini-CoT.172 .135β.214 n = 40n = All Count-GBM.119 .089β.152 .169 .125β.219 CLMBR-T.131 .098β.170 .191 .142β.247 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.158 .120β.206 .120 .093β.154 .179 .134β.229 .145 .108β.189 Local-Rubric-Generic.154 .117β.201 .109 .085β.137 .196 .145β.255 .177 .131β.232 Local-Rubric.136 .103β.177 .143 .106β.185 .177 .134β.222 .179 .137β.223 Global-Rubric-Blind.128 .099β.163 .174 .129β.232 .174 .130β.225 .163 .122β.209 Global-Rubric.137 .106β.175 .122 .095β.153 .194 .146β.253 .176 .132β.228 Global-Rubric-Auto.139 .106β.179 .154 .118β.197 .168 .130β.211 .206 .155β.266 Global-Rubric-Tabular.145 .111β.184 β.176 .136β.221 β 22 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 7. Celiac disease. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.497 .495β.498 GPT5-Mini-CoT.497 .496β.499 n = 40n = All Count-GBM.450 .342β.565 .671 .510β.828 CLMBR-T.567 .454β.666 .543 .415β.671 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.486 .371β.605 .415 .296β.537 .614 .482β.736 .602 .451β.751 Local-Rubric-Generic.673 .544β.812 .658 .522β.801 .690 .571β.817 .653 .510β.796 Local-Rubric.658 .514β.795 .668 .534β.791 .670 .532β.799 .684 .545β.807 Global-Rubric-Blind.556 .439β.666 .495 .390β.597 .702 .578β.810 .705 .577β.812 Global-Rubric.612 .483β.747 .570 .407β.721 .663 .505β.814 .678 .532β.804 Global-Rubric-Auto.542 .422β.653 .391 .275β.501 .644 .499β.774 .486 .365β.608 Global-Rubric-Tabular.523 .397β.645 β.569 .455β.689 β (b) AUPRC Qwen3-8B-CoT.009 .005β.014 GPT5-Mini-CoT.010 .005β.014 n = 40n = All Count-GBM.011 .005β.022 .077 .024β.160 CLMBR-T.012 .007β.019 .017 .007β.036 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.023 .006β.074 .013 .005β.030 .030 .009β.087 .021 .010β.038 Local-Rubric-Generic.065 .018β.147 .036 .013β.069 .065 .020β.138 .047 .014β.094 Local-Rubric.040 .013β.087 .060 .015β.156 .028 .012β.052 .040 .015β.091 Global-Rubric-Blind.013 .007β.022 .011 .006β.017 .043 .015β.092 .030 .013β.055 Global-Rubric.165 .026β.346 .023 .009β.045 .174 .040β.372 .042 .016β.081 Global-Rubric-Auto.012 .007β.020 .008 .005β.013 .032 .012β.060 .018 .006β.052 Global-Rubric-Tabular.013 .006β.022 β.103 .008β.244 β Table 8. Hyperlipidemia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.500 .452β.546 GPT5-Mini-CoT.640 .601β.678 n = 40n = All Count-GBM.627 .580β.675 .702 .662β.745 CLMBR-T.632 .591β.674 .689 .647β.733 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.616 .573β.658 .587 .544β.631 .722 .684β.762 .669 .626β.709 Local-Rubric-Generic.679 .638β.719 .634 .592β.676 .727 .686β.762 .688 .648β.723 Local-Rubric.673 .630β.712 .680 .641β.716 .740 .701β.776 .728 .689β.767 Global-Rubric-Blind.627 .583β.669 .649 .601β.694 .710 .668β.750 .690 .644β.739 Global-Rubric.674 .635β.711 .654 .611β.695 .734 .698β.769 .712 .672β.754 Global-Rubric-Auto.664 .621β.708 .680 .638β.722 .711 .669β.753 .713 .670β.754 Global-Rubric-Tabular.655 .613β.696 β .745 .704β.784 β (b) AUPRC Qwen3-8B-CoT.132 .109β.157 GPT5-Mini-CoT.176 .148β.205 n = 40n = All Count-GBM.211 .168β.257 .287 .225β.349 CLMBR-T.194 .159β.237 .251 .202β.307 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.195 .152β.239 .184 .146β.232 .263 .213β.320 .231 .183β.279 Local-Rubric-Generic.257 .201β.317 .207 .161β.253 .302 .241β.369 .246 .197β.302 Local-Rubric.229 .180β.283 .220 .178β.266 .316 .251β.382 .308 .245β.374 Global-Rubric-Blind.206 .161β.255 .262 .203β.322 .297 .236β.364 .290 .229β.358 Global-Rubric.232 .184β.283 .227 .178β.283 .291 .228β.355 .273 .217β.332 Global-Rubric-Auto.244 .187β.306 .244 .193β.304 .286 .225β.349 .297 .232β.367 Global-Rubric-Tabular.227 .179β.281 β .319 .252β.386 β 23 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 9. Hypertension. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.552 .502β.600 GPT5-Mini-CoT.646 .609β.684 n = 40n = All Count-GBM.660 .617β.705 .693 .653β.732 CLMBR-T.660 .618β.705 .721 .682β.759 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.619 .569β.670 .576 .526β.623 .620 .574β.669 .585 .535β.633 Local-Rubric-Generic.645 .603β.688 .612 .569β.658 .714 .674β.752 .668 .628β.708 Local-Rubric.703 .662β.740 .674 .638β.711 .747 .713β.782 .738 .704β.771 Global-Rubric-Blind.684 .639β.725 .655 .610β.697 .722 .679β.761 .699 .651β.741 Global-Rubric.662 .613β.708 .645 .600β.687 .702 .658β.742 .663 .615β.708 Global-Rubric-Auto.696 .652β.740 .663 .619β.705 .721 .680β.759 .692 .649β.732 Global-Rubric-Tabular.642 .589β.690 β.720 .682β.761 β (b) AUPRC Qwen3-8B-CoT.142 .116β.168 GPT5-Mini-CoT.172 .145β.205 n = 40n = All Count-GBM.277 .214β.339 .258 .203β.317 CLMBR-T.198 .164β.235 .263 .213β.324 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.231 .180β.291 .183 .145β.231 .234 .183β.291 .208 .161β.259 Local-Rubric-Generic.226 .180β.283 .224 .177β.280 .274 .218β.338 .254 .200β.317 Local-Rubric.225 .181β.275 .199 .160β.244 .265 .215β.320 .261 .207β.322 Global-Rubric-Blind.237 .193β.287 .206 .166β.248 .285 .226β.347 .256 .204β.312 Global-Rubric.219 .177β.269 .206 .162β.258 .263 .204β.323 .243 .188β.300 Global-Rubric-Auto.248 .199β.302 .189 .157β.225 .272 .219β.327 .247 .196β.305 Global-Rubric-Tabular.280 .217β.348 β .297 .236β.361 β Table 10. Lupus. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.500 .499β.500 GPT5-Mini-CoT.550 .499β.625 n = 40n = All Count-GBM.704 .586β.810 .823 .749β.893 CLMBR-T.697 .596β.802 .681 .588β.768 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.634 .514β.752 .634 .510β.756 .695 .556β.822 .667 .545β.778 Local-Rubric-Generic.684 .560β.802 .484 .361β.598 .713 .610β.818 .648 .515β.770 Local-Rubric.782 .669β.881 .755 .658β.846 .801 .676β.906 .781 .652β.886 Global-Rubric-Blind.678 .573β.774 .640 .541β.741 .750 .650β.831 .738 .635β.828 Global-Rubric.753 .631β.864 .710 .619β.797 .807 .708β.892 .800 .717β.880 Global-Rubric-Auto.789 .693β.872 .732 .626β.831 .818 .739β.886 .742 .631β.845 Global-Rubric-Tabular.751 .618β.864 β.812 .728β.886 β (b) AUPRC Qwen3-8B-CoT.009 .005β.013 GPT5-Mini-CoT.084 .007β.235 n = 40n = All Count-GBM.040 .011β.105 .060 .021β.133 CLMBR-T.035 .011β.092 .020 .009β.037 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.042 .009β.121 .025 .009β.051 .047 .013β.112 .055 .010β.156 Local-Rubric-Generic.021 .010β.036 .010 .005β.017 .026 .011β.050 .019 .008β.036 Local-Rubric.041 .019β.073 .030 .014β.053 .060 .024β.113 .044 .020β.078 Global-Rubric-Blind.019 .009β.033 .015 .008β.025 .076 .013β.198 .027 .013β.050 Global-Rubric .089 .017β.211 .068 .010β.189 .058 .020β.132 .041 .018β.076 Global-Rubric-Auto.034 .015β.063 .026 .012β.046 .075 .020β.186 .031 .014β.055 Global-Rubric-Tabular.058 .018β.135 β.038 .017β.066 β 24 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 11. Pancreatic cancer. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.630 .574β.694 GPT5-Mini-CoT.604 .554β.662 n = 40n = All Count-GBM.733 .649β.810 .933 .895β.964 CLMBR-T.734 .651β.815 .812 .741β.876 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.798 .738β.854 .767 .709β.815 .860 .803β.908 .827 .770β.877 Local-Rubric-Generic.801 .726β.868 .718 .639β.792 .853 .790β.908 .859 .800β.911 Local-Rubric.842 .772β.907 .824 .755β.887 .909 .865β.947 .909 .865β.946 Global-Rubric-Blind.832 .780β.884 .790 .723β.852 .864 .805β.916 .860 .810β.901 Global-Rubric.840 .775β.898 .760 .684β.836 .874 .813β.928 .834 .764β.894 Global-Rubric-Auto.832 .776β.880 .774 .709β.832 .866 .806β.915 .814 .751β.870 Global-Rubric-Tabular.811 .751β.865 β.843 .778β.900 β (b) AUPRC Qwen3-8B-CoT.086 .043β.155 GPT5-Mini-CoT.205 .099β.314 n = 40n = All Count-GBM.127 .070β.195 .469 .322β.605 CLMBR-T.191 .103β.290 .276 .165β.407 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.239 .134β.359 .101 .053β.175 .321 .202β.449 .166 .094β.258 Local-Rubric-Generic.399 .257β.529 .170 .082β.273 .415 .273β.547 .381 .248β.514 Local-Rubric.400 .272β.533 .336 .214β.465 .491 .362β.622 .419 .291β.552 Global-Rubric-Blind.192 .106β.305 .170 .090β.270 .232 .134β.346 .186 .109β.283 Global-Rubric.310 .194β.439 .275 .157β.402 .438 .301β.575 .325 .202β.448 Global-Rubric-Auto.264 .148β.383 .170 .085β.271 .335 .205β.464 .240 .135β.353 Global-Rubric-Tabular.184 .102β.291 β.300 .189β.429 β Table 12. Anemia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.500 .478β.523 GPT5-Mini-CoT.530 .510β.549 n = 40n = All Count-GBM.535 .511β.562 .562 .537β.587 CLMBR-T.699 .675β.721 .821 .803β.837 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.578 .553β.602 .552 .529β.579 .649 .625β.673 .615 .592β.640 Local-Rubric-Generic.683 .660β.706 .580 .556β.604 .732 .712β.754 .653 .630β.676 Local-Rubric.631 .607β.654 .678 .657β.700 .752 .731β.773 .737 .717β.759 Global-Rubric-Blind.723 .700β.746 .742 .721β.764 .776 .756β.797 .784 .765β.804 Global-Rubric.713 .691β.737 .734 .711β.757 .764 .744β.785 .771 .750β.791 Global-Rubric-Auto.692 .669β.714 .717 .692β.739 .769 .750β.790 .763 .741β.784 Global-Rubric-Tabular.746 .727β.767 β.790 .771β.810 β (b) AUPRC Qwen3-8B-CoT.495 .468β.520 GPT5-Mini-CoT.514 .490β.538 n = 40n = All Count-GBM.517 .485β.549 .533 .502β.563 CLMBR-T.669 .636β.702 .788 .759β.816 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.554 .523β.586 .550 .517β.580 .634 .602β.667 .595 .564β.626 Local-Rubric-Generic.674 .641β.706 .550 .518β.581 .705 .673β.737 .623 .591β.653 Local-Rubric.631 .597β.662 .671 .640β.703 .710 .677β.740 .694 .662β.727 Global-Rubric-Blind.698 .665β.728 .721 .691β.751 .724 .691β.757 .732 .700β.764 Global-Rubric.694 .661β.725 .720 .689β.751 .728 .696β.758 .723 .690β.755 Global-Rubric-Auto.671 .643β.701 .718 .689β.745 .734 .703β.764 .730 .699β.758 Global-Rubric-Tabular.721 .693β.751 β.754 .725β.784 β 25 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 13. Hyperkalemia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.758 .738β.779 GPT5-Mini-CoT.786 .766β.805 n = 40n = All Count-GBM.595 .570β.620 .665 .640β.687 CLMBR-T.594 .568β.620 .752 .730β.774 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.671 .648β.696 .606 .581β.631 .748 .726β.769 .691 .669β.715 Local-Rubric-Generic.770 .750β.791 .696 .673β.721 .817 .797β.835 .772 .751β.794 Local-Rubric.808 .788β.828 .816 .797β.835 .832 .814β.850 .832 .814β.850 Global-Rubric-Blind.760 .739β.782 .668 .644β.694 .806 .787β.826 .786 .767β.806 Global-Rubric.755 .732β.777 .716 .694β.739 .825 .805β.844 .799 .776β.820 Global-Rubric-Auto.784 .762β.805 .771 .750β.793 .824 .806β.842 .823 .804β.842 Global-Rubric-Tabular.794 .774β.814 β .833 .815β.850 β (b) AUPRC Qwen3-8B-CoT.735 .705β.763 GPT5-Mini-CoT.744 .717β.771 n = 40n = All Count-GBM.582 .550β.616 .644 .612β.677 CLMBR-T.585 .550β.617 .763 .735β.791 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.672 .638β.706 .579 .545β.612 .750 .719β.779 .687 .653β.720 Local-Rubric-Generic.757 .727β.788 .674 .638β.706 .808 .779β.837 .755 .721β.786 Local-Rubric.807 .777β.834 .790 .759β.820 .821 .791β.848 .819 .789β.847 Global-Rubric-Blind.742 .709β.775 .673 .639β.708 .798 .767β.826 .782 .751β.810 Global-Rubric.733 .702β.765 .703 .670β.736 .818 .791β.846 .786 .756β.816 Global-Rubric-Auto.770 .739β.799 .750 .717β.780 .812 .784β.838 .809 .780β.836 Global-Rubric-Tabular.769 .738β.798 β.815 .786β.843 β Table 14. Hypoglycemia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.687 .665β.710 GPT5-Mini-CoT.662 .636β.687 n = 40n = All Count-GBM.566 .538β.592 .622 .594β.651 CLMBR-T.618 .591β.644 .777 .755β.799 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.568 .537β.598 .540 .512β.569 .680 .654β.706 .633 .605β.659 Local-Rubric-Generic.623 .595β.649 .567 .539β.595 .702 .674β.727 .660 .632β.685 Local-Rubric.709 .684β.734 .700 .674β.726 .780 .757β.803 .769 .747β.792 Global-Rubric-Blind.738 .711β.764 .659 .631β.685 .769 .746β.795 .731 .706β.756 Global-Rubric.693 .667β.718 .582 .553β.608 .794 .774β.816 .688 .660β.713 Global-Rubric-Auto.618 .590β.646 .606 .578β.635 .727 .703β.752 .662 .636β.688 Global-Rubric-Tabular.677 .649β.702 β.752 .727β.775 β (b) AUPRC Qwen3-8B-CoT.662 .629β.693 GPT5-Mini-CoT.650 .617β.683 n = 40n = All Count-GBM.547 .512β.582 .640 .602β.680 CLMBR-T.593 .556β.629 .764 .730β.797 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.573 .535β.611 .535 .499β.572 .675 .638β.709 .632 .595β.668 Local-Rubric-Generic.631 .593β.667 .559 .523β.595 .705 .668β.738 .655 .617β.691 Local-Rubric.703 .667β.739 .690 .653β.728 .767 .732β.798 .755 .719β.790 Global-Rubric-Blind.702 .663β.739 .640 .600β.676 .750 .713β.784 .709 .670β.743 Global-Rubric.682 .647β.716 .585 .548β.620 .783 .750β.814 .684 .647β.718 Global-Rubric-Auto.592 .555β.630 .581 .544β.621 .733 .697β.763 .655 .617β.692 Global-Rubric-Tabular.663 .627β.697 β.739 .702β.773 β 26 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 15. Hyponatremia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.686 .666β.707 GPT5-Mini-CoT.706 .685β.727 n = 40n = All Count-GBM.498 .473β.523 .539 .514β.563 CLMBR-T.577 .551β.602 .658 .633β.681 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.562 .536β.587 .549 .525β.575 .595 .571β.619 .551 .526β.575 Local-Rubric-Generic.591 .566β.615 .546 .520β.571 .644 .621β.667 .583 .559β.608 Local-Rubric.720 .697β.742 .713 .690β.735 .740 .718β.763 .741 .718β.763 Global-Rubric-Blind.634 .611β.657 .631 .608β.653 .706 .684β.729 .725 .704β.746 Global-Rubric.620 .596β.643 .557 .531β.581 .719 .697β.740 .670 .645β.694 Global-Rubric-Auto.658 .633β.682 .615 .591β.640 .733 .713β.756 .732 .709β.753 Global-Rubric-Tabular.706 .684β.728 β .757 .737β.778 β (b) AUPRC Qwen3-8B-CoT.655 .626β.683 GPT5-Mini-CoT.657 .629β.687 n = 40n = All Count-GBM.519 .488β.552 .526 .495β.558 CLMBR-T.560 .528β.594 .641 .608β.674 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.563 .530β.594 .550 .518β.583 .595 .563β.626 .554 .522β.587 Local-Rubric-Generic.572 .539β.606 .541 .509β.576 .634 .602β.666 .571 .539β.604 Local-Rubric.709 .678β.739 .683 .650β.717 .703 .668β.737 .688 .652β.723 Global-Rubric-Blind.643 .611β.675 .600 .568β.632 .690 .659β.722 .703 .673β.736 Global-Rubric.625 .592β.658 .554 .520β.588 .703 .672β.735 .639 .606β.673 Global-Rubric-Auto.647 .614β.681 .607 .573β.641 .720 .691β.751 .719 .690β.750 Global-Rubric-Tabular.651 .618β.683 β.713 .685β.742 β Table 16. Thrombocytopenia. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.705 .683β.725 GPT5-Mini-CoT.739 .719β.759 n = 40n = All Count-GBM.486 .461β.511 .586 .562β.610 CLMBR-T.552 .525β.578 .627 .604β.654 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.550 .525β.577 .542 .517β.567 .612 .586β.635 .580 .555β.604 Local-Rubric-Generic.701 .679β.723 .529 .504β.554 .787 .767β.805 .679 .655β.701 Local-Rubric.774 .754β.794 .747 .727β.768 .841 .824β.859 .815 .797β.833 Global-Rubric-Blind.715 .693β.735 .694 .670β.716 .828 .810β.847 .826 .808β.844 Global-Rubric.693 .670β.716 .632 .607β.656 .851 .834β.868 .857 .841β.873 Global-Rubric-Auto.708 .684β.731 .640 .616β.663 .828 .809β.846 .828 .810β.845 Global-Rubric-Tabular.819 .801β.837 β .863 .846β.879 β (b) AUPRC Qwen3-8B-CoT.637 .608β.666 GPT5-Mini-CoT.670 .645β.696 n = 40n = All Count-GBM.488 .460β.517 .573 .540β.605 CLMBR-T.558 .527β.588 .606 .575β.640 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.536 .502β.569 .550 .517β.583 .588 .556β.622 .580 .548β.612 Local-Rubric-Generic.678 .645β.711 .518 .489β.552 .768 .740β.797 .654 .621β.685 Local-Rubric.745 .715β.774 .708 .677β.740 .810 .781β.836 .782 .753β.810 Global-Rubric-Blind.685 .652β.716 .701 .671β.729 .804 .776β.830 .809 .781β.835 Global-Rubric.670 .638β.702 .664 .632β.694 .834 .807β.859 .830 .802β.855 Global-Rubric-Auto.666 .632β.701 .639 .605β.671 .800 .770β.829 .792 .762β.822 Global-Rubric-Tabular.791 .761β.820 β .839 .814β.864 β 27 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 17. Chest X-ray. AUROC and AUPRC with 95% bootstrap CI. Best result highlighted. (a) AUROC Qwen3-8B-CoT.546 .519β.573 GPT5-Mini-CoT.520 .498β.541 n = 40n = All Count-GBM.581 .550β.612 .609 .576β.642 CLMBR-T.606 .574β.638 .630 .599β.660 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.559 .535β.583 .528 .496β.560 .616 .592β.640 .538 .505β.569 Local-Rubric-Generic.561 .537β.584 .551 .519β.583 .608 .583β.631 .557 .526β.589 Local-Rubric.533 .507β.559 .509 .478β.541 .606 .583β.630 .582 .550β.612 Global-Rubric-Blind.505 .480β.530 β.585 .559β.608 β Global-Rubric.526 .502β.551 β.594 .567β.619 β Global-Rubric-Auto.510 .479β.540 .489 .458β.522 .575 .543β.605 .546 .515β.576 Global-Rubric-Tabular.471 .441β.506 β.551 .518β.583 β (b) AUPRC Qwen3-8B-CoT.526 .491β.560 GPT5-Mini-CoT.507 .476β.538 n = 40n = All Count-GBM.541 .502β.581 .582 .540β.624 CLMBR-T.600 .558β.642 .623 .584β.666 Qwen3-8BQwen3-0.6BQwen3-8BQwen3-0.6B NaiveText.549 .517β.582 .521 .483β.561 .609 .577β.641 .528 .489β.567 Local-Rubric-Generic.562 .528β.595 .522 .483β.557 .601 .567β.632 .555 .514β.597 Local-Rubric.526 .494β.556 .511 .472β.548 .605 .571β.638 .571 .530β.610 Global-Rubric-Blind.502 .472β.533 β.587 .556β.620 β Global-Rubric.544 .514β.578 β.582 .550β.614 β Global-Rubric-Auto.507 .467β.547 .499 .463β.541 .557 .518β.596 .535 .492β.577 Global-Rubric-Tabular.464 .429β.500 β.539 .499β.579 β 28 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 18. Full-dataset EHRSHOT results for Global-Rubric-Tabular. TaskAUROCAUPRC Operational Outcomes ICU transfer.800 .752β.846 .195 .130β.277 Long length of stay.737 .714β.758 .464 .421β.509 30-day readmission.757 .725β.787 .357 .300β.419 Task-group Avg..765 .745β.786 .340 .308β.374 Assignment of New Diagnoses Acute MI.765 .726β.803 .183 .140β.225 Lupus.819 .728β.897 .048 .020β.078 Hyperlipidemia.736 .697β.772 .312 .248β.383 Hypertension.718 .677β.757 .299 .239β.368 Celiac disease.654 .540β.772 .098 .016β.190 Pancreatic cancer.859 .798β.915 .378 .239β.518 Task-group Avg..759 .729β.790 .216 .185β.248 Anticipating Lab Results Anemia.810 .806β.815 .361 .351β.372 Hyponatremia.815 .811β.818 .560 .552β.568 Thrombocytopenia.885 .881β.888 .571 .558β.582 Hyperkalemia.853 .840β.865 .097 .086β.109 Hypoglycemia.751 .733β.768 .031 .026β.037 Task-group Avg..823 .818β.827 .324 .319β.329 Chest X-ray findings Chest X-ray.584 .572β.597 .743 .731β.756 Task-group Avg..584 .571β.597 .743 .731β.756 Overall Avg. (15 tasks) .770 .756β.783 .312 .297β.327 B. Full EHRSHOT Evaluation Results for Global-Rubric-Tabular Once a global rubric and its associated tabularization script have been learned, applying the transformation to additional examples is inexpensive and deterministic. This makes it practical to evaluate Global-Rubric-Tabular on the full EHRSHOT dataset without subsampling. Table 18 reports results for all 15 benchmark tasks. The model achieves a mean AUROC of 0.770 and mean AUPRC of 0.312 across tasks. 29 LLMs can construct powerful representations and streamline sample-efficient supervised learning C. Results using a smaller text-embedding model Table 19. Qwen3-8B vs Qwen3-0.6B Embedding (AUROC). LLM-based embedding models, 95% bootstrap CI. For each method, first row: 8B; second row: 0.6B. Best per column for each sample size regime (n = 40, n = All) is highlighted. AUROC Overall (15) Operational Outcomes (3) Assignment of New Diag. (6) Anticipating Labs (5) Chest X-ray Findings (1) n = 40 NaiveText .638 .623β.652 .732 .710β.753 .648 .616β.680 .586 .574β.598 .559 .535β.583 .600 .586β.615 .684 .662β.707 .605 .574β.639 .558 .546β.569 .528 .496β.560 Local-Rubric-Generic .688 .673β.703 .734 .711β.758 .700 .665β.733 .673 .663β.684 .561 .537β.584 .622 .606β.639 .703 .679β.727 .625 .589β.662 .584 .572β.595 .551 .519β.583 Local-Rubric .717 .703β.730 .744 .721β.766 .725 .692β.757 .729 .718β.739 .533 .507β.559 .709 .696β.722 .727 .702β.750 .715 .685β.744 .731 .722β.741 .509 .478β.541 Global-Rubric-Blind .682 .669β.696 .708 .683β.732 .673 .644β.701 .714 .704β.725 .505 .480β.530 .664 .649β.679 .660 .635β.684 .654 .623β.686 .679 .667β.690 β Global-Rubric .700 .685β.714 .752 .731β.772 .708 .676β.742 .695 .684β.705 .526 .502β.551 .665 .648β.680 .686 .664β.708 .672 .635β.705 .644 .634β.655 β Global-Rubric-Auto .694 .681β.708 .740 .720β.760 .704 .675β.732 .692 .680β.703 .510 .479β.540 .658 .645β.672 .688 .666β.710 .662 .633β.691 .670 .659β.681 .489 .458β.522 Global-Rubric-Tabular .686 .672β.701 .675 .649β.697 .683 .651β.714 .749 .738β.758 .471 .441β.506 β n = All NaiveText .699 .684β.714 .775 .754β.793 .709 .674β.744 .657 .646β.668 .616 .592β.640 .657 .641β.672 .738 .717β.758 .671 .635β.705 .614 .602β.625 .538 .505β.569 Local-Rubric-Generic .738 .725β.751 .781 .761β.800 .739 .711β.770 .736 .726β.746 .608 .583β.631 .697 .682β.712 .768 .745β.788 .708 .674β.741 .669 .658β.679 .557 .526β.589 Local-Rubric .772 .758β.784 .802 .786β.818 .770 .738β.799 .789 .780β.798 .606 .583β.630 .763 .749β.776 .790 .774β.807 .767 .734β.797 .779 .770β.788 .582 .550β.612 Global-Rubric-Blind .751 .738β.764 .776 .755β.794 .745 .716β.775 .777 .768β.787 .585 .559β.608 .747 .732β.761 .735 .713β.755 .733 .702β.762 .770 .761β.780 β Global-Rubric .763 .748β.777 .786 .768β.805 .756 .723β.789 .791 .781β.800 .594 .567β.619 .748 .734β.762 .759 .737β.779 .736 .705β.766 .757 .748β.766 β Global-Rubric-Auto .756 .743β.769 .790 .772β.807 .752 .722β.782 .776 .766β.786 .575 .543β.605 .720 .706β.735 .755 .736β.776 .698 .665β.728 .761 .751β.772 .546 .515β.576 Global-Rubric-Tabular .751 .739β.764 .756 .734β.778 .742 .713β.769 .799 .790β.808 .552 .519β.582 β 30 LLMs can construct powerful representations and streamline sample-efficient supervised learning Table 20. Qwen3-8B vs Qwen3-0.6B Embedding (AUPRC). LLM-based embedding models, 95% bootstrap CI. For each method, first row: 8B; second row: 0.6B. Best per column for each sample size regime (n = 40, n = All) is highlighted. AUPRC Overall (15) Operational Outcomes (3) Assignment of New Diag. (6) Anticipating Labs (5) Chest X-ray Findings (1) n = 40 NaiveText .343 .331β.356 .271 .245β.301 .148 .123β.177 .580 .565β.595 .549 .517β.582 .312 .302β.323 .254 .223β.286 .104 .089β.123 .553 .538β.568 .521 .483β.561 Local-Rubric-Generic .392 .378β.407 .296 .268β.326 .187 .158β.217 .662 .647β.676 .562 .528β.595 .327 .316β.339 .263 .238β.293 .126 .107β.148 .568 .553β.582 .522 .483β.557 Local-Rubric .406 .393β.420 .299 .272β.328 .178 .152β.206 .719 .705β.732 .526 .494β.556 .395 .382β.408 .293 .266β.323 .165 .138β.194 .709 .694β.723 .511 .472β.548 Global-Rubric-Blind .370 .360β.382 .263 .237β.293 .133 .113β.153 .694 .679β.708 .502 .472β.533 .349 .337β.360 .235 .213β.260 .140 .118β.162 .667 .652β.682 β Global-Rubric .400 .382β.418 .298 .270β.327 .192 .154β.233 .681 .666β.696 .544 .514β.578 .352 .337β.367 .259 .233β.286 .154 .125β.186 .645 .631β.659 β Global-Rubric-Auto .377 .365β.389 .285 .259β.314 .157 .133β.181 .669 .654β.685 .507 .467β.547 .357 .346β.368 .254 .230β.281 .132 .112β.153 .659 .644β.675 .499 .463β.541 Global-Rubric-Tabular .380 .369β.392 .245 .221β.269 .151 .128β.178 .720 .706β.733 .462 .426β.497 β n = All NaiveText .391 .377β.406 .315 .283β.346 .179 .151β.208 .649 .634β.664 .609 .577β.641 .352 .340β.365 .290 .260β.325 .138 .117β.163 .610 .595β.624 .528 .489β.567 Local-Rubric-Generic .434 .419β.448 .335 .307β.364 .213 .184β.243 .724 .710β.738 .601 .567β.632 .397 .383β.412 .338 .305β.375 .188 .160β.216 .652 .637β.667 .555 .514β.597 Local-Rubric .452 .439β.466 .341 .310β.374 .223 .194β.251 .762 .748β.776 .605 .571β.638 .440 .427β.454 .347 .315β.383 .209 .181β.239 .748 .734β.761 .571 .530β.610 Global-Rubric-Blind .428 .415β.443 .321 .292β.351 .185 .155β.217 .753 .739β.767 .587 .556β.620 .397 .385β.410 .291 .264β.320 .159 .137β.182 .747 .732β.762 β Global-Rubric .459 .442β.478 .339 .309β.371 .236 .200β.276 .773 .760β.786 .582 .550β.614 .408 .394β.423 .316 .286β.349 .183 .156β.209 .732 .718β.746 β Global-Rubric-Auto .437 .422β.452 .343 .312β.378 .195 .164β.226 .760 .745β.773 .557 .518β.596 .416 .402β.430 .320 .290β.354 .173 .147β.200 .741 .727β.755 .535 .492β.577 Global-Rubric-Tabular .441 .426β.456 .328 .295β.368 .204 .173β.237 .772 .758β.785 .538 .501β.576 β 31 LLMs can construct powerful representations and streamline sample-efficient supervised learning D. Prompts Used in Global Rubric Representation Learning Methods D.1. Prompt to LLM for Global Rubric Creation (Figure 3, Panel (B)) # Prompt used with GPT-5-mini for global rubric synthesis You are a medical expert designing a structured rubric for a clinical prediction task. ## Task - Name: task name - Query: taskquery ## Context You will be given 40 labeled patient EHR examples (20 positive, 20 negative). Another model will later use your rubric to transform new patient EHRs into structured summaries, which will then serve as input to a supervised classifier. ## What You Must Do Study the examples below. Combine what you observe in them with your medical knowledge to design a rubric template -- a set of named fields that, when filled in for any patient, produce a structured summary optimized for this prediction task. The rubric should: 1. ** Be data-driven and discriminative. ** Identify which features, patterns, and interactions actually separate the positive and negative cases. The rubric should capture not just obvious indicators but also subtler or compound features you notice. At the same time, do not overfit to these 40 cases -- use your clinical knowledge to include factors that are generally relevant even if not prominent in this sample. 2. ** Be structured and consistent. ** Every rubricified output must follow the same field names and order. For each field, specify what to extract from the EHR and how to format it. Specify what to write when data is absent. 3. ** Extract facts only. ** The evaluator filling in the rubric must extract and organize information from the EHR. It must NOT make predictions, assign risk levels, or draw conclusions. 4. ** Be concise. ** The rubric should focus on extracting information that is relevant to the task. It should not ask the evaluator to reproduce the entire EHR. ## Positive Examples (Ground Truth: Yes) NaiveText EHR serializations of 20 positive examples concatenated (x text format) ## Negative Examples (Ground Truth: No) NaiveText EHR serializations of 20 negative examples concatenated (x text format) ## Output Output ONLY the rubric template itself -- the instructions another model will follow to transform a patient EHR. No preamble, no explanation of your reasoning. The template must be self-contained and directly usable. Figure 12. Prompt used with GPT-5-mini to guide global rubric creation from NaiveText serializations (x text ) of EHR examples. 32 LLMs can construct powerful representations and streamline sample-efficient supervised learning D.2. Prompt to LLM for Global Rubric Application (Figure 3, Panel (D)) # Prompt used with GPT-5-mini for global rubric application You are a medical data extraction specialist. ## Task taskquery ## Rubric Template (follow this exactly) rubric instructions ## Patient EHR ehr text (x text format) ## Instructions Fill in every field of the rubric template above using ONLY information from this patientβs EHR. Rules: - Follow the exact field order and section structure of the rubric. - Be concise: use short phrases, numbers, and dates. Do not write paragraphs. - If data for a field is not present in the EHR, write βNo dataβ. - Do NOT add commentary, predictions, risk assessments, or conclusions. - Do NOT include any information not found in the EHR above. Rubric output: Figure 13. Prompt used with GPT-5-mini for transforming a naive text serialized input (x text ) into its rubric text serialization version (x rubric ). 33 LLMs can construct powerful representations and streamline sample-efficient supervised learning D.3. Prompt to LLM for Creating a Global Rubric Application Parser (Figure 3, Panel (E)) # Prompt used with GPT-5.2 for generating a parser script for rubric application You are an expert Python developer and medical informaticist. ## Your Task Write a complete, self-contained Python script that reads patient EHR serializations and fills in a structured clinical rubric template using ** deterministic string/regex parsing only ** --- no LLM API calls, no network requests. ## Clinical Task Context - Task name: task name - Prediction query: task query ## Rubric Template to Fill The script must fill in every field defined in the following rubric instructions: rubricinstructions, R ## EHR Serialization Format Below are 40 example patient EHR serializations from the training cohort, labeled by ground-truth outcome. For each patient you are shown BOTH: 1. The raw naive text EHR serialization. 2. The LLM-produced rubric fill for that exact patient --- showing you how the fields should be extracted from the raw text. Use these paired examples to understand the extraction mapping precisely. 40 paired examples of naive text serializations (x text ) and LLM-filled rubric text serializations (x rubric ). ## Required Script Interface The generated script must: 1. Accept the following command-line arguments via argparse: - β--input dirβ : root directory of naivetext serializations - β--outputdirβ : root directory for llmrubric-parser outputs - β--taskβ : task name - β--splitsβ : one or more of βtrain val testβ 2. For each split, read βinputdir/task/split.jsonβ --- a JSON array where each element has: - βpatientidβ (int) - βpredictiontimeβ (ISO datetime string) - βtaskβ (str) - βsplitβ (str) - βlabelβ (bool) - βserializationβ (str)β the EHR text to parse 3. For each patient call βfill rubric(serialization: str) -> strβ, which: - Extracts all rubric fields from the EHR text using regex and string operations - Returns a filled-in rubric string that follows the exact field names, order, and format from the rubric template above - Writes "NA" for any field whose data is absent from the EHR 4. Write output to βoutput dir/task/split.jsonβ --- a JSON array where each element has: - βpatientidβ (int) - βpredictiontimeβ (str) - βtaskβ (str) - βsplitβ (str) - βlabelβ (bool) - βrubricified textβ (str)β output of fillrubric() 5. Create output directories as needed (parents=True, existok=True). 6. Print progress to stdout: total patients processed per split. ## Constraints - Use only Python standard library plus βreβ, βjsonβ, βargparseβ, βpathlibβ, βsysβ. No third-party packages. - No LLM API calls, network requests, external tools. - The βfillrubricβ function must be deterministic and handle missing data gracefully (write "NA" rather than raising exceptions). - The script must be syntactically valid Python 3.8+. - Do NOT hardcode file paths --- use the argparse arguments. ## Output Output ONLY the Python script, with no explanation, no preamble, and no markdown fences. The output must start with β#!/usr/bin/env python3β and be directly writable to a .py file. Figure 14. Prompt used with GPT-5.2 to create a parser script for transforming a naive text serialized input (x text ) into its rubric text serialization version (x rubric ). 34 LLMs can construct powerful representations and streamline sample-efficient supervised learning D.4. Prompt to LLM for Creating a Global Rubric Tabularization Parser (Figure 3, Panel (F)) # Prompt used with GPT-5.2 for generating a parser script to transform rubric serializations to tabular features You are an expert Python developer and medical informaticist. ## Your Task Write a complete, self-contained Python featurizer script that reads rubric-formatted patient EHR texts and converts each one into a ** fixed-dimension numeric feature vector ** using deterministic string/regex parsing | no LLM calls, no network requests. ## Clinical Task Context - Task name: task name - Prediction query: taskquery ## Rubric Parser Source (shows all rubric field names and their text formats) The following is the parser that generates the rubric text. Study it to understand which fields exist and how their values are formatted in the text. This is the ** ground truth ** for what fields can appear in a rubric text and how their values are formatted. βpython task-specific rubric parser generated via prompt in Section D.3 β ## Reference Rubric Texts (20 positive, 20 negative) ** Important context: ** These 40 patients are the cohort that was used to * design * the rubric itself. They are provided as examples so you can calibrate your regex patterns against actual data. ** However ** , the featurizer you write will be applied to a ** much larger dataset ** (thousands of patients). Your feature extraction logic must therefore be: - ** General ** : handle any value the rubric parser could plausibly produce, not just the values seen in these 40 patients - ** Robust ** : gracefully handle missing, NA, or unexpected values for every field - ** Comprehensive ** : derive features from every field in the rubric, even if that field happens to be NA for all 40 examples shown here Use the parser source above as the authoritative specification of fields and value formats; use the examples below to validate and calibrate your regex patterns. 40 example text serialization in x rubric format ## Required Script Interface The generated script must: 1. Accept CLI arguments via argparse: - β--input dirβ - β--outputdirβ - β--taskβ - β--splitsβ 2. For each split, read βinput dir/split/task.jsonβ | a JSON array where each element has: - βpatientidβ (int) - βlabeltimeβ (ISO datetime string) - βlabelvalueβ (bool) - βconversationsβ (list) | rubric text is in βconversations[1]["content"]β between β--- Patient EHR ---β and β--- End of EHR ---β 3. Implement βdef extractfeatures(rubrictext: str) -> dict[str, float]β: - Parse every rubric field from the text - Return a flat dict mapping feature nameβ float value - For ** numeric fields ** : extract the number; if missing/NA write β0.0β and set βfieldmissing = 1.0β - For ** categorical / Yes/No fields ** : one-hot encode all known values; unknown/NAβ all zeros plus a βfieldmissing = 1.0β indicator - All returned values must be float (0.0 or 1.0 for binary, numeric otherwise) - The dict must have the ** same keys in the same order ** for every call (fixed schema) 4. Define βSCHEMA: list[dict]β at module level | one entry per feature with keys: - β"name"β: feature name (matches key in extract features output) - β"type"β: β"numeric"β, β"binary"β, or β"categorical"β - β"description"β: short human-readable description - β"possiblevalues"β: list of string values for categorical/binary fields, omit for numeric 35 LLMs can construct powerful representations and streamline sample-efficient supervised learning 5. For each split, build an NΓF float32 matrix from βextractfeaturesβ, save as: - βoutputdir/task/split.npzβ with numpy keys: - βembeddingsβ: shape (N, F) float32 - βlabelsβ: shape (N,) int32 - βpatient idsβ: shape (N,) int64 - βpredictiontimesβ: shape (N,) object (strings) 6. Save βoutputdir/task/featureschema.jsonβ once (after processing the first split): βjson "task": "task", "task query": "taskquery", "numfeatures": <F>, "features": <SCHEMA list> β 7. Create output directories as needed. Print progress to stdout. ## Constraints - Use only Python standard library plus βreβ, βjsonβ, βnumpyβ, βargparseβ, βpathlibβ, βsysβ. No third-party packages beyond numpy. - No LLM API calls, no network requests. - βextract featuresβ must be deterministic and never raise exceptions on any input (catch all errors, default to 0.0). - The script must be syntactically valid Python 3.8+. - Do NOT hardcode file paths | use the argparse arguments. - Aim for ** at least 30 features ** to capture the richness of the rubric. Include all numeric fields, all categorical fields (one-hot), and Yes/No procedure/comorbidity flags. ## Output Output ONLY the Python script, with no explanation, no preamble, and no markdown fences. Start with β#!/usr/bin/env python3β. Figure 15. Prompt used with GPT-5.2 to create a parser script that transforms rubric-transformed inputs (x rubric ) into a fixed-dimensional tabular feature vector. 36 LLMs can construct powerful representations and streamline sample-efficient supervised learning E. Full Global Rubric Examples E.1. Full Global Rubric for the Hypertension Diagnosis Task RUBRIC INSTRUCTIONS FOR TASK: HYPERTENSION Rubric purpose - Provide a reproducible, stepwise process to transform any EHR into a structured, clinical-evidence summary useful for assessing the likelihood that a patient will develop hypertension in the next year. - The rubric standardizes what to extract, how to summarize trends and risk factors, and how to record uncertainty and provenance so downstream models or clinicians can apply consistent reasoning. How to use this rubric - Follow the numbered extraction and analysis steps for each new patient. - Populate the structured template fields exactly (use units shown). If data are missing, enter βmissingβ and note time windows attempted. - Do NOT make a final yes/no prediction inside the form. Instead, produce the structured summary and quantitative or qualitative risk-domain scores for downstream modeling. A. Preparation (before extracting) 1. Define the prediction window: βnext yearβ relative to the EHR reference date and time. 2. Define time windows to extract: - Very recent: last 30 days - Recent: 31--180 days - Baseline/remote: >180 days up to available history 3. Standardize units and formats: - Blood pressure: mmHg (systolic/diastolic) - Weight: kg or oz β convert to kg if numeric calculations needed - Height: cm or in β convert to meters - Labs: use usual clinical units (creatinine mg/dL, A1c %, etc.) 4. Log data sources (vitals, problem list, medications, laboratory, procedures, notes) and timestamp of extraction. B. Step-by-step extraction & transformation procedure Step 1 --- Demographics and baseline context - Extract: - Age (years) - Sex / gender - Race / ethnicity (if available) - Relevant social history: tobacco (current/former/never), alcohol (heavy/regular/rare/none), illicit drug use, tobacco product types - Pregnancy status (current or past complications such as pre-eclampsia) - Baseline height and weight; calculate BMI (kg/m 2 ) and BMI category - Record date of last update for each demographic item. Step 2 --- Blood pressure (BP) data extraction and normalization - Extract all systolic and diastolic BP values with timestamps and context (office, inpatient, ED, home, ambulatory, perioperative). - Normalize values: remove implausible readings (document them) and ensure mmHg units. - For each time window: - Compute count, mean, median, standard deviation, minimum, and maximum. - Identify last available BP and date. - Flag highest recent systolic and diastolic values with dates. - Compute trend metrics: - Recent slope = (mean recent β meanbaseline) / time (mmHg per month); indicate direction only if clinically meaningful (e.g., β₯3 mmHg/year). - BP variability indicator: SD of systolic BP in recent window; flag high variability if SD >10 mmHg. - Categorize BP using ACC/AHA thresholds: - Normal (<120/<80) - Elevated (120--129/<80) - Stage 1 Hypertension (130--139 or 80--89) - Stage 2 Hypertension (β₯140 or β₯90) - If mixed, note βdiscordantβ and list counts per category. Step 3 --- Antihypertensive and BP-impacting medications - Extract current and recent medications with start and stop dates if available. - Flag antihypertensives (ACEi, ARBs, beta-blockers, diuretics, CCBs, vasodilators). - Flag BP-raising agents (systemic corticosteroids, NSAIDs, decongestants, stimulants, calcineurin inhibitors, SNRIs, MAOIs, some oral contraceptives). - For each flagged medication record name, dose, dates, indication, and temporal relation to BP changes. Step 4 --- Comorbidities associated with increased HTN risk - Extract diagnoses and ICD codes with dates: - Major risk: CKD, diabetes, CVD, PAD, OSA, endocrine causes, pregnancy or pre-eclampsia, obesity (BMI 37 LLMs can construct powerful representations and streamline sample-efficient supervised learning β₯30), heavy alcohol use. - Moderate risk: hyperlipidemia, metabolic syndrome, thyroid disease, autoimmune disease with renal involvement. - Secondary HTN clues: resistant BP, hypokalemia, episodic symptoms. - Record first documentation date, last active date, and severity when available. Step 5 --- Relevant laboratory data - Extract labs with dates grouped by time window: - Creatinine / eGFR - Electrolytes (Na, K, HCO 3 ) - Glucose, HbA1c - Lipids - Urine albumin or protein - Thyroid tests - Aldosterone/renin, cortisol, catecholamines if available - Flag abnormal values with interpretation (e.g., eGFR <60 ml/min). Step 6 --- Procedures and objective testing - Extract echocardiography, renal imaging, sleep studies, ABPM. - Note evidence of end-organ effects (LVH, albuminuria, renal disease). Step 7 --- Social, behavioral, and family data - Smoking status and intensity. - Alcohol use severity. - Family history of HTN or early CVD. - Adherence or socioeconomic barriers if documented. Step 8 --- Acute confounders - Identify acute illness, pain, surgery, sepsis, AKI, or inpatient context affecting BP interpretation. - Avoid using isolated inpatient readings without outpatient corroboration. Step 9 --- Domain synthesis and scoring - For each domain, record evidence, recency, and confidence (High/Moderate/Low). - Domains include BP phenotype, medications, metabolic risk, kidney function, secondary HTN, end-organ disease, behavior, and acute confounders. - Assign severity (Major/Moderate/Minor) and create a domain scorecard; do NOT generate a final binary label. Step 10 --- Evidence provenance and missing data - Record source, timestamp, and confidence for each major item. - Explicitly flag critical missing data (e.g., no outpatient BP in 12 months). Step 11 --- Structured output - Produce a standardized summary with demographics, BP summary, medications, comorbidities, labs, procedures, lifestyle, acute confounders, domain scorecard, missing data, and a 2--4 sentence neutral text summary. Step 12 --- Guidance notes - Recommend confirmatory testing or review where appropriate (e.g., home BP, med review, nephrology referral). - Do not conclude final risk. Final note to the user - Use this rubric to populate the structured template for every patient. Do not record a final hypertension risk classification here; the output is intended for downstream models or clinician judgment. Figure 16. Global rubric instructions for extracting structured patient profiles for 1-year hypertension diagnosis prediction. 38 LLMs can construct powerful representations and streamline sample-efficient supervised learning E.2. Full Global Rubric for the Hyponatremia Lab Result Prediction Task RUBRIC INSTRUCTIONS FOR TASK: HYPONATREMIA LAB RESULT PredictionDate: [Extract the βcurrent timeβ / prediction timestamp from the EHR header]. Format: Y-M-D. If not present write NA. Patient: - Age: [years as integer from EHR]. If not present write NA. - Sex: [as documented: MALE / FEMALE / Other / Unknown]. If not present write NA. - Race/Ethnicity: [as documented]. If not present write NA. ProblemListFlags (presence and dates): - Chronic kidney disease / End-stage renal disease (CKD/ESRD): [Yes / No]. If Yes, list documented term(s) and most recent date(s) (Y-M-D). If none write No. - Dialysis history/procedure in record: [Yes / No]. If Yes, list procedure name(s) and most recent date(s). If none write No. - Prior documented hyponatremia / -osmolality and or hyponatremia": [Yes / No]. If Yes, give the documentation text and date(s). If none write No. - Active malignancy listed in Problem List or current visits: [Yes / No]. If Yes, list malignancy type(s) and most recent date(s). If none write No. SerumSodium Last3 (most recent first): For up to 3 most recent serum/plasma/blood sodium measurements, extract a line per measurement in this exact format: - Y-M-D (daysbeforeprediction): [value] mmol/L ; Specimen=[serum/plasma/blood] ; Setting=[ED/Inpatient/Outpatient/Lab] ; Note=[any explicit result comment if present] If fewer than 3 measurements exist, include those available; if none write NA. SerumSodium Min90: - Lowest documented serum/plasma/blood sodium value in the prior 90 days (value mmol/L) and date (Y-M-D). If none write NA. SerumOsmolality Last3: - For up to 3 most recent serum osmolality measurements: Y-M-D (daysbeforeprediction): [value] mOsm/kg ; Setting=[as above] If none write NA. UrineStudies Last3: - For up to 3 most recent urine study sets, extract for each available element on one line: - Y-M-D (daysbeforeprediction): UrineNa=[value] mmol/L ; UrineOsm=[value] mOsm/kg ; SpecificGravity=[value] ; Setting=[ED/Inpatient/Outpatient/Lab] Only include elements that are present for that date. If no urine studies documented write NA. RenalFunction: - Most recent serum creatinine (mg/dL) and date: Y-M-D: [value] mg/dL. If none write NA. - Most recent BUN (mg/dL) and date: Y-M-D: [value] mg/dL. If none write NA. - Recent acute renal failure / acute kidney injury entries within 30 days: [Yes / No]. If Yes include diagnosis text and date(s). If none write No. VolumeRelatedFindings (documented in problem lists or visit notes within past 30 days): - Extract presence with dates for these items (list each if present as "Item: Y-M-D;"): Edema, Ascites, Hypotension (documented low BP or explicit "hypotension"), Dehydration, Vomiting, Diarrhea, Nasogastric/feeding tube, Ileostomy/colostomy, Recent large-volume paracentesis. If none of these documented in past 30 days write NA. Medications PotentiallyAffectingSodium (recent administrations | extract from medication list / inpatient meds / discharge meds): - Time window: last 14 days before PredictionDate (if EHR supports more granular times use those). For each relevant med/class present include one line: - [Y-M-D last administration if available] : [Medication name] ; Class=[thiazide/loop diuretic / SSRI / SNRI / TCA / anticonvulsant (carbamazepine/oxcarbazepine) / NSAID / SSRI, etc.] ; Route=[oral/IV] ; Dose if documented=[text] - If none of these medication classes documented in last 14 days write NA. - Also include "Chronic diuretic use noted (Yes/No) and last documentation date" (e.g., long-term thiazide). IVFluids Last72h: - List IV fluid administrations in last 72 hours (date/time if available) in the format: - Y-M-D: [fluid type as documented, e.g., D5W / D5NS / 0.9% NaCl / hypotonic saline / LR / "glucose 39 LLMs can construct powerful representations and streamline sample-efficient supervised learning 50 mg/mL prefills"] ; Volume if documented. - If none documented write NA. AcuteConditionsAssociatedWithSIADHorHyponatremia (documented within 30 days): - For each present within 30 days, list as "Condition: Y-M-D" from problem/visit notes: - Pulmonary infection / pneumonia / pulmonary disease - CNS disorder (stroke, hemorrhage, encephalopathy) - Sepsis / severe infection - Recent major surgery / Postoperative state - Pain / Severe nausea (if explicitly documented) - Malignancy active (if not already in ProblemListFlags) If none documented write NA. RecentProcedures ChemotherapyTransfusion (last 30 days): - List any of: major surgery, chemotherapy, recent blood transfusion, paracentesis, TPN (total parenteral nutrition), plasmapheresis, hemodialysis | format: - Y-M-D: [procedure name / chemo agent e.g., paclitaxel] ; Notes=[if available] If none write NA. Glucose Last3: - Up to 3 most recent serum/plasma or point-of-care glucose values (most recent first): - Y-M-D: [value] mg/dL ; Type=[serum/plasma/glucometer] ; Setting=[ED/Inpatient/Outpatient] If none write NA. SerumProteinOrLipidExtremes: - If very high triglycerides or abnormal total protein/albumin documented close to sodium measurement, extract: - Y-M-D: Triglycerides=[value] mg/dL ; TotalProtein=[value] g/dL ; Albumin=[value] g/dL If none documented write NA. PriorHyponatremiaHistory: - Any historical low sodium episodes before 90 days (brief): list lowest prior value and date(s) or write NA. LabQualityNotes: - Any documented lab-quality flags on sodium measurement (e.g., hemolysis, lipemia, blood collection tube" note, specimen issues): extract verbatim note and date(s). If none write NA. RelevantVitalSigns NearMostRecentSodium: - From the same encounter as the most recent sodium (if identifiable), extract: systolic/diastolic BP (mmHg), heart rate (bpm), and whether on oxygen or dialysis that encounter. Format: - Date: Y-M-D ; SBP=[value] ; DBP=[value] ; HR=[value] ; Oxygen=[yes/no] with O2 sat if given ; DialysisThisEncounter=[yes/no] If not available write NA. FreeText FindingsCues: - Extract any verbatim phrases (short quotes) that explicitly mention hyponatremia-related language in notes or problem list (e.g., "hyponatremia", "hypo-osmolality", "SIADH", "hypotonic fluids", "low sodium") with the date and the note type. Format: - Y-M-D ; Source=[ProblemList/VisitNote/LabComment] ; Text="[exact phrase]" If none write NA. DataCompleteness: - For each of the following categories indicate [Present / Absent / Not documented]: Serum sodium labs, urine sodium/osmolality, serum osmolality, recent meds list, dialysis record, IV fluids record, creatinine/BUN. Example: SerumSodium: Present ; UrineSodium: Absent ; etc. ExtractionRules / Formatting Rules (must follow exactly): - Always extract facts only; do not add interpretation, risk assessment, or predictions. - Dates: use Y-M-D as in EHR; if EHR provides relative days include "(N days before prediction)" after date. - When multiple values on same date, include all values separated by ";". - If an item not found anywhere in the EHR, write exactly "NA". - Keep each field on a single line (except the repeated-measure lists which may have up to three lines as specified). - Use units exactly as specified (mmol/L for Na and UrineNa; mOsm/kg for osmolality; mg/dL for glucose/BUN/creatinine; mg/dL for triglycerides). - Do not synthesize or infer ranges; extract only documented numeric values and verbatim text. EndOfTemplate. Figure 17. Global rubric instructions for extracting structured patient profiles for hyponatremia lab result prediction (abnormal vs. normal). 40