Paper deep dive
EarthAlign: Learning Language-Aligned Representations of Earth System Data
Fadillah Adamsyah Maani, Numan Saeed, Jiaqi Ma, Dat Tien Nguyen, Youssef Wehbe, Duncan Watson-Parris, Salman Khan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/14/2026, 10:14:07 AM
Summary
The paper introduces EarthAlign, a large-scale dataset (553,738 samples, 2000-2024) combining natural event/disaster records with climatology-derived normal-weather samples, paired with 63-variable ERA5 meteorological data and four grounded caption styles. It proposes MeteoLIP, a dual-encoder framework using a novel MetTok tokenizer to aggregate variable-specific patch embeddings into compact meteorology tokens conditioned on geolocation and time. MeteoLIP aligns meteorological data with language representations, outperforming CLIP and SigLIP baselines on cross-modal retrieval, weather-condition classification, and physical variable estimation tasks.
Entities (11)
Relation Signals (10)
EarthAlign ā containsdatafrom ā ERA5
confidence 95% Ā· Each sample includes temporal and geospatial information and four grounded caption styles aligned with 63-variable ERA5 meteorological data.
MeteoLIP ā usesmodule ā MetTok
confidence 95% Ā· MeteoLIP incorporates MetTok, which uses a geolocation- and time-conditioned query to aggregate variable-specific patch embeddings...
MeteoLIP ā evaluatedon ā Cross-modal Retrieval
confidence 93% Ā· To comprehensively assess Earth system multimodal representations, we introduce three downstream tasks: cross-modal retrieval...
MeteoLIP ā evaluatedon ā weather-condition classification
confidence 93% Ā· To comprehensively assess Earth system multimodal representations, we introduce three downstream tasks: ... weather-condition classification...
MeteoLIP ā evaluatedon ā physical variable estimation
confidence 93% Ā· To comprehensively assess Earth system multimodal representations, we introduce three downstream tasks: ... physical variable estimation.
MeteoLIP ā outperforms ā SigLIP
confidence 92% Ā· Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks.
MeteoLIP ā outperforms ā CLIP
confidence 92% Ā· Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Interpreting Earth system conditions is central to understanding environmental risks and informing decisions across weather- and climate-sensitive sectors. Existing multimodal weather systems primarily target task-specific prediction, generation, or question answering, while general-purpose alignment between Earth system data and language remains underexplored due to the limited availability of diverse language supervision grounded in meteorological conditions. To fill this gap, we introduce EarthAlign, a large-scale dataset with broad geographic coverage spanning 2000-2024 that combines 10 expert-curated natural event and disaster data sources with climatology-derived normal-weather samples. Each sample includes temporal and geospatial information and four grounded caption styles aligned with 63-variable ERA5 meteorological data. Building on EarthAlign, we introduce MeteoLIP, a dual-encoder framework for learning aligned meteorology-language representations. MeteoLIP incorporates MetTok, which uses a geolocation- and time-conditioned query to aggregate variable-specific patch embeddings into a compact sequence of meteorology tokens. To comprehensively assess Earth system multimodal representations, we introduce three downstream tasks: cross-modal retrieval, weather-condition classification, and physical variable estimation. Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks. Further analyses highlight the importance of meteorological input, temporal and geospatial conditioning, and multi-style supervision for meteorology-language alignment. Thus, EarthAlign and MeteoLIP provide a foundation for learning and evaluating language-aligned representations of Earth system data. Our code, dataset, and weights are publicly available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.14624v1
- Canonical: https://arxiv.org/abs/2610.14624v1
Trouble viewing inline? Open PDF directly ā
Full Text
136,388 characters extracted from source content.
Expand or collapse full text
Preprint EARTHALIGN: LEARNING LANGUAGE-ALIGNED REP- RESENTATIONS OF EARTH SYSTEM DATA Fadillah Adamsyah Maani 1 Numan Saeed 1 Jiaqi Ma 1,ā Dat Tien Nguyen 1 Youssef Wehbe 2 Duncan Watson-Parris 3 Salman Khan 1,2 1 Mohamed bin Zayed University of Artificial Intelligence 2 ADIA Lab 3 University of California San Diego ā Corresponding authors: jiaqi.ma@mbzuai.ac.ae Ā WebsiteĀ§ CodeƵ Dataset & Weights ABSTRACT Interpreting Earth system conditions is central to understanding environmental risks and informing decisions across weather- and climate-sensitive sectors. Ex- isting multimodal weather systems primarily target task-specific prediction, gen- eration, or question answering, while general-purpose alignment between Earth system data and language remains underexplored due to the limited availability of diverse language supervision grounded in meteorological conditions. To fill this gap, we introduce EarthAlign, a large-scale dataset with broad geographic coverage spanning 2000ā2024 that combines 10 expert-curated natural event and disaster data sources with climatology-derived normal-weather samples. Each sample includes temporal and geospatial information and four grounded caption styles aligned with 63-variable ERA5 meteorological data. Building on Earth- Align, we introduce MeteoLIP, a dual-encoder framework for learning aligned meteorology-language representations. MeteoLIP incorporates MetTok, which uses a geolocation- and time-conditioned query to aggregate variable-specific patch embeddings into a compact sequence of meteorology tokens. To com- prehensively assess Earth system multimodal representations, we introduce three downstream tasks: cross-modal retrieval, weather-condition classification, and physical variable estimation. Our experiments show that MeteoLIP improves upon the corresponding CLIP and SigLIP baselines across these tasks. Further anal- yses highlight the importance of meteorological input, temporal and geospatial conditioning, and multi-style supervision for meteorology-language alignment. Thus, EarthAlign and MeteoLIP provide a foundation for learning and evaluat- ing language-aligned representations of Earth system data. Our code, dataset, and weights are publicly available at https://github.com/mbzuai-oryx/EarthAlign. 1INTRODUCTION Earth system forecasts and analyses play a critical role in supporting decision-making across sectors such as agriculture, water, energy, and disaster management (Brasseur & Gallardo, 2016; White et al., 2017). Recent data-driven weather models and Earth system foundation models, including FourCastNet (Pathak et al., 2022), Pangu-Weather (Bi et al., 2023), ClimaX (Nguyen et al., 2023), and Aurora (Bodnar et al., 2025), have achieved strong performance in forecasting atmospheric and broader Earth system states. Their representations are primarily learned through objectives defined over physical variables, such as forecasting future atmospheric states (Nguyen et al., 2023; Bodnar et al., 2025) and reconstructing masked meteorological fields (Lessig et al., 2023; Schmude et al., 2024). While these objectives effectively model relationships within numerical Earth system data, they do not explicitly align the learned representations with natural-language descriptions of weather conditions and natural phenomena. Natural language provides a valuable source of supervision that can align multivariate meteorologi- cal data with descriptions of weather conditions and natural phenomena. In natural-image domains, 1 arXiv:2610.14624v1 [cs.CV] 10 Oct 2026 Preprint Figure 1: EarthAlign Overview. EarthAlign is a vision-language alignment dataset for the Earth system, comprising 553,738 samples that span 2000ā2024, with a broad taxonomy of natural phe- nomena, broad geographic coverage, and diverse caption content to enable multimodal alignment. CLIP (Radford et al., 2021) has demonstrated that language supervision can support shared rep- resentations for cross-modal retrieval and text-based classification, and such visual representations have subsequently become key components of multimodal models across different domains (Liu et al., 2023; Shabbir et al., 2025). Despite this progress, general-purpose alignment between Earth system data and language remains underexplored, partly due to the lack of diverse language super- vision grounded in meteorological conditions and natural phenomena. Existing multimodal datasets such as WeatherQA (Ma et al., 2024), CLLMate (Li et al., 2025), and MeteorPred (Tang et al., 2026) have enabled progress in severe-weather question answering, event prediction, and warning genera- tion, but are primarily designed around task-specific prediction, generation, or question answering. Learning robust meteorology-language representations requires rich language supervision covering diverse weather conditions and natural phenomena, with descriptions from different perspectives and temporal and geographic context to associate them with the corresponding Earth system states. To address this, we construct EarthAlign (Figure 1), which brings heterogeneous natural event, dis- aster, and normal-weather records into an Earth systemālanguage setting. EarthAlign comprises 553,738 samples spanning 2000ā2024, with broad geographic coverage and diverse natural phe- nomena. We compile natural event and disaster records from governmental, institutional, and pub- lished sources, and complement them with normal-weather samples identified relative to local cli- matology. For each record, we generate four metadata-grounded caption styles to provide different semantic perspectives on the underlying Earth system condition. Each record is also paired with a 63-variable ERA5 (Hersbach et al., 2020) snapshot at the corresponding time and spatial extent to enable meteorology-language alignment. To learn semantically aligned representations of me- teorological data and language, we introduce MeteoLIP, a dual-encoder framework that maps both modalities into a shared representation space. As meteorological inputs comprise 63 surface and atmospheric variables, MeteoLIP uses MetTok to aggregate variable-specific patch embeddings into compact meteorology tokens conditioned on temporal and geospatial information. During alignment training, MeteoLIP samples one of the four caption styles for each meteorological input, exposing the model to complementary descriptions of the same Earth system condition. Our contributions can be summarized as follows: (1) We introduce EarthAlign, a large-scale Earth systemālanguage dataset with broad geographic coverage, diverse natural phenomena and normal- weather conditions, metadata-grounded multi-style captions, and alignment with multivariate ERA5 data; (2) We introduce MeteoLIP for meteorology-language representation learning, and train 2 Preprint Meteo-CLIP and Meteo-SigLIP variants that improve over their corresponding CLIP and SigLIP baselines; (3) We establish three evaluation tasks spanning cross-modal alignment, semantic under- standing, and quantitative inference through retrieval, weather-condition classification, and physical variable estimation. 2RELATED WORK Vision-language alignment. Vision-language pretraining learns shared representations by align- ing visual data with rich natural language descriptions. CLIP (Radford et al., 2021) demonstrated the effectiveness of large-scale image-text contrastive learning, while SigLIP (Zhai et al., 2023) and InternVL (Chen et al., 2024) further improved its training objective and model scaling. Beyond natu- ral images, this paradigm has been adapted to specialized domains such as medical imaging (Zhang et al., 2025; Khattak et al., 2024) and remote sensing (Liu et al., 2024; Xiong et al., 2025). Due to their transferable representations, these visual encoders are widely adopted in multimodal lan- guage models across domains (Liu et al., 2023; Wang et al., 2025; Sellergren et al., 2026; Soni et al., 2025). Despite these developments, vision-language alignment for Earth system data remains underexplored, primarily due to the lack of large-scale language supervision paired with geoloca- tion and time information. EarthAlign addresses this gap by introducing geocoded natural language captions describing diverse historical weather conditions and natural phenomena from around the world, enabling their alignment with Earth system data, such as gridded meteorological data. Multimodal weather and climate datasets and benchmarks. As summarized in Table 8, re- cent work has increasingly connected meteorological data with natural language for weather- and climate-related tasks. WeatherQA (Ma et al., 2024) pairs expert forecast analyses with mesoscale analysis images for severe-weather question answering. ClimateIQA (Chen et al., 2025) generates meteorological heatmaps from four ERA5 variables (Hersbach et al., 2020) to detect three types of severe weather events. Aquilon (Varambally et al., 2025) defines seven question types and con- structs samples from ERA5 and an expert-curated event record. CLLMate (Li et al., 2025) curates expert-verified weather and climate events from environmental news and pairs them with four ERA5 variables for event prediction. MeteorPred (Tang et al., 2026) uses meteorological agency warnings as supervision and ERA5 variables as inputs for warning generation in China. While these datasets vary in scope and task formulation, they are primarily designed for instruction following or question answering. In contrast, EarthAlign provides large-scale dataset spanning 2000ā2024, with broad geographic coverage and natural phenomena, with four grounded caption styles per record to enable vision-language alignment with Earth system data. We further introduce retrieval, classification, and physical variable estimation tasks to evaluate multimodal representations in this domain. 3DATASET AND EVALUATION TASKS 3.1DATASET CONSTRUCTION To address the limitations of existing multimodal climate datasets, we present EarthAlign, a large- scale dataset designed to facilitate alignment of Earth-system data with language. We build the dataset based on two types of sources: records of natural events and disasters, and climatological data for normal weather samples. EarthAlign comprises 553,738 samples with broad geographic coverage spanning 2000ā2024 and encompasses a diverse set of keywords describing a wide range of natural phenomena (Fig. 1). Each sample contains temporal information, a latitude-longitude bounding box, and four captions in different styles. Our data curation pipeline is shown in Fig. 2. Natural event and disaster records. Natural events and disasters constitute the majority of sam- ples in EarthAlign, providing large-scale and diverse observations of natural phenomena. These records were collected from 10 expert-curated sources (Appendix Table 9): 1) EM-DAT (Delforge et al., 2025), 2) NOAA Storm Event Database (NOAA NCEI), 3) IBTrACS (Knapp et al., 2010; Gahtan et al., 2024), 4) COOLR (Juang et al., 2019), 5) PIKART (Vallejo-Bernal et al., 2025), 6) AusDE (Australian Government, Attorney-Generalās Department, 2023), 7) MexDE (Centro Na- cional de Prevenci Ģ on de Desastres (CENAPRED), 2023), 8) CanDE (Public Safety Canada, 2026), 9) IndDE (Badan Nasional Penanggulangan Bencana, 2026), and 10) FraDE (Minist ` ere de la Tran- sition Ģ ecologique, 2026). Since these sources provide different types of information in different 3 Preprint Climatology-Based Normal-Weather Sampling Natural Event and Disaster Data Curation High Quality Natural Event and Hazard Records Metadata Harmonization Geocoding latitude-longitude alignment ERA5 Grid Sampling sample cells per ADM1 ERA5 Hourly Variables Climatology Threshold 1991-2020, DOY Ā± 2 Normality Filter & Sampling Global Administrative Units Multi-Style Grounded Captioning Multi-Style Grounded Caption Generation Quality Audit 10 records per source EarthAlign LocationLabelTime Physical Framing Atmospheric Pattern Impact Focused Natural Event Focused Multi-Style Captions Figure 2: Data curation pipeline. EarthAlign combines expert-curated natural-event records with climatology-based normal-weather samples. The resulting records contain standardized spatial and temporal information, labels, and four grounded caption styles for alignment with Earth system data. formats, we apply source-specific processing steps (Appendix B.1). In general, we begin by man- ually filtering the records to retain only natural phenomena. We then extract all available metadata to support the grounded generation of detailed descriptions. For sources that provide only textual location names, we apply geocoding to obtain coordinates and latitude-longitude bounding boxes. After processing, we obtain 489,185 records with standardized phenomenon categories, associated metadata, as well as temporal and geospatial information. Climatology-based normal-weather sampling. To complement the natural event and disaster records, we identify normal-weather conditions relative to local climatology. We construct local climatologies from ERA5 (Hersbach et al., 2020) hourly variables over the 1991ā2020 WMO refer- ence period (WMO, 2017). Adapting the centered 5-day window proposed in Zhang et al. (2005), we compute each variable-specific threshold for each day of year and hour using a centeredĀ±2-day window. For scalability, we sample 20% of the ERA5 grid cells within each first-level administra- tive unit (ADM1), based on administrative boundaries from geoBoundaries (Runfola et al., 2020), with at least 10 cells whenever possible, and retrieve hourly variables from 1991 to 2025. Depend- ing on the variable, we define normality by P05āP95,ā¤P90, orā„P10 thresholds. A grid-cell hour is considered normal only if all variables satisfy their respective thresholds, and an ADM1-hour is retained when at least 80% of its sampled cells are normal. Using this conservative filter, we retain 1.6% and 11% of candidate hours in California and Abu Dhabi, respectively. We then sam- ple eligible normal-weather hours globally over 2000ā2024 with broad country coverage and area- proportional sampling, yielding 64,553 samples. Further details on the meteorological variables and their variable-specific normality criteria are provided in Appendix B.2. Grounded caption generation. From the metadata, we generate four stylistically different captions (i.e. natural-event-focused, physical-framing, impact-focused, and atmospheric-pattern) to provide complementary semantic perspectives of each record and support alignment with Earth system data. We use Gemini 3.1 Pro with a prompt that requires every caption to be directly supported by the input metadata and excludes unsupported causes, impacts, forecasts, and descriptive modifiers. To assess caption grounding and adherence to the target styles, a trained graduate annotator audited a stratified sample of 440 captions across all data sources and caption styles. The overall pass rate was 95.68% (421/440), with style-specific pass rates of 100%, 98%, 89%, and 95%. We provide the generation prompt in Appendix C.1, and the audit rubric and error analysis in Appendix C.2. Earth system data alignment. Earth system data are heterogeneous across data types and modal- ities, variables, and spatial and time resolutions. ERA5 (Hersbach et al., 2020) provides globally gridded hourly data at 0.25 ā¦ resolution and is widely used in recent Earth system models and mul- timodal meteorological benchmarks (Rasp et al., 2024; Bodnar et al., 2025; Li et al., 2025; Tang et al., 2026). Motivated by this, we adopt ERA5 as a common baseline while preserving compatibil- ity with other Earth system products. For each record, we extract a single multivariate ERA5 patch at the corresponding timestamp and spatial extent. Each patch contains 28 single-level variables and 7 pressure-level variables at 5 pressure levels, resulting in 63 input variables as listed in Appendix D. 4 Preprint V W H Meteorology Tokenizer (MetTok) Meteorology-Language Alignment "A blizzard brought winter weather to southeast Minnesota ..." "Torrential rain from a tropical cyclone drenched ..." Natural Event Focused: "A blizzard brought winter weather to southeast ..." Physical Framing: "Widespread wind gusts combined with falling ..." Impact Focused: "Falling snow and high winds resulted in severe ..." Atmospheric Pattern: "A powerful mid-latitude cyclone progressed ..." Natural Event Focused: "On 15 March 2010, a complex landslide ..." Physical Framing: "Torrential rain from a tropical cyclone ..." Impact Focused: "The passage of Tropical Storm Hubert brought ..." Atmospheric Pattern: "A tropical cyclone named Hubert produced ..." Meteorology Encoder Text Tokenizer Aa Text Encoder Random style sampling Meteorology Tokens Meteorology Representations Text Tokens Text Representations Figure 3: MeteoLIP Overview. MeteoLIP consists of transformer-based meteorological and text encoders. MetTok converts multivariate meteorological input into a compact token sequence. Dur- ing training, we randomly sample a caption style for each meteorological input and optimize a con- trastive objective to align meteorological and textual representations in a shared embedding space. 3.2EVALUATION TASKS To comprehensively evaluate Earth system multimodal representations, we devise three downstream tasks assessing their cross-modal alignment, semantic understanding, and quantitative inference capability: (1) Retrieval evaluates the alignment between Earth system (e.g. ERA5) representa- tions and their corresponding textual descriptions using bidirectional text-to-meteorology (T2M) and meteorology-to-text (M2T) retrieval; (2) Classification evaluates whether Earth system repre- sentations can capture category-level semantics of meteorological conditions. Given an Earth system input (e.g. ERA5), the model predicts one of 11 weather conditions: normal weather, drought, heat wave, cold wave, frost, atmospheric river, cyclone, convective storm, high wind, fog, and winter weather; and (3) Physical variable estimation evaluates quantitative inference by estimating cy- clone maximum sustained wind speed and minimum central pressure from an Earth system input. 4METEOLIP: ALIGNING METEOROLOGICAL DATA WITH LANGUAGE EarthAlign comprises paired ERA5 data and complementary textual descriptions. To learn aligned representations of these descriptions and their meteorological conditions, we introduce MeteoLIP, a dual-encoder framework for aligning meteorological data with natural language. As illustrated in Fig. 3, MeteoLIP utilizes separate transformer-based meteorology and text encoders and maps their outputs into a shared representation space. To process the 63-variable ERA5 input, MeteoLIP uses MetTok, which provides a mechanism for incorporating geospatial and temporal information while producing a compact token sequence whose length depends on the spatial resolution. Through this alignment, MeteoLIP learns cross-modal representations that enable a range of downstream tasks. 4.1MODEL ARCHITECTURE AND TRAINING OBJECTIVE Dual-encoder formulation. Let x T denote a textual input and x M āR VĆHĆW denote meteo- rological data, where V is the number of variables and H Ć W are the spatial dimensions. Each meteorological input is supplemented with temporal information tāR 3 , corresponding to year, day of year, and hour of day, and latitudeālongitude geolocation informationā āR 2ĆHĆW . MeteoLIP employs dual encoders to map the two modalities into a shared D-dimensional representation space, with z M = F M (T M (x M ,t,ā)) and z T = F T (T T (x T )), where z M ,z T āR D denote the mete- orological and textual representations, respectively. T M and T T denote the meteorology and text tokenizers, whileF M andF T denote the transformer-based meteorology and text encoders. Meteorology tokenization (MetTok). Unlike natural images, meteorological inputs contain many physical variables. Let ps be the patch size and P = HW/ps 2 be the number of patches. In- dependently patchifying each variable would produce V P tokens, which would increase the self- attention complexity in the meteorology encoder from O(P 2 D) to O(V 2 P 2 D). MetTok instead 5 Preprint aggregates the variables within each patch, thereby retaining a sequence of P tokens. Specifi- cally, we independently patchify each variable in x M using a variable-specific linear projection and subsequently add a learnable variable-identity embedding, producing variable-patch embeddings E M āR VĆPĆd , where d = D/8 is the MetTok embedding dimension. We encode the tempo- ral information t by linearly normalizing the year and applying sine-cosine transformations to the day of year and hour of day, resulting in e t āR 5 , which is spatially expanded toR 5ĆHĆW . Sim- ilarly, we apply sine-cosine transformations to latitude and longitude, resulting in e ā āR 4ĆHĆW . To obtain temporal-geospatial patch embeddings E tā āR PĆd , we concatenate the spatially ex- panded e t and e ā along the channel dimension and jointly patchify them using a linearāGELUālinear projection, allowing temporal and geospatial features to interact nonlinearly. Inspired by Bodnar et al. (2025); Jaegle et al. (2021), we introduce a learnable query embedding e q āR d to aggregate the variable embeddings in E M through cross-attention. Since meteorological conditions depend on geographic location and temporal context, we condition the query on E tā to obtain time- and location-aware queries E q = [e q + Ī±e tā,1 ;... ;e q + Ī±e tā,P ] āR PĆd , where Ī± āR is a learn- able scalar. We then apply cross-attention independently within each patch, using the time- and location-aware query as the query and the corresponding variable embeddings as keys and val- ues, yielding e o,p = e q,p + CrossAttn(e q,p ,e M,p ,e M,p ) for p = 1,...,P , where e q,p āR d and e M,p āR VĆd are the query and variable embeddings for patch p, respectively. We then project each aggregated representation to the meteorology encoder dimension using a linear layer, b e o,p = W o e o,p + b o āR D . The resulting meteorology tokens [ b e o,1 ;... ; b e o,P ] are prepended with a class token and passed to the meteorology encoder, thereby avoiding the additional quadratic self-attention cost induced by variable-wise tokenization. Learning objective. During training, we sample one caption style for each meteorological input in a minibatch to expose the model to diverse linguistic descriptions from different perspectives and improve robustness. Given a minibatch of B paired meteorological and textual inputs, we compute their pairwise cosine similarities as s ij = sim z (i) M ,z (j) T . Following Radford et al. (2021); Zhai et al. (2023), we optimize a cross-modal contrastive objective that increases s i for paired meteorological and textual representations while decreasing s ij for unmatched pairs iĢø= j. 4.2TASK-SPECIFIC INFERENCE Cross-modal retrieval. Given a retrieval set of N paired samples (x (i) M ,x (i) T ) N i=1 , MeteoLIP encodes each meteorological and textual input into shared representations z (i) M and z (i) T , respectively. We then compute the cosine similarity between the two modalities as s ij = sim z (i) M ,z (j) T . For T2M retrieval, we rank meteorological candidates by their similarity to a textual query, and reverse the direction for M2T retrieval. Classification. We construct K textual promptsp k (c) K k=1 for each class cāC and apply prompt ensembling to obtain a class prototype z (c) T = 1 K P K k=1 F T (T T (p k (c))). Given a meteorological representation z M , we predict its class as Ėc = arg max cāC sim z M , z (c) T . Physical variable estimation. Inspired by Christensen et al. (2024); Maani et al. (2026), we esti- mate cyclone maximum sustained wind speed and minimum central pressure from x M using value- conditioned text prototypes. We begin by constructing K = 80 textual templates p k (v) K k=1 , such as āa cyclone with sustained wind speeds of v knotsā for each candidate value v ā V , where V =v min ,v min +1,...,v max , and obtain a value prototypez (v) T = 1 K P K k=1 F T (T T (p k (v))). We then compute the similarity score for each v as s v (z M ) = sim z M , z (v) T . LetS Ļ (x M ) denote the top-Ļ candidate values by score, where Ļ is the nearest odd integer to 0.05|V|. For robust estimation, we take the median ofS Ļ (x M ) as the final estimate. 6 Preprint 5EXPERIMENTS AND RESULTS 5.1EXPERIMENTAL SETUP Meteorological data processing. We use the 63 ERA5 meteorological variables listed in Ap- pendix D. We use the specified time for records with an exact timestamp, while we sample a timestamp within the reported interval for records spanning a time range. We standardize each meteorological input to 56Ć 56 grid cells. For samples spanning less than 14 ā¦ Ć 14 ā¦ , we expand the bounding box to this extent. For larger samples, we construct a square region using the larger latitudinal or longitudinal extent and linearly interpolate it to 56Ć 56 to avoid introducing extremes. We also normalize each variable using its mean and standard deviation computed across samples. Dataset split. We employ a year-based train-test split to assess whether the learned representa- tions remain effective across years, using 2000ā2010 and 2014ā2024 for training and withholding 2011ā2013 for evaluation on unseen intermediate years. This results in 490,395 training samples, with 63,343, 48,645, 7,353, and 8,114 test samples for retrieval, classification, cyclone maximum sustained wind speed estimation, and cyclone minimum central pressure estimation, respectively. MeteoLIP and training configuration. We investigate two MeteoLIP variants based on the CLIP (Radford et al., 2021) and SigLIP (Zhai et al., 2023) architectures and their respective learning objectives, denoted as Meteo-CLIP and Meteo-SigLIP. We initialize the encoders from the corre- sponding OpenAI CLIP and SigLIP WebLI checkpoints and use a text context length of 77 tokens. For the meteorology encoder, we use a patch size of 4Ć 4, which results in (56/4) 2 = 196 spatial patches. Unless otherwise specified, we train all models for 30 epochs on 4ĆA100 (40GB) GPUs with a batch size of 256 per GPU, using the AdamW optimizer with a learning rate of 1Ć 10 ā4 and weight decay of 1Ć 10 ā2 . We also apply a cosine learning-rate schedule with 2,000 warmup steps. We report the mean and standard deviation over three training runs using the final epoch checkpoint. Baselines. For the CLIP and SigLIP baselines, following the multichannel patchification used in FourCastNet (Pathak et al., 2022), we concatenate all meteorological variables along the channel dimension and jointly project each spatial patch using a linear layer, producing one token per patch. To follow the original image-text pair formulation of CLIP and SigLIP, we train the baselines using only the natural-event-focused caption, which identifies the natural phenomenon together with its temporal and geospatial context when available. 5.2MAIN RESULTS Retrieval. To evaluate meteorology-language alignment, we first assess bidirectional cross-modal retrieval. As shown in Table 1a, Meteo-CLIP and Meteo-SigLIP improve over their corresponding CLIP and SigLIP baselines across all retrieval metrics. Meteo-CLIP improves M2T and T2M R@1 from 20.25 and 20.03 to 58.79 and 60.10 respectively, while Meteo-SigLIP achieves the best perfor- mance with 66.07 and 66.79. Similar improvements are observed at R@5 and R@10, demonstrating strong meteorology-language alignment. Since the default baselines use a single caption style, we further examine the effects of multi-style supervision and MetTok conditioning in Section 5.3. Classification. Table 1b evaluates category-level meteorological representations under two pro- tocols: text-prototype inference, which applies the text-based inference described in Section 4.2 without task-specific post-training, and linear probing, which trains a linear prediction head on top of the frozen meteorology encoder. Under text-prototype inference, Meteo-CLIP improves F1 from 30.15 to 47.91 and Top-1 accuracy from 33.69 to 57.69, while Meteo-SigLIP improves from 21.56 to 50.19 and Top-1 accuracy from 22.13 to 55.59. The improvements can also be observed under linear probing, where Meteo-SigLIP achieves the highest F1 of 80.75 and Top-1 accuracy of 92.35. These results indicate that MeteoLIP improves both language-based inference and the category- level semantic information encoded in the frozen meteorology representations. We provide detailed category-wise results in Appendix F.1, where drought, heat wave, and cold wave are identified as the most challenging categories. Physical variable estimation. As provided in Table 1c, under text-prototype inference, Meteo-CLIP improves R 2 for cyclone maximum sustained wind speed from 5.29 to 62.27 and for minimum cen- tral pressure from 26.81 to 67.64, while Meteo-SigLIP improves them from ā11.59 to 60.17 and from 45.12 to 73.39, respectively. Under linear probing, Meteo-SigLIP achieves the best perfor- 7 Preprint Table 1: Comparison of CLIP and SigLIP baselines with their MeteoLIP counterparts. (a) Cross-modal retrieval. (b) Classification under text-prototype inference and linear probing. (c) Phys- ical variable estimation of cyclone maximum sustained wind speed and minimum central pressure under the same two protocols. Bold and underline indicate the best and second-best results. (a) Meteorology-to-TextText-to-Meteorology ModelR@1āR@5āR@10āR@1āR@5āR@10ā CLIP20.25Ā± 0.13 45.67Ā± 0.19 58.25Ā± 0.27 20.03Ā± 0.14 45.05Ā± 0.39 57.43Ā± 0.47 Meteo-CLIP58.79Ā± 1.4388.13Ā± 0.6094.09Ā± 0.3260.10Ā± 0.2387.68Ā± 0.1393.50Ā± 0.12 SigLIP32.06Ā± 0.56 61.11Ā± 0.75 72.07Ā± 0.69 30.23Ā± 0.22 58.12Ā± 0.25 68.98Ā± 0.25 Meteo-SigLIP66.07Ā± 0.6491.77Ā± 0.2996.14Ā± 0.1566.79Ā± 0.5091.27Ā± 0.3195.62Ā± 0.18 (b) Text-Prototype InferenceLinear Probing ModelF1āTop-1 Acc. āTop-3 Acc. āF1āTop-1 Acc. āTop-3 Acc. ā CLIP30.15Ā± 2.83 33.69Ā± 3.10 66.95Ā± 2.53 75.04Ā± 0.20 90.61Ā± 0.05 98.62Ā± 0.06 Meteo-CLIP47.91Ā± 2.1057.69Ā± 1.4385.98Ā± 0.9779.98Ā± 0.2692.16Ā± 0.1198.82Ā± 0.05 SigLIP21.56Ā± 1.48 22.13Ā± 2.95 50.15Ā± 3.52 76.30Ā± 0.54 91.06Ā± 0.08 98.61Ā± 0.05 Meteo-SigLIP50.19Ā± 2.9555.59Ā± 5.3182.29Ā± 3.6380.75Ā± 0.1492.35Ā± 0.0498.78Ā± 0.04 (c) Text-Prototype InferenceLinear Probing VariableModelR 2 āMAEāRMSEāR 2 āMAEāRMSEā Sustained Wind CLIP5.29Ā± 19.85 14.62Ā± 1.38 21.38Ā± 2.24 68.09Ā± 1.04 9.42Ā± 0.17 12.46Ā± 0.20 Meteo-CLIP62.27Ā± 4.759.35Ā± 0.4213.53Ā± 0.8471.34Ā± 0.518.65Ā± 0.0811.80Ā± 0.10 SigLIP ā11.59Ā± 16.49 14.46Ā± 0.88 23.25Ā± 1.72 70.52Ā± 0.48 8.91Ā± 0.11 11.97Ā± 0.10 Meteo-SigLIP60.17Ā± 1.739.32Ā± 0.2113.91Ā± 0.3071.85Ā± 0.378.50Ā± 0.0411.70Ā± 0.08 Central Pressure CLIP26.81Ā± 14.46 9.70Ā± 0.80 16.54Ā± 1.60 74.08Ā± 0.82 7.04Ā± 0.11 9.87Ā± 0.16 Meteo-CLIP67.64Ā± 4.236.37Ā± 0.2211.01Ā± 0.7276.36Ā± 0.466.41Ā± 0.099.43Ā± 0.09 SigLIP45.12Ā± 3.00 7.96Ā± 0.20 14.36Ā± 0.39 75.98Ā± 0.35 6.60Ā± 0.06 9.50Ā± 0.07 Meteo-SigLIP73.39Ā± 1.615.96Ā± 0.0910.00Ā± 0.3077.36Ā± 0.456.16Ā± 0.099.23Ā± 0.09 mance for both variables, with R 2 values of 71.85 for maximum sustained wind speed and 77.36 for minimum central pressure. We further compare against physical baselines derived directly from ERA5 in Appendix F.2. These results demonstrate that the learned meteorology-language represen- tations are capable of capturing physically meaningful quantitative information. 5.3ABLATION STUDIES Caption supervision. We study the effects of caption style for retrieval and multi-style supervision. Table 2 shows that the natural-event-focused caption results in the highest retrieval performance, achieving 58.79 M2T and 60.12 T2M R@1. In comparison, the other caption styles yield sub- stantially lower retrieval performance, suggesting that explicitly describing the natural phenomenon together with its temporal and geospatial context provides more instance-discriminative informa- tion for retrieval. Table 3 further demonstrates the benefit of multi-style supervision during train- ing. Increasing from one to four caption styles improves both model architectures: CLIP increases from 20.25/20.03 to 31.64/31.10 M2T/T2M R@1, while Meteo-CLIP improves from 34.24/55.55 to 58.79/60.10. Moreover, Meteo-CLIP consistently outperforms CLIP under both one-style and four- style training, demonstrating its effectiveness across different levels of caption-style supervision. Model initialization. As shown in Table 4, initializing from the OpenAI CLIP checkpoint improves M2T R@1 from 34.46 to 58.79 and T2M R@1 from 41.43 to 60.10 compared with training from scratch. This suggests that, despite the modality gap between meteorological data and RGB im- ages, pretrained visual representations provide a useful initialization for learning spatial structure in meteorological fields. One possible explanation is that some meteorological variables can reflect co- herent spatial patterns associated with natural phenomena such as cyclones and atmospheric rivers, which can benefit from the spatial pattern recognition capabilities of pretrained image encoders. MetTok conditioning. Table 5 highlights the contribution of meteorological input and MetTokās conditioning components across retrieval, weather condition classification, and maximum sustained wind estimation. For retrieval, variable-identity embedding and geolocation provide relatively mod- est improvements, whereas temporal conditioning contributes substantially. Notably, using geolo- cation and time alone without meteorological input achieves 50.41/58.49 M2T/T2M R@1, showing that temporal and geographic metadata provide strong cues for matching natural event captions. 8 Preprint Table 2: Meteo-CLIP retrieval perfor- mance by caption style. Caption StyleM2T R@1āT2M R@1ā Natural Event Focused58.79Ā± 1.4460.12Ā± 0.25 Physical Framing21.18Ā± 0.18 21.67Ā± 0.08 Impact Focused19.09Ā± 0.17 18.55Ā± 0.08 Atmospheric Pattern 22.24Ā± 0.41 21.52Ā± 0.08 Table 3: Ablation on multi- style caption supervision during training. #: # Styles. Model#M2T R@1āT2M R@1ā 1 20.25Ā± 0.13 20.03Ā± 0.14 CLIP 4 31.64Ā± 0.24 31.10Ā± 0.05 1 34.24Ā± 8.52 55.55Ā± 4.57 Meteo-CLIP 458.79Ā± 1.4360.10Ā± 0.23 Table 4: Ablation on model initialization for Meteo-CLIP training. Init.M2T R@1āT2M R@1ā Scratch 34.46Ā± 1.78 41.43Ā± 2.21 OpenAI58.79Ā± 1.4360.10Ā± 0.23 Table 5: Ablation of MetTok components. We train Meteo-CLIP under different MetTok configurations to assess the contribution of meteorological input, variable-identity embedding, geoloca- tion, and temporal conditioning. Bold and underlined values in- dicate the best and second-best results, respectively. Input & EncodingRetrievalWeather ConditionSustained Wind Met. Var. ID Geo. TimeM2T R@1āT2M R@1āF1āAcc. āR 2 āMAEā āā 32.02Ā± 0.33 31.74Ā± 0.19 47.04Ā± 3.67 56.01Ā± 6.4952.66Ā± 5.20 10.33Ā± 0.40 āā 32.44Ā± 0.34 32.71Ā± 0.21 48.17Ā± 1.56 57.23Ā± 2.3246.74Ā± 13.37 10.44Ā± 0.67 āā 50.41Ā± 2.43 58.49Ā± 1.22 32.64Ā± 4.30 45.12Ā± 11.06 ā155.14Ā± 34.45 25.75Ā± 1.93 āā 33.21Ā± 0.69 32.83Ā± 0.44 47.25Ā± 1.41 54.93Ā± 3.20 62.51Ā± 3.87 9.43Ā± 0.15 āāā 57.30Ā± 1.6559.17Ā± 0.4441.27Ā± 1.19 47.47Ā± 4.8859.82Ā± 4.569.79Ā± 0.47 ā58.79Ā± 1.4360.10Ā± 0.2347.91Ā± 2.1057.69Ā± 1.4362.27Ā± 4.759.35Ā± 0.42 Table 6: Retrieval cue analysis. M2T R@1 of Full MetTok and metadata only using original and time-removed captions under all test and same month candidate pools. Candidate PoolCaptionFull MetTokMetadata onlyRel. Gap All test samples Original58.79Ā± 1.43 50.41Ā± 2.4314.3% Time removed 15.41Ā± 0.02 6.57Ā± 0.1857.4% Same month Original61.84Ā± 2.48 53.02Ā± 3.4114.3% Time removed 38.65Ā± 4.06 23.26Ā± 2.9939.8% However, this metadata-only configuration achieves substantially lower performance on weather condition classification and fails to provide useful sustained wind estimates, achieving 32.64 F1 and an R 2 of ā155.14, compared with 47.91 F1 and 62.27 R 2 for the full MetTok configuration. To further examine the role of metadata in retrieval, Table 6 compares the full MetTok configuration with metadata only using original and time-removed captions under different candidate pools. With the original captions, the relative gap between the two configurations is 14.3% under both candidate pools. When temporal information is removed from the captions, the gap increases substantially: the full configuration achieves 15.41 M2T R@1 compared with 6.57 using metadata only over all test samples, and 38.65 compared with 23.26 when candidates are restricted to the same calendar month. Qualitative examples in Figure 7 further show that the full configuration retrieves meteorologically related events, whereas using metadata alone can retrieve unrelated phenomena. These results and analysis highlight the importance of meteorological input for semantic and physical inference and show that it also contributes substantially to retrieval when direct temporal cues are removed. Training data scaling. We study the effect of training data scale using 10%, 25%, 50%, and 100% of the training set. Figure 4 demonstrates that performance generally improves across all tasks with more training data, with the full training set achieving the best performance. The improvements are particularly notable for retrieval and maximum sustained wind speed estimation, while classifica- tion and minimum central pressure estimation also benefit from increased data scale. These results suggest that the model can further benefit from additional meteorology-language supervision. Source-held-out generalization. We study source-level general- ization by training Meteo-CLIP while simultaneously removing AusDE, CanDE, FraDE, IndDE, and MexDE from training, and then evaluating retrieval on the test sets of the five held-out record sources. Table 7 shows that the decrease in retrieval performance varies substantially across sources, which may reflect differences in source-specific reporting criteria, local terminology, and spatial granularity. These results highlight the importance of local record sources for meteorology-language alignment, while showing that cross-source generalization remains challenging. Table 7: Source-held-out M2T R1; mean over 3 runs. Training Data Test DataFullHeld-Out AusDE (n=27) 80.2 72.8 (-7.4) CanDE (n=47) 64.5 63.1 (-1.4) FraDE (n=966) 24.0 10.6 (-13.4) IndDE (n=3,835) 69.6 13.1 (-56.5) MexDE (n=336) 48.1 37.5 (-10.6) 6CONCLUSION We present EarthAlign, a large-scale Earth systemālanguage dataset that pairs meteorological data with multi-style natural language descriptions of diverse natural phenomena and normal-weather conditions. Alongside EarthAlign, we develop MeteoLIP to learn language-aligned representa- 9 Preprint tions of high-dimensional meteorological data while incorporating variable-specific, temporal, and geospatial information. Across cross-modal retrieval, weather-condition classification, and phys- ical variable estimation, MeteoLIP improves over the corresponding CLIP and SigLIP baselines. Our analyses further demonstrate the importance of meteorological input, temporal and geospatial conditioning, multi-style language supervision, and diverse training data for effective meteorology- language alignment. We believe EarthAlign can provide a useful foundation for advancing language- aligned multimodal representation learning of Earth system data. ACKNOWLEDGMENTS This material is based on work supported by ADIA Lab, Abu Dhabi under a collaborative research project titled Climate Co-Pilot: A Multimodal AI Agent for Climate Resilience in the MENA Re- gion. AI USE STATEMENT In this work, we used generative AI tools to generate captions from available metadata, assist with coding for experiments and visualizations, help identify record data sources and relevant literature, assist with translation when working with source-specific records in Mexico, France, and Indonesia, and provide writing assistance to improve clarity and readability. We have reviewed the AI-assisted work. For example, a subset of the generated captions was manually audited for consistency with the underlying metadata across the record data sources; AI-assisted code was manually inspected and executed; identified record data sources and literature were independently verified; and AI-assisted writing refinements were manually reviewed and further revised. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI. REPRODUCIBILITY STATEMENT The preprocessed data,source code,and model weights are publicly available at https://github.com/mbzuai-oryx/EarthAlign.The distribution of EarthAlign is reported in Figure 1 and Appendix E, with representative examples illustrated in Figure 5.The overall dataset construction is described in Section 3.1, while Appendices B, C, and D provide further details on source-specific record processing, grounded multi-style caption generation, and ERA5 meteorological data alignment and preprocessing, respectively. MeteoLIP model configurations and training hyperparameters are provided in Section 5.1. We report experimental results as mean and standard deviation across independent runs to account for variation across training runs. REFERENCES Australian Government, Attorney-Generalās Department. Disaster events with category impact and location. data.gov.au, 2023. URL https://data.gov.au/data/dataset/disaste r-events-with-category-impact-and-location. Accessed May 2026. Badan Nasional Penanggulangan Bencana. Data informasi bencana indonesia (DIBI). Badan Na- sional Penanggulangan Bencana, 2026. URL https://dibi.bnpb.go.id/superset/d ashboard/2/. Accessed May 2026. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qi- uyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Hu- men Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium- range global weather forecasting with 3d neural networks. Nature, 619(7970):533ā538, 2023. 10 Preprint Cristian Bodnar, Wessel P. Bruinsma, Ana Lucic, Megan Stanley, Anna Allen, Johannes Brandstet- ter, Patrick Garvan, Maik Riechert, Jonathan A. Weyn, Haiyu Dong, Jayesh K. Gupta, Kit Tham- biratnam, Alexander T. Archibald, Chun-Chieh Wu, Elizabeth Heider, Max Welling, Richard E. Turner, and Paris Perdikaris. A foundation model for the earth system. Nature, 641(8065): 1180ā1187, May 2025.ISSN 1476-4687.doi: 1 0 . 1 0 3 8 / s 4 1 5 8 6 - 0 2 5 - 0 9 0 0 5 -y.URL https://doi.org/10.1038/s41586-025-09005-y. Guy P. Brasseur and Laura Gallardo. Climate services: Lessons learned and future prospects. Earthās Future, 4(3):79ā89, 2016. doi: https://doi.org/10.1002/2015EF000338. URL https://agup ubs.onlinelibrary.wiley.com/doi/abs/10.1002/2015EF000338. Centro Nacional de Prevenci Ģ on de Desastres (CENAPRED). Declaratorias sobre emergencia, desas- tre y contingencia climatol Ģ ogica a nivel municipal entre 2000 y 2023. Datos Abiertos de M Ģ exico, Gobierno de M Ģ exico, 2023. URL https://historico.datos.gob.mx/busca/dat aset/declaratorias-sobre-emergencia-desastre-y-contingencia-cli matologica/resource/946ad8a6-abe4-4ba8-b96b-7a45bfe687ea. Accessed May 2026. Jian Chen, Peilin Zhou, Yining Hua, Dading Chong, Meng Cao, Yaowei Li, Wei Chen, Bing Zhu, Junwei Liang, and Zixuan Yuan. Climateiqa: A new dataset and benchmark to advance vision- language models in meteorology anomalies analysis. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD ā25, p. 5322ā5333, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400714542. doi: 10.1145/37 11896.3737406. URL https://doi.org/10.1145/3711896.3737406. Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 24185ā24198, 2024. Matthew Christensen, Milos Vukadinovic, Neal Yuan, and David Ouyang. Visionālanguage foun- dation model for echocardiogram interpretation. Nature Medicine, 30(5):1481ā1488, May 2024. ISSN 1546-170X. doi: 10.1038/s41591-024-02959-y. URL https://doi.org/10.1038/ s41591-024-02959-y. Damien Delforge, Valentin Wathelet, Regina Below, Cinzia Lanfredi Sofia, Margo Tonnelier, Joris A.F. van Loenhout, and Niko Speybroeck. Em-dat: the emergency events database. In- ternational Journal of Disaster Risk Reduction, 124:105509, 2025. ISSN 2212-4209. doi: https://doi.org/10.1016/j.ijdrr.2025.105509. URL https://w.sciencedirect. com/science/article/pii/S2212420925003334. J. Gahtan, K. R. Knapp, C. J. Schreck, H. J. Diamond, J. P. Kossin, and M. C. Kruk. International best track archive for climate stewardship (ibtracs) project, version 4r01, 2024. URL https: //w.ncei.noaa.gov/products/international-best-track-archive. since1980; accessed March 2026. Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun. Rotary position embedding for vision transformer. In European Conference on Computer Vision (ECCV), 2024. Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, Andr Ģ as Hor Ģ anyi, Joaqu Ģ ın Mu Ģ noz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, Adrian Simmons, Cornel Soci, Saleh Abdalla, Xavier Abellan, Gianpaolo Balsamo, Peter Bechtold, Gionata Biavati, Jean Bidlot, Massimo Bonavita, Giovanna De Chiara, Per Dahlgren, Dick Dee, Michail Diamantakis, Rossana Dragani, Johannes Flemming, Richard Forbes, Manuel Fuentes, Alan Geer, Leo Haimberger, Sean Healy, Robin J. Hogan, El Ģ ıas H Ģ olm, Marta Janiskov Ģ a, Sarah Keeley, Patrick Laloyaux, Philippe Lopez, Cristina Lupu, Gabor Radnoti, Patricia de Rosnay, Iryna Rozum, Freja Vamborg, Sebastien Villaume, and Jean-No Ģ el Th Ģ epaut. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999ā2049, 2020. doi: https://doi.org/10.1002/qj.3803. URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.380 3. 11 Preprint Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL https://openreview.net/forum?i d=nZeVKeeFYf9. Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Car- reira. Perceiver: General perception with iterative attention. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, p. 4651ā4664. PMLR, 18ā24 Jul 2021. URL https://proceedings.mlr.press/v139/jaegle21a.html. Caroline S. Juang, Thomas A. Stanley, and Dalia B. Kirschbaum. Using citizen science to expand the global map of landslides: Introducing the cooperative open online landslide repository (coolr). PLOS ONE, 14(7):1ā28, 07 2019. doi: 10.1371/journal.pone.0218657. URL https://doi. org/10.1371/journal.pone.0218657. Muhammad Uzair Khattak, Shahina Kunhimon, Muzammal Naseer, Salman Khan, and Fahad Shah- baz Khan. Unimed-clip: Towards a unified image-text pretraining paradigm for diverse medical imaging modalities, 2024. URL https://arxiv.org/abs/2412.10372. Kenneth R. Knapp, Michael C. Kruk, David H. Levinson, Howard J. Diamond, and Charles J. Neu- mann. The international best track archive for climate stewardship (ibtracs): Unifying tropical cyclone best track data. Bulletin of the American Meteorological Society, 91(3):363ā376, 2010. doi: 10.1175/2009BAMS2755.1. Christian Lessig, Ilaria Luise, Bing Gong, Michael Langguth, Scarlet Stadtler, and Martin Schultz. Atmorep: A stochastic model of atmosphere dynamics using large scale representation learning, 2023. URL https://arxiv.org/abs/2308.13280. Haobo Li, Zhaowei Wang, Jiachen Wang, Yueya Wang, Alexis Kai Hon Lau, and Huamin Qu. CLLMate: A multimodal benchmark for weather and climate events forecasting. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 17536ā17562, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176- 332-6. doi: 10.18653/v1/2025.emnlp-main.886. URL https://aclanthology.org/202 5.emnlp-main.886/. Fan Liu, Delong Chen, Zhangqingyun Guan, Xiaocong Zhou, Jiale Zhu, Qiaolin Ye, Liyong Fu, and Jun Zhou. Remoteclip: A vision language foundation model for remote sensing. IEEE Transactions on Geoscience and Remote Sensing, 62:1ā16, 2024. doi: 10.1109/TGRS.2024.33 90838. URL https://doi.org/10.1109/TGRS.2024.3390838. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. Chengqian Ma, Zhanxiang Hua, Alexandra Anderson-Frey, Vikram Iyer, Xin Liu, and Lianhui Qin. Weatherqa: Can multimodal language models reason about severe weather?, 2024. URL https: //arxiv.org/abs/2406.11217. Fadillah Maani, Numan Saeed, Tausifa Jan Saleem, Zaid Farooq, Hussain Alasmawi, Werner Diehl, Ameera Mohammad, Gareth Waring, Saudabi Valappil, Leanne Bricker, and Mohammad Yaqub. Fetalclip: a visual-language foundation model for fetal ultrasound image analysis. npj Digital Medicine, Jun 2026. ISSN 2398-6352. doi: 10.1038/s41746- 026- 02907-9. URL https: //doi.org/10.1038/s41746-026-02907-9. Minist ` ere de la Transition Ģ ecologique. Base nationale de gestion ASsist Ģ e des proc Ģ edures adminis- tratives relatives aux risques (GASPAR). data.gouv.fr, 2026. URL https://w.data.gou v.fr/datasets/base-nationale-de-gestion-assistee-des-procedure s-administratives-relatives-aux-risques-gaspar. Accessed May 2026. Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh K Gupta, and Aditya Grover. Climax: A foundation model for weather and climate. arXiv preprint arXiv:2301.10343, 2023. 12 Preprint NOAA NCEI. Storm events database. https://w.ncei.noaa.gov/stormevents/. Accessed March 2026. Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar. Fourcastnet: A global data- driven high-resolution weather model using adaptive fourier neural operators, 2022. URL http s://arxiv.org/abs/2202.11214. Public Safety Canada. Canadian disaster database (cdd). Government of Canada Open Government Portal, 2026. URL https://open.canada.ca/data/en/dataset/1c3d15f9-9cf a-4010-8462-0d67e493d9b9. Accessed May 2026. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Interna- tional Conference on Machine Learning, 2021. URL https://api.semanticscholar. org/CorpusID:231591445. Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Russell, Alvaro Sanchez-Gonzalez, Vivian Yang, Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sha. Weatherbench 2: A benchmark for the next generation of data-driven global weather models. Journal of Advances in Modeling Earth Systems, 16(6):e2023MS004019, 2024. doi: https://doi.org/10.1029/2023MS004019. URL https://agupubs.onlinelibrary.wi ley.com/doi/abs/10.1029/2023MS004019. Daniel Runfola, Austin Anderson, Heather Baier, Matt Crittenden, Elizabeth Dowker, Sydney Fuhrig, Seth Goodman, Grace Grimsley, Rachel Layko, Graham Melville, Maddy Mulder, Rachel Oberman, Joshua Panganiban, Andrew Peck, Leigh Seitz, Sylvia Shea, Hannah Slevin, Rebecca Youngerman, and Lauren Hobbs. geoboundaries: A global database of political administrative boundaries. PLOS ONE, 15(4):1ā9, 04 2020. doi: 10.1371/journal.pone.0231866. URL https://doi.org/10.1371/journal.pone.0231866. Johannes Schmude, Sujit Roy, Will Trojak, Johannes Jakubik, Daniel Salles Civitarese, Shraddha Singh, Julian Kuehnert, Kumar Ankur, Aman Gupta, Christopher E Phillips, Romeo Kienzler, Daniela Szwarcman, Vishal Gaur, Rajat Shinde, Rohit Lal, Arlindo Da Silva, Jorge Luis Guevara Diaz, Anne Jones, Simon Pfreundschuh, Amy Lin, Aditi Sheshadri, Udaysankar Nair, Valentine Anantharaj, Hendrik Hamann, Campbell Watson, Manil Maskey, Tsengdar J Lee, Juan Bernabe Moreno, and Rahul Ramachandran. Prithvi wxc: Foundation model for weather and climate, 2024. URL https://arxiv.org/abs/2409.13598. Andrew Sellergren, Sahar Kazemzadeh, Tiam Jaroensri, Atilla Kiraly, Madeleine Traverse, Timo Kohlberger, Shawn Xu, Fayaz Jamil, C Ģ ıan Hughes, Charles Lau, Justin Chen, Fereshteh Mahvar, Liron Yatziv, Tiffany Chen, Bram Sterling, Stefanie Anna Baby, Susanna Maria Baby, Jeremy Lai, Samuel Schmidgall, Lu Yang, Kejia Chen, Per Bjornsson, Shashir Reddy, Ryan Brush, Ken- neth Philbrick, Mercy Asiedu, Ines Mezerreg, Howard Hu, Howard Yang, Richa Tiwari, Sunny Jansen, Preeti Singh, Yun Liu, Shekoofeh Azizi, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram Ģ e, Morgane Riviere, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Elena Buchatskaya, Jean-Baptiste Alayrac, Dmitry Lepikhin, Vlad Feinberg, Sebastian Borgeaud, Alek Andreev, Cassidy Hardin, Robert Dadashi, L Ģ eonard Hussenot, Armand Joulin, Olivier Bachem, Yossi Matias, Katherine Chou, Avinatan Hassidim, Kavi Goel, Clement Farabet, Joelle Barral, Tris Warkentin, Jonathon Shlens, David Fleet, Vic- tor Cotruta, Omar Sanseviero, Gus Martins, Phoebe Kirk, Anand Rao, Shravya Shetty, David F. Steiner, Can Kirmizibayrak, Rory Pilgrim, Daniel Golden, and Lin Yang. Medgemma technical report, 2026. URL https://arxiv.org/abs/2507.05201. Akashah Shabbir, Mohammed Zumri, Mohammed Bennamoun, Fahad Shahbaz Khan, and Salman Khan. Geopixel: Pixel grounding large multimodal model in remote sensing. In Forty-second International Conference on Machine Learning, ICMLā25, 2025. 13 Preprint Sagar Soni, Akshay Dudhane, Hiyam Debary, Mustansar Fiaz, Muhammad Akhtar Munir, Muham- mad Sohail Danish, Paolo Fraccaro, Campbell D Watson, Levente J Klein, Fahad Shahbaz Khan, and Salman Khan. Earthdial: Turning multi-sensory earth observations to interactive dialogues. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14303ā 14313, 2025. doi: 10.1109/CVPR52734.2025.01334. Shuo Tang, Jian Xu, Jiadong Zhang, Yi Chen, Qizhao Jin, Lingdong Shen, Chenglin Liu, and Shim- ing Xiang. Meteorpred: A meteorological multimodal large model and dataset for severe weather event prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 22910ā22919, June 2026. S. M. Vallejo-Bernal, T. Braun, N. Marwan, and J. Kurths. Pikart: A comprehensive global catalog of atmospheric rivers. Journal of Geophysical Research: Atmospheres, 130(15):e2024JD041869, 2025. doi: https://doi.org/10.1029/2024JD041869. URL https://agupubs.onlinelibr ary.wiley.com/doi/abs/10.1029/2024JD041869. Sumanth Varambally, Veeramakali Vignesh Manivannan, Yasaman Jafari, Luyu Han, Zachary No- vack, Zhirui Xia, Salva R Ģ uhling Cachay, Srikar Eranky, Ruijia Niu, Taylor Berg-Kirkpatrick, Duncan Watson-Parris, Yian Ma, and Rose Yu. Aquilon: Towards building multimodal weather LLMs. In ICML 2025 Workshop on Assessing World Models, 2025. URL https://openre view.net/forum?id=KVxOwEYAF4. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. Internvl3.5: Advancing open-source multimodal models in versatil- ity, reasoning, and efficiency, 2025. URL https://arxiv.org/abs/2508.18265. Christopher J. White, Henrik Carlsen, Andrew W. Robertson, Richard J.T. Klein, Jeffrey K. Lazo, Arun Kumar, Frederic Vitart, Erin Coughlan de Perez, Andrea J. Ray, Virginia Murray, Sukaina Bharwani, Dave MacLeod, Rachel James, Lora Fleming, Andrew P. Morse, Bernd Eggen, Richard Graham, Erik Kjellstr Ģ om, Emily Becker, Kathleen V. Pegion, Neil J. Holbrook, Darryn McEvoy, Michael Depledge, Sarah Perkins-Kirkpatrick, Timothy J. Brown, Roger Street, Lindsey Jones, Tomas A. Remenyi, Indi Hodgson-Johnston, Carlo Buontempo, Rob Lamb, Holger Meinke, Berit Arheimer, and Stephen E. Zebiak. Potential applications of subseasonal-to-seasonal (s2s) predic- tions. Meteorological Applications, 24(3):315ā325, 2017. doi: https://doi.org/10.1002/met.1654. URL https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/met.16 54. WMO. WMO Guidelines on the Calculation of Climate Normals. World Meteorological Organiza- tion, Geneva, Switzerland, 2017. URL https://library.wmo.int/idurl/4/55797. WMO-No. 1203. Zhitong Xiong, Yi Wang, Weikang Yu, Adam J Stewart, Jie Zhao, Nils Lehmann, Thomas Dujardin, Zhenghang Yuan, Pedram Ghamisi, and Xiao Xiang Zhu. Dofa-clip: Multimodal vision-language foundation models for earth observation, 2025. URL https://arxiv.org/abs/2503.0 6312. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 11975ā11986, October 2023. Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Pre- ston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, Andrea Tupini, Yu Wang, Matt Mazzola, Swadheen Shukla, Lars Liden, Jianfeng Gao, Angela Crabtree, Brian Piening, Carlo Bifulco, 14 Preprint Matthew P. Lungren, Tristan Naumann, Sheng Wang, and Hoifung Poon. A multimodal biomedi- cal foundation model trained from fifteen million imageātext pairs. NEJM AI, 2(1):AIoa2400640, 2025. doi: 10.1056/AIoa2400640. URL https://ai.nejm.org/doi/full/10.1056 /AIoa2400640. Xuebin Zhang, Gabriele Hegerl, Francis W. Zwiers, and Jesse Kenyon. Avoiding inhomogeneity in percentile-based indices of temperature extremes. Journal of Climate, 18:1641ā-1651, 2005. doi: 10.1175/JCLI3366.1. URL https://doi.org/10.1175/JCLI3366.1. 15 Preprint APPENDIX Table 8: Comparison of recent existing multimodal weather and climate datasets by temporal and geographic coverage, evidence source, natural-phenomenon types, dataset size, and evaluation tasks. DatasetYearsGeo. Scope Evidence Source# NP Types SizeTasks WeatherQA2014ā2020 CONUSExpert forecast analyses 78,511Severe weather MC-VQA and mesoscale case study generation ClimateIQA2023GlobalMeteorological heatmaps 326.3k img. (762.1k QA) Meteorological anomaly VQA and description generation Aquilon1979ā2022 GlobalExpert-curated records & ERA5 N/R444,244Weather analysis, forecasting, and extreme event VQA CLLMate2015ā2023 GlobalExpert-verified news137,747Weather and climate phenomena and conse- quences MC-VQA MeteorPred2023ā2024 ChinaAgency warnings7421,363Severe weather VQA, MC-VQA, and structured generation tasks EarthAlign2000ā2024GlobalExpert-curated records & ERA5 93553,738VL alignment: retrieval, classification, and physical variable estimation NP: Natural phenomena. N/R: not reported or unknown. VQA: visual question answering. MC: multiple-choice. VL: vision-language. 102550100 35 40 45 50 55 60 65 M2T R@1 102550100 35 40 45 50 55 60 65 T2M R@1 102550100 35.0 37.5 40.0 42.5 45.0 47.5 50.0 F1 102550100 ā100 ā50 0 50 100 Wind R 2 102550100 20 30 40 50 60 70 80 Pressure R 2 Score (%) Training data (%) Figure 4: Effect of training data scale. We train Meteo-CLIP with 10%, 25%, 50%, and 100% of the training data and evaluate model performance across retrieval, classification, and physical variable estimation. Error bars denote standard deviation over three training runs. 16 Preprint AQUALITATIVE EXAMPLES S Between 29 May and 01 June 2021, heavy rains triggered general flooding across New Zealand, affecting Otago, the West Coast, Auckland, and the Canterbury Region. In July 2010, a cold wave brought extreme temperatures to Argentina, affecting numerous provinces including Buenos Aires, Cordoba, and Santa Fe. In March 2015, volcanic activity at Villarrica resulted in ash fall across multiple locations in Chile, including Pucon, Curarrehue, and Temuco. On 23 September 2013, excessive rains triggered a wet landslide in Caranavi, Bolivia. A rotating dust cloud, identified as a dust devil with estimated winds around 60 knots, moved through Happy Camp in Western Siskiyou County, California, on March 28, 2000. A slow-moving winter storm brought heavy snow and whiteout conditions to Edmonton, Alberta, and surrounding areas from March 21 to March 23, 2013. On 09 October 2000, a widespread frost and freeze affected southern Illinois between 08 UTC and 13 UTC. Severe weather and storms affected multiple regions of Venezuela, including Caracas, Miranda, and Zulia, from 24 to 27 October 2022. A drought triggered by a lack of rain affected Angola from March 2012 to May 2013. On 02 February 2000, a tornado accompanied by rain struck the town of Newcastle in KwaZulu-Natal, South Africa. From July to August 2003, a heat wave brought extreme temperatures to Slovenia. Between February 5 and February 7, 2011, two major bushfires broke out across metropolitan Perth, Western Australia, affecting the Perth Hills area. Between 15 April 2024 and 16 April 2024, heavy rains triggered flash flooding and storm conditions in Dubai, United Arab Emirates. Tropical Storm Jade impacted Madagascar between April 6 and April 7, 2009, reaching speeds of 160 km/h. On 02 September 2009, a 7.0 moment magnitude earthquake struck Indonesia, affecting areas across Jawa Barat, Jawa Tengah, and DKI Jakarta. From May to October 2015, an El Nino-driven drought affected numerous districts across Papua New Guinea. Between 12 and 17 October 2019, Tropical Cyclone Hagibis struck Japan, bringing 160 km/h winds to multiple prefectures including Tokyo, Nagano, and Fukushima. On 28 November 2007, a coastal flood affected several provinces in the Philippines, including Pangasinan, Antique, and Zamboanga Del Sur. A flash flood occurred in Karkaraly, Karaganda Region, Kazakhstan, spanning from 08 April to 12 April 2015. On 13 November 2015, a wet landslide occurred in Lindong, Zhejiang, China, following a flood. Example: Tornado and Hail Location: [43.6, 44.0, -89.6, -88.9]Start Time: 00:00 UTC, 01 May 2010End Time: 01:00 UTC, 01 May 2010 Captions: Natural Event Focused On May 1, 2010, three weak EF0 and EF1 tornadoes and one- inch hail occurred in Green Lake and Marquette counties in Wisconsin. Physical Framing Two lines of thunderstorms developed across Wisconsin, with the second line generating three weak tornadoes and one-inch hail over Green Lake and Marquette counties. Impact Focused Three weak EF0 and EF1 tornadoes and one-inch diameter hail impacted Green Lake and Marquette counties as a line of thunderstorms moved through the area. Atmospheric Pattern Parts of Wisconsin were positioned in the warm sector of a mid-latitude cyclone with persistent southerly winds ahead of a cold front, where moderate CAPE and strong shear supported the development of thunderstorms. Example: Landslide Location: [-21.2, 48.0]Start Time: 15 March 2010End Time: 15 March 2010 Captions: Natural Event Focused On 15 March 2010, a complex landslide occurred in the Mananjary District of Vatovavy, Madagascar, triggered by Tropical Storm Hubert. Physical Framing Torrential rain from a tropical cyclone drenched south-east Madagascar, resulting in mudslides across the Mananjary District. Impact Focused The passage of Tropical Storm Hubert brought torrential rainfall to Madagascar, triggering complex mudslides in the Vatovavy region. Atmospheric Pattern A tropical cyclone named Hubert produced torrential rain over south-east Madagascar, creating the conditions for a complex landslide. Figure 5: Examples from EarthAlign. The global map illustrates geographically diverse natural- event records, while the bottom panels provide detailed examples of a tornado and hail event, and a landslide with their spatial and temporal metadata along with four grounded caption styles. 17 Preprint 120Ā°W118Ā°W116Ā°W114Ā°W112Ā°W110Ā°W 44Ā°N 46Ā°N 48Ā°N 50Ā°N 52Ā°N 54Ā°N 56Ā°N 1016 IVT direction 0100200300400 IVT Magnitude (kg m ā1 s ā1 ) Natural Event Focused 1. Between June 19 and June 28, 2013, a massive storm system caused significant flooding across southern Alberta, Canada. Torrential rain led to a mudslide and caused the Bow and Elbow Rivers to flood. ā Source: CanDE Ā· Ground truth 2. A riverine flood occurred in Calgary, Alberta, Canada, between 20 June 2013 and 27 June 2013. ā Source: EMāDAT 3. On 20 June 2013, a downpour triggered a mudslide near Mt. Norquay in Alberta, Canada, while heavy rains caused Cougar Creek in Canmore to overflow. ā Source: COOLR Physical Framing 1. Floodwaters covered an area of 307,926 square kilometers during a riverine flood in Calgary, Alberta, Canada. ā Source: EMāDAT 2. Riverine flooding persisted in Alberta, Canada, continuing until May 31, 2011. ā Source: EMāDAT 3. Heavy rain spanned a region from Canmore to Calgary, producing an average of 75 to 150 m over a three-day period. The storm system initially affected northern Alberta, where the Hanging Stone River breached its banks, before moving south and lingering over the southern part of the province. ā Source: CanDE Ā· Ground truth Impact Focused 1. Torrential rain from the storm system triggered a mudslide and caused severe river flooding in Alberta. The Hanging Stone River breached its banks in the north, while the Bow and Elbow Rivers flooded further south, alongside significant flooding in High River and Canmore. ā Source: CanDE Ā· Ground truth 2. A downpour in Alberta resulted in overflowing floodwaters from Cougar Creek and a mudslide in Improvement District No. 9. ā Source: COOLR 3. Riverine flooding impacted Calgary, Alberta, Canada. ā Source: EMāDAT Atmospheric Pattern 1. Riverine flooding conditions persisted in Calgary, Alberta, Canada, ending on 27 June 2013. ā Source: EMāDAT 2. A massive storm system crept through Alberta, moving south on June 19 and remaining over southern Alberta for three days, delivering 75 to 150 m of heavy rain. ā Source: CanDE Ā· Ground truth 3. Heavy rains and a downpour over Alberta, Canada, drove the destabilization of local creek banks and triggered a mudslide. ā Source: COOLR Query Image Top-3 Retrieval Texts (a) 16Ā°E18Ā°E20Ā°E22Ā°E24Ā°E26Ā°E28Ā°E 48Ā°N 50Ā°N 52Ā°N 54Ā°N 56Ā°N 58Ā°N 60Ā°N 1004 1012 10 m s ā1 wind 020406080 Probability (%) Cold Wave Winter Weather Frost High Wind Heat Wave ā® 73.6% 17.5% 4.9% ~0% ~0% -12.3-7.1-2.03.28.313.4 2m Temperature (Ā°C) (b) 134Ā°E136Ā°E138Ā°E140Ā°E142Ā°E144Ā°E 30Ā°N 32Ā°N 34Ā°N 36Ā°N 38Ā°N 40Ā°N 42Ā°N 988 996 1004 1012 20 m s ā1 wind 9209409609801000 Central Pressure (hPa) Similarity Groundtruth: 980.0 Prediction: 980.0 20406080100120 Maximum Sustained Wind (kt) Similarity Groundtruth: 55.0 Prediction: 55.0 980988996100410121020 Mean Sea-Level Pressure (hPa) (c) Figure 6: Qualitative results using the MeteoLIP meteorology and text encoders. (a) M2T re- trieval retrieves captions describing the same or closely related event across different caption styles. (b) The cold-wave example assigns the highest probability to cold wave, followed by winter weather and frost. (c) The similarity profiles over candidate physical values show peaks around the predicted minimum central pressure and maximum sustained wind speed. 18 Preprint Ground TruthFull MetTok Geo. + Time Only Example 1 Abushfirebroke containmentlinesfrom aprescribedburnin Leeuwin-Naturaliste NationalParkin WesternAustralia.A subsequentfire startedatNannup, contributingtoa broaderoutbreakof firesacrossthe region. 1.Abushfirebrokecontainmentlinesfromaprescribedburnin Leeuwin-NaturalisteNationalParkinWesternAustralia.A subsequentfirestartedatNannup,contributingtoabroader outbreakoffiresacrossthe region. 2.Landfiresinvolvingbrushandpastureburnedacrossthe TasmanianlocationsofSommers Bay, Dunalley,andBichenoin Australia. 3.Two major bushfiresbrokeoutacrossmetropolitanPerth, WesternAustralia,affectingthePerthHillsarea. 4.Alandfireburnedthroughbrushandpastureinthe Margaret River region ofWesternAustralia. 5.AnextremeheatwaveaffectedSouthAustraliaandVictoria. Temperaturesexceeded40degreesCelsiusinmultiplelocations, includingAdelaide,Melbourne,andDartmoor,whichrecordeda maximumof43degreesCelsius. 1.AseriesofseverestormsstruckPerthandSouthernWestern Australia.Theeventsincludedabrieftornadowith125kmperhour windsinDianellaandMorley,followedbyheavyrain,thunder,and wind gusts upto140kmperhour. 2.Acyclone,reportedaseithertropicalorextratropical,wasactive intheSouthIndianbasinoffWesternAustralia,withmaximum sustainedwindsof20knotsandacentralpressureof1000mbar. 3.AtropicalcyclonewasactiveintheWesternAustraliasub-basin oftheSouthIndianOcean,withmaximumsustainedwindsof20 knotsandacentralpressureof1000mbar. 4.AtropicalcyclonewasactiveintheWesternAustraliasub-basin oftheSouthIndianOcean,withmaximumsustainedwindsof20 knotsandacentralpressureof1000mbar. 5.AtropicalcyclonewasactiveintheWesternAustraliasub-basin oftheSouthIndianOcean,withmaximumsustainedwindsof20 knotsandacentralpressureof1000mbar. Example 2 Post-TropicalStorm Irenemovednorth intoCanada,bringing heavyrainand strong wind gusts to Quebecandthe Maritimeprovinces. 1.Post-TropicalStormIrenemovednorthintoCanada,bringing heavyrainand strong wind gusts toQuebecandtheMaritime provinces. 2. Heavy rainsfromatropicalcyclonetriggeredalandslidein L'Assomption,Quebec,Canada. 3.ExtratropicalcycloneconditionswereobservedoverQuebec, Canada,withmaximumsustainedwindsof40knotsandacentral pressureof987millibars. 4.Wildfiresandassociatedsmokeaffectedthecommunitiesof NorthWestRiverandSheshatshiuinNewfoundlandandLabrador, Canada. 5.AnextratropicalcyclonewaslocatedoverNewBrunswick, Canada,withmaximumsustainedwindsof40knotsandacentral pressureof1002mbar. 1.Normalweatherconditionswereobservedacrosspartsof Canada,includingQuebec,NewBrunswick,NovaScotia,Prince EdwardIsland,andNewfoundlandandLabrador. 2.Normalweatherconditionswereobservedacrosspartsof Canada,includingNewBrunswick,Ontario,PrinceEdwardIsland, andQuebec. 3.NormalweatherconditionswereobservedacrossNewfoundland andLabradorandQuebec,Canada. 4.NormalweatherconditionswereobservedacrossNew BrunswickandQuebec,Canada. 5.NormalweatherconditionswereobservedacrossNew BrunswickandQuebec,Canada. Example 3 Adownpourtriggered alandslideinSanta Cruz County, California, causing a combinationofshale andmudto giveway. 1.AdownpourtriggeredalandslideinSantaCruz County, California, causing acombinationofshaleandmudto giveway. 2.AdownpourtriggeredatleasttwomudslidesintheSantaCruz mountainsofCalifornia. 3.DownpourstriggeredlandslidesinMarin County andalongthe CentralCaliforniacoastnear Big SurandCarmel. 4.AstormbroughtheavyrainandwindstoCentralandNorthern California, triggering amudslideanddowningtreesinMonteRio. 5.Adownpourtriggeredacomplexlandslidealongthesteepcliffs southofCarmelin MontereyCounty, California. 1. Strong windswithestimated gusts around35knotsaffected Northern MontereyBay, California. 2.A strong windeventbroughtestimated gusts of33to61knotsto theSanFranciscoand MontereyBay areasofCalifornia. 3.Aseriesof systems broughtheavyrainand strong windstothe SanFranciscoand MontereyBay areasofCalifornia,resultingin floods,flashfloods,anddebrisflows. 4.DownpourstriggeredlandslidesinMarin County andalongthe CentralCaliforniacoastnear Big SurandCarmel. 5.Aseriesofstorm systems brought strong winds,heavyrain, anddebrisflowstotheSanFranciscoand MontereyBay areasof California. Example 4 Anextremeheatwave affectednortheast Iowa,bringing maximumapparent temperaturesbetween 100and110eachday. 1.ExcessiveheatsettledovernortheastIowa,producingafternoon heatindicesbetween110and120. 2.AnexcessiveheateventaffectedIowa,featuringdaily temperaturesinthe90sandovernightlows remaining inthe70s. 3.AnextremeheatwaveaffectednortheastIowa,bringing maximumapparenttemperaturesbetween100and110eachday. 4.AnexcessiveheateventaffectedmultiplecountiesinIowa,with afternoontemperaturesreachingtheupper90stolower100s. 5.AsignificantheatwaveaffectedmultiplecountiesinIowa, bringingthehottestweatherwithmaximumtemperatures soaring intothe90sandlower100s. 1.Amudslidetriggeredbyadownpourandflashflooding uprootedatreein Lansing Township,Iowa. 2.Severalroundsofthunderstormsbroughtheavyrain,hailupto 1inch,andwind gusts of55to70knotstonortheastIowa. 3.Thunderstormsbroughtheavyrain,lightning,andhailupto oneinchindiameter,leadingtoflashfloodingacrossFloydand WinneshiekcountiesinIowa. 4.Acoupleroundsofthunderstormsbroughtheavyrain,hail, and strong windstonortheastIowa. 5.Severethunderstormsproducedestimatedwind gusts around 52to55knotsacrosspartsofnortheastIowa. Figure 7: Qualitative comparison of M2T retrieval using the full MetTok configuration and geoloca- tion+time only, showing the top-5 retrieved texts for each ground-truth event, with green indicating meteorologically related events and red indicating unrelated phenomena. Full MetTok retrieves more meteorologically relevant events, while geolocation and time alone can retrieve unrelated phenom- ena. 19 Preprint Table 9: Expert-curated natural event and disaster sources used in EarthAlign. The table reports the geographic scope, provider type, natural phenomena type, years used, and number of records after source-specific processing. SourceGeographic Scope Provider TypeNatural Phenomena TypeYears Used Num. of Records EM-DAT GlobalResearch institutionNatural hazards and disasters 2000ā202425,172 NOAAUnited StatesGovernment agencySevere weather events2000ā2024286,292 IBTrACS GlobalGovernment agencies Cyclones2000ā202484,764 COOLRGlobalGovernment agencyLandslides2000ā202412,511 PIKARTGlobalResearch institutionAtmospheric rivers2000ā202336,840 AusDEAustraliaGovernment agencyNatural hazards and disasters 2000ā201495 MexDEMexicoGovernment agencyNatural hazards and disasters 2000ā20161,628 CanDECanadaGovernment agencyNatural hazards and disasters 2000ā2022385 IndDEIndonesiaGovernment agencyNatural hazards and disasters 2008ā202434,623 FraDEFranceGovernment agencyNatural hazards and disasters 2000ā20246,875 BDATASET SAMPLES RECORD CONSTRUCTION B.1EXPERT-CURATED NATURAL EVENT AND DISASTER RECORDS We collect natural event and disaster records from 10 expert-curated sources, summarized in Table 9. We apply source-specific processing to construct standardized metadata records. The resulting meta- data is subsequently used to ground multi-style caption generation. We describe the processing for each source below. NOAA Storm Events Database. The event record source (NOAA NCEI) provides records of severe weather events across the United States. We retain records from 2000ā2024 and extract the available event metadata, including the episode and event narratives, event type, state and county, start and end times, event magnitude and magnitude type, flood cause, and tornado F/EF-scale rating. We further group event records using the combination of EPISODE ID and EPISODENARRATIVE, and aggregate the associated event types, measurements, event narratives, locations, and start and end times. We additionally convert all reported local start and end times to UTC. To obtain a stan- dardized geospatial extent for each episode, we map the reported state and county or weather-zone information to official U.S. geographic boundary data 1 using GeoPandas, and convert the matched geographic boundaries to their enclosing latitudeālongitude bounding box. We then construct a stan- dardized metadata record containing the episode start and end times, event types, location, available physical measurements, flood cause, tornado F/EF-scale rating, and narrative. IBTrACS. IBTrACS (Knapp et al., 2010; Gahtan et al., 2024) provides the most comprehensive global archive of tropical cyclones. We retain records from 2000ā2024 and extract the available metadata, including basin and sub-basin, cyclone nature, time, center location, WMO wind and pressure, storm translation speed and direction, and agency-provided cyclone classifications. We further retain only main-track records with agency-reported position and intensity values, i.e. those whose TRACK TYPE is main and whose IFLAG contains O or V. We exclude records with an unknown or unreported cyclone nature. To obtain more detailed location information, we reverse geocode cyclone center coordinates using OpenStreetMap 2 for records whose DIST2LAND is less than 50 km. We additionally derive a more specific cyclone subcategory from the available agency- specific classifications while preserving differences among agency classifications. We then construct a standardized metadata record containing the snapshot time, basin and sub-basin, center location, cyclone nature and subcategory, WMO wind and pressure, and storm translation speed and direction. EM-DAT. EM-DAT (Delforge et al., 2025) provides records of major disasters worldwide. We retain records from 2000ā2024 whose disaster group is Natural and whose disaster subgroup 1 https://w2.census.gov/geo/tiger/GENZ2024/shp/cb_2024_us_county_500k. zip,https://w.weather.gov/source/gis/Shapefiles/WSOM/z_18mr25.zip,https: //w.weather.gov/source/gis/Shapefiles/WSOM/mz18mr25.zip,https://w.weat her.gov/source/gis/Shapefiles/WSOM/oz18mr25.zip,https://w.weather.gov/so urce/gis/Shapefiles/WSOM/hz20fe25.zip 2 https://w.openstreetmap.org/ 20 Preprint is Climatological, Meteorological, Hydrological, or Geophysical, and exclude records with missing start or end years. We extract the available metadata, including disaster type and subtype, event name, country, location, origin, associated disaster types, magnitude and magni- tude scale, and start and end dates. Each disaster record is uniquely identified by its DisNo.. To obtain a standardized geospatial extent for each record, we geocode each reported location using OpenStreetMap and aggregate the resulting geospatial boundaries to derive an enclosing latitudeā longitude bounding box. For disasters spanning large geographic extents, we further perform spatial upsampling by generating localized bounding boxes for subregions within the original extent. Start- ing from 9,380 filtered records, our upsampling strategy generates 15,792 additional samples from 2,041 large-extent records. We then construct a standardized metadata record containing the disas- ter type and subtype, event name, origin, associated disaster types, magnitude and magnitude scale, start and end dates, and locations. COOLR. COOLR (Juang et al., 2019) provides records of reported landslides and mass movements worldwide. We retain records from 2000ā2024 with available latitude and longitude coordinates and extract the available metadata, including event date, event title and description, landslide category and trigger, and coordinates. To obtain a textual location for each event, we reverse geocode the reported latitudeālongitude coordinates using OpenStreetMap and exclude records for which reverse geocoding does not return a location. We then construct a standardized metadata record containing the landslide category and trigger, location, event date, event title, and event description. PIKART. PIKART (Vallejo-Bernal et al., 2025) provides a global high-resolution catalog of atmo- spheric rivers derived from ERA5 meteorological data based on integrated vapor transport (IVT). We retain records from 2000ā2023 and select one timestamp from each atmospheric river track. For each selected timestamp, we extract the available metadata, including intersected continents, mean IVT, track centroid speed, and track centroid positions, and use the corresponding atmospheric river extent as the geospatial extent. To provide a qualitative characterization of IVT magnitude, we com- pute the distribution of mean IVT across all timestamps and tracks and categorize values below the 10th percentile as low, above the 90th percentile as high, and between the 40th and 60th per- centiles as moderate. We additionally derive the atmospheric river propagation direction from the displacement of the track centroid between the beginning and end of each track. We then con- struct a standardized metadata record containing the snapshot time, intersected continents, mean IVT magnitude and category, and track-mean atmospheric river propagation speed and direction. AusDE. AusDE (Australian Government, Attorney-Generalās Department, 2023) provides records of events and disasters across Australia from a government-curated source. We retain records from 2000ā2014, exclude events occurring outside Australia, and remove non-natural events such as transport accidents, criminal acts, and industrial incidents. We extract the available metadata, in- cluding event title and description, start and end dates, and latitudeālongitude coordinates. To obtain a standardized geospatial extent for each record, we use GPT-5.5 Pro with the event title, description, and provided coordinates to derive a latitudeālongitude bounding box corresponding to the affected area. We then construct a standardized metadata record containing the disaster title, start and end dates, and event description. MexDE. MexDE (Centro Nacional de Prevenci Ģ on de Desastres (CENAPRED), 2023) provides records of events and disasters across Mexico from a government-curated source. We retain records from 2000ā2024 and exclude non-natural events such as chemical events. We extract the available metadata, including state, municipality, start and end dates, phenomenon type, classification, and observations, and translate the original Spanish-language metadata into English using GPT-5.5. We further group records sharing the same start date, end date, phenomenon type, classification, and observations. To obtain a standardized geospatial extent for each record, we geocode each reported municipality and state using OpenStreetMap, and aggregate the resulting bounding boxes to derive an enclosing latitudeālongitude bounding box. We then construct a standardized metadata record containing the phenomenon type, observations, start and end dates, and location. CanDE. CanDE (Public Safety Canada, 2026) provides records of events and disasters across Canada from a government-curated source. We retain records from 2000ā2024 whose event group is Natural and exclude records whose event subgroup is Biological. We extract the avail- able metadata, including event group and subgroup, disaster type, start and end dates, reported places, affected provinces, and event comments. To obtain a standardized geospatial extent for each record, we use GPT-5.5 Pro with the reported places, affected provinces, and event comments to 21 Preprint Figure 8: Examples of spatial grid-cell sampling for climatology-based normal-weather identifica- tion on California and Abu Dhabi. derive a latitudeālongitude bounding box corresponding to the affected area. We then construct a standardized metadata record containing the disaster type, locations, start and end dates, and event description. IndDE. IndDE (Badan Nasional Penanggulangan Bencana, 2026) provides records of events and disasters across Indonesia from a government-curated source. We retain records from 2000ā2024 and exclude non-natural incidents such as residential fires, social conflicts, and technological fail- ures. We extract the available metadata, including disaster type, event type, province, regency or city, date, and time, and translate the disaster and event types from Indonesian into English using GPT-5.5. We further group records sharing the same disaster type, event type, province, and date, and aggregate their associated locations. We also remove reported times of 08:08:08 AM and 12:01:00 AM, which occur anomalously frequently in the source, and convert the remaining lo- cal times to UTC. To obtain a geospatial extent for each record, we geocode each reported regency or city using OpenStreetMap, and aggregate the resulting bounding boxes to derive an enclosing latitudeālongitude bounding box. We then construct a standardized metadata record containing the disaster type and subtype, date, and locations. FraDE. FraDE (Minist ` ere de la Transition Ģ ecologique, 2026) provides records of events and dis- asters across France from a government-curated source. We retain records from 2000ā2024 and extract the available metadata, including disaster type, start and end dates, and municipality name and code. We further group records sharing the same disaster type, start date, and end date, and aggregate their associated municipalities. We translate the disaster type names from French into English using GPT-5.5. To obtain a standardized geospatial extent for each record, we match each reported municipality to its corresponding French commune 3 and aggregate the resulting adminis- trative boundaries to derive a latitudeālongitude bounding box. We then construct a standardized metadata record containing the disaster type, start and end dates, and locations. B.2CLIMATOLOGY-BASED NORMAL-WEATHER SAMPLING For each first-level administrative unit (ADM1), we identify ERA5 grid cells within its geoBound- aries administrative boundary (Runfola et al., 2020) and randomly sample 20% of the cells, with at least 10 cells whenever possible. For instance, as illustrated in Figure 8, this yields 147 sampled grid cells for California and 22 for Abu Dhabi. For each sampled cell, we retrieve ERA5 hourly variables from 1991 to 2025, including 2m temperature, 2m dewpoint temperature, skin temperature, mean sea-level pressure, surface pressure, sea-surface temperature, 10m and 100m wind speed, 10m wind gust, total precipitation, surface solar radiation downwards, cloud base height, significant height of 3 French commune boundary data 22 Preprint Table 10: Variable-specific climatological criteria used for normal-weather sampling. Percentiles are computed independently for each ERA5 grid cell, day of year, UTC hour, and variable over the 1991ā2020 reference period using a centeredĀ±2-day window. CriterionVariables P05āP952m temperature; 2m dewpoint temperature; skin temperature; mean sea-level pres- sure; surface pressure; sea-surface temperature ā¤ P9010m wind speed; 100m wind speed; 10m wind gust; total precipitation (1h, 3h, 6h, and 24h); 3h mean sea-level pressure change; significant height of combined wind waves and swell; mean wave period ā„ P10Dewpoint depression; surface solar radiation downwards; cloud base height combined wind waves and swell, and mean wave period, when applicable. We additionally derive 3h, 6h, and 24h precipitation accumulations, 3h mean sea-level pressure change, and dewpoint de- pression. The complete set of variables and their corresponding normality criteria is summarized in Table 10. For each grid cell, variable, day of year, and UTC hour, we estimate the climatological thresholds over the 1991ā2020 WMO reference period (WMO, 2017) using observations pooled from the centeredĀ±2-day window, yielding 150 observations for each threshold estimate. We then apply the grid-cell- and ADM1-level normality rules described in Section 3.1 to identify eligible normal-weather hours. We further restrict the resulting candidate pool to 2000ā2024 and perform global sampling, yielding 64,553 normal-weather samples. CMULTI-STYLE GROUNDED CAPTION GENERATION AND AUDIT C.1CAPTION GENERATION For each metadata record, we use Gemini 3.1 Pro to generate one caption in each of four styles. Natural-event-focused captions describe the event or phenomenon directly, with time and location information when available; physical-framing captions emphasize its supported spatial or tempo- ral organization; impact-focused captions emphasize supported weather-, climate-, environmental-, or hazard-related effects; and atmospheric-pattern captions emphasize provided meteorological or environmental conditions and setup. The generation prompt requires all captions to be directly sup- ported by the input metadata and prohibits unsupported causes, impacts, forecasts, and descriptive modifiers. When the metadata does not sufficiently support a requested style, the model is instructed to produce a minimal grounded caption rather than infer missing style-specific content. The repre- sentative metadata and generated captions are shown in Figure 9. We further report the distribution of caption lengths in Figure 10. The full generation prompt is provided below. Grounded Caption Generation Prompt You are an expert caption writer for climate and weather data. Task: Generate grounded captions from the provided metadata/context. Goal: Produce stylistically diverse captions that are strictly supported by the input, preserve informa- tive uncertainty when present, and are suitable for text-ERA5 climate data alignment. Hard constraints: ā¢ Do not add any fact that is not directly supported by the input. ā¢ Do not infer specific causes, future outcomes, identities, or impacts unless explicitly given. ā¢ Do not describe objects, actions, regions, or conditions not present in the metadata/context. ā¢ If a detail is missing, leave it out. ā¢ If the input contains explicit uncertainty, disagreement, multiple reported classifications, or alterna- tive labels, do not replace them with a single definite claim. Retain the ambiguity when it is relevant to the caption. ā¢ Do not mention missing, unspecified, or unavailable information. ā¢ Do not force all supported metadata into every caption. Include only the details that best fit the requested style. ā¢ Do not refer to the metadata, narrative, source text, or report inside the caption. 23 Preprint MetadataGenerated Captions EM - DAT Disaster Type: Storm Disaster Subtype: Lightning/Thunderstorms Start Time: 24 April 2023 End Time: 25 April 2023 Locations: Sri Lanka Natural Event Focused: Lightning and thunderstorms occurred in Sri Lanka from 24 April 2023 to 25 April 2023. Physical Framing: Storm activity featuring lightning spanned across Sri Lanka. Impact Focused: Sri Lanka was affected by a storm producing lightning and thunderstorms. Atmospheric Pattern: Storm conditions over Sri Lanka were characterized by lightning and thunderstorms. NOAA Begin Time: 18:00 UTC, 29 June 2023 End Time: 19:00 UTC, 29 June 2023 Event Type: Hail State (County): WISCONSIN (MILWAUKEE and WASHINGTON) Measurement(s): largest hail magnitudes around 0.75-1 inches Narrative: Isolated strong thunderstorms developed in the... Natural Event Focused: Between 18 UTC and 19 UTC on 29 June 2023, isolated strong thunderstorms produced hail up to 1 inch in magnitude across Milwaukee and Washington counties in Wisconsin. Physical Framing: Large hail measuring 0.75 to 1 inch fell across the Milwaukee Metropolitan area as isolated strong thunderstorms developed over the region. Impact Focused: Hailstones reaching up to 1 inch in magnitude struck parts of Milwaukee and Washington counties during isolated strong thunderstorms. Atmospheric Pattern: Isolated strong thunderstorms developed along or near a lake breeze front or surface trough in the Milwaukee Metropolitan area, producing large hail. IBTrACS Snapshot Time: 03:00 UTC, 17 October 2022 Basin: Western North Pacific Cyclone Nature: tropical cyclone Cyclone Subnature: typhoon Storm Speed: 7.0 Storm Direction: 250.0 Natural Event Focused: A typhoon was active in the Western North Pacific at 03 UTC on 17 October 2022. Physical Framing: The tropical cyclone tracked across the Western North Pacific at a speed of 7 knots toward 250 degrees. Impact Focused: A typhoon moved through the Western North Pacific on 17 October 2022. Atmospheric Pattern: A tropical cyclone in the Western North Pacific was moving at 7 knots on a 250-degree heading. COOLR Landslide Category: Landslide Location: Tijuana, Municipio de Tijuana, Baja... Event Date: 27 March 2023 Event Title: Fraccionamiento La Sierra Event Description: El deslizamiento provocĆ³ el colapso de un... Natural Event Focused: On 27 March 2023, a landslide occurred in the Fraccionamiento La Sierra area of Tijuana, Baja California, Mexico. Physical Framing: A landslide was recorded within the municipality of Tijuana, Baja California. Impact Focused: A landslide event struck Tijuana, Mexico, on 27 March 2023. Atmospheric Pattern: Landslide conditions developed in the Fraccionamiento La Sierra area of Mexico in late March 2023. PIKART Record: Atmospheric River (AR) Snapshot Snapshot Time: 21:00 UTC, 26 March 2023 Intersected Continents: asia Mean IVT Magnitude: 276.623 Mean IVT Category: weak IVT Propagation Speed: 39.328 Propagation Direction: eastward Natural Event Focused: A weak atmospheric river moved eastward across Asia at 21 UTC on 26 March 2023. Physical Framing: The atmospheric river propagated eastward over the Asian continent at approximately 39 km/h, maintaining a mean integrated vapor transport magnitude of roughly 276 kg/(mĀ·s). Impact Focused: A weak atmospheric river event was recorded over Asia on 26 March 2023. Atmospheric Pattern: An eastward-propagating atmospheric river exhibited weak integrated vapor transport over Asia. AusDE Disaster: Landslide -Somersby Start Date: 2007-06-08 End Date: 2007-06-08 Description: On the 8 June 2007, Piles Creek was in flood... Natural Event Focused: On June 8, 2007, heavy rain caused Piles Creek to flood, resulting in a landslide in Somersby. Physical Framing: Excessive rainfall drove soil erosion and a landslide around the flooded Piles Creek in Somersby. Impact Focused: Flooding along Piles Creek resulted in soil erosion and a landslide. Atmospheric Pattern: An east coast low generated the heavy rain that led to flooded conditions at Piles Creek. MexDE Phenomenon: Tropical cyclone Observation(s): Tropical wave Start Date: 2005-08-24 End Date: 2005-08-25 Location: Mexico, Michoacan, Aguililla Natural Event Focused: A tropical cyclone, observed as a tropical wave, affected Aguililla, Michoacan, Mexico, from August 24 to August 25, 2005. Physical Framing: Over the course of two days, a tropical wave moved through the region of Aguililla in Michoacan, Mexico. Impact Focused: Tropical cyclone conditions were observed in Aguililla, Mexico, in late August 2005. Atmospheric Pattern: A tropical wave associated with a tropical cyclone was observed over Michoacan, Mexico, between August 24 and August 25, 2005. CanDE Disaster Type: Flood Location(s): Canada: Yukon Start Date: 2022-04-30 End Date: 2022-07-20 Description: The winter and spring leading up to the flood... Natural Event Focused: Between April 30 and July 20, 2022, a series of 16 flood events occurred across the Yukon in Canada following higher than normal snowpacks. Physical Framing: Flooding conditions extended across the entire Yukon territory, encompassing 16 distinct flood events by July 13, 2022. Impact Focused: Higher than normal winter and spring snowpacks resulted in an increasing flood risk and ultimately 16 flood events across the Yukon. Atmospheric Pattern: The winter and spring seasons preceding the 2022 flooding featured higher than normal snowpacks across many regions of the Yukon. IndDE Disaster Type: Drought Disaster Subtype: Drought Date: 5 May 2023 Locations: Country: Indonesia, Province: Jawa Barat,... Natural Event Focused: A drought occurred in Bogor, Jawa Barat, Indonesia, on 5 May 2023. Physical Framing: Drought conditions affected the region of Bogor in Jawa Barat, Indonesia. Impact Focused: The area of Bogor, Jawa Barat, experienced a drought on 5 May 2023. Atmospheric Pattern: Dry conditions characterized a drought event in Bogor, Jawa Barat, Indonesia. FraDE Disaster Type: Drought Start Date: 2022-01-01 End Date: 2022-12-31 Locations (France): Grand Est, Meuse, Douaumont-Vaux Natural Event Focused: A drought occurred in Douaumont-Vaux, Meuse, within the Grand Est region of France, lasting throughout 2022. Physical Framing: Drought conditions extended across the Grand Est region of France, including the Meuse area of Douaumont-Vaux, persisting from January 1 to December 31, 2022 Impact Focused: The Grand Est region of France, specifically Douaumont-Vaux in Meuse, experienced a prolonged period of drought during 2022. Atmospheric Pattern: Persistent dry conditions characterized a year-long drought in Douaumont-Vaux, France, spanning the entirety of 2022. Normal Weather Condition: Normal Weather Start Time: 9 UTC, 2021-11-16 End Time: 11 UTC, 2021-11-16 Location(s): Mexico, Sonora Natural Event Focused: Normal weather conditions were observed in Sonora, Mexico, between 9 UTC and 11 UTC on November 16, 2021. Physical Framing: Typical weather conditions extended across Sonora, Mexico, on November 16, 2021. Impact Focused: Normal weather prevailed in Sonora, Mexico, on November 16, 2021. Atmospheric Pattern: A period of normal weather was recorded over Sonora, Mexico, from 9 UTC to 11 UTC. Figure 9: Representative metadata records and their generated captions across record sources. ā¢ Do not convert numeric or categorical metadata into qualitative descriptions unless that wording is directly supported by the input. 24 Preprint ā¢ Do not add unsupported descriptive modifiers such as compact, broad, organized, intense, weak, rapidly, slowly, strengthening, or weakening. ā¢ Do not mention vehicles, casualties, injuries, deaths, missing people, houses, buildings, infrastruc- ture, property damage, economic losses, or other human-impact or built-environment details. Focus only on the supported natural phenomena, weather conditions, environmental conditions, or natural hazard type. ā¢ If a requested style is not supported, still produce that caption, but reduce it to a minimal, fully grounded restatement of the event rather than inventing missing structure, impacts, or atmospheric setup. ā¢ Do not make every caption a full recap of the same event metadata. ā¢ When mentioning time, use hour-only format such as 21 UTC, not 21:00 UTC. ā¢ Do not repeat the same system name, event name, storm name, phenomenon name, or cyclone name in every caption. ā¢ Do not mention exact time in every caption. Use time only when it is important to the grounded description or helps distinguish the event more clearly. Writing requirements: ā¢ Write exactly one caption for each style. ā¢ Keep each caption natural, clear, and concise. ā¢ Ensure each caption reads smoothly as a standalone caption. ā¢ Each caption must stand on its own and should not assume prior context. ā¢ Avoid generic referents such as āthe systemā when a more natural grounded alternative can be used. ā¢ Prefer caption-like phrasing over report-like phrasing. ā¢ Make the captions stylistically different by varying emphasis, structure, and phrasing, not by intro- ducing unsupported facts or unsupported interpretations. ā¢ Avoid repeating the same wording, sentence structure, or lead phrase across captions. ā¢ Prefer a single sentence, but use 2 to 3 sentences when needed to capture supported details. ā¢ Correct obvious surface errors such as typos, stray punctuation, spacing issues, duplicated words, and minor formatting problems when the intended meaning is clear. ā¢ Write all location names in plain ASCII. Use common English names or ASCII romanized forms for locations that contain accents, non-Latin scripts, or special punctuation. ā¢ Do not discard informative input details solely because they are uncertain or reported differently; maintain their uncertainty status when relevant to the caption. ā¢ If the input includes too many location names, you may summarize them with a shorter shared parent location only when that summary is clearly supported by the input and does not introduce a new regional interpretation. ā¢ If you mention a date, you must include the full supported date including the year. ā¢ Before writing each caption, select only the input details most relevant to that style, and do not reuse the same full set of details across all four captions unless the input does not provide enough distinct supported details to differentiate them further. ā¢ When the input lacks enough distinct supported details for strong stylistic separation, keep weakly supported captions shorter and more selective rather than repeating the same full metadata summary. ā¢ Express numeric metadata naturally, and avoid raw technical notation when it does not read smoothly as a caption. Allowed caption styles: 1. natural eventfocused ā¢ Focus on what is happening in a straightforward and natural way. ā¢ Do not include unrelated human-interest details, casualty counts, or incidental impacts. ā¢ Date and time may be included when it helps anchor the event naturally. 2. physical framing ā¢ Focus on how the event or conditions were distributed, organized, extended, or expressed across space or time. ā¢ Prefer wording about extent, spread, concentration, coverage, or visible physical conditions when supported. ā¢ Do not simply restate the natural-event caption with reordered wording. ā¢ Do not include time unless it is needed to describe the supported spatial or temporal arrangement. 3. impact focused ā¢ Focus on supported climate-, weather-, environmental, or hazard-related effects of the event. ā¢ Avoid named individuals, casualty counts, and highly itemized damage unless explicitly needed. 25 Preprint ā¢ Do not center unrelated human-interest or incidental impacts. ā¢ If only human-loss impacts are supported, describe the event itself without adding consequences. ā¢ Do not include time unless it is needed to anchor the reported impacts. 4. atmospheric pattern ā¢ Focus on atmospheric, meteorological, or environmental setup only when it is explicitly described in the input. ā¢ Do not introduce meteorological structure unless it is supported. ā¢ If no such pattern is explicitly supported, keep the caption grounded by describing the reported conditions. ā¢ Do not include time unless it is needed to ground the reported setup or evolution. Final check before answering: ā¢ Is every statement directly supported by the input? ā¢ Are unsupported causes, impacts, and forecasts excluded? ā¢ Were obvious surface errors cleaned up without changing the meaning or introducing unsupported interpretations? ā¢ If multiple locations were summarized, is the shared location label clearly supported by the input? ā¢ Is each caption aligned with its style? ā¢ Is each caption concise and natural? ā¢ Is the output valid JSON with exactly the required keys? Output format: Return valid JSON only, with exactly the required keys and string values. Do not output markdown, comments, code fences, or any text outside the JSON object. Input metadata/context: input placeholder 0.00 0.01 0.02 0.03 0.04 Overall Token-Length Distribution 0.000 0.015 0.030 0.045 0.060 Token-Length Distribution by Caption Style Caption style atmospheric_patternimpact_focusednatural_event_focusedphysical_framing 10203040506070 0.00 0.02 0.04 0.06 0.08 Token-Length Distribution by Dataset Dataset AusDE CanDE COOLR IndDE EM-DAT FraDE IBTrACS MexDE Negative Samples NOAA PIKART Number of CLIP Tokens Density Figure 10: Distribution of generated caption lengths measured by using the CLIP text tokenizer. We report the overall distribution, distributions across the four caption styles, and distributions across record data sources. C.2CAPTION AUDIT AND ERROR ANALYSIS Caption audit. A trained graduate annotator manually audits 440 captions, comprising 10 records from each of 11 record sources (i.e. 10 expert-curated natural event and disaster sources and the 26 Preprint normal-weather samples) across all four caption styles. Each caption is evaluated for date/time ac- curacy, location accuracy, event/phenomenon accuracy, trigger/cause accuracy, unsupported added detail, unsupported modifiers, weather/climate focus, and standalone clarity. Table 11 summarizes the results, with a caption counted as passing only when all audit criteria are satisfied. Error analysis. Among the erroneous captions, the most frequent audit failure category is weath- er/climate focus (7 cases), followed by location accuracy (6), unsupported modifiers (2), trigger/- cause accuracy (2), unsupported added detail (1), and event/phenomenon accuracy (1). Table 12 provides representative examples. In particular, several impact-focused failures emphasize human consequences such as drowning or transport accidents rather than focusing on weather, climate, envi- ronmental, or hazard-related effects. Other errors arise from overgeneralized locations, unsupported qualitative modifiers, or descriptions that alter the identity of the reported phenomenon. Table 11: Manual audit results for generated captions. A trained graduate annotator verified 10 samples from each record source across all caption styles, totaling 440 captions. Caption StyleAuditedPassedPass Rate Natural Event Focused110110100.00% Physical Framing11010898.18% Impact Focused1109889.09% Atmospheric Pattern11010595.45% Overall44042195.68% DERA5 DATA ALIGNMENT AND PREPROCESSING ERA5 variables. We retrieve ERA5 hourly data on single and pressure levels through the Coper- nicus Climate Data Store (CDS) API. Each meteorological input contains 28 single-level variables, comprising 25 atmospheric and land variables and 3 wave variables, together with 7 pressure-level variables at 925, 850, 700, 500, and 300 hPa. This results in 63 variables in total. The complete variable set is listed in Table 13. Temporal alignment. We align each processed record to a single ERA5 hour based on its available temporal information. For IBTrACS, PIKART, and NOAA or IndDE records with an exact times- tamp, we use the reported timestamp directly. For NOAA and climatology-based normal-weather records specified over a startāend time interval, we uniformly sample one hourly timestamp within the reported interval. COOLR and some IndDE records provide only a date; for these records, we uniformly sample one of the 24 hours within the reported day in local time and subsequently con- vert it to UTC. EM-DAT, FraDE, CanDE, AusDE, and MexDE provide date ranges, from which we uniformly sample one hourly timestamp within the reported interval. All reported or sampled timestamps are standardized to UTC before ERA5 retrieval. Spatial alignment. The processed records contain either a spatial bounding-box extent or a sin- gle center location. NOAA, EM-DAT, PIKART, AusDE, MexDE, CanDE, IndDE, FraDE, and climatology-based normal-weather records provide bounding-box extents, whereas IBTrACS and COOLR provide only a center latitudeālongitude coordinate. For IBTrACS and COOLR, we con- struct a 14 ā¦ Ć14 ā¦ region centered on the reported coordinate. For records with an existing bounding box smaller than 14 ā¦ Ć 14 ā¦ , we preserve its center and symmetrically expand each dimension to a minimum extent of 14 ā¦ whenever possible. Near the northern or southern geographic boundary, where symmetric expansion would extend beyond 90 ā¦ orā90 ā¦ latitude, we shift the excess latitu- dinal extent in the opposite direction. For records whose extent exceeds 14 ā¦ in either dimension, we use the larger of the latitudinal and longitudinal extents and symmetrically expand the shorter dimension to construct a square region containing the original bounding box. Standardization. We retrieve ERA5 patches based on the selected timestamp and spatial extent. We then standardize each extracted patch to 56Ć56 grid cells using linear interpolation for all variables except mean wave direction, for which we use nearest-neighbor interpolation. We further normalize each variable using its mean and standard deviation computed across samples. 27 Preprint Table 12: Representative caption audit failures and their corresponding failure categories. StyleGenerated CaptionAudit Comment Impact Focused A rip current resulted in a drowning in Coastal Horry, South Carolina, on 13 June 2022. Weather/climate focus: drowning is a human-impact outcome rather than a weather-, climate-, or environmental effect. Impact Focused A weak IVT event carried moisture at a mean magnitude of 249.224 kg/(mĀ·s). Event/phenomenon accuracy: a weak IVT event does not necessarily correspond to an atmospheric river and changes the identity of the reported phenomenon. Impact Focused Hazardous snow and ice accumulations from Storm Bettina led to transport accidents in German locations like Schwabisch Hall and Denzeheide. Weather/climate focus: transport accidents are incidental human impacts and should not be the focus of the caption. Impact Focused The winter storm produced extreme snow drifts in Prince Edward County, alongside significant snow accumulation and drifting along the north shore of Lake Ontario. Unsupported modifier: āextremeā is not supported by the input metadata. Physical Framing Significant precipitation and lake effect snow spread across Eastern Canada, with intense snow drifting extending along the north shore of Lake Ontario through to Kingston. Location/modifier: the lake-effect snow is described more specifically along Lake Ontario through Kingston, thus overgeneralizing its spatial extent; Unsupported modifier: āintenseā is unsupported. Table 13: ERA5 variables used to construct the meteorological inputs. The 28 single-level variables are combined with 7 pressure-level variables at 5 pressure levels, yielding 63 variables in total. Variable GroupVariables Single-level atmospheric and land 2m temperature; 2m dewpoint temperature; 10m u-component of wind; 10m v- component of wind; instantaneous 10m wind gust; mean sea-level pressure; total precipitation; convective precipitation; snowfall; snow depth; snowmelt; runoff; volumetric soil water layer 1; volumetric soil water layer 2; volumetric soil water layer 3; volumetric soil water layer 4; convective available potential energy; con- vective inhibition; boundary layer height; cloud base height; total cloud cover; zero-degree level; total column water vapour; vertical integral of eastward water vapour flux; vertical integral of northward water vapour flux Single-level waveSignificant height of combined wind waves and swell; mean wave direction; peak wave period Pressure-levelGeopotential; temperature; u-component of wind; v-component of wind; specific humidity; vertical velocity; vorticity, each at 925, 850, 700, 500, and 300 hPa EADDITIONAL DATASET STATISTICS AND DISTRIBUTIONS Regional distribution. Figure 11 shows the geographic distribution of EarthAlign across 19 re- gions. As illustrated, EarthAlign spans a broad range of geographic regions with North America being the most represented, largely reflecting the NOAA Storm Events Database which contributes 286,292 records from the United States. The remaining global and country-specific sources pro- vide additional coverage across other regions, resulting in broad geographic coverage with a notable concentration in North America. Weather condition class distribution. Figure 12 shows the class distribution for the weather- condition classification task across the training and test sets. The two splits exhibit broadly similar distributions across most classes, with drought showing the largest relative difference. The resulting class imbalance reflects the underlying distribution of records across the constituent record sources. 28 Preprint Arctic Ocean 1,708 Europe 24,813 Central Asia 4,122 North America 295,016 Eastern Asia 13,039 North Atlantic Ocean 52,617 Western Asia 4,864 North Pacific Ocean 63,065 Northern Africa 6,849 Southern Asia 12,463 South-Eastern Asia 41,951 Sub-Saharan Africa 23,665 Latin America & Caribbean 24,356 Oceania 5,721 South Atlantic Ocean 6,972 South Pacific Ocean 17,915 Indian Ocean 43,960 Southern Ocean 2,134 Antarctica 99 One count per region with at least 10.53% bbox overlap. Regional memberships: 645,329; unique records: 553,738. Pillars use square-root scaling with a fixed minimum height. Figure 11: Regional distribution of EarthAlign records. Records spanning multiple regions con- tribute to more than one regional count. Normal Weather DroughtHeat Wave Cold Wave FrostAtmospheric River CycloneConvective Storm High Wind Fog Winter Weather 0% 10% 20% 30% 40% Sample Proportion (%) Train (2000ā2010, 2014ā2024)Test (2011ā2013) 56,742 7,816 4,672 3,211 1,866 32,197 80,138 158,320 15,459 2,247 28,360 7,811 2,299 684 466 422 4,643 10,262 16,092 1,864 330 3,772 Figure 12: Class distribution for the weather condition classification task in the training and test sets. FADDITIONAL ANALYSIS F.1WEATHER CONDITION CLASSIFICATION ANALYSIS Table 14 provides category-wise classification results, with the corresponding confusion matrices shown in Figure 13. Under text-prototype inference, Meteo-CLIP and Meteo-SigLIP generally improve category-level performance over their respective CLIP and SigLIP baselines. The im- provements are particularly notable for atmospheric river, convective storm, high wind, and win- ter weather. Under linear probing, performance increases substantially across almost all categories, with Meteo-CLIP and Meteo-SigLIP achieving macro F1 scores of 79.98 and 80.75 respectively. The consistently stronger linear-probing results indicate that the frozen meteorology representations encode substantial category-level information. Across weather conditions, drought, heat wave, and cold wave remain the most challenging cate- gories. Even under linear probing, their F1 scores are substantially below those of atmospheric river, cyclone, and convective storm, which exceed 94 F1. The lower performance of drought, heat wave, and cold wave cannot be solely attributed to limited test data, as frost achieves above 82 F1 with both MeteoLIP variants despite having only 422 test samples, fewer than drought, heat wave, and cold wave. The confusion matrices further show that these challenging categories are frequently confused with other meteorological conditions, including normal weather. The lower performance on drought, heat wave, and cold wave potentially reflects their inherently temporal characteristics. For instance, the Copernicus Warm and Cold Spell indicators define such 29 Preprint Table 14: Per-category classification performance under text-prototype inference and linear probing across 11 weather conditions. We report class-wise F1 scores and macro F1, with the number of test samples shown in parentheses. Bold indicates the best result and underline indicates the second-best. CategoryCLIPMeteo-CLIPSigLIPMeteo-SigLIP Text-Prototype Inference Normal weather (n=7,811)42.59Ā± 4.77 50.84Ā± 2.6534.07Ā± 7.09 56.29Ā± 5.64 Drought (n=2,299)18.66Ā± 1.33 23.42Ā± 4.1414.20Ā± 2.15 25.36Ā± 3.51 Heat wave (n=684)16.81Ā± 3.61 28.14Ā± 3.7915.45Ā± 6.20 43.04Ā± 8.05 Cold wave (n=466) 25.49Ā± 5.61 17.98Ā± 0.71 14.83Ā± 0.81 24.77Ā± 1.52 Frost (n=422)12.46Ā± 1.73 33.32Ā± 6.9317.74Ā± 7.77 48.48Ā± 15.29 Atmospheric river (n=4,643) 31.37Ā± 10.18 72.37Ā± 4.42 12.70Ā± 1.29 69.19Ā± 4.15 Cyclone (n=10,262)54.07Ā± 7.1461.67Ā± 5.80 27.12Ā± 4.06 42.07Ā± 8.38 Convective storm (n=16,092) 25.20Ā± 4.59 65.39Ā± 0.30 12.11Ā± 8.89 57.70Ā± 15.64 High wind (n=1,864)30.31Ā± 1.76 59.29Ā± 4.2429.29Ā± 6.24 66.00Ā± 6.36 Fog (n=330)8.55Ā± 2.22 33.77Ā± 7.336.92Ā± 2.27 35.31Ā± 10.16 Winter weather (n=3,772)65.89Ā± 3.33 80.70Ā± 3.24 52.59Ā± 7.47 84.09Ā± 1.83 Macro F1ā30.13Ā± 2.8347.90Ā± 2.0921.55Ā± 1.4850.21Ā± 2.95 Linear Probing Normal weather (n=7,811)90.37Ā± 0.13 91.74Ā± 0.1390.69Ā± 0.12 93.00Ā± 0.04 Drought (n=2,299)48.80Ā± 0.46 52.54Ā± 0.8951.41Ā± 1.17 56.70Ā± 0.61 Heat wave (n=684)47.30Ā± 1.35 61.94Ā± 1.15 52.84Ā± 1.49 62.41Ā± 0.86 Cold wave (n=466)48.43Ā± 0.79 49.49Ā± 0.95 48.98Ā± 1.93 54.59Ā± 1.23 Frost (n=422)75.38Ā± 1.69 83.14Ā± 0.54 75.40Ā± 2.56 82.12Ā± 0.96 Atmospheric river (n=4,643) 96.46Ā± 0.18 97.92Ā± 0.10 96.86Ā± 0.14 97.73Ā± 0.12 Cyclone (n=10,262)98.11Ā± 0.06 98.31Ā± 0.04 98.24Ā± 0.0498.24Ā± 0.02 Convective storm (n=16,092) 94.07Ā± 0.09 95.08Ā± 0.16 94.25Ā± 0.03 95.03Ā± 0.04 High wind (n=1,864)74.86Ā± 0.55 80.84Ā± 0.74 78.00Ā± 0.22 80.03Ā± 0.22 Fog (n=330)62.75Ā± 0.96 76.81Ā± 1.63 62.36Ā± 0.55 76.70Ā± 0.55 Winter weather (n=3,772)88.95Ā± 0.24 91.98Ā± 0.08 90.31Ā± 0.15 91.73Ā± 0.22 Macro F1ā75.04Ā± 0.2079.98Ā± 0.2676.30Ā± 0.5480.75Ā± 0.14 events from sequences of at least three hot or cold days based on daily minimum and maximum temperature anomalies 4 . Similarly, meteorological drought indicators such as the Standardized Pre- cipitation Index (SPI) characterize precipitation deficits over accumulated time scales ranging from months to years 5 . In contrast, MeteoLIP currently takes a single meteorological snapshot as input, thereby limiting its ability to capture the temporal persistence and accumulated anomalies underly- ing these conditions. These findings motivate future extensions of MeteoLIP to incorporate multiple timestamps or temporal sequences, which can help analyze meteorological conditions over time. F.2PHYSICAL BASELINES FOR CYCLONE INTENSITY ESTIMATION Since mean sea-level pressure and 10-m wind are included in the ERA5 input, we additionally compare MeteoLIP with physical baselines derived directly from these variables. For minimum central pressure, the direct ERA5 baseline takes the minimum mean sea-level pressure within each input patch, while for maximum sustained wind speed, it takes the maximum 10-m wind speed. We also fit a linear regression from each ERA5-derived quantity to its corresponding IBTrACS target using the training set. As shown in Table 15, the direct ERA5 baselines achieve R 2 values of 51.96 for minimum central pressure and 10.78 for maximum sustained wind speed, which increase to 63.78 and 48.02 with linear regression. In comparison, linear probing of Meteo-CLIP achieves R 2 values of 76.36 and 71.34, while Meteo-SigLIP achieves 77.36 and 71.85, respectively. These results show that linear probing of the learned meteorological representations achieves better predictive performance than directly extracting or linearly calibrating the corresponding ERA5 variables. 4 EDO INDICATOR FACTSHEET - Warm Spell and Cold Spell Duration and Intensity Indicators 5 EDO AND GDO INDICATOR FACTSHEET - Standardized Precipitation Index (SPI) 30 Preprint Table 15: Physical baselines for cyclone intensity estimation. Bold and underlinedvalues indicate the best and second-best results, respectively. VariableMethodR 2 āMAEāRMSEā Wind Direct ERA510.78 14.5020.83 ERA5 Linear Regression 48.02 12.2715.90 Meteo-CLIP71.348.6511.80 Meteo-SigLIP71.858.5011.70 Pressure Direct ERA551.967.6113.44 ERA5 Linear Regression 63.788.4311.67 Meteo-CLIP76.366.419.43 Meteo-SigLIP77.366.169.23 Table 16: Effect of initialization on downstream fine-tuning using the same Meteo-SigLIP architec- ture. Bold and underlineindicate the best and second-best results, respectively. ClassificationSustained WindCentral Pressure InitializationF1āTop-1 Acc. āR 2 āMAEāR 2 āMAEā Scratch77.02Ā± 0.7691.56Ā± 0.1672.55Ā± 1.048.23Ā± 0.1878.89Ā± 0.745.65Ā± 0.13 WebLI78.38Ā± 0.64 91.88Ā± 0.2074.31Ā± 0.287.89Ā± 0.0880.65Ā± 0.375.36Ā± 0.05 MeteoLIP81.31Ā± 0.4792.97Ā± 0.1576.15Ā± 0.357.56Ā± 0.0681.24Ā± 0.385.24Ā± 0.03 F.3DOWNSTREAM FINE-TUNING Using the same Meteo-SigLIP architecture, we fine-tune models initialized from scratch, the SigLIP WebLI checkpoint, and our Meteo-SigLIP pretrained checkpoint on the classification and physical variable estimation tasks. As shown in Table 16, MeteoLIP initialization achieves the best perfor- mance, achieving 81.31 F1 and 92.97 Top-1 accuracy for classification, R 2 of 76.15 with 7.56 MAE for maximum sustained wind speed, and R 2 of 81.24 with 5.24 MAE for minimum central pressure. These results indicate that meteorology-language pretraining provides a stronger initialization for task-specific fine-tuning, further demonstrating the transferability of the learned multimodal repre- sentations. F.4DOWNSTREAM TRANSFER TO WEATHERQA We further evaluate the transferability of Meteo-CLIP representations on WeatherQA (Ma et al., 2024), a severe-weather reasoning benchmark over the CONUS, using 7,304 training samples and 600 test samples. WeatherQA comprises a multiple-choice question answering task for the affected geographic region (Area Affected) and a classification task for the primary weather concern (Con- cerning). For each WeatherQA sample, we extract spatial patch representations from the frozen Meteo-CLIP meteorology encoder, map them to the hidden space of Qwen3-VL-4B-Instruct (Bai et al., 2025) using a learnable projection layer, and adapt the language model using LoRA (Hu et al., 2022) with rank 16 and Ī± = 32. We train for five epochs with AdamW using a batch size of 16, learning rates of 5Ć10 ā5 and 2Ć10 ā4 for LoRA and the projection layer, respectively, and a cosine learning-rate schedule. Meteo-CLIP achieves 49.22Ā± 4.45% accuracy on Area Affected, which requires selecting the af- fected region from four candidate options, and 51.73Ā± 2.41% conditional accuracy on Concerning, an 11-category weather-concern classification task evaluated when the affected area is correctly predicted. These results show that the frozen Meteo-CLIP representations encode spatial and mete- orological information that can be effectively transferred to downstream severe-weather reasoning, further demonstrating the usefulness of the learned meteorological representations beyond the Earth- Align evaluation tasks. 31 Preprint WeatherQA Qualitative Example System Prompt: You are an AI assistant with expertise in severe weather analysis and forecasting. You are provided with a meteorological representation describing atmospheric and surface conditions across the contiguous United States. Your responsibility is to assess potential severe weather conditions, affected areas, and weather concerns. User Prompt: <Parameters> The provided meteorological representation describes atmospheric and surface conditions across the contiguous United States at 11 / 12, 16UTC. The meteorological representation is derived from a 63-variable ERA5 snapshot. Pressure-level variables at 300, 500, 700, 850, and 925 hPa: geopotential, temperature, zonal wind, meridional wind, specific humidity, vertical velocity, and rela- tive vorticity. Surface and single-level variables: volumetric soil water content in layers 1ā4; vertically integrated eastward and northward water-vapour flux; significant wave height; peak wave period; mean wave direction; snowmelt; convective precipita- tion; snowfall; runoff; total precipitation; convective inhibition; cloud-base height; zero-degree level; instantaneous 10-m wind gust; convective available potential energy; total column water vapour; snow depth; mean sea-level pressure; boundary-layer height; total cloud cover; 10-m zonal and meridional wind; 2-m temperature; and 2-m dewpoint temperature. <meteorological representation> </Parameters> <Thought process> Use the provided meteorological representation to answer the following multiple-choice and classifi- cation questions: 1. Analyze the spatial and meteorological conditions represented in <Parameters> and identify regions showing signals associated with severe or hazardous weather. 2. Combine the relevant meteorological information to assess the location and severity of the weather conditions. 3. Map the conclusions to the provided options. Only give the correct option characters, without any other details or explanations. Use JSON format with the keys āArea Affectedā for the first answer and āConcerningā for the second answer. </Thought process> <Item number="1"> Question: Choose one geographical area(s) most likely to be impacted by the severe weather event from the four options provided. A southern/eastern North Carolina and northeastern South Carolina B Central Kansas and Oklahoma C portions of northeast Iowa into far southwest Wisconsin D Far southeast Nebraska...northeast Kansas...extreme northwest Missouri Answer: <answer>Option</answer> </Item> <Item number="2"> Question: Choose the most likely scenario regarding severe weather concerns from the options pro- vided. Options include whether a convective watch has benn issued, the probability of a future watch, or, in winter, potential weather phenomena related to winter storms. A Severe potential...Watch unlikely (confidence level on the expectation of a watch is 5 or 20%) B Severe potential...Watch possible (confidence level on the expectation of a watch is 40 or 60%) C Severe potential...Watch likely (confidence level on the expectation of a watch is 80 or 95%) D Heavy snow E Severe potential...tornado watch likely (confidence level on the expectation of a watch is 80 or 95%) F Severe potential...severe thunderstorm watch likely (confidence level on the expectation of a watch is 80 or 95%) G Winter mixed precipitation H Freezing rain I Severe potential...watch needed soon (confidence level on the expectation of a watch is 95%) J Blizzard K Snow squall 32 Preprint Answer: <answer>Option</answer> </Item> Model Output: "Area Affected": "A", "Concerning": "A" F.5TRAINING DYNAMICS Figure 14a shows the training dynamics of Meteo-CLIP. As training progresses, the CLIP test loss decreases and gradually converges, while retrieval performance generally improves. We observe a similar trend for physical variable estimation, where the R 2 values increase substantially after the early training epochs and then remain relatively stable. In contrast, the classification F1 exhibits larger fluctuations and does not show a similarly consistent increasing trend. To further examine whether these fluctuations reflect the quality of the learned meteorological representations, we perform linear probing on the frozen meteorology encoder checkpoints from epochs 6, 12, 18, 24, and 30. As shown in Figure 14b, both F1 and Top-1 accuracy generally im- prove with training, while Top-3 accuracy remains stable at approximately 98.8%. This indicates that although classification performance under text-prototype inference in the learned meteorology- language space can fluctuate across epochs, the frozen meteorology representations continue to be- come more discriminative for category-level prediction. The nearly saturated Top-3 accuracy further suggests that later training primarily improves the ranking of the correct class among the top candi- date categories, which is better reflected by F1 and Top-1 accuracy. F.6POSITIONAL ENCODING IN THE METEOROLOGICAL ENCODER We investigate two positional encoding approaches for the meteorological encoder, i.e. learnable positional embed- dings (LPE) and 2D rotary positional embeddings (RoPE) (Heo et al., 2024). For 2D RoPE, we adopt the RoPE- Mixed variant, which has been shown to outperform the axial formulation in vision transformers (Heo et al., 2024). Table 17 compares LPE, 2D RoPE, and their combination under both OpenAI pre-trained and scratch initialization. LPE achieves the best retrieval performance across both settings, with a larger performance gap when training from scratch. In addition, combining LPE with 2D RoPE does not provide additional gains. Table 17: Ablation on positional em- bedding under pre-trained and scratch initialization. Init.Pos. EncodingM2T R@1āT2M R@1ā OpenAI LPE 58.79Ā± 1.43 60.10Ā± 0.23 2D RoPE54.93Ā± 1.8156.29Ā± 1.20 LPE + 2D RoPE 53.40Ā± 3.19 56.87Ā± 0.19 Scratch LPE 34.46Ā± 1.78 41.43Ā± 2.21 2D RoPE27.74Ā± 0.6936.03Ā± 0.90 LPE + 2D RoPE 25.61Ā± 5.37 35.23Ā± 2.18 F.7MODEL CAPACITY Figure 15 studies the effect of scaling the encoder from ViT-B to ViT-L. For pretrained Meteo- CLIP, ViT-L achieves a lower training loss than ViT-B (0.0490 vs. 0.0802), but a higher test loss (0.73 vs. 0.65) and lower M2T/T2M R@1 (54.24/57.21 vs. 58.79/60.10). We observe a similar trend for Meteo-SigLIP, where scaling to ViT-L reduces the training loss from 0.2395 to 0.1864, while increasing the test loss from 0.56 to 0.61 and reducing M2T/T2M R@1 from 66.07/66.79 to 63.28/61.66. In comparison, scaling the pretrained CLIP baseline from ViT-B to ViT-L slightly improves both retrieval performance and test loss. When Meteo-CLIP is trained from scratch, ViT-L achieves a lower test loss than ViT-B (0.90 vs. 0.95) and improves M2T/T2M R@1 from 34.46/41.43 to 36.68/41.86. This suggests that although increased capacity can benefit Meteo-CLIP and pre- trained visual representations provide a useful initialization (Section 5.3), scaling larger pretrained vision-language encoders may require greater adaptation and additional meteorology-language su- pervision due to the substantial modality gap between RGB images and multivariate meteorological data. These results position EarthAlign as a challenging setting, where simply fine-tuning a larger model does not guarantee better performance. 33 Preprint GLIMITATIONS AND FUTURE DIRECTIONS Although EarthAlign curates natural event and disaster records from established expert-curated databases and published datasets, their underlying reporting and coverage patterns inevitably shape the geographic, temporal, and event-distribution biases of the dataset. In particular, the United States, Indonesia, and France contain more records than many other countries due to the exten- sive country-level coverage provided by the NOAA Storm Events Database, IndDE, and FraDE, respectively (Table 9). In addition, MeteoLIP currently represents each sample using a single me- teorological snapshot. Future work can extend this setting to multiple timestamps and temporally aggregated inputs, as well as broader Earth-system data products such as operational forecasts and other reanalysis datasets. Incorporating regional or downscaled products with higher spatial resolu- tion may further improve the representation of localized phenomena and enable more comprehensive alignment across temporal scales, spatial resolutions, and Earth-system data sources. 34 Preprint Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 2,3651,914347330971917924231,502167 2209013212318916556989281135 12215950211056292 3250198140031352042 71020719507115835 481,1494151482861,01630836047079365 1696794961434093714,3872,470331531276 4022,2504,39040502028862,7962,8661,644566 2104324413841591681,04929134 9384341041320325 81850168305411274171562,689 Text-Prototype Inference Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 7,40693101213443610551076 6128526451227294965223127 30312620039350000 149140167118828493 442501629115411916 6774104,4624836909 77121212010,0704616017 84898101264115,583108970 7231644183431,2995154 27310090040318930 771601361423295213,496 Linear Probing Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 3,3231,5252141,392351213013341483253 303846161157249136718769138109 4387377310261039350 2790282372257492 77031355071437 14731710304,0789211650396 191878332111612,2365,187969152113130 1,2791,4111,0751944523137668,7071,222549124 4421121401491051,47118100 640437001325619 12210278931815117443,182 Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 7,5647731941322763525 58991673271710154694333108 31253780009239210 15980189173924273 23120203380622612 4981204,5143421608 88132701810,067392107 80707504113115,5931651646 4264510222281,476394 13160360019325514 78160274632163143,541 Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 2,2942,9456054718482392097273215 3461,14426647178124351049459 49884895150352522 1474918410004136638 91431244194010965 1841,33933126758318202608814345177 7372,5911,314225151221,7324972,21669286 1,6514,4683,81225399223151,1783,103747443 13236871811052221,397918 33109135710521857 5368025511181065263281571,746 Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 7,456881111937241095951 5989316381324214524328118 42353190028277100 14513017110101225593 502101629215471016 5573104,49236298013 7181602110,0834613013 10213412102253315,4671311067 10344224382781,3745113 323202170036018725 732001471752870153,523 Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 6,779372593617611513225016 8866171217410118231653217388 11558448212844170 10360289281255621 275065304051186 1271213217903,956896746819 2,875572573542642,2792,943277242274124 5,430656763174615373627,01358145955 35591266139791091,3822970 114061400202884 1322703173518401381783,030 Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather Normal Weather Drought Heat Wave Cold Wave Frost Atmospheric River Cyclone Convective Storm High Wind Fog Winter Weather 7,52011092351722683728 4211,07059332414224763731112 28463750019224110 12690227186525265 24130193371643511 35161104,5024428708 65151701610,079591308 62108671183515,610151938 23431110312451,4283106 12232470018025113 65310396642655153,526 Predicted Category True Category CLIP Meteo-CLIP SigLIP Meteo-SigLIP Figure 13: Confusion matrices under text-prototype inference and linear probing. Cell values report mean counts, while color intensity is normalized by true category to account for class imbalance. 35 Preprint 081624 0.8 1.2 1.6 2.0 CLIP Test Loss 081624 15 30 45 60 M2T R@1 081624 15 30 45 60 T2M R@1 081624 40 45 50 55 60 F1 081624 ā180 ā120 ā60 0 60 Wind R 2 081624 ā100 ā50 0 50 100 Pres R 2 Metric value Epoch (a) 612182430 77 78 79 80 F1 612182430 91.2 91.5 91.8 92.1 92.4 Acc Top-1 612182430 98.7 98.8 98.9 99.0 Acc Top-3 Score (%) Epoch (b) Figure 14: Training dynamics of Meteo-CLIP. (a) CLIP test loss and retrieval, classification, and physical variable estimation performance across epochs. (b) Linear-probing classification perfor- mance using frozen meteorology encoder checkpoints at epochs 6, 12, 18, 24, and 30. 36 Preprint CLIP ViT-B Pretrained CLIP ViT-L Pretrained Meteo-CLIP ViT-B Pretrained Meteo-CLIP ViT-L Pretrained Meteo-CLIP ViT-B Scratch Meteo-CLIP ViT-L Scratch SigLIP ViT-B Pretrained SigLIP ViT-L Pretrained Meteo-SigLIP ViT-B Pretrained Meteo-SigLIP ViT-L Pretrained M2T R@1 āT2M R@1 āTest Loss āTrain Loss ā 20.25 21.62 58.79 54.24 34.46 36.68 32.06 25.09 66.07 63.28 20.03 21.49 60.10 57.21 41.43 41.86 30.23 24.63 66.79 61.66 2.13 2.04 0.65 0.73 0.95 0.90 2.05 2.42 0.56 0.61 0.0053 0.0050 0.0802 0.0490 0.1308 0.1163 0.0373 0.0393 0.2395 0.1864 Figure 15: We compare ViT-B and ViT-L encoder variants using pretrained initialization, and ad- ditionally compare Meteo-CLIP models trained from scratch. We report M2T and T2M R@1 along with training and test loss to examine how model capacity interacts with initialization and meteorological-language inputs. The reported values are averaged over three training runs. 37