Paper deep dive

Automatic Knowledge Graph Construction and Query for Earthquake Catalogs

Yuxin Zhou, Huai Zhang, S. Mostafa Mousavi

Year: 2026Venue: arXiv preprintArea: physics.geo-phType: PreprintEmbeddings: 42

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 8/1/2026, 10:53:49 AM

Summary

The paper presents a systematic application of GraphRAG to raw, tabular earthquake catalogs to automatically construct queryable knowledge graphs. Using three case studies (Qiaojia-Dongchuan, Ridgecrest, Maduo), the authors demonstrate that GraphRAG can generate structurally complete graphs without manual structuring. The study evaluates the pipeline against ground truth, identifying failure modes such as hallucinations and mechanism errors, which are mitigated through seismology-informed prompt engineering. The results show GraphRAG offers a practical, near-zero-cost interface for catalog analysis, outperforming vector-RAG baselines in holistic summarization.

Entities (8)

GraphRAG · method · 98%Maduo · catalog · 95%Qiaojia-Dongchuan · catalog · 95%Ridgecrest · catalog · 95%GPT-4o-mini · model · 90%Leiden algorithm · algorithm · 90%Vector-RAG · baseline · 90%Baihetan Reservoir · location · 85%

Relation Signals (6)

GraphRAG → appliedto → Earthquake Catalogs

confidence 95% · We present the first systematic application of graph based retrieval augmented generation GraphRAG directly to raw, tabular catalog records

GraphRAG → constructs → Knowledge Graphs

confidence 95% · the pipeline builds structurally complete, queryable knowledge graphs for all three

GraphRAG → uses → Leiden algorithm

confidence 90% · detects communities via the Leiden algorithm

GraphRAG → uses → GPT-4o-mini

confidence 90% · LLM-based entity and relationship extraction (using gpt-4o-mini)

Qiaojia-Dongchuan → locatednear → Baihetan Reservoir

confidence 85% · Qiaojia-Dongchuan relocation catalog... adjacent to the Baihetan Reservoir

GraphRAG → outperforms → Vector-RAG

confidence 80% · A vector RAG baseline demonstrates the graph layers distinctive value

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:In recent years, the number of events in earthquake catalogs has significantly increased due to the utilization of more effective deep learning based detectors and phase pickers but answering open ended questions such as what characterizes this sequence? remains constrained by rigid spatiotemporal windowing and subjective expert interpretation. We present the first systematic application of graph based retrieval augmented generation GraphRAG directly to raw, tabular catalog records across three independently featured catalogs, a reservoir adjacent swarm, the 2019 Ridgecrest tectonic sequence, and the 2021 Maduo Mw7.4 aftershock sequence. Without the need for manual data structuring, the pipeline builds structurally complete, queryable knowledge graphs for all three. Rigorous evaluation individually verified against catalog derived ground truth and a rule based reference graph exposes failure modes, and four seismology informed prompt fixes eliminate all targeted fabrications while sharply improving mechanism reasoning. A vector RAG baseline demonstrates the graph layers distinctive value, catalog wide summarization and temporal stage comparison. In addition, we have identified two main pitfalls that need attention. GraphRAG thus offers a practical, transferable, near zero cost query interface for earthquake catalogs, where careful prompting ensures the results are consistently accurate and trustworthy.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

40,632 characters extracted from source content.

Expand or collapse full text

Automatic Knowledge Graph Construction and Query for Earthquake Catalogs Yuxin Zhou 1,2 , Huai Zhang *1 , and S.Mostafa Mousavi 2 Abstract In recent years, the number of events in earthquake catalogs has significantly increased due to the utilization of more effective deep-learning-based detectors and phase pickers but answering open-ended questions such as “what characterizes this sequence?” remains con- strained by rigid spatiotemporal windowing and subjective expert interpretation. We present the first systematic application of graph-based retrieval-augmented generation (GraphRAG) directly to raw, tabular catalog records across three independently featured catalogs: a reservoir-adjacent swarm (Qiaojia-Dongchuan), the 2019 Ridgecrest tectonic sequence, and the 2021 Maduo Mw 7.4 aftershock sequence. Without the need for manual data structuring, the pipeline builds structurally complete, queryable knowledge graphs for all three. Rigorous evaluation — 1,200 answers individually verified against catalog-derived ground truth and a rule-based reference graph — exposes failure modes, and four seismology-informed prompt fixes eliminate all targeted fabrications while sharply improving mechanism reasoning (up to 2.90/3). A vector-RAG baseline demonstrates the graph layer’s distinctive value: catalog- wide summarization and temporal-stage comparison. In addition, we have identified two main pitfalls that need attention. GraphRAG thus offers a practical, transferable, near-zero-cost query interface for earthquake catalogs, where careful prompting ensures the results are consistently accurate and trustworthy. Keywords: knowledge graph; retrieval-augmented generation; GraphRAG; earthquake cata- log; large language model; benchmark evaluation Cite this article as Zhou, Y., H. Zhang, and S.M. Mousavi (2026). Automatic Knowledge Graph Construction and Query for Earthquake Catalogs, The Seismic Record 0(0), 1–10, doi: 00.0000/000000000. Supplemental Material Introduction Dense-array monitoring (Ross et al., 2019; Shelly, 2020) and deep-learning phase detection/picking (Zhu and Beroza, 2019; Mousavi et al., 2020; Ross et al., 2018; Mousavi and Beroza, 2022) have together driven an order-of-magnitude expansion in earthquake-catalog size, for example, the 1. State Key Laboratory of Earth System Numerical Modeling and Application, College of Earth and Planetary Sciences, University of Chinese Academy of Sciences, Beijing 100049, China,https://orcid.org/0009-0000-2877-9173 (FA)https://orcid.org/0000-0003-0411-4841 (SA) 2. Department of Earth and Planetary Sciences, Harvard University, Cambridge, MA 02138, USA, https://orcid.org/0009-0000-2877-9173 (FA)https://orcid.org/0000-0001- 5091-5370 (TA) *Corresponding author: H. Zhang, hzhang@ucas.ac.cn © 2026. The Authors. This is an open access article distributed under the terms of the C-BY license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 2021 Maduo 푀 푤 7.4 aftershock sequence contains over 10,000 relocated events (Guan et al., 2024). Beyond scale, a deeper challenge is characterizing what high-resolution catalogs represent physically: mainshock-aftershock decay (Gutenberg and Richter, 1944; Utsu et al., 1995), swarms, foreshock sequences identifiable only in hindsight, and induced seismicity (Gupta, 2002) are still distinguished mainly by coarse qualitative criteria, or point-process mod- els fit after the fact (Ogata, 1988), that does not allow quanti- tative, reproducible cross-sequence comparison in real time. However, this increase in the typical seizes of earthquake catalogs has not helped yet to this major challenge. Our analyses and interpretations of the high-resolution deep- learning based catalogs remain more or less limited to a few classical engineered features. https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record1 arXiv:2607.24984v1 [physics.geo-ph] 27 Jul 2026 Knowledge graphs organize such semantically linked information as reasoning-ready (entity, relationship, entity) triples (Hogan et al., 2021), and have been applied to earth- quake emergency response (Qiu et al., 2024) and seismic metadata organization (Davis and Hunt, 2024). Retrieval- Augmented Generation (Lewis et al., 2020) is an artificial intelligence architecture that enhances the accuracy and physical reliability of large language models by grounding their responses in dynamically retrieved, domain-specific external data. Microsoft’s GraphRAG (Edge et al., 2024), rely on RAG systems to automatically extracting enti- ties/relationships, detects communities via the Leiden algo- rithm (Traag et al., 2019), and generates natural-language summaries and query access, with no predefined ontology – part of a broader effort to unify LLMs and knowledge graphs (Pan et al., 2024). This raises a natural question: can GraphRAG be applied directly, end-to-end, to earth- quake catalogs of varying scale, region, and character to produce a working natural-language query interface, and can its most damaging failure modes be suppressed with modest, seismology-specific engineering effort rather than a full custom rebuild? RAG has recently begun to be applied within seis- mology and geohazard research specifically. Yao et al. (2025) combine knowledge-graph construction with a hybrid RAG strategy for earthquake emergency response, extracting entities and relationships from thousands of professional emergency-management documents; Orantes- Jiménez (2025) use LLMs to build knowledge graphs from earthquake news articles. Recent reviews (Yu et al., 2025; Li and Zhou, 2026) note that RAG in the geosciences remains applied mainly to narrative, prose-style sources – news, reports, technical literature – rather than to the raw, tab- ular observational records a catalog itself consists of, and call for RAG architectures tailored to structured geoscien- tific data. Our study differs in three respects: it applies RAG directly to structured, numerical catalog records rather than narrative text describing events after the fact; it tar- gets fully automatic, schema-free construction with no per- catalog ontology or extraction-rule engineering, as opposed to the hand-designed ontologies underlying prior earth- quake knowledge-graph work; and it evaluates GraphRAG specifically, whose automatic community detection and hierarchical summarization are built to transform thou- sands of brief catalog records into comprehensive, sequence- level summaries—a core capability that our benchmark explicitly evaluates. Section 3 demonstrates that fully automatic indexing and natural-language querying run end-to-end on all three catalogs under default prompts, establishing mechani- cal transferability while exposing low baseline answer quality (catalog-only averages 0.76–1.22/3). Section 4 quantifies the catalog-dependent effect of narrative-text enrichment. Section 5 describes an iterative, seismology- oriented prompt-engineering scheme that eliminates the targeted fabrication modes across all 600 post-fix answers. Furthermore, this method introduced a baseline vector- RAG comparison (embedding-similarity retrieval over the same text chunks, with no graph layer) to quantify the community-report layer’s impact on holistic questions. Section 6 details the specific catalog errors identified during verification—such as the systematic misdating of the Ridgecrest mainshock and the hallucinated inclusion of an unlisted Maduo event. We report these issues to guide safe deployment, rather than as the study’s central finding. Extended examples and full per-condition bench- mark results as well as prompt details are provided in the Supplementary Material (SM). Materials and Methods Case Studies We use three independent catalogs differing in scale, region, and origin, totaling 20,027 events (Figure 1; overview table in SM Table S4): (1) the Qiaojia-Dongchuan relocation cat- alog, 5,218 events recorded by a temporary dense array adjacent to the Baihetan Reservoir between 23 Aug 2022 and 17 Mar 2023; (2) the 2019 Ridgecrest sequence, 4,188 events (푀≥2.0) spanning July 2019, a standard catalog (Shelly, 2020) on a purely tectonic strike-slip system in the Mojave Desert; and (3) the 2021 Maduo 푀 푤 7.4 sequence, the largest of the three at 10,621 events (푀≥0.5, 1 Jun 2021–8 Jun 2023), relocated by Guan et al. (2024). This aftershocks-only cat- alog begins after the true 22 May 2021 mainshock and contains no May-2021 data. GraphRAG pipeline and evaluation design To effectively translate complex seismicity data into a structured, queryable knowledge base, the system architec- ture relies on two primary phases: Indexing and Retrieval (Figure 2a).The indexing phase transforms raw, disparate https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record2 (a) Qiaojia-Dongchuan (n=5,218) 102°30'E103°00'E103°30'E 26°00'N 26°30'N 27°00'N 27°30'N 0246 M (b) Ridgecrest (n=4,188) 117°45'W117°30'W117°15'W 35°30'N 35°45'N 36°00'N 0246 M (c) Maduo (n=10,621) 98°E99°E 34°N 35°N 0246 M Figure 1. Spatial distribution of seismic events. (a) Qiaojia-Dongchuan; (b) Ridgecrest; (c) Maduo. Epicenters colored by magnitude. Alt text: Three side-by-side scatter maps with latitude/longitude axes, each titled with catalog name and event count. Dots mark epicenters, colored on a dark-purple-to-yellow magnitude scale (0–7) per a small color-bar legend in each panel’s corner. (a) Qiaojia-Dongchuan: a roughly circular, dense blob of points near 103 ◦ E, 27 ◦ N. (b) Ridgecrest: a narrow, elongated diagonal band of points trending NW-SE. (c) Maduo: a long, thin, mostly east-west line of points spanning about 98–99.5 ◦ E, with two small blue lake outlines in the upper left. earthquake catalog entries into a deeply connected net- work. It begins with LLM-based entity and relationship extraction (using gpt-4o-mini) to identify key seismic features—such as specific earthquakes, fault structures, and their spatiotemporal links. Next, Leiden community detec- tion groups these interconnected events into highly related, tectonic or sequence-based clusters (e.g., distinct swarms or aftershock zones). The system then processes these clusters through automated community-report generation, creating high-level textual summaries that describe the defining characteristics of each localized sequence. During the retrieval phase, the system supports two com- plementary search modes for interacting with the indexed seismic data. Global Search performs a holistic synthesis across all community reports, making it ideal for answer- ing broad, sequence-level questions regarding overarch- ing migration patterns or aggregate statistics. In contrast, Local Search executes precise entity retrieval to isolate exact details concerning specific mainshocks, stations, or local- ized catalog anomalies. Full parameter configurations for both the indexing and retrieval pipelines are detailed in the Supplementary Material (SM). Evaluation proceeds in three stages: (i) capability demon- stration under default prompts (Section 3); (i) narrative- text enrichment, evaluated with a 100-question benchmark spanning five categories – (A) overall summarization, (B) precise statistics, (C) local retrieval, (D) physical mecha- nism, (E) temporal-stage comparison; 20 questions each (Section 4); and (i) iterative, seismology-oriented prompt engineering targeting the exposed failure modes (Section 5). Two independent ground truths anchor scoring: statistics computed directly from each catalog’s raw records, and, for Qiaojia-Dongchuan, a rule-based knowledge graph (10,766 nodes; 113,620 edges) answering the same questions deter- ministically. To provide a point of comparison, we identi- cally scored a standard vector-RAG baseline. This baseline used the exact same text chunks, embeddings, and LLM as our GraphRAG pipeline, but relied on top-10 chunk similar- ity for retrieval rather than a graph layer We employed an LLM assistant to score the 1,200 GraphRAG and 300 baseline responses (from 0 to 3) against two ground truths. Following this automated scoring, the authors manually validated all refusals and Category-B/D answers against raw catalog statistics, and the first author resolved all borderline evaluations. To validate the auto- mated scoring, the first author conducted a blind audit of 29 stratified-random answers. When adjudicated against the ground truth, this audit showed 76% exact agreement with the LLM workflow and 100% agreement within one point, with zero unresolved fabrication disputes. In contrast, a purely unaided human review achieved only 52% exact agreement and erroneously accepted two fabricated answers as correct, highlighting the critical need for ground-truth anchoring over subjective human judgment. Responses were graded using the following rubric: a score of 3 indicates a correct, independently verifiable central claim; a score of https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record3 INDEXING (one-time, fully automatic) RETRIEVAL (interactive queries) Earthquake catalog (+ narrative text) Text chunking Entity & relationship extraction (gpt-4o-mini) Leiden community detection Knowledge graph (entities + relationships, communities colored) Community reports (LLM summaries) Global Search map-reduce over community reports Local Search direct entity retrieval Grounded natural-language answer user query (a) 123 4 (b) Figure 2. (a) Schematic of the GraphRAG indexing and retrieval pipeline. (b) Entity-relationship graph automatically constructed by GraphRAG from the Qiaojia catalog (5,247 entities; 3,920 relationships), with no hand-designed schema; GEO (coordinate/date tags), EVENT (individual earthquake records), and a small residual of Unclassified/Organization entities from generic extraction. Alt text: Two-panel figure. (a) Flowchart: a blue dashed box ‘INDEXING’ contains four linked boxes–catalog input, text chunking, LLM entity/relationship extraction, Leiden community detection–feeding a small node-cluster icon labeled ‘Knowledge graph’ and a ‘Community reports’ box; an orange dashed box ‘RETRIEVAL’ shows ‘Global Search’ and ‘Local Search’ boxes both arrowing into a ‘Grounded natural-language answer’ box, with a user-query arrow above. (b) A dense force-directed node-link network on a gray background: small dots joined by thin gray edges, colored by legend–blue GEO, teal EVENT, orange Unclassified, green Organization–with large teal hub nodes near the center. 2 denotes a correct central claim grounded in the data, but containing secondary errors, omissions, or unverifiable ele- ments; a score of 1 applies when the central claim is wrong or missing (e.g., refusing an answerable question or misrep- resenting data subsets) despite using genuine records; and a score of 0 is reserved for fabricated values, hallucinated records, or false claims presented as factual. Refusals earn a 3 only if the required information is completely absent from the catalog. https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record4 To quantify uncertainty, 95% bootstrap confidence inter- vals were calculated for the per-answer scores. For indi- vidual category cells (푛=20), the interval half-widths reached a maximum of 0.66, meaning score differences below ∼0.5 are indistinguishable from scoring noise. For broader condition averages (푛=100), this resolu- tion tightens to ±0.22, establishing a ∼0.3 threshold for significance. Consequently, minor variations—such as Maduo’s +0.04 gain, the narrative-enrichment declines, and GraphRAG’s overall edge over the baseline on Qiaojia and Ridgecrest—fall within this noise margin. However, the sharp improvements in Category D, the overall score gains for Qiaojia (+0.73) and Ridgecrest (+0.45), and the base- line’s outperformance of GraphRAG on Maduo remain sta- tistically robust (Table 2) Results: Capability Demonstration With no hand-designed schema and no catalog-specific con- figuration, GraphRAG automatically built structurally com- plete, queryable knowledge graphs for all three catalogs from raw tabular records alone: Qiaojia-Dongchuan (5,247 entities, 3,920 relationships, 146 communities; Figure 2b), Ridgecrest (1,985 entities, 2,449 relationships, 83 commu- nities), and Maduo (15,402 entities, 11,784 relationships, 600 communities). Entity types are consistently dominated by coordinate/date (GEO) tags (∼72–83%) across all three, confirming that indexing capability is general and does not depend on catalog region or scale. Per-catalog entity- type and community-size distributions are given in the SM (Figures S1–S2). The resulting graphs support natural- language queries that a fixed-field database cannot: Local Search queries retrieve precise, traceable individual records (e.g., correctly naming Ridgecrest’s 푀7.1 mainshock and its 6 July date), and Global Search synthesizes holistic, catalog-wide descriptions of sequence character. However, this out-of-the-box functionality has reliability issues. When reviewing 80 Ridgecrest benchmark answers (excluding local retrieval), the system frequently misidentified smaller earthquakes (푀3.5–푀5.5) as the sequence’s largest event. Additionally, default prompts caused the model to hallu- cinate fluent but baseless physical mechanisms, such as falsely attributing Ridgecrest’s seismicity to reservoir or fluid processes. Under the default, catalog-only condition, the average scores were generally poor: 0.96 for Qiaojia, 1.22 for Ridgecrest, and 0.76 for Maduo out of a possi- ble 3. Qiaojia struggled most with mechanism questions (Category D at 0.55/3), while Maduo suffered from uni- formly low scores across all categories (0.45–0.95/3). These baseline deficits, detailed in SM Table S5, motivated the targeted prompt-engineering strategy in Section 5, which successfully doubled several of these scores. Table 1 gives six representative question/answer pairs – four correct, high- quality responses spanning extremal retrieval, aftershock association, narrative-grounded mechanism reasoning, and a correct refusal, plus the two residual errors detailed in Section 6 – to illustrate concretely what the model’s output looks like; extended examples are given in the SM. Results: Narrative-Text Enrichment To assess the effect of narrative enrichment, each cata- log was re-indexed with contextual literature—Baihetan Reservoir background for Qiaojia, Shelly (2020) for Ridgecrest, and Guan et al. (2024) for Maduo—and re-evaluated. The effect is catalog-dependent and non- monotonic (Category-D scores in SM Table S6). Qiaojia’s Category-D score rises (0.55→1.35) but mainly reflects a more cautious tone matching the text’s “contested” framing rather than genuine use of its specific named entities. For Ridgecrest, the score remained essentially unchanged (1.45 to 1.40). When asked about industrial activity, the model simply refused to answer instead of using the newly provided text by Shelly (2020) to confidently deny the claim. While a blank refusal is the correct response when using only the raw catalog—since the bare catalog lacks mech- anism data—it represents a failure to utilize the enriched text, which contained the evidence needed to explicitly rule out the mechanism. While Maduo’s score increased (0.75 to 1.65), the enriched text introduced a major new mistake: the model confused the natural deep-fluid processes described by Guan et al.. with human-induced hydraulic fracturing. This error did not occur when the model relied on the cat- alog alone. Additionally, under default settings, the model hallucinated the answer for the "largest-magnitude event". It correctly named the real-world May 2021 mainshock, but because the provided catalog data did not start until June 1, the model pulled this fact from its outside memory rather than the data—a "coincidentally correct" error that persisted even after prompt engineering. https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record5 Table 1. Representative example queries and GraphRAG answers, spanning correct, high-quality responses and the two residual error classes of Section 6 (translated from the original Chinese-language queries; retrieval mode and condition in parentheses; all answers from the fully re-verified post-fix benchmark). QuestionAnswer (abridged)Outcome When, where, and at what mag- nitude did the largest event in the Qiaojia-Dongchuan catalog occur (Global Search, catalog-only)? “The largest-magnitude event in the catalog occurred on 18 October 2022 at 22:12:35.080..., with magnitude 4.2.” Correct (true: 푀4.18, 18 Oct 2022 22:12) Which events are directly related to the largest-magnitude event in the 2019 Ridgecrest sequence (Local Search, catalog-only)? “The largest-magnitude event ... is the M7.10 earthquake on 2019-07-06 at 03:19:52.860. This event is a major mainshock, followed by a series of aftershocks directly related to it...” Correct What is the approximate rup- ture/triggeringmechanism ofthe2021Maduo푀 푤 7.4 sequence(GlobalSearch, narrative-enriched)? “...generated by the KMPJ fault, clearly identified as the main seismogenic fault, ... interacting with secondary faults (the Maduo-Gande and Jiuzhi faults)... the aftershock sequence is complex, with a bifurcating geometry...” Correct;grounded in the enrichment textratherthan pretrained priors Is the 2019 Ridgecrest seismic- ity associated with industrial activ- ity such as hydraulic fracturing, wastewater injection, or reser- voir impoundment (Global Search, catalog-only)? “I am sorry but I am unable to answer this question given the provided data.” Correctrefusal: the purely tectonic catalogcontains no such evidence; defaultprompts hadanswered frombackground knowledge What is the trend in seismic- ity rate over time in the 2019 Ridgecrestsequence(Global Search, catalog-only)? “...on July 4, 36 events were recorded, with a maximum magni- tude of 4.6. Activity then peaked on July 6, with multiple nearby epicenters recorded, maximum magnitude 4.97...” Wrong: omits the true푀7.1main- shock on its own day (Section 6) Whatisthelargest- magnitude event in the Maduo sequence(GlobalSearch, narrative-enriched)? “...the sequence’s main event is the 푀 푤 7.4 earthquake of 22 May 2021... recorded as EVENT 2021-05-22T00:00:00...” Wrong: catalog con- tains no May-2021 data (Section 6) Results: Iterative Prompt Engineering Following the identification of mechanistically distinct fail- ure modes in Sections 3 and 4, we deployed iteratively verified prompt and configuration adjustments to address four specific issues identically across all conditions. These targeted errors consisted of magnitude fabrication via field confusion, historical-earthquake conflation, cross-catalog place-name contamination, and a scope-collapse bug in community detection. Each intervention was tailored to a diagnosed root cause; for instance, the scope-collapse issue was traced to a default GraphRAG clustering parameter that inadvertently discarded the majority of catalog entities. Comprehensive diagnostic evidence, exact prompt modifi- cations, and mechanistic details are provided in SM Text S5. We re-indexed every affected condition and re-ran the full benchmark, individually re-verifying all answers (Figure 3; Table 2). All three targeted fabrication modes were effec- tively eliminated: no impossible magnitude value appeared in any of the 600 re-verified post-fix answers (versus repeated “푀10.4”/“푀12.8” before), no cross-catalog place- name contamination was found, and historical-earthquake conflation (previously ∼1 in 5 relevant answers) disap- peared. Despite the prompt fixes, two distinct types of hallu- cinations remained: presenting out-of-catalog events as real data, and inventing aggregate counts (detailed in Section 6). While scores for physical mechanism questions (Category https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record6 D) improved dramatically—jumping from 0.55 to 2.90 for Qiaojia and 1.45 to 2.85 for Ridgecrest—precise statistics (Category B) remained persistently weak. Category B scored between 0.50 and 1.04 out of 3, making it the lowest- performing category in five out of six test conditions. The main issue is that the model severely undercounts totals and rates. This happens because a retrieval-then-synthesize sys- tem tries to answer math questions using a few text snippets rather than scanning the entire database—a fundamental architectural flaw rather than a prompt issue (detailed fur- ther in SM Text S5.4). The vector-RAG baseline (Table 2) shows that GraphRAG’s advantage concentrates precisely where its community reports are designed to help: holistic sum- marization (Category A: 1.85 vs. 1.30 on Qiaojia) and temporal-stage comparison (Category E: 1.70 vs. 1.10 on Ridgecrest), where top-푘 chunk retrieval cannot synthe- size catalog-wide structure and mostly returns hedged partial descriptions or refusals. Category B is comparably weak for both architectures (0.85–1.05 across the Qiaojia and Ridgecrest catalog-only conditions; lower still for GraphRAG on Maduo), confirming this limitation is common to retrieval-then-synthesize systems rather than specific to GraphRAG. The baseline’s perfect Category-D scores (3.00) reflect uniform honest refusals scored as cor- rect because the catalogs contain no mechanism evidence; GraphRAG’s 2.85–2.90 comes from substantive grounded answers, so Category D is uninformative here – the architec- tural comparison rests on Categories A and E. On Maduo, however, the baseline’s uniformly honest refusals outscore GraphRAG’s degraded index (average 1.40 vs. 0.80): the graph layer’s value is conditional on a healthy index. The ranking is sensitive to the refusal-scoring convention (Table 2, note), so we report both. Notably, the baseline reproduces the famous-mainshock intrusion of Section 6 – misdating Ridgecrest’s 푀7.1 from pretrained knowledge while explicitly admitting the retrieved chunks lack it – evidence that this hazard is intrinsic to LLM synthesis, not to GraphRAG. Residual Catalog-Specific Errors Our per-answer verification of the fully fixed pipeline (Section 5) also surfaced two narrower, catalog-specific errors, concentrated on the two catalogs tied to a globally famous mainshock – though not exclusively: a mechani- 0123 Mean verified score (0–3) +0.55 +0.50 +0.50 A Summarization +0.29 −0.25 −0.25 B Statistics +0.45 +0.15 −0.45 C Local retrieval +2.35 +1.40 +0.45 D Mechanism ±0.00 +0.45 −0.05 E Stage comparison Δ Qiaojia Ridgecrest Maduo Before (default prompts) After (targeted fixes) Figure 3. Per-category verified benchmark scores before (open circles; default prompts) and after (filled circles; targeted fixes) the Section 5 prompt-engineering scheme, catalog-only condition. Arrows show the direction of change for each catalog; the right column (∆) gives the per-catalog score change. Physical-mechanism questions (Category D) improve most; precise-statistics questions (Category B) remain architecture-limited. Alt text: Dumbbell (dot-and-arrow) plot. X-axis: mean verified score, 0 to 3. Y-axis: five question categories (A–E), each with three colored rows for Qiaojia (blue), Ridgecrest (orange), and Maduo (green). Each row shows an open circle (default prompts) joined by an arrow to a filled circle (after fixes), pointing right for gains and left for losses; a right-hand ∆ column lists each numeric change. Category D shows the longest rightward arrows, up to +2.35; Category B shows short arrows, including two negative changes. cal scan of all 200 post-fix Qiaojia answers for dates or coordinates outside the catalog’s range found six (3%, all Category-C local-retrieval questions, in both conditions) list- ing wholly invented event records; the same scan flags zero Ridgecrest answers. We report these to inform responsible deployment, not as a limitation of the pipeline’s core query capability demonstrated in Sections 3–5. On Ridgecrest, the true 푀7.1 mainshock (6 Jul 2019, the busiest day) is placed on 4 or 5 July instead in 50– 70% of relevant re-verified answers, often merged with the 푀6.4 foreshock date; narrative enrichment did not reduce this. The correct answers in Section 3 and Table 1 are drawn from the complementary 30–50%: the same ques- tion type yields the correct date in a substantial minority of answers, which is precisely what makes this error hazardous – spot-checking a few correct answers cannot exclude it. On Maduo, whose catalog begins 1 June 2021 with no May data, the real out-of-catalog 22-May-2021 mainshock is nonethe- less presented as the catalog’s own largest event in up to 60% https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record7 Table 2. Verified category-average scores after the targeted fixes (Section 5), all catalogs/conditions, and the vanilla vector-RAG baseline (catalog-only). ConditionABCDEAvg Qiaojia catalog-only1.851.041.352.901.301.69 Qiaojia +narrative1.850.901.301.601.501.43 Ridgecrest catalog-only1.700.851.252.851.701.67 Ridgecrest +narrative1.230.901.052.601.101.38 Maduo catalog-only1.200.700.501.200.400.80 Maduo +narrative0.650.500.902.151.151.07 Qiaojia vector-RAG baseline1.301.001.153.001.001.49 Ridgecrest vector-RAG baseline1.451.051.203.001.101.56 Maduo vector-RAG baseline1.001.001.003.001.001.40 A: summarization; B: statistics; C: local retrieval; D: mechanism; E: stage comparison. Baseline rows share chunks, embed- dings, LLM, and questions with the corresponding catalog-only condition (Section 2); their Category-D scores reflect uniform honest refusals scored as correct. Under the stricter convention scoring refusals on answerable questions 0 rather than 1, the baseline averages fall to 0.98, 1.20, and 0.69. of narrative-enriched answers (versus≤15% catalog-only), alongside an independently fabricated “78,832 aftershocks” figure recurring in ∼30% of enriched mechanism/stage- comparison answers. We interpret the catalog-dependent asymmetry as evidence that an LLM’s pretrained knowledge of a globally famous, heavily reported event can, in a minor- ity of answers, override retrieved catalog-grounded evidence during synthesis – a narrower and more specific hazard than the general “ungrounded generation” hallucination described in the broader literature (Ji et al., 2023), and one that our Section 5 HISTORICAL_EVENT fix, designed for a structurally similar problem (historical-earthquake con- flation), did not fully generalize to suppress. These hazards resist casual review: in our scoring audit (Section 2), two fabricated-record answers were initially rated fully correct by an unaided human pass and were caught only when their dates and coordinates were checked against the catalog’s actual range. Extended examples are given in the SM. Discussion and Conclusions This study demonstrates that GraphRAG can be applied directly, end-to-end, to earthquake catalogs of markedly dif- ferent scale, region, and origin – with no hand-designed schema, no custom extraction rules, and no per-catalog reconfiguration – to produce structurally complete knowl- edge graphs supporting flexible natural-language query, with verified instances of precise record retrieval and grounded mechanism reasoning (Table 1), though answer reliability varies by catalog and question category (Table 2). This is, to our knowledge, the first systematic demonstration of GraphRAG’s schema-free construction capability trans- ferring across independent earthquake catalogs at essen- tially zero ontology-engineering cost, although downstream answer quality does not transfer uniformly (post-fix catalog- only averages of 1.69, 1.67, and 0.80/3). Beyond capability, we show that reliability is not fixed once GraphRAG is applied out of the box, but can be mea- surably improved with modest, seismology-oriented prompt engineering: our four targeted fixes (Section 5) eliminated three concrete error types from the re-verified answer set entirely and raised mean scores from 0.76–1.22/3 (SM Table S5) to 0.80–1.69/3 (Table 2) – catalog-only gains of +0.73 for Qiaojia and +0.45 for Ridgecrest, but an essen- tially flat +0.04 for Maduo, where gains in summarization and mechanism reasoning were offset by declines in statis- tics, retrieval, and stage comparison. These findings point to a practical and affordable strategy for deploying LLM query tools on earthquake catalogs: start with the default https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record8 GraphRAG pipeline, test it against a small ground-truth benchmark, and refine the prompts instead of building a new system from scratch. However, there is a catch: while this approach successfully eliminated the targeted hallu- cinations across all test cases, it only produced signifi- cant overall score improvements for two of the three cata- logs. Narrative-text enrichment, by contrast, reduced post- fix overall scores for Qiaojia (1.69→1.43/3) and Ridgecrest (1.67→1.38/3); it raised Maduo’s (0.80→1.07/3), but at the cost of more frequent out-of-catalog mainshock intrusions there (up to 60% of enriched answers versus≤15% catalog- only), so it should be adopted only with per-catalog verifica- tion (Section 4). Deployment is limited by two main caveats, the first being that precise-statistics queries (Category B) performed poorly across all conditions even after applying fixes, scoring just 0.50–1.04 out of 3. This category scored the lowest in five out of six evaluation conditions, which points to a funda- mental structural flaw in retrieval-then-synthesize architec- tures when handling aggregate or ranking tasks. Therefore, rather than relying on prompt engineering to fix this issue, we recommend routing these specific mathematical ques- tions directly to deterministic database or graph queries. The second caveat involves the residual errors detailed in Section 6, which primarily affected catalogs associated with well-known mainshocks. These errors show that verifying the output for each specific catalog remains a necessary step before deployment, especially when analyzing well- known events. A promising future fix would be explicitly instructing the model to never use its pre-trained back- ground knowledge to "correct" the retrieved data, though we did not have the chance to design and test that approach in this study. Because all our results rely on a single model (gpt-4o-mini), the rate of hallucinations and the effec- tiveness of our fixes might vary if a different model is used. However, since the vector-RAG baseline reproduced the exact same hallucination (Section 6), we know this specific flaw stems from the LLM itself, not GraphRAG’s architecture. Ultimately, neither limitation invalidates our main conclusion: GraphRAG is a highly useful tool for auto- matically building earthquake-catalog knowledge graphs, performing strongly for Qiaojia and Ridgecrest, and more modestly for Maduo. When the index is healthy (as with Qiaojia and Ridgecrest), GraphRAG’s community-report layer noticeably outperforms a standard vector-RAG base- line on complex, holistic questions. Conversely, when the index degrades (as with Maduo), the baseline matches or even beats GraphRAG. Overall, combining this architecture with targeted prompt engineering yields genuine, verifiable improvements in reliability. Data and Resources All data and codes are available at https://doi. org/10.5281/zenodo.21459373. The Supplementary Material accompanying this article provides full pipeline and benchmark parameters, per-category and per-catalog score tables, entity-type and community-size distributions, exact prompt-engineering modifications, and extended question/answer examples supporting Sections 3–6. Declaration of Competing Interests The authors acknowledge that there are no conflicts of interest recorded. Acknowledgments Y.Z. and S.M.M were supported by Harvard Milton Fund. The authors thank the developers of the GraphRAG frame- work and the seismological data providers whose catalogs and publications made this study possible. We also thank the members of Plantcore.AI, whose insights during our discussions helped inspire this work. References Davis, W. and C. R. Hunt (2024). Knowledge graphs for seismic data and metadata. Appl. Comput. Geosci. 21, 100151. Edge, D., H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson (2024). From local to global: A graph RAG approach to query-focused summarization. Guan, P. H., J. S. Lei, and D. P. Zhao (2024). Machine-learning based location of the 2021 Mw 7.4 Maduo earthquake sequence: Insight into intraplate seismogenesis. Tectonophysics 888, 230458. Gupta, H. K. (2002). A review of recent studies of triggered earth- quakes by artificial water reservoirs with special emphasis on earthquakes in Koyna, India. Earth-Sci. Rev. 58(3–4), 279–310. Gutenberg, B. and C. F. Richter (1944). Frequency of earthquakes in California. Bull. Seismol. Soc. Am. 34(4), 185–188. Hogan, A., E. Blomqvist, M. Cochez, C. d’Amato, G. de Melo, C. Gutierrez, S. Kirrane, J. E. L. Gayo, R. Navigli, S. Neumaier, A. C. N. Ngomo, A. Polleres, S. M. Rashid, A. Rula, L. Schmelzeisen, J. Sequeda, S. Staab, and A. Zimmermann (2021). Knowledge graphs. ACM Comput. Surv. 54(4), 1–71. https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record9 Ji, Z., N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung (2023). Survey of hallucination in natural language generation. ACM Comput. Surv. 55(12), 248. Lewis, P., E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2020). Retrieval-augmented generation for knowledge- intensive NLP tasks. In Advances in Neural Information Processing Systems, Volume 33, p. 9459–9474. Li, W. and Y. Zhou (2026). Toward knowledge-enhanced geohazard intelligence: A review of knowledge graphs and large language models. GeoHazards 7(2), 40. Mousavi, S. M. and G. C. Beroza (2022). Deep-learning seismology. Science 377(6607), eabm4470. Mousavi, S. M., W. L. Ellsworth, W. Zhu, L. Y. Chuang, and G. C. Beroza (2020). Earthquake transformer – an attentive deep-learning model for simultaneous earthquake detection and phase picking. Nat. Commun. 11, 3952. Ogata, Y. (1988). Statistical models for earthquake occurrences and residual analysis for point processes. J. Am. Stat. Assoc. 83(401), 9–27. Orantes-Jiménez, D. (2025). Harnessing large language models to build knowledge graphs from earthquake news. Int. J. Digit. Earth 18(2), 2594950. Pan, S., L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu (2024). Unifying large language models and knowledge graphs: A roadmap. IEEE Trans. Knowl. Data Eng. 36(7), 3580–3599. Qiu, P. Y., L. K. Pang, Y. Luo, Y. H. Liu, H. Q. Xing, K. Liu, and G. L. Zhuang (2024). Earthquake event knowledge graph con- struction and reasoning. Geomat. Nat. Hazards Risk 15(1), 2383768. Ross, Z. E., M. A. Meier, E. Hauksson, and T. H. Heaton (2018). Generalized seismic phase detection with deep learning. Bull. Seismol. Soc. Am. 108(5A), 2894–2901. Ross, Z. E., D. T. Trugman, E. Hauksson, and P. M. Shearer (2019). Searching for hidden earthquakes in Southern California. Science 364(6442), 767–771. Shelly, D. R. (2020). A high-resolution seismic catalog for the initial 2019 Ridgecrest earthquake sequence: Foreshocks, aftershocks, and faulting complexity. Seismol. Res. Lett. 91(4), 1971–1978. Traag, V. A., L. Waltman, and N. J. van Eck (2019). From Louvain to Leiden: Guaranteeing well-connected communities. Sci. Rep. 9, 5233. Utsu, T., Y. Ogata, and R. S. Matsu’ura (1995). The centenary of the Omori formula for a decay law of aftershock activity. J. Phys. Earth 43(1), 1–33. Yao, L., F. Ren, and K. Du (2025). From knowledge graph con- struction to retrieval-augmented generation: A framework for comprehensive earthquake emergency support. Geo-Spat. Inf. Sci. 29(1). Yu, R., S. Luo, R. Ghosh, L. Li, Y. Xie, and X. Jia (2025). RAG for geoscience: What we expect, gaps and opportunities. Zhu, W. Q. and G. C. Beroza (2019). PhaseNet: A deep-neural- network-based seismic arrival-time picking method. Geophys. J. Int. 216(1), 261–273. Manuscript Received 00 Month 0000 https://w.seismosoc.org/publications/the-seismic-record/•DOI: 00.0000/000000000The Seismic Record10

Instant research discovery