Paper deep dive

Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction

Zhuowen Liang, Zhengxuan Zhang, Jiayang Wang, Jiazhuo Chen, Nan Tang

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 104

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 8/12/2026, 1:40:38 AM

Summary

The paper introduces Doc2DB-Bench, a benchmark for evaluating the construction of relational databases from long, heterogeneous documents, addressing the limitations of existing Document-to-Table benchmarks. It also presents DataMosaic, a system that uses an extract-verify-iterate loop to ensure relational faithfulness, integrity constraints, and semantic consistency in the generated database instances.

Entities (9)

Doc2DB-Bench · benchmark · 98%DataMosaic · system · 97%Doc2DB · problemtask · 96%Company · entity-table · 92%Hold · relationship-table · 92%Alpha Holdings · entity-instance · 90%Arena Tech · entity-instance · 90%Coyni Corp · entity-instance · 90%Coyni Inc · entity-instance · 85%

Relation Signals (6)

Doc2DB-Bench → contains → 203 long-document instances

confidence 95% · Doc2DB-Bench, a benchmark for Document-to-Database construction, containing 203 long-document instances across 42 schemas and seven domain groups

Doc2DB-Bench → evaluates → Document-to-Database construction

confidence 95% · We introduce Doc2DB-Bench, a benchmark for Document-to-Database construction

DataMosaic → uses → extract-verify-iterate loop

confidence 94% · DataMosaic... organized as an extract-verify-iterate control loop. An orchestrator coordinates entity and relationship extractors with verification and repair

Doc2DB → requires → relational faithfulness

confidence 93% · Doc2DB-Bench thus provides a testbed for reliable, auditable, and relationally faithful LLM-based data systems.

Alpha Holdings → investsin → Arena Tech

confidence 91% · Alpha Holdings is a long-term investor with a significant stake in Arena Tech.

Arena Tech → investsin → Coyni Corp

confidence 91% · The report further notes that Arena Tech invested in Coyni Corp.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domains such as finance, healthcare, education, transportation, and enterprise operations, downstream workflows rely on normalized schemas, entity identities, keys, cross-table relationships, and integrity constraints for analytics, compliance, auditing, and SQL-backed decision making. Existing Document-to-Table benchmarks are insufficient for this setting: flattening evidence into single tables can duplicate entities, obscure many-to-many relationships, create sparse records, and avoid testing whether extracted facts form a valid database instance. This creates an urgent need to evaluate document understanding as database construction rather than field extraction. We introduce Doc2DB-Bench, a benchmark for Document-to-Database construction, containing 203 long-document instances across 42 schemas and seven domain groups, with 117 entity tables, 132 relationship tables, 7,341 rows, and 41,935 cells. Built through a controllable DB-to-Doc synthesis pipeline and organized by a taxonomy of intra-table extraction and inter-table reasoning, the generated documents undergo authenticity verification, proving indistinguishable from real-world references. Doc2DB-Bench thus provides a testbed for reliable, auditable, and relationally faithful LLM-based data systems. The benchmark is publicly available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

102,339 characters extracted from source content.

Expand or collapse full text

Beyond Tables: Doc2DB-Bench for Relationally Faithful Document-to-Database Construction Zhuowen Liang 1 , Zhengxuan Zhang 1 , Jiayang Wang 1 , Jiazhuo Chen 1 , Nan Tang 1∗ 1 The Hong Kong University of Science and Technology (Guangzhou) Abstract Practical AI systems increasingly need to turn long, heterogeneous documents into queryable relational databases, not isolated spreadsheets. In domains such as finance, healthcare, education, transportation, and enterprise operations, down- stream workflows rely on normalized schemas, entity identities, keys, cross-table relationships, and integrity constraints for analytics, compliance, auditing, and SQL-backed decision making. Existing Document-to-Table benchmarks are in- sufficient for this setting: flattening evidence into single tables can duplicate en- tities, obscure many-to-many relationships, create sparse records, and avoid test- ing whether extracted facts form a valid database instance. This creates an ur- gent need to evaluate document understanding as database construction rather than field extraction. We introduceDoc2DB-Bench, a benchmark for Document-to- Database construction, containing 203 long-document instances across 42 schemas and seven domain groups, with 117 entity tables, 132 relationship tables, 7,341 rows, and 41,935 cells. Built through a controllable DB-to-Doc synthesis pipeline and organized by a taxonomy of intra-table extraction and inter-table reasoning, the generated documents undergo authenticity verification, proving indistinguishable from real-world references. Doc2DB-Bench thus provides a testbed for reliable, auditable, and relationally faithful LLM-based data systems. The benchmark is publicly available at https://github.com/SetonLiang/Doc2DB-Bench 1 Introduction Modern organizations rely on documents as the primary carrier of operational knowledge, from clin- ical notes and commercial contracts to financial reports and enterprise records [ 30,6,22]. Yet down- stream applications rarely consume free-form text directly: analytics pipelines, Business Intelligence dashboards, compliance workflows, and SQL-backed systems require structured, queryable, and au- ditable data [28,48,46]. As large language models become increasingly capable of processing long and heterogeneous documents, converting document evidence into reliable structured data has be- come a central goal of information extraction [ 43,26,25]. Document-to-Table and Its Limitations.Most existing benchmarks study this problem under the Document-to-Table(Doc2Table) setting, where systems extract fields or populate a single flattened table [51,9,19,47]. This setting is useful for isolated record extraction, but it is insufficient when downstream workflows require relational databases. Flattening multi-entity evidence can duplicate entities, obscure many-to-many relationships, introduce sparse records with excessiveNULLvalues, and avoid testing whether extracted facts satisfy schema-level constraints. Example 1Consider a financial report in Fig.1(a) describing multiple companies, their cash posi- tions across fiscal years, and investment relationships among them. The target schema in Fig.1(b) contains an entity tableCompanyand a self-referencing relationship tableHold, where company identifiers are system-generated keys and do not appear in the text. ∗ Corresponding author: Nan Tang (E-mail: nantang@hkust-gz.edu.cn) arXiv:2608.08459v1 [cs.CL] 9 Aug 2026 Document-to-Database: Extraction Meets Relational Semantics Zhengxuan Zhang 1 , Zhuowen Liang 1 , Jiazhuo Chen 1 , Haixun Wang 2 , Nan Tang 1 1 HKUST(GZ), China, 2 EvenUp, USA ABSTRACT Bridging the gap between unstructured documents and relational databases is challenging because document extraction operates lo- cally, whereas databases enforce global semantics through schemas, keys, and constraints. Consequently, existing one-shot large lan- guage model (LLM) extraction approaches often fail to reconcile results with relational semantics, yielding inconsistent and hard- to-audit outputs. We present DataMosaic, a document-to-database (Doc2DB) system that explicitly mediates between extraction and database semantics. Given a database schema and constraints, a central orchestrator coordinates entity and relationship extraction alongside veri!cation, repair, and targeted re-extraction within a closedextract-verify-iterateloop. By systematically resolving docu- ment ambiguity and constraint violations, DataMosaic incremen- tally constructs accurate and semantically consistent databases. Featuring pluggable extractors, veri!ers and repair operators, ex- periments in diverse datasets show that DataMosaic substantially reduces constraint violations and improves database-level accuracy over strong Doc2DB baselines. PVLDB Reference Format: Zhengxuan Zhang 1 , Zhuowen Liang 1 , Jiazhuo Chen 1 , Haixun Wang 2 , Nan Tang 1 . Document-to-Database: Extraction Meets Relational Semantics. PVLDB, 14(1): X-X, 2020. doi:X.X/X.X PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/HKUSTDial/DataMosaic. 1 INTRODUCTION In enterprises and scienti!c organizations, analytics, compliance, and decision-making ultimately operate over relational databases, not over raw text or unconstrained model outputs. Yet much of the underlying evidence remains embedded in unstructured doc- uments [24,26]. Bridging this gap is challenging because docu- ment extraction is inherently local—operating over isolated snip- pets [14,48]—whereas relational databases enforce global semantics through schemas, keys, and integrity constraints. This fundamental mismatch means that one-shot extraction, even when powered by LLMs, often fails to produce databases that are consistent, auditable, or query-ready [22, 47]. This tension motivatesdocument-to-database (Doc2DB)con- struction. Given a document collection퐿, a target schema푀, and This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing info@vldb.org. Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097. doi:X.X/X.X integrity and business constraintsω, the goal of Doc2DB is to con- struct a relational database instance that is both factually accurate and semantically valid:(퐿,푀,ω) D!"2DB →↑푁[15,33]. Crucially, Doc2DB is not a text-to-table mapping task. It is a database con- struction problem: entity identities and keys are typically absent from text, relationships must be assembled across documents [35], and global constraints must hold over the entire database instance. Treating Doc2DB as a pure extraction problem collapses these re- quirements into a single local decision, obscuring the role of data- base semantics [39]. E#$%&’(1.Consider a!nancial report describing multiple com- panies, their cash positions across!scal years, and investment rela- tionships among them. The target schema consists of an entity table Companyand a self-referencing relationship tableHold, with primary keys that are system-generated and do not appear in the text. Blue denotes entity mentions; purple denotes at- tribute values; red underlined values indicate sources of errors Arena Techreported solid performance in!scal year2023, with cash increasing from9,867,000at the beginning of the year to11,200,000at year end. Alpha Holdings!led its annual report for2024. The report states that cash began at18,524(in thousand)and ended at32,368,000. Alpha Holdingsis a long-term investor with a signi!cant stake inArena Tech. According to the company overview, Arena Techis among the core holdings associated with Alpha Holdings. The report further notes thatArena Tech invested inCoyni Corp. RegardingCoyni Corp, the report records that its cash position declined from4,544,000to4,062,000during!scal year2023. Elsewhere,Coyni Incis mentioned as an a"liate company, but no corresponding!nancial!gures are reported. The goal is to construct a relational database instance under a given target schema, consisting of an entity tableCompanyand a self- referencing relationship tableHold(Figure??(c)). Crucially, company primary keys (cid) aresystem-generatedsurrogate identi!ers and do not appear in the source text. cid name!scal_year cash_begin cash_end 1Arena Tech20239,867,000 11,200,000 2Alpha Holdings2024 18,524 18,524,000 32,368,000 3Coyni Corp20234,544,0004,062,000 4Coyni Inc2023NULLNULL (c) ExtractedCompanytable: red (error); blue (correct). investor_cid investee_cid 12 21 13 23 (d) ExtractedHoldrelationships: red (spurious), orange (missing). !Entity Table: –Company(cid, name, fiscal_year, cash_begin, cash_end) –Key generation:cid GENERATED ALWAYS AS IDENTITY. !Relationship Table: –Hold(investor_id, investee_id) –Foreign keys: investor_id,investee_id→Company.cid !Integrity Constraints & Business Rules: 퐿 1 :No mutual investment: ¬(Hold(푀,푁)↑Hold(푁,푀)) 퐿 2 :Cash sanity bound: 푂푀푃푄_푅푆푇/푂푀푃푄_푁푅푈푉푆<10 퐿 3 :Recursive investment: Hold(푀,푁)↑Hold(푁,푂)↓Hold(푀,푂) Constructing such a database instance requires aligning extracted evidence with schema-level semantics. (1) Entity extraction.Figure??(a) shows the entity table produced by an LLM extractor. While many company mentions and attribute values are correctly extracted, the result violates basic data semantics. Con!icting evidence leads to incorrect assignments, such as unit mis- match forAlpha Holdings w.r.t.퐿 2 . Moreover, entities mentioned only partially, such asCoyni Inc, are materialized as separate tuples with missing attributes, introducing both incompleteness and duplication. (2) Relationship extraction.Figure??(b) shows the output of a straightforward relationship extractor based on surface-level textual cues. The extractedHoldtable reveals two fundamental failures. First, contradictory phrasing induces a spurious mutual investment (e.g., both(1,2)and(2,1)), violating business rule퐿 1 that prohibits reci- procity. Second, although the document implies a multi-hop invest- ment chain (퐿 3 ), the extractor fails to recover the transitive relation- ship(2,3), leaving an implied edge missing.! Example 1 illustrates why Doc2DB is challenging. Local ex- traction decisions produce tables that violate global integrity con- straints, resulting in hallucinated entities and relationships, missing implied links, and semantically inconsistent attributes. Such errors cannot be reliably addressed within extractors alone, motivating a separate veri!cation stage with explicit feedback loops. Error Patterns in Entity Extraction.Although extracting a single entity table is often treated as schema “!lling”, entity extraction is brittle in practice and prone to systematic semantic errors. (1) Hallucinated entity extraction.Extractors may extract tuples that should not exist, including duplicate tuples caused by unre- solved aliasing and spurious entities inferred from weak mentions. (2) Attribute misinterpretation.Even when entity identity is cor- rect, extracted attributes may violate intended semantics, for exam- ple by ignoring units or temporal quali!ers, yielding syntactically valid but semantically inconsistent values. (3) Incomplete entity extraction.Entity information may be miss- ing or partial when mentions are brief or indirect. Error Patterns in Relationship Extraction.Relationship extrac- tion is harder, as it requires enforcing directionality, business rules, and semantics that extend beyond local textual evidence. (1) Hallucinated relationship extraction.Extractors may con- struct relationships that violate target semantics, such as spurious reciprocal links that contradict no-reciprocity constraints. (2) Missing relationship extraction.Extractors may also fail to construct relationships that are only implicitly supported, such as transitive links, re"ecting the limits of local reasoning. Why Doc2DB Is Fundamentally Hard.These examples reveal that Doc2DB is not merely “information extraction at scale”, but a full-"edged database construction problem characterized by three intertwined challenges. Challenge 1: Document–schema mismatch.Documents are written as narratives or event sequences, whereas schemas are normalized around entities, keys, and relationships. A single paragraph may populate multiple tables, while a single tuple may be scattered across documents, making just extraction brittle. Challenge 2: Things, not strings.Databases operate on entities and keys, whereas documents provide only surface mentions. Since keys are absent from text, mapping mentions to entities is inherently uncertain; forcing early key assignment fragments facts and makes key- and join-consistent extraction brittle. Challenge 3: No one-shot guarantees.Extraction operates over local snippets, whereas integrity constraints and business rules require global veri"cation over the entire database instance. LLMs and tra- ditional IE pipelines can extract fragments of structure, but they provide no guarantees of integrity or consistency. Taken together, these challenges tell us that Doc2DB requires system-levelsupport that goes beyond single-pass extraction. Contributions.We present DataMosaic, a Doc2DB system for con- structing relational databases from documents, with the following notable contributions. (1) The Doc2DB problem.We formalize the Doc2DB problem and articulate key system requirements for practical deployments: key- consistent entity resolution, relationship construction, and global constraint satisfaction (Section 2). (2) The DataMosaic approach.We design DataMosaic (Section 3), a Doc2DB system organized as anextract-verify-iteratecontrol loop. An orchestrator coordinates entity and relationship extractors with veri!cation and repair, triggering targeted re-extraction under detected violations to ensure database-level consistency. 2 cid name!scal_year cash_begin cash_end 1Arena Tech20239,867,000 11,200,000 2Alpha Holdings2024 18,524 18,524,000 32,368,000 3Coyni Corp20234,544,0004,062,000 4Coyni Inc2023NULLNULL (c) ExtractedCompanytable: red (error); blue (correct). investor_cid investee_cid 12 21 13 23 (d) ExtractedHoldrelationships: red (spurious), orange (missing). !Entity Table: –Company(cid, name, fiscal_year, cash_begin, cash_end) –Key generation:cid GENERATED ALWAYS AS IDENTITY. !Relationship Table: –Hold(investor_id, investee_id) –Foreign keys: investor_id,investee_id→Company.cid !Integrity Constraints & Business Rules: 퐿 1 :No mutual investment: ¬(Hold(푀,푁)↑Hold(푁,푀)) 퐿 2 :Cash sanity bound: 푂푀푃푄_푅푆푇/푂푀푃푄_푁푅푈푉푆<10 퐿 3 :Recursive investment: Hold(푀,푁)↑Hold(푁,푂)↓Hold(푀,푂) Constructing such a database instance requires aligning extracted evidence with schema-level semantics. (1) Entity extraction.Figure??(a) shows the entity table produced by an LLM extractor. While many company mentions and attribute values are correctly extracted, the result violates basic data semantics. Con!icting evidence leads to incorrect assignments, such as unit mis- match forAlpha Holdings w.r.t.퐿 2 . Moreover, entities mentioned only partially, such asCoyni Inc, are materialized as separate tuples with missing attributes, introducing both incompleteness and duplication. (2) Relationship extraction.Figure??(b) shows the output of a straightforward relationship extractor based on surface-level textual cues. The extractedHoldtable reveals two fundamental failures. First, contradictory phrasing induces a spurious mutual investment (e.g., both(1,2)and(2,1)), violating business rule퐿 1 that prohibits reci- procity. Second, although the document implies a multi-hop invest- ment chain (퐿 3 ), the extractor fails to recover the transitive relation- ship(2,3), leaving an implied edge missing.! Example 1 illustrates why Doc2DB is challenging. Local ex- traction decisions produce tables that violate global integrity con- straints, resulting in hallucinated entities and relationships, missing implied links, and semantically inconsistent attributes. Such errors cannot be reliably addressed within extractors alone, motivating a separate veri!cation stage with explicit feedback loops. Error Patterns in Entity Extraction.Although extracting a single entity table is often treated as schema “!lling”, entity extraction is brittle in practice and prone to systematic semantic errors. (1) Hallucinated entity extraction.Extractors may extract tuples that should not exist, including duplicate tuples caused by unre- solved aliasing and spurious entities inferred from weak mentions. (2) Attribute misinterpretation.Even when entity identity is cor- rect, extracted attributes may violate intended semantics, for exam- ple by ignoring units or temporal quali!ers, yielding syntactically valid but semantically inconsistent values. (3) Incomplete entity extraction.Entity information may be miss- ing or partial when mentions are brief or indirect. Error Patterns in Relationship Extraction.Relationship extrac- tion is harder, as it requires enforcing directionality, business rules, and semantics that extend beyond local textual evidence. (1) Hallucinated relationship extraction.Extractors may con- struct relationships that violate target semantics, such as spurious reciprocal links that contradict no-reciprocity constraints. (2) Missing relationship extraction.Extractors may also fail to construct relationships that are only implicitly supported, such as transitive links, re"ecting the limits of local reasoning. Why Doc2DB Is Fundamentally Hard.These examples reveal that Doc2DB is not merely “information extraction at scale”, but a full-"edged database construction problem characterized by three intertwined challenges. Challenge 1: Document–schema mismatch.Documents are written as narratives or event sequences, whereas schemas are normalized around entities, keys, and relationships. A single paragraph may populate multiple tables, while a single tuple may be scattered across documents, making just extraction brittle. Challenge 2: Things, not strings.Databases operate on entities and keys, whereas documents provide only surface mentions. Since keys are absent from text, mapping mentions to entities is inherently uncertain; forcing early key assignment fragments facts and makes key- and join-consistent extraction brittle. Challenge 3: No one-shot guarantees.Extraction operates over local snippets, whereas integrity constraints and business rules require global veri"cation over the entire database instance. LLMs and tra- ditional IE pipelines can extract fragments of structure, but they provide no guarantees of integrity or consistency. Taken together, these challenges tell us that Doc2DB requires system-levelsupport that goes beyond single-pass extraction. Contributions.We present DataMosaic, a Doc2DB system for con- structing relational databases from documents, with the following notable contributions. (1) The Doc2DB problem.We formalize the Doc2DB problem and articulate key system requirements for practical deployments: key- consistent entity resolution, relationship construction, and global constraint satisfaction (Section 2). (2) The DataMosaic approach.We design DataMosaic (Section 3), a Doc2DB system organized as anextract-verify-iteratecontrol loop. An orchestrator coordinates entity and relationship extractors with veri!cation and repair, triggering targeted re-extraction under detected violations to ensure database-level consistency. 2 cid name!scal_year cash_begin cash_end 1Arena Tech20239,867,000 11,200,000 2Alpha Holdings2024 18,524 18,524,000 32,368,000 3Coyni Corp20234,544,0004,062,000 4Coyni Inc2023NULLNULL (c) ExtractedCompanytable: red (error); blue (correct). investor_cid investee_cid 12 21 13 23 (d) ExtractedHoldrelationships: red (spurious), orange (missing). !Entity Table: –Company(cid, name, fiscal_year, cash_begin, cash_end) –Key generation:cid GENERATED ALWAYS AS IDENTITY. !Relationship Table: –Hold(investor_id, investee_id) –Foreign keys: investor_id,investee_id→Company.cid !Integrity Constraints & Business Rules: 퐿 1 :No mutual investment: ¬(Hold(푀,푁)↑Hold(푁,푀)) 퐿 2 :Cash sanity bound: 푂푀푃푄_푅푆푇/푂푀푃푄_푁푅푈푉푆<10 퐿 3 :Recursive investment: Hold(푀,푁)↑Hold(푁,푂)↓Hold(푀,푂) Constructing such a database instance requires aligning extracted evidence with schema-level semantics. (1) Entity extraction.Figure??(a) shows the entity table produced by an LLM extractor. While many company mentions and attribute values are correctly extracted, the result violates basic data semantics. Con!icting evidence leads to incorrect assignments, such as unit mis- match forAlpha Holdings w.r.t.퐿 2 . Moreover, entities mentioned only partially, such asCoyni Inc, are materialized as separate tuples with missing attributes, introducing both incompleteness and duplication. (2) Relationship extraction.Figure??(b) shows the output of a straightforward relationship extractor based on surface-level textual cues. The extractedHoldtable reveals two fundamental failures. First, contradictory phrasing induces a spurious mutual investment (e.g., both(1,2)and(2,1)), violating business rule퐿 1 that prohibits reci- procity. Second, although the document implies a multi-hop invest- ment chain (퐿 3 ), the extractor fails to recover the transitive relation- ship(2,3), leaving an implied edge missing.! Example 1 illustrates why Doc2DB is challenging. Local ex- traction decisions produce tables that violate global integrity con- straints, resulting in hallucinated entities and relationships, missing implied links, and semantically inconsistent attributes. Such errors cannot be reliably addressed within extractors alone, motivating a separate veri!cation stage with explicit feedback loops. Error Patterns in Entity Extraction.Although extracting a single entity table is often treated as schema “!lling”, entity extraction is brittle in practice and prone to systematic semantic errors. (1) Hallucinated entity extraction.Extractors may extract tuples that should not exist, including duplicate tuples caused by unre- solved aliasing and spurious entities inferred from weak mentions. (2) Attribute misinterpretation.Even when entity identity is cor- rect, extracted attributes may violate intended semantics, for exam- ple by ignoring units or temporal quali!ers, yielding syntactically valid but semantically inconsistent values. (3) Incomplete entity extraction.Entity information may be miss- ing or partial when mentions are brief or indirect. Error Patterns in Relationship Extraction.Relationship extrac- tion is harder, as it requires enforcing directionality, business rules, and semantics that extend beyond local textual evidence. (1) Hallucinated relationship extraction.Extractors may con- struct relationships that violate target semantics, such as spurious reciprocal links that contradict no-reciprocity constraints. (2) Missing relationship extraction.Extractors may also fail to construct relationships that are only implicitly supported, such as transitive links, re"ecting the limits of local reasoning. Why Doc2DB Is Fundamentally Hard.These examples reveal that Doc2DB is not merely “information extraction at scale”, but a full-"edged database construction problem characterized by three intertwined challenges. Challenge 1: Document–schema mismatch.Documents are written as narratives or event sequences, whereas schemas are normalized around entities, keys, and relationships. A single paragraph may populate multiple tables, while a single tuple may be scattered across documents, making just extraction brittle. Challenge 2: Things, not strings.Databases operate on entities and keys, whereas documents provide only surface mentions. Since keys are absent from text, mapping mentions to entities is inherently uncertain; forcing early key assignment fragments facts and makes key- and join-consistent extraction brittle. Challenge 3: No one-shot guarantees.Extraction operates over local snippets, whereas integrity constraints and business rules require global veri"cation over the entire database instance. LLMs and tra- ditional IE pipelines can extract fragments of structure, but they provide no guarantees of integrity or consistency. Taken together, these challenges tell us that Doc2DB requires system-levelsupport that goes beyond single-pass extraction. Contributions.We present DataMosaic, a Doc2DB system for con- structing relational databases from documents, with the following notable contributions. (1) The Doc2DB problem.We formalize the Doc2DB problem and articulate key system requirements for practical deployments: key- consistent entity resolution, relationship construction, and global constraint satisfaction (Section 2). (2) The DataMosaic approach.We design DataMosaic (Section 3), a Doc2DB system organized as anextract-verify-iteratecontrol loop. An orchestrator coordinates entity and relationship extractors with veri!cation and repair, triggering targeted re-extraction under detected violations to ensure database-level consistency. 2 (a) A financial report(b) Database schema definition (c) Extracted Company table: red (error); blue (correct). (d) Extracted Hold relationships: red (spurious), orange (missing) Figure 1:Document-to-Database extraction: (a) document(s); (b) target database schema; (c) Ex- tracted Entity Tables; (d) Extracted Relationship Tables. Constructing the database requires more than extracting local values. An LLM extractor may cor- rectly identify many company mentions and numbers, but still produce semantically invalid tuples. For entity extraction (Fig.1(c)), it may misinterpret units, such as treating “18,524 (in thousand)” as a raw value, or materialize weak mentions such asCoyni Incas incomplete duplicate entities. For relationship extraction (Fig.1(d)), surface-level cues may induce spurious mutual investments, while implicit multi-hop links may be missed. These errors show that local extraction decisions must be reconciled with schema-level semantics and integrity constraints.□ From Extraction to Database Construction.This motivatesDocument-to-Database(Doc2DB): given a document collection퐹, a target schema퐸, and integrity or business constraintsΣ, the goal is to construct a relational database instance퐷that is both faithful to the documents and valid under the schema:(퐹, 퐸,Σ) DOC2DB −→퐷.Unlike Doc2Table, Doc2DB is a database construction problem. Entity identifiers are often implicit, relationships may be distributed across document segments, and valid outputs must satisfy global constraints rather than independent field-level decisions [ 11,34,33, 37]. Therefore, evaluating document understanding at the database level requires testing not only value extraction, but also entity alignment, relationship construction, and relational validity. Despite its practical importance, Doc2DB remains underexplored as a benchmark task. Existing in- formation extraction benchmarks, including Rotowire [ 40], E2E [31], LiveSum [9], InstructIE [18], StructText [19], and DTBench [51], mainly focus on flat tables, single-table extraction, or simplified generation tasks. SQUiD [35] explores text-to-relational database generation, but mainly targets log- ical relational view recovery rather than realistic Doc2DB construction. As summarized in Table1, they do not fully evaluate cross-table schema construction (i.e., normalized multi-table structures with inter-table dependencies), long-context evidence aggregation, and database-level correctness. Challenges.Building a comprehensive Doc2DB benchmark is challenging for two reasons. First, direct human annotation is difficult to scale: annotators must collect documents, define schemas, and manually construct ground-truth entity and relationship tables. Second, realistic Doc2DB instances must cover diverse reasoning requirements, including unit normalization, ambiguity resolution, multi- hop relation construction, temporal changes, and constraint satisfaction, while also spanning domains with different narrative styles and schema structures. Our Proposal.We presentDoc2DB-Bench, a benchmark for evaluating Doc2DB capabilities be- yond flat table extraction. To avoid the scalability bottleneck of manual annotation, we design a controllable DB2Doc reverse synthesis pipeline grounded in existing relational databases such as BIRD [ 23] and Spider [45]. Starting from schemas and database instances, the pipeline decomposes tuples into atomic evidence, assigns capability labels, serializes evidence into document plans, gen- erates style-conditioned documents, and validates the generated documents through coverage and extraction-consistency checks. To model realistic document complexity, Doc2DB-Bench introduces a two-pillar Doc2DB taxon- omy.Intra-Table Capabilitiescover cell-level extraction, normalization, inference, disambiguation, and faithfulness.Inter-Table Capabilitiescapture database-specific reasoning, including identity res- olution, relationship linking, multi-hop composition, dynamic change, and integrity constraints. This taxonomy supports controlled benchmark construction and fine-grained diagnosis of model failures. Finally, Doc2DB-Bench evaluates outputs at the database level by measuring both entity integrity Table 1:Comparison of existing benchmarks and ourDoc2DB-Bench. Benchmark Single-Table Extraction (ST) Multi- Domain (MD) Cross-Table Reasoning (CR) Long- Context (LC) DB-Level Eval (DC) Rotowire [40]37777 E2E [31]37777 LiveSum [9]37777 InstructIE [18]33777 StructText [19]33777 DTBench [51]33737 SQUiD [35]33773 Doc2DB-Bench(ours)33333 and relational fidelity, thereby testing whether extracted facts form a schema-conformant, queryable, and relationally faithful database instance. Contributions.We summarize our contributions as follows: 1.Doc2DB Capability Taxonomy.We introduce a two-pillar taxonomy of the key capabilities re- quired for Doc2DB construction, spanningIntra-Tablevalue extraction andInter-Tablerelational reasoning, to guide benchmark design and support fine-grained evaluation. 2.Controllable DB2Doc Synthesis Pipeline.We propose a reverse-synthesis pipeline that gen- erates long-document Doc2DB instances from real relational schemas and database instances, reducing manual annotation cost while controlling document complexity. 3.The Doc2DB-Bench Benchmark.We construct a multi-domain benchmark for evaluating LLMs on relationally faithful Doc2DB tasks, going beyond flat, single-table extraction benchmarks. 4.Extensive Experiments.We evaluate a broad range of mainstream LLMs on Doc2DB-Bench and conduct in-depth analyses across entity- and relation-level capabilities, revealing their strengths, limitations, and open research opportunities for Doc2DB extraction. 2 Document-to-Database: The Problem 2.1 Problem Statement Database Specification.We formalize the target database specification asS=(퐸,Σ), where퐸= (E,R)denotes the Entity-Relationship schema.EandRare the sets of entity and relationship tables, respectively, andT=E∪Rdenotes the set of tables. Each table푇∈Tis defined by an attribute set퐴 푇 and a primary key퐾 푇 ⊆퐴 푇 .The constraint setΣ=Σ rel ∪Σ app consists of standard relational constraints, such as type, domain, and inclusion constraints, and application-specific rules, such as temporal conditions. Together,Sspecifies the target database structure and its validity conditions. Document Corpus.We consider a collection of heterogeneous documents퐹=푓 1 , 푓 2 ,..., 푓 푚 that provide the evidence for instantiating the target schema. We assume low-level processing, such as text extraction and layout analysis, has been completed. Since real-world documents may contain incomplete information, unobserved attributes naturally remainNULL. Accordingly, every extracted value should be grounded in the source documents without hallucinating unsupported information. Definition 1 (The Doc2DB Problem).Given a database specification(퐸,Σ)and a document corpus 퐹=푓 1 ,..., 푓 푚 , the Doc2DB task is to construct a relational database instance(퐹, 퐸,Σ) 퐷표푐2퐷퐵 −→ 퐷,where퐷=퐼 푥 |푥∈퐸is the collection of instantiated entity and relationship tables. The target instance퐷should satisfy (1)Constraint satisfaction:퐷|=Σ; and (2)Closeness to ground truth:퐷 is as close as possible to the ground-truth database퐷 ∗ . 2.2 DB2Doc Synthesis Paradigm To construct a benchmark for evaluating Doc2DB extraction, we adopt a reverse synthesis perspective: given a ground-truth database instance퐷=퐼 푥 |푥∈퐸with specificationS=(퐸,Σ), we synthesize a source document corpus퐹from which퐷 ∗ can be faithfully recovered. Definition 2 (DB2Doc Synthesis).Given a ground-truth database퐷 ∗ , its specificationS, and a ca- pability taxonomyH, DB2Doc synthesizes a document corpus퐹satisfying three properties: (1) Figure 2:A Taxonomy of Doc2DB Extraction Capabilities. completeness, every ground-truth value and relational tuple in퐷 ∗ is supported by evidence in퐹; (2) exclusiveness,퐹contains no evidence supporting database facts beyond퐷 ∗ underS; and (3)capa- bility awareness, the evidence for each target value/relational tuple is constructed to require specific reasoning capabilities fromHfor its recovery, withHformalized as a two-pillar taxonomy in Sec.3. The completeness and exclusiveness conditions jointly ensure that퐷 ∗ is the unique ground-truth database recoverable from퐹underS, while capability awareness enables fine-grained evaluation across taxonomy levels. Each instance may exercise any subset of capabilities fromH, reflecting the naturally uneven distribution of requirements across real-world documents. 3 Doc2DB Taxonomy Doc2DB requires models to construct both values and relations. Accordingly, we organize the re- quired capabilities into two complementary pillars. Intra-Table Capabilities (Pillar 1) askswhat val- ues should populate an entity record, whereas Inter-Table (Pillar 2) askshow entity records should be connected. Figure 2summarizes the resulting taxonomy and its fine-grained sub-capabilities. 3.1 Pillar1: Intra-Table Capabilities Pillar 1 focuses on what values should populate an entity record, covering four complementary chal- lenges.Transformative Alignment (TA).Normalizes observed values into schema-compatible rep- resentations (e.g.,“18,524 in thousands”→18,524,000).Reasoning & Inference (RI).Derives values that are not explicitly stated (e.g.,computing a target value from several reported quantities). Target Discrimination (TD).Identifies the intended value among distracting or conflicting evidence (e.g.,selecting the current-year rather than a historical figure).Evidence Faithfulness (EF).Avoids unsupported outputs (e.g.,keeping an unreported attribute asNULL). Pillar 1 extends DTBench [ 51] from single-table extraction to schema-compliant entity construction in Doc2DB. 3.2 Pillar 2: Inter-Table Capabilities Pillar 2 focuses on how entity records should be connected, covering four complementary challenges. Identity Resolution (IDR).Resolves ambiguous mentions to unique database entities (e.g.,identi- fying “Lao Zhang” as the employee in the technical department among records with the same name). Relationship Linking (RL).Maps entity mentions and interactions into normalized relational tuples (e.g.,expanding “Alice manages X, Y, and Z” into three separate relationship rows).Structural Reasoning (SR).Infers relations from distributed or compositional evidence (e.g.,“Alice leads the Mobile Team” and “the Mobile Team manages Project X” imply(Alice,Project X)).Integrity Con- straint (IC).Suppresses relations that violate negative evidence or explicit constraints (e.g.,not linking Bob to Project X when the document states that all managersexcept Bobare assigned to it). Detailed sub-capabilities and examples are provided in AppendixB. 4 Document-to-Database (Db2Doc) Synthesis Given a ground-truth relational database instance퐷, its specification푆=(퐸,Σ), and the capability taxonomyH, our goal is to synthesize a document corpus퐹from which퐷can be faithfully recovered. As shown in Fig.3, our DB2Doc pipeline consists of five stages: capability assignment, evidence decomposition, strategic serialization, iterative document generation, and dual validation. Figure 3:Overview of the benchmark construction framework. The rationale behind this design is to separatewhatshould be tested fromhowit is expressed in documents. Capability assignment determines the target reasoning requirements; evidence decom- position grounds each target value or tuple in atomic evidence; serialization controls the document structure; generation turns the evidence into realistic long-form text; and validation checks that the final document remains complete, faithful, and recoverable. This staged design makes the synthesis process controllable, reproducible, and suitable for fine-grained Doc2DB evaluation. Step 1: Capability Assignment.ALabeling Agent scans each cell푣 푖푗 ∈퐷and assigns a label setL푖푗⊆Hto produce an annotated matrixM 퐷 . The agent employs an LLM as a semantic router that jointly conditions on the cell value푣푖푗, its attribute context fromS, and the enclosing relational tuple푟, reasoning over both intra-table semantics and inter-table relational structure to determineL푖푗. Multi-label assignment captures compositional requirements,e.g.,a cell demanding summation followed by currency conversion is labeledL푖푗=RI-AR,TA-UT, as illustrated in Sec. 3. Step 2: Joint Evidence Decomposition.üARefiner Agent processes each annotated cell퐶 푖푗 and decomposes it into an atomic evidence set퐸 푖푗 =푒 1 ,푒 2 ,...,푒 푘 , forming the global evidence poolP. Critically, the agent performsjoint executionover two orthogonal constraint dimensions simultane- ously: theattribute-level label(Pillar 1,e.g.,TA-EMfor enumeration mapping) and therow-level label(Pillar 2,e.g.,SRfor structural reasoning). To ensure evidence quality, we apply checklist-based verification along four dimensions: (i)value/relation correctness, ensuring factual consistency with target values and relations; (i)label faithfulness, verifying that the assigned capability is properly instantiated; (i)schema linkage, checking consistency with valid schema elements and cross-table references; and (iv)data integrity and null faithfulness, preventing unsupported content and pre- serving missing values. Failed cases receive diagnostic feedback and are iteratively revised until verification succeeds or a retry limit is reached. Detailed criteria are provided in Appendix E.2. Step 3: Strategic Serialization. ̊ASerializer Agent linearizes the verified evidence poolPinto coherent blocks, controlling the document’s macro-structure and evidence organization. Each ev- idence item푒∈Pis indexed by its entity푖and attribute푗, and serialized according to푉(푒)= (휋 1 ,휋 2 ,휖), where(휋 1 ,휋 2 )denotes a mode-specific ordering of the entity and attribute indices(푖, 푗), and휖∼푈(0,1)serves as a stochastic tie-breaker. We consider three serialization strategies:Horizon- tal(푉=(푖, 푗,휖)), which groups evidence by entity;Vertical(푉=(푗,푖,휖)), which groups evidence by attribute across entities; andStochastic Vertical(푉=(휋(푗),푖,휖)), which follows the vertical scheme while randomly permuting the attribute order. Together, these strategies introduce controllable vari- ation in document structure while maintaining coherent evidence organization. Step 4: Iterative Document Generation.AWriter Agentgenerates the final document block by block following the serialized plan. A context-aware sliding window conditions each block on the current evidence퐸 푏푙표푐푘 and a bounded history of previously generated text퐻 푝푟푒푣 , promoting fluent transitions and consistent cross-block coreference. Furthermore, to better emulate the complexity of real-world documents, generation is con- trolled by aplug-and-play configuration module. It supports configurable parameters such as Table 2:Comprehensive statistics of the Doc2DB-Bench benchmark. The#denotes the total count, andAvg. Len.indicates the average document length per sample in tokens. Domain DocumentSchemaTable Scale (Total) # Docs Avg. Len. # DBs # Ent. # Rel. # Rows # Cols # Cells Education1340,42431212830963,451 Finance5940,0731023391,91835310,454 Medical1143,93039123431993,522 University3133,913617218602094,339 Sports2750,991520161,0851775,772 Transportation1753,09151267181284,678 Others4346,5661024261,5873139,719 Total/Avg.20343,326421171327,3411,475 41,935 Document_Length,Noise_Level(∈[0,1]), andDocument_Style. The noise level controls the frequency of natural distractors, including irrelevant background content, formatting artifacts, and header/footer interference, while the style parameter follows domain-specific reference documents (e.g.,formal legal writing or concise clinical shorthand). Together, these controls increase document realism and diversity without violating the factual constraints in the evidence poolP. Step 5: Dual Validation.To guarantee that the synthesized document퐷 푠푦푛 satisfies the rigorous constraints of the Doc2DB paradigm,ϻaVerifier Agentconducts a final dual-validation pass. This phase operates as an inverse-consistency check: •Coverage Verification:Ensures that no required atomic evidence fromPis omitted or seman- tically distorted, utilizing explicit boundary tags (e.g.,<frag_푖>...</frag_푖>) injected during generation to deterministically trace textual spans back to their source atoms. •Extraction Consistency:An evaluation mechanism is deployed to re-extract the database instance 퐷 ′ 푠푦푛 from퐷 푠푦푛 . We rigorously verify the structural isomorphism and value equivalence between the re-extracted퐷 ′ 푠푦푛 and the ground-truth database퐷. If퐷 syn fails either validation criterion due to hallucinated relations, missing evidence, or referential inconsistencies, it is routed back to Step 4 for targeted rewriting with error feedback. This closed-loop validation minimizes unrecoverable or contradictory information in the final benchmark corpus. Doc2DB-Bench Statistics.We synthesize documents from databases sourced from BIRD [ 23] and Spider [45], using them as ground truth. Specifically, we curate 42 high-quality databases across seven domains, including education and medical. Details of database collection and processing are provided in Appendix A. All agents use Gemini-2.5-Pro [7] as the backbone model, selected based on preliminary experiments balancing quality and cost. All synthesized cases are further verified by seven computer science PhD candidates, each specializing in a corresponding domain. As summarized in Table 2, Doc2DB-Bench contains 42 synthesized (Document, Specification, Database) samples, covering 117 entity tables, 132 relationship tables, 7,341 rows, and 41,935 cells across seven domains. The benchmark targets schema understanding, long-context reason- ing, and cross-table dependency modeling, with fine-grained capability annotations for 11,205 cells (>25%) and 3,129 rows (>40%). To guide realistic synthesis, we collect reference documents from DocBench [52], MMLongBench [29], MMDocRAG [10], and web sources, spanning financial and legal reports to research abstracts. Fig.2shows the capability distribution; unannotated cells and rows correspond to directly extractable cases requiring verbatim recovery. 5 Experiments In this section, we systematically evaluate the performance of various LLMs on Doc2DB-Bench for the Doc2DB extraction task. Specifically, we seek to answer the following research questions: RQ1:How well do different LLMs perform on Doc2DB extraction at the entity and relation levels? RQ2:Beyond overall performance, what specific capabilities and vulnerabilities do these models exhibit across the fine-grained dimensions defined in our taxonomy? RQ3:How realistic and authentic are the documents generated by our automated pipeline? Table 3:Comparative performance of LLMs on Doc2DB-Bench.Greenhighlights the best. ModelSize Entity-LevelRelation-LevelOverall Performance P.R.F1P.R.F1P.R.F1LS Open-source Models Qwen2.5-14b-Ins14b42.8760.1744.0727.4543.5428.9229.4050.3632.9739.39 LLaMA-3.1-70b-Ins 70b14.9658.8917.2520.2938.7916.9712.1247.4614.2922.76 Qwen2.5-72b-Ins72b41.6266.5745.0328.4854.6331.9830.9659.5336.0739.53 Proprietary Models GPT-4o [16]-72.3970.9270.3062.9446.7850.6865.6655.9959.0547.52 GPT-5.4-79.9381.1280.0980.0969.6372.3378.4773.9375.2569.02 Gemini-2.5-flash [7]-75.8975.4874.2173.6057.3559.5071.3364.5865.9953.74 Gemini-2.5-pro [7]-76.8082.2478.1367.9867.0165.4669.8373.8070.8157.19 Claude-opus-4-6-87.1983.7084.9366.7162.1663.1777.3371.5773.6061.95 Qwen3-max-76.5177.0075.7757.0159.7455.9663.4866.4263.4452.48 Deepseek-V4-flash-76.6566.0667.3067.6743.2348.9369.0352.1457.0649.05 Specialized IE Systems LangExtract [12]-29.7240.5033.4933.3419.0822.0036.0625.2028.7223.55 DocETL [36]-78.2076.8574.9146.7541.2641.5165.2760.9760.8249.87 LangChain [14]-60.7783.0969.0933.5042.4736.3347.2864.3853.8344.14 LlamaExtract [28]-52.2868.9556.5847.4148.7347.3945.3556.4949.1040.26 5.1 Setup Baselines.We benchmark a broad range of state-of-the-art LLMs, categorized into three groups: (1)Open-weight models, including Qwen2.5-14B/72B-Instruct [44] and Llama-3.1-70B-Instruct; (2) Proprietary APIs, including GPT-4o [16], GPT-5.4, Gemini-2.5-Flash/Pro [7], Claude-opus-4-6 [2], Qwen3-Max, and DeepSeek-V4-Flash [42]; and (3)Specialized IE systems, including LlamaEx- tract [28], a commercial extraction service with iterative refinement and citations; LangExtract [12], which uses chunking and few-shot prompting for long-document extraction; LangChain [ 14], a gen- eral framework with schema-constrained function calling for structured outputs; and DocETL [36], a declarative LLM-based data processing framework with modular extraction pipelines. Evaluation Details.For fair and reproducible comparison, we use identical prompts and greedy decoding with temperature0across all experiments. All systems use GPT-5.4 as the backbone model, except LlamaExtract, which uses its proprietary model. To isolate relational reasoning from upstream entity extraction errors, we additionally introduce anoracle entity setting, where the modelM receives ground-truth entity tables퐸 ∗ and predicts only relation tables:푇 rel =M(퐷,푆, 퐸 ∗ ). Metrics.To evaluate extraction at a granular level, we employ the following metrics: •Cell-level Accuracy.We evaluate fine-grained attribute extraction using cell-level Precision, Re- call, and F 1 . Rather than relying on row-level matching, which is sensitive to tuple ordering and may cause cascading alignment errors, we employ aGlobal Maximum Weight Matchingalgo- rithm to establish one-to-one tuple alignment between predicted (푇) and ground-truth (푇 ∗ ) tables (details in AppendixC.1). Let퐶 match denote the number of cells satisfying our matching criteria (e.g.,, exact numeric equality or≥90%string similarity). The metrics are defined as: 푃= |퐶 match | 퐶 total (푇) , 푅= |퐶 match | 퐶 total (푇 ∗ ) , 퐹 1 = 2푃푅 푃+푅 , where퐶 total (·)denotes the number of non-empty cells in the corresponding table. •Semantic Quality.We employ GPT-4o as an evaluator to assess semantic equivalence beyond ex- act cell matching. It assigns a score from 0 to 100 based on factual correctness, schema conformity, and robustness to paraphrasing, capturing semantically equivalent but structurally varied predic- tions. The evaluation prompt is provided in AppendixC.2, with further results in AppendixD. 5.2 Experimental Results and Analysis Overall Performance.As shown in Table3, proprietary models consistently outperform open- source counterparts across all metrics. In particular, GPT-5.4 achieves the best overall F1 score Table 4:The Comparison of Capabilities Across Different Models for Different Categories. Cell colors represent performance levels (Blue: High; Red: Low). CapabilitySub-capabilities Qwen2.5- 14B-Ins LLaMA-3.1- 70B-Ins GPT-4o DeepSeek- V4-Flash Qwen3-Max Claude- Opus-4-6 Gemini- 2.5-Pro GPT-5.4 Entity Disambiguation51.5859.9269.2455.2564.5772.9680.0379.49Identity ResolutionCoreference Resolution37.8450.1361.1462.1056.7667.6177.7182.18 One-to-Many Allocation49.0651.9065.6362.2764.7770.3878.1881.83 Multi-Entity Binding47.6553.8560.1853.8358.5764.8267.2682.97 Relationship Linking Conditional Linkage52.0257.4769.1062.6561.8069.6475.8179.16 Transitive Inference40.8057.8365.5157.3553.7859.4870.8278.47 Global Aggregation49.2348.3071.5959.8362.1564.2573.0382.20 Structural Reasoning Dynamic Change36.5151.8358.1146.9358.3666.2678.6979.27 Referential Integrity--------Integrity Constraint Mutual Exclusion36.4156.9460.1858.4554.6161.9670.3974.46 Figure 4:Radar chart of different models across various capabilities. GPT-5.4Qwen3-max 50 55 60 65 70 75 80 Entity-Level w/o label w/ label GPT-5.4Qwen3-maxGemini-2.5-pro 50 55 60 65 70 75 80 Relation-Level w/o label w/ label Figure 5:Impact of capability annotations on ex- traction performance. (75.25) and the highest llm score (69.02), indicating strong capability in structured extraction. Claude-opus-4-6 and Gemini-2.5-pro also deliver competitive performance. Notably, GPT-4o shows only moderate results (Overall F1 59.05), falling noticeably short of reasoning-capable models de- spite its scale. In contrast, open-source models such as Qwen2.5-14B-Ins exhibit significantly lower overall F1 (32.97), highlighting the persistent performance gap under complex extraction settings. Finding 1.Models with stronger reasoning capabilities consistently achieve better performance on Doc2DB extraction, where structural reasoning proves critical for schema-compliant output. Entity-level performance.Entity-level results reveal a clear advantage of high-capacity proprietary models. Claude-opus-4-6 achieves the highest entity-level F1 (84.95), followed by GPT-5.4 (80.09) and Gemini-2.5-pro (78.13). These models maintain a good balance between precision and recall, indicating robust span detection and boundary alignment. Compared to them, Qwen2.5-14B shows notably lower precision (42.87) despite relatively higher recall (60.17), suggesting that open-source models tend to over-generate entity spans, leading to reduced accuracy. Relationship level performance.Relation extraction is evaluated under an oracle entity setting: given the relational schema, the source document, and the ground-truth entity tables, the model gen- erates only the relation tables. This setting isolates relational reasoning ability from upstream entity extraction errors. As shown in Table3, relation-level extraction remains substantially more challeng- ing than entity extraction for most models. A key reason is that relation prediction requires not only identifying the correct records, but also globally aligning related entities and tuples under a struc- tured schema, which is more error-prone than localized entity extraction. GPT-5.4 achieves the best relation-level F1 (72.33), outperforming GPT-4o (50.68), Gemini-2.5-pro (65.46), and Claude-opus- 4-6 (63.13), indicating stronger capability in global alignment and structured relational reasoning. Finding 2.Relation-level extraction is a major bottleneck in Doc2DB extraction. Compared with entity extraction, it is substantially more challenging because it requires accurate global alignment across entities and tuples, leading to consistently lower performance. Table 5:Quality and Authenticity evaluation of Doc2DB-Bench documents using (a) rubric-based LLM and human assessment and (b) commercial AI detectors. Judger Lexical LogicalTextual Genre Average Richness Consistency Coherence Fidelity LLM4.464.283.844.944.38 Human4.173.973.984.004.03 MethodDoc2DB-Bench Reference ZeroGPT35.2023.12 FastGPT [4]37.5035.38 5.3 Capability-Level Performance As revealed in Section5.2, relation extraction is the primary bottleneck in Doc2DB extraction, yet overall metrics do not revealwhich specific inter-table capabilitiesremain challenging for LLMs. We therefore conduct a fine-grained evaluation over Pillar 2 capabilities in our taxonomy. For each capability, we compute cell-level Precision, Recall, and F 1 on the corresponding annotated relation tables using the same matching criteria, enabling fine-grained analysis of inter-table reason- ing abilities such as entity linking, relation alignment, and multi-hop dependency recovery. Capability assignment introduces diﬀiculty.As shown in Fig.5, models perform consistently worse on documents with explicit capability labels than on those without. The drop is substantial at the entity level, with GPT-5.4 decreasing from 74.15 to 56.75 and Qwen3-max from 66.70 to 52.37. At the relation level, the decline is smaller but consistent across models (GPT-5.4: 67.77 to 58.69; Qwen3-max: 56.63 to 51.23; Gemini-2.5-pro: 66.49 to 60.26). These results confirm that capability-annotated documents are genuinely harder and validate the effectiveness of our taxonomy. Detailed Error Analysis.Fig. 4shows that GPT-5.4 performs best across all capabilities, while Table4reports fine-grained results across four inter-table capabilities, revealing three key findings: (i) Implicit evidence amplifies model disparities.Surface-level tasks rely on explicit cues, whereas coreference resolution and multi-entity binding require aggregating scattered implicit evidence, sig- nificantly widening the gap between strong and weak models.(i) Structural reasoning remains a major bottleneck.Even SOTA models plateau at around 80% on Dynamic Change and Transitive Inference, while smaller models fall below 40%, indicating limited multi-step relational reasoning ability.(i) Integrity Constraint (IC) verification is consistently weak.Models often hallucinate dependencies in Referential Integrity tests and violate exclusivity rules in Mutual Exclusion tasks, highlighting the lack of constraint-aware generation and potential risks to database reliability. Finding 3..Implicit reasoning over distributed and compositional evidence increases difficulty and widens model gaps, with structural reasoning and integrity constraints as the primary failure modes. 5.4 Authenticity Verification To evaluate document quality and realism, we conduct a rubric-based assessment using both an LLM judge and human annotators. Documents are evaluated along four dimensions:lexical richness, log- ical consistency, textual coherence, and genre fidelity, using a five-point scale. We use DeepSeek- V3.2 as the automatic evaluator and recruit five graduate students document intelligence research ex- perience to independently assess 50 randomly sampled documents using the same rubrics. As shown in Table 5(a), the LLM and human evaluations consistently demonstrate the high linguistic quality and domain authenticity. Furthermore, commercial AI detectors (ZeroGPT and FastGPT [ 4]) show that Doc2DB-Bench closely matches real-world distributions in AI-generation scores (Table 5(b)). Notably, the FastGPT score (37.50%) is nearly identical to the real-data baseline (35.38%). Overall, these results demonstrate that Doc2DB-Bench preserves the lexical diversity and stylistic nuances of human-written reports, achieving high fidelity. More details are provided in AppendixF. 6 Related Work Structured Information Extraction.Traditional IE decomposes the problem into sub-tasks such as named entity recognition and relation extraction, evolving from rule-based methods [ 15,21] to pre- trained language model-based approaches [24,41,49]. Such pipelines are difficult to customize for schema-driven extraction, where target attributes may fall outside existing ontologies [18]. Recent work leverages LLMs for structured extraction: some advanced prompting methods [9,1,39] de- compose complex extraction tasks into intermediate steps; StructSum [17] generates structured sum- maries as tables and mind maps; InstructIE [18], Lotus [32], and PZ [27] extract specified attributes through natural-language queries; Doctopus [5] combines LLMs with traditional IE tools to reduce cost. However, existing work focuses on Doc2Table extraction over single flat tables [39,9,18,26], while the Doc2DB setting, which requires constructing multi-table relational databases with inter- table dependencies and integrity constraints from documents, remains largely unexplored. Existing Benchmarks.Prior benchmarks largely repurpose table-to-text datasets such as Ro- towire [40] and Wiki40B [13], which support only trivial extraction reducible to direct text replication. LiveSum [9] covers limited domains, while synthetic benchmarks InstructIE [18] and StructText [19] suffer from simplified schemata and trivial text–tuple alignments. Meanwhile, SemBench [20] and UDA-Bench [38] target unstructured document analysis but lack fine-grained capability evaluation. More recent DTBench [51] introduces capability-aware evaluation for document-to-table extraction, but remains limited to flat-table extraction without cross-table relational construction. SQUiD [35] further explores text-to-relational database generation with multi-table schema synthesis, yet mainly evaluates logical relational view recovery from simple descriptions rather than realistic Doc2DB task requiring cross-table reasoning and database-level consistency. These limitations motivate Doc2DB- Bench, the first benchmark for systematic LLM evaluation on document-to-database extraction. 7 Conclusion We introduce Doc2DB-Bench, a novel benchmark advancing information extraction from isolated flat tables to relationally faithful document-to-database construction. To overcome manual anno- tation bottlenecks, we propose a controllable reverse-synthesis pipeline grounded in a comprehen- sive taxonomy of intra-table and inter-table capabilities. Extensive evaluations reveal that while state-of-the-art LLMs excel at localized entity extraction, they fundamentally struggle with global relational alignment, multi-hop structural reasoning, and adherence to database integrity constraints. Doc2DB-Bench provides a rigorous and realistic testbed for advancing reliable, schema-compliant, and auditable LLM-based data systems, with further dataset expansion underway. Limitations While Doc2DB-Bench provides broad coverage across diverse schemas and domains, several aspects remain open for further exploration. The current DB2Doc synthesis pipeline relies on LLM-based generation, making human verification important for ensuring the quality and realism of synthesized documents. Beyond synthesis quality, we also plan to expand the benchmark by incorporating more domain-specific databases and broader real-world sources, strengthening its coverage and diversity. References [1]N. Ahuja, F. Bardoliya, C. Baral, and V. Gupta. Map&make: Schema guided text to table generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 30249–30262, 2025. [2]Anthropic.Claude opus 4.6 system card.https://w-cdn.anthropic.com/ 6a5fa276ac68b9aeb0c8b6af5fa36326e0e166d/Claude%20Opus%204.6%20System% 20Card.pdf , 2026. [3]H. Babaei Giglou, J. D’Souza, and S. Auer. Llms4synthesis: Leveraging large language models for scientific synthesis. InProceedings of the 24th ACM/IEEE Joint Conference on Digital Libraries, pages 1–12, 2024. [4]G. Bao, Y. Zhao, Z. Teng, L. Yang, and Y. Zhang. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature.arXiv preprint arXiv:2310.05130, 2023. [5]C. Chai, J. Li, Y. Deng, Y. Zhong, Y. Yuan, G. Wang, and L. Cao. Doctopus: Budget-aware structural table extraction from unstructured documents.Proceedings of the VLDB Endowment, 18(11):3695–3707, 2025. [6]Z. Chen, Z. Gu, L. Cao, J. Fan, S. Madden, and N. Tang. Symphony: Towards natural language query answering over multi-modal data lakes. InCIDR, pages 1–7, 2023. [7]G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced rea- soning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025. [8]C. Deng, J. Yuan, P. Bu, P. Wang, Z.-Z. Li, J. Xu, X.-H. Li, Y. Gao, J. Song, B. Zheng, et al. Longdocurl: a comprehensive multimodal long document benchmark integrating understand- ing, reasoning, and locating. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1135–1159, 2025. [9]Z. Deng, C. Chan, W. Wang, Y. Sun, W. Fan, T. Zheng, Y. Yim, and Y. Song. Text-tuple- table: Towards information integration in text-to-table generation via global tuple extraction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9300–9322. Association for Computational Linguistics, 2024. [10]K. Dong, Y. Chang, S. Huang, Y. Wang, R. Tang, and Y. Liu. Benchmarking retrieval- augmented multimodal generation for document question answering.arXiv preprint arXiv:2505.16470, 2025. [11]X. Dong, E. Gabrilovich, G. Heitz, W. Horn, N. Lao, K. Murphy, T. Strohmann, S. Sun, and W. Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 601–610, 2014. [12]Google. Introducing langextract.https://langextract.net, 2025. [13]M. Guo, Z. Dai, D. Vrandečić, and R. Al-Rfou. Wiki-40b: Multilingual language model dataset. InProceedings of the Twelfth Language Resources and Evaluation Conference, pages 2440– 2452, 2020. [14]Harrison Chase. Langchain.https://github.com/langchain-ai/langchain, 2022. [15]J. Hobbs. Fastus: A finite-state processor for information extraction from real-world text. In JOINT CONFERENCE ON, 1993. [16]A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. [17]P. Jain, A. Marzoca, and F. Piccinno. Structsum generation for faster text comprehension. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 7876–7896, 2024. [18]Y. Jiao, M. Zhong, S. Li, R. Zhao, S. Ouyang, H. Ji, and J. Han. Instruct and extract: In- struction tuning for on-demand information extraction. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10030–10051. Association for Computational Linguistics, 2023. [19]S. Kashyap, S. Shirai, N. Mihindukulasooriya, and H. Samulowitz. Structtext: A synthetic table-to-text approach for benchmark generation with multi-dimensional evaluation.CoRR, abs/2507.21340, 2025. [20]J. Lao, A. Zimmerer, O. Ovcharenko, T. Cong, M. Russo, G. Vitagliano, M. Cochez, F. Özcan, G. Gupta, T. Hottelier, et al. Sembench: A benchmark for semantic query processing engines. arXiv preprint arXiv:2511.01716, 2025. [21]T. Lee, Z. Wang, H. Wang, and S.-w. Hwang. Attribute extraction and scoring: A probabilistic approach. In2013 IEEE 29th International Conference on Data Engineering (ICDE), pages 194–205. IEEE, 2013. [22]B. Li, Z. Liang, Y. Xie, X. Lin, T. Luo, X. Liu, Y. Zhu, Z. Peng, Y. Li, Z. Zhang, et al. Datas- pace: Benchmarking data agents for verifiable analytics over heterogeneous workspaces.arXiv preprint arXiv:2608.03451, 2026. [23]J. Li, B. Hui, G. Qu, J. Yang, B. Li, B. Li, B. Wang, B. Qin, R. Geng, N. Huo, et al. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems, 36:42330–42357, 2023. [24]Y. Li, L. Song, and C. Zhang. Sparse conditional hidden markov model for weakly supervised named entity recognition. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 978–988, 2022. [25]Z. Li, X. Chen, H. Yu, H. Lin, Y. Lu, Q. Tang, F. Huang, X. Han, L. Sun, and Y. Li. Struc- trag: Boosting knowledge intensive reasoning of llms via inference-time hybrid information structurization. InInternational Conference on Learning Representations, volume 2025, pages 36107–36124, 2025. [26]Z. Liang, X. Lin, Z. Zhang, Y. Luo, H. Wang, and N. Tang. Long-document qa with chain-of- structured-thought and fine-tuned slms.arXiv preprint arXiv:2603.29232, 2026. [27]C. Liu, M. Russo, M. Cafarella, L. Cao, P. B. Chen, Z. Chen, M. Franklin, T. Kraska, S. Madden, R. Shahout, et al. Palimpzest: Optimizing ai-powered analytics with declarative query process- ing. InProceedings of the Conference on Innovative Database Research (CIDR), page 2, 2025. [28]LlamaIndex. Unstructured data extraction: Turn documents into insights.https://w. llamaindex.ai/blog/unstructured-data-extraction, 2024. [29]Y. Ma, Y. Zang, L. Chen, M. Chen, Y. Jiao, X. Li, X. Lu, Z. Liu, Y. Ma, X. Dong, et al. Mmlongbench-doc: Benchmarking long-context document understanding with visualizations. Advances in Neural Information Processing Systems, 37:95963–96010, 2024. [30]H. Muscolino, A. Machado, D. Vesset, and J. Rydning.What every executive needs to know about unstructured data.Technical report, IDC Corp., sponsored by Box Inc., Aug. 2023.https://w.storagenewsletter.com/2023/09/07/ what-every-executive-needs-to-know-about-unstructured-data/ . [31]J. Novikova, O. Dusek, and V. Rieser. The E2E dataset: New challenges for end-to-end gener- ation. InProceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pages 201–206. Association for Computational Linguistics, 2017. [32]L. Patel, S. Jha, M. Pan, H. Gupta, P. Asawa, C. Guestrin, and M. Zaharia. Semantic operators and their optimization: Enabling llm-based data processing with accuracy guarantees in lotus. Proceedings of the VLDB Endowment, 18(11):4171–4184, 2025. [33]N. Peng, H. Poon, C. Quirk, K. Toutanova, and W.-t. Yih. Cross-sentence n-ary relation extrac- tion with graph lstms.Transactions of the Association for Computational Linguistics, 5:101– 115, 2017. [34]C. D. Sa, C. Zhang, J. Shin, and C. Ré. Deepdive: Declarative knowledge base construction. SIGMOD Record, 45(1):60–67, 2016. [35]M. Sadia, Z. Yang, Y. Xiao, A. Chen, and A. R. Chowdhury. Squid: Synthesizing relational databases from unstructured text. In C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2025, Suzhou, China, November 4-9, 2025, pages 31987–32012. Association for Computational Linguistics, 2025. [36]S. Shankar, T. Chambers, T. Shah, A. G. Parameswaran, and E. Wu. Docetl: Agentic query rewriting and evaluation for complex document processing.arXiv preprint arXiv:2410.12189, 2024. [37]J. Shin, S. Wu, F. Wang, C. De Sa, C. Zhang, and C. Ré. Incremental knowledge base con- struction using deepdive. InProceedings of the VLDB Endowment International Conference on Very Large Data Bases, volume 8, page 1310, 2015. [38]V. Subramaniyaswamy, V. Vijayakumar, R. Logesh, and V. Indragandhi. Unstructured data analysis on big data using map reduce.Procedia Computer Science, 50:456–465, 2015. [39]X. Tang, Y. Zong, J. Phang, Y. Zhao, W. Zhou, A. Cohan, and M. Gerstein. Struc-bench: Are large language models good at generating complex structured tabular data? InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2024. [40]S. Wiseman, S. M. Shieber, and A. M. Rush. Challenges in data-to-document generation.arXiv preprint arXiv:1707.08052, 2017. [41]F. Wu, R. Hoffmann, and D. S. Weld. Information extraction from wikipedia: Moving down the long tail. InProceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 731–739, 2008. [42]A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, et al. Deepseek-v4: Towards highly efficient million-token context intelligence.arXiv preprint arXiv:2606.19348, 2026. [43]D. Xu, W. Chen, W. Peng, et al. Large language models for generative information extraction: A survey.Frontiers of Computer Science, 18, 2024.https://journal.hep.com.cn/fcs/ EN/10.1007/s11704-024-40555-y. [44]A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu. Qwen2.5 technical report. CoRR , abs/2412.15115, 2024. [45]T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. InProceedings of the 2018 conference on empirical methods in natural language processing, pages 3911–3921, 2018. [46]T. Zeng, Y. Hong, Z. Ding, D. Liu, Y. Mei, Y. Su, Y. Wang, X. Zhang, J. Zhu, J. Zhu, et al. Qwenpaw-data: Bridging facts, methodology, and execution for autonomous enterprise data analytics.arXiv preprint arXiv:2607.11019, 2026. [47]Z. Zhang, Z. Liang, J. Chen, H. Wang, and N. Tang. Document-to-database: Extraction meets relational semantics. volume 19, pages 2522–2535. VLDB Endowment, 2026. [48]Z. Zhang, Z. Liang, Y. Wu, T. Lin, Y. Luo, and N. Tang. Datamosaic: Explainable and verifiable multi-modal data analytics through extract-reason-verify.arXiv e-prints, pages arXiv–2504, 2025. [49]G. Zheng, S. Mukherjee, X. L. Dong, and F. Li. Opentag: Open attribute value extraction from product profiles. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1049–1058, 2018. [50]L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information processing systems, 36:46595–46623, 2023. [51]ZJU-DAILY. DTBench: A synthetic benchmark for document-to-table extraction, 2026. https://arxiv.org/abs/2602.13812. [52]A. Zou, W. Yu, H. Zhang, K. Ma, D. Cai, Z. Zhang, H. Zhao, and D. Yu. Docbench: A bench- mark for evaluating llm-based document reading systems. InProceedings of the 4th Interna- tional Workshop on Knowledge-Augmented Methods for Natural Language Processing, pages 359–373, 2025. A Dataset Collection and Processing To establish a relationally faithful ground truth for Doc2DB-Bench, we curated a collection of com- plex relational databases from two predominant text-to-SQL benchmarks: BIRD [ 23] and Spider [45]. The collection process followed a multi-stage pipeline designed to maximize domain diversity and structural complexity. 050100150200250300350400>450 # Rows 0 2 4 6 8 10 Count 10152025303540455055>60 # Columns 0 1 2 3 4 5 6 7 8 9 Count 02505007501000125015001750>2000 # Cells 0 2 4 6 8 10 12 Count Figure 6:Distribution of database scale in Doc2DB-Bench in terms of rows, columns, and cells. A.1 Database Selection and Sourcing We first screen the database pools from BIRD and Spider, selecting databases that are suitable for document-to-database evaluation. Specifically, we prioritize databases with at least three intercon- nected tables, non-trivial schemas involving multiple entity types, relationship tables, and foreign-key dependencies, as well as diverse cross-table dependencies. To ensure broad coverage, the final collec- tion spans seven domains, including education, finance, healthcare, university, sports, transportation, and others. Unlike conventional benchmarks that focus on flat-table extraction, our selection favors databases with rich entity-relationship (ER) structures and explicit relational dependencies, enabling the evaluation of cross-table reasoning and database construction capabilities. A.2 Data Cleansing and Schema Refinement The raw databases underwent several refinement steps to ensure they were suitable for document synthesis: •Schema Standardization:We normalize attribute names, unify schema representations, and en- sure that primary and foreign keys are explicitly defined. This step reduces schema ambiguity and facilitates reliable DB-to-document synthesis. •Tuple Sampling and Instance Construction:Since using all database tuples may produce ex- cessively long documents and dilute evidence density, we sample representative tuples while pre- serving entity coverage, inter-table relationships, and referential integrity. The resulting instances remain complete relational databases rather than isolated table fragments, yielding 117 entity ta- bles and 132 relationship tables with 7,341 rows and 41,935 cells. Fig. 6further illustrates the distribution of database scales in terms of rows, columns, and cells. •Integrity Verification:We perform SQL-based sanity checks on the sampled databases to en- sure that all instances satisfy relational constraints, including primary-key uniqueness, foreign-key referential integrity, and attribute-level validity. We further verify schema consistency, table con- nectivity, and tuple completeness to ensure that sampled instances preserve the original relational structures without introducing broken dependencies or isolated fragments. Only verified database instances are retained for subsequent document synthesis. A.3 Domain-Specific Style Grounding Although databases from BIRD [23] and Spider [45] provide diverse domain schemas and struc- tured facts, they are not designed to capture realistic document styles and writing conventions. To improve document realism, we augment the structured data with domain-specific narrative styles by collecting reference documents from LongDocURL [8], DocBench [52], MMLongBench [29], and MMDocRAG [10]. These references cover diverse document formats, ranging from academic papers and financial reports to posters and restaurant menus, serving as stylistic anchors for the syn- thesis pipeline. They enable the generation of authentic domain-specific documents while strictly preserving the factual constraints of the underlying databases. A.4 Verification Procedure. Although our dataset is generated through a multi-agent synthesis pipeline, we incorporate a rigorous verification procedure to ensure the quality and reliability of the synthesized documents. Figure 7:Representative examples from the finance, healthcare, and education domains. Each case pairs a multi-table relational schema with a domain-specific synthesized document, where colored links illustrate the grounding between database records and document evidence. First, our framework includes a dedicatedValidator Agentthat performs automatic verification from two perspectives. At the fine-grained level, it checks whether generated sentences and paragraphs faithfully cover the information contained in the source databases, preventing missing or incomplete evidence. At the database reconstruction level, it evaluates whether the complete database can be accurately recovered from the generated document. Any detected inconsistencies or missing infor- mation trigger an iterative repair process to refine the generated documents. Detailed verification and repair procedures are provided in Appendix E.4. Second, we conduct human verification on the generated documents. We recruit seven PhD candi- dates in computer science, with each reviewer responsible for one specific domain. Each reviewer evaluates the synthesized documents from two aspects: (1) whether the document style and content align with realistic domain-specific documents encountered in practice; and (2) whether the underly- ing database information can be reliably reconstructed from the generated document. Any cases that fail these criteria, such as unrealistic document patterns, missing database evidence, or inconsistent entity relationships, are filtered out to ensure the quality and reliability of the final dataset. A.5 Dataset Visualization. Figure7presents representative examples from the finance, healthcare, and education domains, each pairing a multi-table relational schema with a realistic domain-specific document. For example, in the finance case, company information is stored in theCompaniestable, quarterly revenue and net income inQuarterly_Financials, and closing prices inStock_Prices. The synthesized equity research report integrates these distributed records into statements such as “Acme Corp reported revenue of $125.0 million and net income of $18.2 million,” while linking them through the shared company_id. Recovering the database therefore requires not only extracting individual values, but also resolving entity references, normalizing expressions such as125,000,000and “$125.0 mil- lion,” and correctly assigning evidence to multiple related tables. Similar examples across the other Figure 8:Challenging examples of relation extraction that require different inter-table capabilities. domains illustrate the diversity of schema structures, document styles, and cross-table dependencies covered by Doc2DB-Bench. B Details of dual-level Taxonomy Pillar 2 inter-table capabilities are organized into the following four types: Relationship Linking (RL).This capability is required when natural language narratives must be parsed and mapped into structured relational tuples, accurately aligning multiple entities across tables based on cardinality and business rules. We categorize this linking into three subcategories. One-to-Many Allocation. The document provides a compressed statement (e.g.,a manager taking over multiple projects), requiring the model to expand it into multiple normalized relationship rows that preserve the underlying 1:N or N:1 cardinality rather than collapsing them. As illustrated in Fig. 8(a), a statement assigning Alice to Projects X, Y, and Z must be expanded into three separate relationship tuples. Multi-Entity Binding.The document describes intertwined N-ary relations (e.g.,distinct suppliers delivering different parts to various projects) in a dense narrative, requiring exact triplet or multi- entity alignment without mismatching pairs or hallucinating spurious links based on text proximity. Conditional Linkage.Requires evaluating existing table attributes against implicit business rules to automatically establish foreign key relations, even without explicit entity name mentions. Identity Resolution (IDR).This capability is required when entity references within the document are ambiguous, pronominal, or use aliases, necessitating cross-referencing with table attributes or surrounding context to ground the mention to a unique, exact database record. Entity Disambiguation. The document uses an ambiguous alias or name that matches multiple database records, requiring contextual clues and schema attributes to identify the correct entity. As shown in Fig.8(b), the alias “Lao Zhang” must be resolved to the employee in the technical depart- ment based on the surrounding “technical issue” context. Coreference Resolution.The target entity is referred to using pronouns, role titles, or implicit aliases scattered across sentences, requiring cross-sentence contextual reasoning to correctly resolve these coreferences to the anchor entity and populate the corresponding relationship fields. Structural Reasoning (SR).This capability is required when relational links cannot be directly extracted from a single text span but must be logically deduced by chaining multiple implicit relations, table structures, or temporal events. Transitive Inference.The document and existing tables provide linked but disjointed relational pairs (e.g.,entity A is linked to B, and B is linked to C), requiring the deduction of implicit transitive relations to populate the target schema. Fig.8(c) illustrates this case: “Alice leads the Mobile Team” and “the Mobile Team manages Project X” must be composed to recover the implicit relation between Alice and Project X. Global Aggregation.The attributes required to form a complete multi-dimensional relationship are scattered across disjointed text spans, requiring multi-hop reading comprehension to logically chain these spans and assemble all necessary foreign keys into a single valid record. Dynamic Change.The document narrative involves cross-table event timelines, sequential actions, or entity state changes over time (e.g., completing a prerequisite course before a new enrollment), requiring reasoning over these temporal dynamics to determine the final, valid relationship status. Integrity Constraint (IC).This capability is required when the extraction of relationships is gov- erned by negative evidence, exceptions, or boundary rules, demanding that the model suppress naive co-occurrence extraction and strictly adhere to database integrity. Null Relation Extraction.The document provides explicit negative evidence, such as cancelled events or unmet prerequisites, requiring the model to identify an empty relation state (e.g.,outputtingNULL) and avoid hallucinating links from superficial keyword co-occurrence. Mutual Exclusion.The document specifies broad relationships alongside explicit exceptions or neg- ative constraints, requiring the model to infer relationship boundaries and exclude invalid relational tuples. As illustrated in Fig. 8(d), although all managers are generally assigned to Project X, the explicit exception “except Bob” requires suppressing the tuple(Bob,Project X). C Experiment Setup C.1 Metric Design Principles: Cell Alignment for Evaluation Naive sequential matching aligns predicted rows to ground-truth rows by position, which is fragile in two ways. Cross-Tuple Hijacking: a model extracts the correct value but assigns it to the wrong row, penalizing an otherwise valid prediction. Cascading Misalignment: a single missed or inserted row shifts all subsequent alignments, inflating both false-positive and false-negative counts. To address both failure modes, we adopt aGlobal Maximum Weight Matchingstrategy over a bipartite graph of ground-truth and predicted rows. Notably, We do not use primary keys as alignment anchors because database-specific identifiers, particularly surrogate keys, are often absent from real-world documents and must be generated dur- ing database construction. Different models may therefore assign different identifiers to otherwise equivalent records. Primary-key-based matching would conflate arbitrary identifier generation with extraction correctness. We instead align rows using shared non-key attributes and evaluate key valid- ity separately through database-integrity metrics. Let푇 ∗ =푟 ∗ 1 ,...,푟 ∗ 푚 and푇=푟 1 ,...,푟 푛 denote the ground-truth and predicted row sets, respec- tively. Matching proceeds in four stages. Stage 1: Global Scoring Matrix.LetKbe the shared column names between푇 ∗ and푇. For each pair ( 푟 ∗ 푖 ,푟 푗 ), we compute a composite similarity score: 푠(푟 ∗ 푖 , 푟 푗 )= 1 |K| ∑ 푘∈K sim ( 푟 ∗ 푖 [푘], 푟 푗 [푘] ) , wheresim(·,·)returns1for exact numeric equality after normalization and uses normalized Leven- shtein similarity for textual values: sim text (푎, 푏)=1− 푑 Lev (norm(푎),norm(푏)) max ( |norm(푎)|,|norm(푏)| ) , where푑 Lev denotes the Levenshtein edit distance andnorm(·)applies lowercasing, whitespace nor- malization, and punctuation normalization. Missing or incompatible values receive a score of0, yielding a global푚×푛scoring matrix. Stage 2: Global Sorting.All pairs with푠(푟 ∗ 푖 ,푟 푗 )≥휏(휏=0.3) are sorted globally in descending order, ensuring high-confidence pairs are resolved first and cannot be preempted by lower-scoring candidates. Stage 3: Double-Locking Assignment.Pairs are assigned greedily from the ranked list. Once (푟 ∗ 푖 ,푟 푗 )is matched, both indices are locked and excluded from further assignment, enforcing strict one-to-one mapping and preventing score inflation via duplicate row generation. Stage 4: Cell-Level Settlement.Within each matched pair, scoring proceeds cell-by-cell as defined in Section5. Unmatched ground-truth rows contribute all non-empty cells to the missing count; unmatched predicted rows contribute to the hallucination count, ensuring every cell is accounted for exactly once. C.2 Prompts for LLM Score. LLM-as-a-Judge: Database Extraction Evaluation Prompt You are a database evaluation expert. Please evaluate the quality of database tables extracted from docu- ments. Evaluation Dimensions: •Accuracy:Semantic consistency, exact numeric matching (including decimal places), and string sim- ilarity≥90%. •Completeness:Presence of all key information from the standard answer and structural integrity. •Standardization:Compliance with Schema definitions and data type/format consistency. Scoring Constraints: •Numeric values must beexactly equal; string similarity below 90% is a mismatch. •Type mismatches are considered serious errors. •Total score ranges from 0 to 100; a perfect match equals 100 points. Output Format:Provide a comprehensive explanation (≤100 words), then output the score as: Evaluation evidence: [explanation] Rating: [[score]] D More Doc2DB-Bench Evaluation D.1 Database Integrity Analysis. Metric.Beyond attribute accuracy, we evaluate whether models can construct structurally complete and constraint-consistent databases. We measureEntity CoverageandRelation Coverageto assess gpt-4o gpt-5.4 gemini2.5-flash gemini2.5-pro claude-opus-4-6 qwen3-max ds-v4-flash 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Score #Entity Coverage (EC) gpt-4o gpt-5.4 gemini2.5-flash gemini2.5-pro claude-opus-4-6 qwen3-max ds-v4-flash 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Score #Relationship Coverage (RC) gpt-4o gpt-5.4 gemini2.5-flash gemini2.5-pro claude-opus-4-6 qwen3-max ds-v4-flash 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Score #Reference Integrity (RRIR) Figure 9:Database-level performance across Entity Coverage, Relation Coverage, and Reference Integrity Rate. the recovery of entities and inter-entity relationships, respectively. We further reportReference In- tegrity Rate, which measures the proportion of foreign-key references that correctly point to existing entities, reflecting the validity and consistency of the constructed database. Result.Database-level results show that reconstruction requires both structural coverage and valid cross-table references. Entity Coverage exceeds Relation Coverage by 12.17–24.52 points across all models, highlighting the greater difficulty of recovering inter-table relationships. Coverage also does not guarantee validity: Gemini-2.5-Pro achieves strong coverage but the lowest RRIR, revealing frequent cross-table reference errors. GPT-5.4 leads all three metrics, demonstrating the strongest ability to recover relational structures while preserving database consistency. Finding 4.Database construction requires more than accurate extraction: maintaining relational consistency and valid entity references remains a critical challenge for LLMs. D.2 Full Results of Different Domains Table 6:Results on DOC2DB-BENCH across various domains.Ver./Sizeshows model version or size. Methods Ver. /Size Domain OverallAcc. Edu. Fin. Med. Uni. Sports Trans. Others Open-source Models Qwen2.514B-Ins13.25 34.21 37.35 36.44 18.16 47.31 34.8232.97 LLaMA-3.170B-Ins10.94 13.18 16.29 21.54 27.86 7.308.1614.29 Qwen2.572B-Ins30.41 34.79 40.98 47.73 31.87 38.80 29.9836.07 Proprietary Models GPT-4o-47.89 57.49 54.20 72.32 62.75 68.02 51.1159.05 GPT-5.4- 67.6768.6566.68 74.7882.2485.2278.4975.25 Gemini 2.5Flash57.63 64.04 41.00 73.13 71.22 78.95 64.5765.99 Gemini 2.5Pro69.8165.4676.5379.75 69.67 74.78 67.9470.81 Claude Opus4-6 67.01 65.38 75.5180.0178.3687.8369.8973.60 Qwen3Max52.06 62.55 56.37 72.15 57.24 70.73 64.1052.80 DeepSeek-V4Flash53.66 59.39 48.23 55.60 57.79 73.56 51.8057.06 The results reveal a substantial performance gap between open-source and proprietary models. Among open-source models, Qwen2.5-72B achieves the highest overall accuracy of 36.07, but still trails GPT-5.4 by 39.18 points. GPT-5.4 obtains the best overall accuracy of 75.25 and leads in finance, sports, and other domains. Nevertheless, no single model dominates across all domains: Gemini 2.5 Pro performs best in education and medicine, while Claude Opus 4-6 leads in university and transportation, suggesting that different domains place distinct demands on schema interpreta- tion, cross-table reasoning, and domain-specific information normalization. Domain-level performance can be further understood from the normalized dataset statistics. Aver- aged across all evaluated models, transportation achieves the highest accuracy (63.25), followed by university (61.35), whereas education obtains the lowest accuracy (47.03). Education contains the largest number of rows per database (276.7), together with 8.0 tables and 4.0 relation tables on av- erage, requiring models to recover a relatively large number of tuples while preserving cross-table associations. Finance is also challenging, but primarily because of its relational density rather than its absolute size: each database contains 6.2 tables and 3.9 relation tables on average, with the high- est relation-to-entity ratio of 1.70. Medical databases present a different difficulty, having the largest average numbers of columns (66.3) and cells (1,174.0) per database, which increases the burden of broad attribute coverage. In contrast, although transportation documents are the longest on average (53,091 tokens), their databases contain only 3.6 tables and 1.2 relation tables per database, suggest- ing that document length alone is not the dominant source of difficulty. Overall, the results indicate that performance is more strongly influenced by the interaction of tuple volume, schema width, rela- tion density, and implicit cross-table dependencies than by document or database scale in isolation. E Implementation of Doc2DB During the entireControllable DB2Doc Reverse-Synthesisprocess, we design specific prompts and rigorous verification protocols across four key agentic modules to ensure generation fidelity and relational complexity. Firstly, in the capability assignment stage, theLabelingAgentacts as a semantic router. We construct prompts to strategically assign fine-grained intra-table (Pillar 1) and inter-table (Pillar 2) capability labels, which directly control the downstream extraction difficulty. Secondly, during the evidence decomposition stage, theRefiner Agentprocesses the annotated tables into atomic evidence. To guarantee the mathematical and semantic rigor of this step, we implement a strictchecklist-based verificationprotocol. We design prompts that explicitly instruct the model to verify four critical di- mensions: Value Correctness, Label Faithfulness, Schema Linkage, and Null Faithfulness. Thirdly, to bridge the gap between discrete evidence and cohesive natural language, theWriter Agentexe- cutes the document generation. We adopt a context-aware generation paradigm where the prompt incorporates a sliding window of historical text and configurable stylistic profiles to emulate real- world document morphological complexity Finally, in order to obtain a high-quality, hallucination- free corpus, theValidator Agentconducts a rigorous dual-validation pass. We construct prompts to evaluate Coverage Verification and Extraction Consistency. Documents that fail these checks trigger an iterative reject-and-rewrite mechanism. E.1 The prompt for Labeling Agent Pillar 1 Assignment Prompt Goal:Assign intra-table (Pillar 1) capability labels to target cells in the entity table. Inputs: •Input Table:markdown_table •Non-target Key Cols:non_target_key_columns •Attribute Schema:attribute_descriptions •Capability Definitions:capability_definitions •Target Label Ratio:label_ratio Instructions: 1.Assign labels that increase single-table extraction dif- ficulty while preserving correctness. 2.Type-Strategy Compatibility (CRITICAL):Inde- pendently analyze the semantic nature and data type of each attribute. Only assign strategies logically/- mathematically suitable for the cell’s value. 3.A cell may have 0, 1, or multiple strategies (default is none). 4.Assign creatively but ensure text remains fluent. Avoid over-complicating simple facts. 5.DO NOT assign labels to Non-target Key Columns. 6.For composite primary keys, use", "as separator. 7.Label coverage: at least the target ratio of assignable cells must have non-empty labels. 8.Return JSON only. Output JSON Schema: "assignments": "<pk_1>": "<attr_1>": [] Pillar 2 Assignment Prompt Goal:Assign inter-table (Pillar 2) capability labels to target cells in the relation table. Inputs: •Input Table:markdown_table •Table Schema Context:relation_context •Attribute Schema:attribute_descriptions •Capability Definitions:capability_definitions •Target Label Ratio:label_ratio Label Assignment Constraints: •IDR_ED,RL_O2M,RL_MB, andRL_MImust each ap- pear alone. Do NOT combine them. •IDR_ED,RL_O2Mrequires that the same foreign key entity appears in≥2 rows. Do NOT assign if unique. Instructions: 1.Assign labels that increase inter-table linking, topol- ogy, and structural extraction difficulty. 2.Topology-Strategy Compatibility (CRITICAL): Analyze structural distribution. Only assign rela- tional strategies logically suitable for the topology. 3.Output ONLY row-level labels per relation row_key. Do NOT output per-attribute labels. 4.A row may have 0, 1, or multiple strategies (default is none). 5.Top-level key MUST use the relation table PK string. 6.Composite keys format:<pk1>,<pk2>(e.g.,12, 10). 7.Label coverage: at least the target ratio of rows must contain non-empty labels. 8.Return JSON only. Output JSON Schema: "assignments": "<pk_1>": [], "<pk_2>": [] The prompt above illustrates how the Labeling Agent acts as a semantic router to assign fine-grained intra-table (Pillar 1) and inter-table (Pillar 2) capability labels. It explicitly instructs the model to analyze data types and topological contexts, ensuring that specific reasoning challenges are injected logically to control downstream extraction difficulty while adhering to target label ratios. E.2 Refiner: Principle of Checklist Given the annotated targets, the Refiner decomposes each target into atomic evidence that realizes its assigned capability labels, forming a global evidence pool for subsequent document generation. When multiple labels are assigned, the evidence must jointly satisfy the corresponding intra-table and inter-table reasoning requirements. To ensure the rigor of the synthesized evidence, we deploy an LLM-as-a-verifier [50] using a strict checklist-based protocol. The verification assesses the generated evidence guidance across four critical dimensions: •Value and Relation Correctness:Verifies that the guidance accurately reflects target cell values and inter-record relations, ensuring numerical correctness for calculated fields and allowing com- plete relational tuples to be unambiguously recovered. •Label Faithfulness:Verifies that the guidance faithfully instantiates the assigned capability labels and rejects cases that omit or misrepresent the intended reasoning requirements. •Schema Linkage:Ensures that the guidance is grounded in valid schema elements and preserves the intended structural dependencies among tables. For relational tables, primary- and foreign-key references must be correctly identified and linked, so that participating entities can be unambigu- ously connected and the resulting evidence remains consistent with the target database schema. •Data Integrity & Null Faithfulness:Ensures that the guidance contains only information sup- ported by the target database and does not introduce values from unrelated cells or records. It also verifies that missing, unavailable, or non-applicable attributes remain uninstantiated, preventing the generation of unsupported values or spurious relational links. On failure, the verifier acts as a critic, returning a specific rationale and actionable revision sugges- tions (e.g., “Instruct the writer to explicitly mention both the supplier and project names”). This process iteratively refines the text until it passes all checks or reaches a retry limit. E.3 The prompt for Writer Agent Writer Agent Prompt [System Role & Objective] You are an expert Writer Agent operating within a parameterized document synthesis pipeline. Your goal is to write ONE coherent, polished, and natural-sounding section of a larger document while perfectly preserving structured ground-truth facts. [Generation Context] •Previous Block Context (Tail Anchor): previous_context •Current Block ID: block_id (Index: block_index of total_blocks) [Base Parameters] •Global Target Document Length: document_length_tokens tokens •Suggested Token Budget for THIS Block: target_block_tokens tokens •Document Style & Tone: document_style •Section Template Hint: section_template [Injection Protocols] •Hard Cases Protocol: hard_cases_protocol •Controlled Noise Protocol: noise_protocol •Representation Complexity Protocol: complexity_protocol [Mandatory Ground Truth (Evidence)] You MUST explicitly embed the following structured facts. Do NOT drop any fact: facts_with_tags [Execution Instructions] 1.FACTUAL FIDELITY: Embed all mandatory facts without altering their meaning. 2.TAG PRESERVATION: Keep facts wrapped in their original tags (e.g.,<rX_cY>). 3.STYLISTIC ADAPTATION: Match the requested style, tone, and template. 4.PROTOCOL COMPLIANCE: Strictly follow any active injection protocols; ignore empty ones. 5.LENGTH & PACING: Stay close to thetarget_block_tokensbudget. 6.SEAMLESS BRIDGING: Continue logically and smoothly from the previous context. 7.FORMATTING: Use natural prose paragraphs. No bullets, markdown, or JSON. 8.OUTPUT STRUCTURE: Start exactly with the Section Title, followed by the body. 9.FINAL OUTPUT: Return ONLY the synthesized text, without conversational filler. The prompt above demonstrates how the Writer Agent transforms serialized atomic evidence into cohesive, long-form natural language documents. It guides the model to employ a context-aware generation paradigm, seamlessly weaving discrete facts into fluent narratives while conforming to designated stylistic profiles and mimicking the morphological complexity of real-world corpora. E.4 The prompt for Validator Agent Verification Agent Prompt Your Goal: Verify whether the generated text is faithful to the table content without fabrication. Table Content:markdown_table Generated Text:generated_text Verification Checks: To ensureCell Extraction Consistency, Key values stated ingenerated_textmust be extractable or in- ferrable fromTable Content. Output Format: Respond with ONLY a single, valid JSON object: "ok": true/false, "errors": [ "description": "Error description", "suggestion": "How to fix" ] FINAL INSTRUCTION:Output ONLY the valid JSON object. If any check fails, setokto false and in- clude all issues inerrors. Repair Agent Prompt Goal:Repair a section of text that failed verification. Input: •Original Text:content •Errors:errors •Required Facts:facts Instructions: 1.Repair the section to fix the reported errors. 2.Ensure all required facts remain correct. 3.Keep the narrative flow natural. 4.Preserve all provenance-style tags already present in the original text. Output Format: Return ONLY the repaired text content. No titles, no metadata, no markdown code blocks. The prompt above details the rigorous dual-validation protocol executed by the Validator Agent to guarantee the fidelity and relational consistency of the synthesized documents. It tasks the model with strictly evaluating coverage verification and extraction consistency, ensuring no evidence is omitted or hallucinated, which inherently serves as the trigger for our iterative reject-and-rewrite mechanism. F Authenticity Verification F.1 LLM & Human Score The synthesized documents in DOC2DB-BENCH should be fluent, logically coherent, and faithful to the conventions of real-world domain documents rather than appearing as artificial or template- based constructions. To evaluate document quality and authenticity, we adopt a four-dimensional framework combining LLM-as-a-judge with human evaluation. As shown in Table 6, the framework assesseslexical richness,logical consistency,textual coherence, andgenre fidelity, each assessed us- ing a five-point rubric, following a similar strategy of previous work (e.g.,LLMs4Synthesis [3] and Table 7:Evaluation rubrics for generated documents (1=worst, 5=best). Score Lexical RichnessLogical Consistency Textual Coherence Genre Fidelity 1Repetitive: Minimal variety, robotic repetition Fragmented: No connections, random claims Incoherent: Difficult to follow, random jumps Mismatch: Incorrect genre conventions 2Limited: Simple vocabulary, narrow range Weak: Loose transitions, vague reasoning Poor flow: Jarring transitions, disconnected Weak: Limited and inconsistent cues 3Acceptable: Adequate variety, standard usage Acceptable: Clear order, basic signaling Acceptable: Some awkward transitions Plausible: Recognizable but generic style 4Versatile: Natural synonyms, precise terminology Compelling: Strong arguments, coherent progression Smooth: Clear progression, minor issues Authentic: Consistent genre-specific conventions 5Sophisticated: Rich nuances, professional mastery Rigorous: Flawless chain, seamless attribution Seamless: Natural flow, effortless transitions Professional: Realistic and polished genre presentation Figure 10:menu case DTBench [51]). We use DeepSeek-V3.2 as the automatic evaluator. In addition, five graduate stu- dents with research experience in document intelligence independently assess 50 randomly sampled documents using the same rubrics. The LLM and human evaluation results in Table5consistently demonstrate the high linguistic quality and domain authenticity of the synthesized documents. F.2 Real Case Comparison Fig. 10–13provide qualitative comparisons between synthesized and real-world documents across diverse domains, including finance, aviation, media, and restaurant menus, further demonstrating the quality and realism of our synthesis pipeline. Figure 11:finance case Figure 12:airline case Figure 13:twitter case

Instant research discovery