Paper deep dive

SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL

Geonho Lee, Min-Soo Kim

Year: 2026Venue: arXiv preprintArea: cs.DBType: PreprintEmbeddings: 86

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task-specific fine-tuning. However, existing LLM-based systems remain unreliable, often generating SQL queries that are invalid under the database schema, referencing non-existent tables, attributes, functions, or values. Such errors persist because interactions with the database management system (DBMS) are typically limited to error messages, leaving it in a largely passive role during query refinement. This paper proposes SafeQL, \textit{a search-based refinement paradigm that redefines the role of the DBMS as an active guide in the refinement process}. Instead of regenerating entire queries after execution failure, SafeQL interprets DBMS feedback to incrementally repair only the erroneous components. Each refinement step is formulated as a guided search within a \textit{safe query space}, where candidate queries are progressively validated through DBMS execution, thereby converging to an executable query and preventing repeated regeneration of errors. Experiments on the Bird and Spider benchmarks show that SafeQL significantly improves execution accuracy and efficiency compared to regeneration-based methods.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

85,008 characters extracted from source content.

Expand or collapse full text

SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL Geonho Lee ghlee5084@kaist.ac.kr Korea Advanced Institute of Science and Technology Republic of Korea Min-Soo Kim ∗ minsoo.k@kaist.ac.kr Korea Advanced Institute of Science and Technology Republic of Korea ABSTRACT Large language models (LLMs) have advanced Text-to-SQL by enabling natural language interfaces to databases without task- specific fine-tuning. However, existing LLM-based systems remain unreliable, often generating SQL queries that are invalid under the database schema, referencing non-existent tables, attributes, functions, or values. Such errors persist because interactions with the database management system (DBMS) are typically limited to error messages, leaving it in a largely passive role during query refinement. This paper proposes SafeQL, a search-based refinement paradigm that redefines the role of the DBMS as an active guide in the refinement process. Instead of regenerating entire queries after execution failure, SafeQL interprets DBMS feedback to incremen- tally repair only the erroneous components. Each refinement step is formulated as a guided search within a safe query space, where candidate queries are progressively validated through DBMS exe- cution, thereby converging to an executable query and preventing repeated regeneration of errors. Experiments on the Bird and Spider benchmarks show that SafeQL significantly improves execution accuracy and efficiency compared to regeneration-based methods. PVLDB Reference Format: Geonho Lee and Min-Soo Kim. SafeQL: Search-based Refinement for Safe and Efficient LLM-based Text-to-SQL. PVLDB, 19(9): 2210-2223, 2026. doi:10.14778/3819518.3819545 PVLDB Artifact Availability: The source code, data, and/or other artifacts have been made available at https://github.com/Geonho-Lee/SafeQL. 1 INTRODUCTION Natural language interfaces to databases (NLIDBs) have long been a key research goal at the intersection of database systems and nat- ural language processing [11,58]. Among them, Text-to-SQL—the task of translating a natural language question into an executable SQL query—represents the most concrete realization of this vision, enabling users to access structured data through natural language rather than query syntax. By eliminating the need for SQL expertise, Text-to-SQL systems make data access more inclusive and have broad applications in analytics, business intelligence, and scientific data exploration [12, 13, 21]. ∗ Corresponding author. This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing info@vldb.org. Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 19, No. 9 ISSN 2150-8097. doi:10.14778/3819518.3819545 Research on Text-to-SQL has evolved through several genera- tions. Early symbolic systems such as PRECISE [29], NaLIR [17], SQLizer [55], and ATHENA [36] relied on rule-based parsing and semantic grammars that explicitly mapped natural language to SQL templates. Although interpretable, these systems were fragile and required manual engineering for each domain. Subsequent neu- ral semantic parsers replaced handcrafted rules with data-driven models trained on paired natural-language and SQL examples. Rep- resentative models include Seq2SQL [63], SQLNet [53], IRNet [9], RAT-SQL [49], and SmBoP [35], which introduced neural encoders and schema linking for cross-domain generalization. Later work incorporated syntax constraints and intermediate representations to improve validity and cross-schema transfer [20,38]. Despite sub- stantial progress, these models still require expensive annotation and retraining whenever the database schemas change, limiting their scalability in practical deployments. The emergence of large language models (LLMs) such as GPT [3] has profoundly transformed the Text-to-SQL landscape. Prompt- based approaches like DAIL-SQL [7] and MCS-SQL [15] demon- strated that general-purpose LLMs can perform Text-to-SQL gen- eration through schema-aware prompting, without task-specific fine-tuning. Building on this paradigm, subsequent prompting tech- niques—including few-shot sampling [3,7,15,19,24,47], chain-of- thought reasoning [50,59], and least-to-most decomposition [31,32, 64]—further improved logical coherence and interpretability. More recently, multi-agent systems including MAC-SQL [48], CHESS- SQL [42], CHASE-SQL [30], Alpha-SQL [16], and OpenSearch- SQL [52] have introduced collaborative reasoning, in which multi- ple LLM agents decompose, verify, and align candidate SQL queries to ensure consistent output. These systems mark a shift from one- shot generation to autonomous reasoning pipelines that incorporate coordination among multiple agents. Despite these advances, reliability remains a critical challenge due to the inherent hallucination tendencies of LLMs. In practice, many failures arise when generated queries cannot be executed, for example, due to references to nonexistent relations, attributes, or functions. Even state-of-the-art LLMs often produce queries that appear plausible but fail at the DBMS level. Such failures expose a mismatch between the user’s intent and the database schema and disrupt downstream workflows, making it essential to refine queries into safe (i.e., executable) ones. Recent studies [31,34,42,48,52] have attempted to mitigate this issue through regeneration-based refinement: executing the gener- ated SQL, capturing DBMS error messages, and prompting the LLM to regenerate a corrected query. However, this paradigm treats the DBMS primarily as a passive checker that provides error messages, and repeatedly regenerates entire queries based on the errors. As a result, the regeneration loop is both inefficient—requiring excessive tokens and computation—and unreliable, often reintroducing previ- ous errors with no guarantee of convergence. Figure 1(a) illustrates a regeneration-based refinement process. The model begins with an initial query푞 0 , which fails during execution. Upon failure, the entire query is discarded, and a new query푞 1 is regenerated based on the previous error message. This cycle repeats (e.g.,푞 2 ,푞 3 . . . ) until an executable query is obtained. Each iteration re-parses and re-executes the entire query, resulting in redundant computation and limited reuse of valid fragments. In contrast, we propose SafeQL, a refinement framework that achieves safe and efficient correction through a search-based re- finement paradigm that fundamentally rethinks how Text-to-SQL errors are corrected. Rather than discarding the initial query푞 0 , SafeQL interprets and incrementally refines it. As shown in Fig- ure 1(b), when푞 0 fails, SafeQL does not regenerate a new푞 1 . Instead, it analyzes the DBMS feedback to precisely locate the faulty com- ponent—whether a relation, attribute, or function—and applies a minimal, structure-preserving correction as depicted in blue arrows in the Figure 1(b). This transforms푞 0 directly into refined and exe- cutable queries, while preserving all valid fragments of the original logic. Through this process, SafeQL replaces the traditional trial- and-error regeneration loop with a guided search over a safe query space as depicted in the green part of the Figure 1(b), where candi- dates are incrementally repaired to satisfy schema-level constraints enforced by the DBMS. Since this safe query space may contain multiple executable refinements for a single input푞 0 —for example, when an unknown attribute can be repaired using several attributes in the schema-—SafeQL selects the candidate that is semantically the most aligned with the initial query푞 0 . This paradigm shift re- defines the DBMS from a passive checker into an active reasoning engine that incrementally narrows the search to the most faithful executable query, improving both safety and efficiency. DBMS DBMS 푞 0 푞 0 (a) Regenera�on-based refinement(b) Search-based refinement execu�on layer 1 (check rela�ons) LLM execu�on layer N (check types) execu�on layer 2 (check a�ributes) 푞 0 Error 푞 2 Error 푞 1 Error execu�on layer 1 (check rela�ons) LLM execu�on layer N (check types) execu�on layer 2 (check a�ributes) Execu�on result Figure 1: High-level comparison of refinement paradigms. Beyond the paradigm itself, the effectiveness of SafeQL critically depends on how search and optimization processes are conducted. The safe query space can grow exponentially, as each detected error may yield multiple candidate substitutions for relations, attributes, or values. To manage this combinatorial complexity, SafeQL em- ploys a best-first search strategy guided by a semantic distance metric that combines structural similarity and embedding-based relevance. This prioritization ensures that the most promising can- didates are explored first, improving convergence while preserving the user’s original query intent. In terms of optimizations, SafeQL in- corporates type-based pruning, which eliminates type-inconsistent refinements, and top-퐾pruning, which restricts candidate substi- tutions to the most semantically relevant ones. Together, these optimizations make search-based refinement both practical and scalable. Equally important is how SafeQL is realized as a practical, system- level framework rather than merely an algorithmic concept. SafeQL is integrated directly into the DBMS, enabling refinement through the database’s own parser, binder, and type analyzer instead of treat- ing SQL as plain text. This tight integration allows for precise error localization, while system-level optimizations—such as caching and vector indexing—make semantic similarity computations efficient at scale. Moreover, SafeQL incorporates a hybrid refinement strategy that combines its search-based refinement with selective regen- eration to handle cases where the initial query is fundamentally misgenerated. Together, these system-level integrations make our SafeQL a complete, end-to-end framework that delivers reliable and efficient Text-to-SQL refinement in real-world environments. Our main contributions are summarized as follows: • We propose a search-based refinement paradigm that reformu- lates Text-to-SQL refinement as a guided exploration of a safe query space, where execution feedback serves as actionable se- mantic guidance rather than passive error reporting. • We propose a best-first search guided by a semantic distance metric, augmented with type-based and top-K pruning, enabling scalable and accurate refinement. •We implement SafeQL as a DBMS-integrated refinement frame- work, incorporating caching, vector indexing, and hybrid refine- ment to achieve efficient and robust execution. •We conduct comprehensive experiments on Bird and Spider benchmarks, demonstrating up to 5.8% improvements in exe- cution accuracy and 15×reductions in token usage compared to regeneration-based baselines. The remainder of this paper is organized as follows. Section 2 provides background and definitions. Section 3 introduces the safe query space and the refinement tree formulation. Section 4 details the search strategy, and Section 5 presents pruning optimizations for scalable refinement. Section 6 describes the system architecture, and Section 7 reports the experimental results. Finally, Section 8 discusses related work, and Section 9 concludes the paper. 2 PRELIMINARIES 2.1 Text-to-SQL Stages The Text-to-SQL task translates a natural language question into an executable SQL query over a given database. Typically, this process can be decomposed into three stages: (1) extracting database information, (2) generating candidate SQL queries, and (3) refining them using DBMS execution. We formalize these stages as follows. Definition 1 (Text-to-SQL Process). Let퐷denote a database, and letN,P, andQdenote the natural language, prompt, and query spaces, respectively. The Text-to-SQL process is defined as N 푓 extract −→P 푓 generate −→Q 푓 refine −→Q,where: • Extraction. 푓 extract :N × 퐷 →P interprets the natural language question in the context of퐷and constructs a prompt enriched with database-level hints. • Generation. 푓 generate :P →Q synthesizes an SQL query using the LLM that reflects the intent expressed in the prompt. • Refinement. 푓 refine :Q× 퐷 →Q corrects the generated query using feedback obtained from DBMS execution over 퐷 .□ Although LLMs can generate plausible SQL queries, they may still suffer from hallucinations. Hence, the refinement stage us- ing DBMS execution is crucial to ensure reliability, regardless of whether the generation is based on the prompt [7,15,19,31,59] or the agent [5, 16, 40, 42, 52]. 2.2 Refinement Stage In the refinement stage, the execution of the DBMS plays a central role in guiding the correction of the query, as shown in Figure 1. Execution errors provide essential feedback signals that enable the Text-to-SQL system to identify and correct invalid parts of a query. By interpreting these errors, the system can transform an invalid query into one that is both executable and semantically faithful to the user’s intent. To formalize how DBMS execution supports this process, we first define the notions of Database and Execution Error, describing how DBMS is utilized during refinement. Here, we adopt a simplified database definition tailored for Text-to-SQL scenarios (e.g., assuming that all attributes in 퐴푡푠 are distinct). Definition 2 (Database). A database퐷is a collection of relations, attributes, and values under schema and type constraints. Formally, 퐷=(푅푒푙푠,퐴푡푠,푉푎푙푠,푇푦푝푠,Λ,Γ), where: • 푅푒푙푠 and퐴푡푠are the sets of relations and attributes, respectively. • 푉푎푙푠 is a set of instance-level values that populate the relations. • 푇푦푝푠 is a set of data types (e.g., int, varchar, text). •Λ:푅푒푙푠 →2 퐴푡푠 is a relation signature mapping each relation to its corresponding set of attributes. •Γ:퐴푡푠 ∪ 푉푎푙푠 → 푇푦푝푠is a type constraint mapping each attribute or value to its data type.□ Figure 2 illustrates a simple schema with two relations, Users and Posts, whereΛ(Users)=ID, name, age andΓ(name)= varchar. �me (date) txt_len (int) txt (text) UserID (int) 2025/7/281,352 1 2025/9/25973 1 2025/10/187,492 3 UsersPosts age (int) name (varchar) ID (int) 24James1 20Leo2 26Hana3 푅푒푙푠 :bold 퐴푡푠 : italic 푉푎푙푠 : plain 푇푦푝푠: (bracket) Figure 2: Example of a database. During query refinement, some candidate queries fail during execution due to schema or type violations. We refer to these cases as execution errors, as formally defined as follows. Definition 3 (Execution Error). Consider a DBMS that manages a database퐷. We denote the execution of푞 ∈ Qby the DBMS as 퐷 ⊢ 푞 ⇒ ⟦푞⟧, where DBMS executes푞on퐷and returns the result⟦푞⟧. An execution error occurs when the execution fails, in which case we write: DBMS : 퐷 ⊢ 푞 ⇒ 휖(푞), where 휖(푞) is the error reported by the DBMS. The most common errors in the Text-to-SQL scenario include: • unknown relation: The query references a non-existent table. • unknown attribute: The query refers to an undefined column. •unknown function (or operator): The query invokes a function (or operator) not supported by the DBMS. • empty result: The query executes successfully, but returns no tuples due to unsatisfiable predicates.□ Figure 3 presents typical examples of execution errors along with their correct (gold) SQL counterparts. The top-left query fails because it references a non-existent attribute (username), whereas the top-right query accesses attributes (txt, time) from another relation without performing a join. In the bottom-left example, the function strftime fails to execute, illustrating how dialect-specific differences in SQL function support can cause errors. Finally, the bottom-right query returns an empty result due to a case-sensitive predicate mismatch (‘leo’ vs. ‘Leo’). Although empty results do not trigger runtime errors in standard DBMSs, many Text-to-SQL benchmarks treat them as errors [34,39]. We follow this convention, while noting that empty results may not always correspond to incorrect queries in practice. Our system also provides configurable controls for refinement per error type, and reports detailed error statistics (Figure 9) to facilitate analysis. Error: unknown a�ribute SELECT ID,username, age FROM Users WHERE age >= 24; SELECT ID,name, age FROM Users WHERE age >= 24; SELECTtxt, time FROM Users WHERE age >= 24; SELECT txt, time FROM Users JOIN Posts ON UserID = ID WHERE age >= 24; SELECT txt FROM Posts WHERE strftime(′y′,time)= 2025 SELECT txt FROM Posts WHERE date_part(′y′,time)= 2025 Generated SQL SELECT name, age FROM Users WHERE name =′leo′; SELECT name, age FROM Users WHERE name =′Leo′; (a) Errors requiring different types of refinements (b) Errors lacking inferable hints from error messages Error: unknown func�on Gold SQL Generated SQL Gold SQL Error: empty result Error: unknown a�ribute Figure 3: Examples of erroneous SQLs and gold SQLs. Regeneration-based Text-to-SQL methods [31,34,42,48,52] attempt to correct these errors through a regeneration-based re- finement process, formally defined as follows. Definition 4 (Regeneration-based Refinement). Let푞 푖 ∈ 푄denote the query at iteration 푖(푖 ≥ 0). The refinement process 푓 regenerate refine :Q× 퐷 →Q,consists of: • Execution. The DBMS executes the current query푞 푖 . If execu- tion succeeds (i.e.,퐷 ⊢ 푞 푖 ⇒ ⟦푞 푖 ⟧), it returns the result⟦푞 푖 ⟧. Otherwise, if execution failed, it returns an execution error휖(푞 푖 ). •Regeneration. The refiner updates the prompt푝 푖 into the new one푝 푖+1 with the error information and regenerates a new query: 푝 푖+1 =(푝 푖 ,푞 푖 ,휖(푞 푖 )), 푞 푖+1 = 푓 generate (푝 푖+1 ) This repeats until a valid query 푞 푘 satisfies 퐷 ⊢ 푞 푘 ⇒⟦푞 푘 ⟧.□ Figure 4 illustrates this common prompt structure adopted by most regeneration-based methods. However, as shown in Figure 3(a), identical error messages Error: unknown attribute may correspond to different underlying causes, each requiring a distinct refinement strategy. Moreover, as shown in Figure 3(b), DBMS error messages often provide little or no actionable guidance, making it difficult for the model to determine the correct fix. To alleviate these limitations, several follow-up methods [34,42,52] enrich the error context. RED- SQL [34] augments the error message with constraints violations details to provide a more informative signal. Meanwhile, CHESS- SQL [42] and OpenSearch-SQL [52] incorporate few-shot refinement examples to guide the model toward more plausible refinements. Despite these improvements, all regeneration-based methods fundamentally rely on the LLM to regenerate an error-free query. Since the model offers no guaranty that a newly generated query will execute successfully, the process often requires multiple it- erations before convergence. This iterative regeneration loop is both computationally expensive and time-consuming, significantly increasing LLM token usage and system latency. Refine the SQL based on the followings. [Question] Show me the posts from people who are older than 24 [Database Schema] CREATE TABLE Users (ID int, name varchar, age int) CREATE TABLE Posts (UserID int, txt text, time date) [Generated SQL] SELECT txt, time FROM Users WHERE age >= 24 [Execution result] Error: unknown attribute - txt [Few shot] refinement examples Figure 4: Example of a regeneration prompt. 3 SAFE QUERY SPACE This section formalizes the search space in which SafeQL performs query refinement. Unlike regenerating entire queries after each failure, SafeQL systematically explores a structured space of re- finements—syntactic transformations that incrementally repair an erroneous query based on its observed error. SafeQL focuses on error-directed transformations, i.e., targeted adjustments that resolve the specific cause of failure while retaining the original query’s structure and intent. We first define by defining the atomic refine- ment step, then construct the safe refinement tree that organizes these steps into error-resolving paths, and finally formalize the safe query space comprising executable, error-free queries. 3.1 Refinement Step We begin with a simplified SQL grammar that captures the core syntactic components. Section 6.4 later extends this grammar to support subquery, aliasing, and other complex SQL structures. Query ::= SELECT S FROM F WHERE W F::= R | F JOIN R ON W W::= 푎 1 ⊕ † 푎 2 | 푎⊕ 푣 S::= 푣 | 푎 | f(푎 1 ,푎 2 , ...) Figure 5: Simplified SQL syntax. † The symbol⊕denotes a comparison operator (e.g., =, <, >). Based on this grammar, we define the refinement stage a se- quence of atomic refinement steps, where each step is a transforma- tion between two SQL queries that modifies exactly one syntactic component while preserving the structural validity of the query. Each refinement step corresponds to one of several canonical oper- ations, formally described in Definition 5. Definition 5 (Refinement Step, 푟 −→ ). A refinement step푟is the application of one of the following operations to a given SQL query. • Relation Refinement: replace a relation 푅 with another 푅 ′ : FROM 푅 푟 −→ FROM 푅 ′ • Join Refinement: add a join with another relation푅 ′ and a suitable condition푊 ′ : FROM 푅 푟 −→ FROM 푅 JOIN 푅 ′ ON푊 ′ • Attribute Refinement: replace an attribute푎with another푎 ′ , in either SELECT or WHERE. Typical cases include: SELECT 푎 푟 −→ SELECT 푎 ′ SELECT 푓(푎 1 ,푎 2 , . . .) 푟 −→ SELECT 푓(푎 ′ 1 ,푎 2 , . . .) WHERE 푎 1 ⊕ 푎 2 푟 −→ WHERE 푎 ′ 1 ⊕ 푎 2 or WHERE 푎 1 ⊕ 푎 ′ 2 WHERE 푎 ⊕ 푣 푟 −→ WHERE 푎 ′ ⊕ 푣 • Value Refinement: replace a constant value 푣 with another 푣 ′ : WHERE 푎 ⊕ 푣 푟 −→ WHERE 푎 ⊕ 푣 ′ •Function Refinement: replace a function symbol푓with another 푓 ′ while preserving its argument structure: SELECT 푓(푎 1 ,푎 2 , . . .) 푟 −→ SELECT 푓 ′ (푎 1 ,푎 2 , . . .)□ 3.2 Safe Refinement Tree Although each refinement step defines a syntactically valid trans- formation, executing all possible steps would lead to an excessively large and semantically irrelevant search space. In practice, most exe- cution errors arise from specific syntactic components of the query (e.g., relations, attributes, or functions), and only refinements target- ing those components can effectively resolve the error [4,26,39]. For instance, an unknown relation error is typically resolved by replacing the relation in theFROMclause, whereas an unknown attribute error is commonly addressed by replacing the invalid attribute in theSELECTclause. These refinements differs from tra- ditional rule-based generation [17,36,55], whose handcrafted rules restrict what queries can be produced in the first place; SafeQL’s refinement act only after a query is generated, correcting errors without limiting the overall expressiveness of the query. This obser- vation motivates the notion of a safe refinement tree, which defines the entire search space of SafeQL that can be explored through admissible refinement steps for a given error. Definition 6 (Safe Refinement Tree,T safe,푞 ). Given an query푞, the safe refinement treeT safe,푞 is a directed tree that represents error- resolving refinement paths from 푞 and is defined as follows: • The root ofT safe,푞 is the initial query 푞. • An edge is added if the parent query produces an execution error. • Each edge corresponds to a refinement step 푞 푟 −→ 푞 ′ for some 푟 ∈ R(휖(푞)). Here,R(휖)denotes the set of admissible refinement operations allowed for each error 휖: R(휖)= ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ Relation Refinement if 휖= unknown relation Relation Refinement Join Refinement Attribute Refinement ⎞ ⎟ ⎠ if 휖= unknown attribute Function Refinement Attribute Refinement )︃ if 휖= unknown function Value Refinement if 휖= empty result □ For unknown attribute and unknown function errors, multiple refinement operations are admissible because their causes are often indirect. An unknown attribute may not only indicate a misspelled or mismatched column name, but also reveal that the necessary relation has not been joined at all. Likewise, an unknown function may arise either from an undefined function name or from invalid arguments within its call—for example, when a missing attribute renders the function signature invalid. Figure 6 illustrates an example Safe Refinement Tree. The root query first fails with an unknown attribute error because the at- tributetxtdoes not exist in theUsersrelation in theFROMclause. Accordingly, SafeQL explores only the admissible refinements for this error type—Attribute, Join, and Relation refinement—and Fig- ure 6 highlights three representative refinement steps: (1) Relation Refinement (Users→ Posts); (2) Join Refinement (Users JOIN Posts ON UserID = ID); (3) Attribute Refinement (txt→ name). After the attribute error is resolved—e.g.,txtis resolved through the middle branch, where a Join Refinement introduces the missing Postsrelation—the next level of the refinement tree produces a new failure in the function call. Although sum is a valid aggregate function, it is not defined over string-typed arguments such astxt, leading to an unknown function error. At this refinement level, both function-level and attribute-level corrections are admissible, and two representative refinements are shown: (1) Function Refinement (sum→ count); (2) Attribute Refinement (txt→ txt_len). SELECT sum(txt) FROM Users WHERE age = 24 SELECT sum(txt) FROMPosts WHEREage= 24 SELECTsum(txt) FROM Users JOIN Posts ON ... WHERE age = 24 SELECTsum(name) FROM Users WHERE age = 24 SELECTcount(txt) FROM Users JOIN Posts ON ... WHERE age = 24 SELECT sum(txt_len) FROM Users JOIN Posts ON ... WHERE age = 24 SELECTcount(name) FROM Users WHERE age = 24 SELECT sum(age) FROM Users WHERE age = 24 unknown a�ribute unknown a�ribute unknown func�on Users→ Posts JOIN Posts txt→ name sum→ countsum→ count unknown func�on txt→ txt_lenname→ age ... ... Safe Query Space ... ... Figure 6: Example of a Safe Refinement Tree. The example above provides the intuition that each execution error can be resolved by one of the admissible refinements defined for its error type. To complete this intuition, we now formalize that such error-resolving refinements are not merely illustrative but are guaranteed to exist within the Safe Refinement Tree. Lemma 7 (Reachability of Error-resolving Refinements). Given a database퐷= (푅푒푙푠,퐴푡푠,푉푎푙푠,푇푦푝푠,Λ,Γ)and the safe re- finement treeT safe,푞 , every execution error휖(푞)at a node푞 ∈ T safe,푞 has a corresponding refinement edge(푞,푞 ′ )such that the refinement operation 푟 ∈ R(휖(푞)) resolves the error under the database 퐷. To formalize this guarantee, we introduce a lightweight opera- tional semantics [10] of SQL execution, in which an error judgment 퐷 ⊢ 푞 ⇒ 휖(푞)is produced when the corresponding error condition holds. Formally: the error condition indicating a violation of execution premise 퐷 ⊢ 푞 ⇒ 휖(푞) Proof.We analyze each error type by making its operational error condition explicit and showing that at least one refinement in R(휖)always restores the violated execution premise. Letrelations(퐹) denote the set of base relations that appear in the FROM clause 퐹 . (1) Unknown Relation. SQL execution requires that every relation referenced in 퐹 exist in the database schema: ∃푅 ∈ relations(퐹), 푅∉ 푅푒푙푠 퐷 ⊢ SELECT S FROM F WHERE W⇒ unknown relation A Relation Refinement replaces the invalid푅with a valid relation푅 ′ ∈ 푅푒푙푠, thereby restoring the failed premise. Thus, at least one refinement edge(푞,푞 ′ ) always exists. (2) Unknown Attribute. An attribute must belong to the union of attribute signatures of all relations in F : 푎∉ ⋃︁ 푅∈relations(퐹) Λ(푅) 퐷 ⊢ SELECT a FROM F WHERE W⇒ unknown attribute This error can be repaired by one of the following refinements: (2.1) Relation Refinement.Replace a relation푅in theFROM clause with a relation푅 ′ such that푎 ∈Λ(푅 ′ ), ensuring that the referenced attribute exists in the revised schema context. (2.2) Join Refinement.Add a relation푅 ′ to the query such that 푎 ∈Λ(푅 ′ ), so thatrelations(퐹)is extended with푅 ′ and the at- tribute reference to be valid. (2.3) Attribute Refinement.Replace the unresolved attribute 푎with an attribute푎 ′ satisfying푎 ′ ∈ ⋃︁ 푅∈relations(퐹) Λ(푅), making the attribute reference to match one of the attributes provided by the current FROM clause. In either case, the violated condition is recoverable: if푎exists elsewhere in the schema, Relation or Join refinement brings the cor- responding relation into the query; otherwise, Attribute refinement replaces푎with a valid attribute. Hence at least one refinement edge (푞,푞 ′ ) always resolves the error. (3) Unknown Function. A function expression may fail for one of two reasons: either the function itself is unsupported by the DBMS, or the function exists, but its argument expressions do not satisfy the type requirements imposed by its signature. Here,퐹푢푛푐푠 denotes the set of all function symbols supported by the DBMS. 푓∉ 퐹푢푛푐푠 퐷 ⊢ 푓(푎 1 , . . .,푎 푘 ) ⇒ unknown function 푓 : 휏 1 , . . .,휏 푘 → 휏 out ,Γ(푎 푖 )=휏 ′ 푖 , (휏 ′ 1 , . . .,휏 ′ 푘 )≠ (휏 1 , . . .,휏 푘 ) 퐷 ⊢ 푓(푎 1 , . . .,푎 푘 ) ⇒ unknown function This error can be repaired by one of the following refinements: (3.1) Function Refinement.If푓∉ 퐹푢푛푐푠, replace푓with a sup- ported function 푓 ′ ∈ 퐹푢푛푐푠, restoring the violated condition. (3.2) Attribute Refinement.If the function symbol exists, but the argument types do not match its signature푓:휏 1 , . . .,휏 푘 → 휏 out , we assume the schema includes at least one attribute of each required type휏 푖 . Under this assumption, the violation is repaired by replacing one or more arguments푎 푖 with attributes푎 ′ 푖 such thatΓ(푎 ′ 푖 )=휏 푖 , thereby restoring the required types. In either case, the violation is recoverable: it is replaced, and a type mismatch is fixed via argument refinement. Thus, at least one refinement edge(푞,푞 ′ ) resolves the error. (4) Empty Result. This error occurs when the query is syntac- tically and semantically valid, but its predicate evaluates to false for all tuples. Typical causes include overly restrictive comparison operators (e.g.,=,>,<applied in a way that no tuple can satisfy), comparisons against values that do not exist in the database, or con- tradictory filter conditions. In such cases, the error can be repaired by adjusting the value conditions, such as relaxing a comparison, modifying a constant, or substituting a value that appears in the data—so that the predicate becomes satisfiable. Accordingly, a Value Refinement typically provides a refinement edge(푞,푞 ′ )that adjusts the predicate to produce a non-empty result. Taken together, these four cases show that every violated execu- tion premise can be repaired by an admissible refinement.□ To illustrate how the operational semantics used in the proof relate to the refinement process, we present two examples that instantiate the refinement steps shown in Figure 6. Example 1 (Join Refinement; Case 2.2). txt∉Λ(Users) 퐷 ⊢ SELECT txt FROM Users⇒ unknown attribute If txt∈Λ(Posts), then Join Refinement extends FROM clause: FROM Users → FROM Users JOIN Posts ON UserID = ID. The violated premise is restored becausetxt ∈Λ(Posts), This corresponds to the middle edge of the first level shown in Figure 6. Example 2 (Attribute Refinement; Case 3.2). sum : int→ int,Γ(name)= text, text≠ int 퐷 ⊢ SELECT sum(name) FROM Users⇒ unknown function If the schema provides an attribute of the required type, such as Γ(age)= int, then Attribute Refinement replaces the argument: sum(name) → sum(age). This corresponds to the rightmost edge of the second level shown in Figure 6. Together, the Lemma 7 ensures that every erroneous node in the safe refinement tree has at least one valid refinement leading to a safe (i.e., executable) query. Consequently, the refine- ment process is guaranteed to reach a region of error-free queries, which we define as the safe query space, formalized below. Definition 8 (Safe Query Space). Every leaf node푞 ′ in the refine- ment treeT safe,푞 satisfies⟦푞 ′ ⟧≠ 휖(푞). We refer to the set of all such leaf nodes as the Safe Query Space: Q safe =푞 ′ | 푞 ′ is a leaf node ofT safe,푞 and⟦푞 ′ ⟧≠ 휖(푞 ′ ). Proof. By construction ofT safe,푞 , a query푞is refined when ⟦푞⟧= 휖(푞). Thus, if a node푞 ′ has no children (i.e., it is a leaf), it must satisfy⟦푞 ′ ⟧≠ 휖(푞 ′ ). Therefore, all leaf nodes inT safe,푞 are error-free, and their set constitutes the safe query space.□ In Figure 6, the green-highlighted queries correspond exactly to such leaf nodes inT safe,푞 and thus serve as concrete instances of queries contained in the safe query space. 4 SEARCH ALGORITHM This section explains how SafeQL navigates the safe query space to identify the most semantically faithful executable query. We first define the search objective based on semantic distance, then describe a best-first search algorithm that prioritizes refinements closest to the original query. 4.1 Search Objective under Semantics Having defined the safe query spaceQ safe , we now formalize the objective that governs the search process within this space. Even among valid candidates inQ safe , refinements may differ in how well they preserve the original intent. To quantify such a deviation, we define a semantic distance that jointly measures structural change and embedding-level drift. Definition 9 (Semantic Distance,훿). For two queries푞 푖 and푞 푗 , the semantic distance 훿(푞 푖 ,푞 푗 ) is defined as 훿(푞 푖 ,푞 푗 )= 훼 ·푑 struct (푞 푖 ,푞 푗 )+(1− 훼)·푑 embed (푞 푖 ,푞 푗 ), 훼 ∈ [0, 1]. • 푑 struct (푞 푖 ,푞 푗 ) measures structural deviation (e.g., relation or join refinements) using the normalized tree edit distance [60] be- tween the abstract syntax trees of푞 푖 and푞 푗 , a standard metric for comparing hierarchical structures such as XML and JSON. • 푑 embed (푞 푖 ,푞 푗 )measures semantic deviation based on embeddings: 푑 embed (푞 푖 ,푞 푗 )= ∑︂ (푡→푡 ′ ) ∥푣 푡 − 푣 푡 ′ ∥, where(푡→푡 ′ )denotes each replaced token (relation, attribute, or value) and푣 푡 is the embedding vector of token푡computed by a neural embedding model [33].□ This hybrid formulation is conceptually analogous to the sparse- dense hybrid retrieval in information retrieval, which combines complementary lexical and semantic signals to improve ranking robustness [6,37]. Similarly, combining푑 struct and푑 embed provides a more robust and balanced measure of syntactic fidelity and seman- tic preservation, while remaining a heuristic rather than a strict notion of semantic equivalence. The former enforces grammatical consistency, while the latter maintains semantic proximity in em- bedding space. The weighting parameter훼controls this trade-off, and we empirically validate its effect in Section 7.4. With semantic distance formalized, we now explain how SafeQL leverages it during the refinement process. Unlike regeneration- based methods that repeatedly applyQ →Qby regenerating full SQLs after every failure, SafeQL redefines refinement as a guided search from the general query spaceQinto the executable subset Q safe under the database퐷. In this view, each erroneous query serves not as a terminal failure but as a starting point from which SafeQL identifies the nearest semantically consistent counterpart inQ safe under the distance metric 훿 . We formalize this as follows: Definition 10 (Search-based Refinement). The refinement process finds the safe query nearest to the input query푞within a distance훿. 푓 search refine :Q× 퐷 →Q safe , 푞 safe = arg min 푞 ′ ∈Q safe 훿(푞,푞 ′ ). 4.2 Best-first Search-based Refinement The overall refinement process follows a best-first exploration of the safe query space, as shown in Algorithm 1. Unlike exhaustive search, which explore all refinements uniformly, best-first search naturally limits exploration in a branch-and-bound style [25], accel- erating convergence. The search initializes a priority queue ordered by semantic distance from the LLM-generated initial query, expand- ing the closest candidates first. This design takes the initial query as the reference point under the assumption that it reflects the LLM’s interpretation of the user’s intent and thus serves as a reasonable available signal for recovering the original intent. When this ini- tial query is fundamentally misgenerated, search-based refinement alone is insufficient, and SafeQL therefore incorporates a hybrid refinement strategy to regenerate the query and establish a better starting point, as described later in Section 6.3. During each iteration of the loop (Lines 4-16), the algorithm dequeues the nearest candidate query푞(Line 5) and attempts to execute it within thetryExecuteblock (Lines 7). If execution suc- ceeds, the algorithm immediately returns푞as the nearest safe query (Line 8), thus guaranteeing that the first valid query discovered is the one most semantically similar to the original input. When execution fails, SafeQL captures the corresponding error type휖(푞)within thecatchErrorblock (Lines 9-16) and uses it to guide subsequent refinements. The GenerateRefinements proce- dure (Line 10) selects the appropriate refinement operations (e.g., relation, join, or attribute refinements) based on휖, while PruneRe- finements (Line 11) filters out redundant or semantically invalid refinements. Each surviving refinement푟is then applied to produce a new candidate query푞 ′ (Line 13), which is enqueued for future exploration only if it has not been visited previously (Lines 16). 5 PRUNING METHODS Although all queries in the safe space are executable, the space can grow rapidly as each refinement generates many candidates. For each error, all schema-level elements (relations, attributes, values) are considered, causing a single step to branch into dozens of can- didates. Value refinements can expand even further, since every possible value in the database becomes a potential substitution. For example, in Figure 7, an unknown attribute error foruser leads SafeQL to try all attributes (e.g.,name,ID,age). Even after correction,WHERE name = ‘leo’may produce an empty result, triggering another round of refinements where‘leo’is substituted with every possible value in the table—‘Leo’,‘Hana’,‘James’, and so on. This results in combinatorial growth of candidates across refinement layers. Proposition 11 (Exponential Size of Safe Query Space). For a safe refinement treeT safe,푞 with branching factor푟and depth푑, the total size of the search space isO(푟 푑 ), since each refinement can generate 푟 new candidates at every level. This exponential growth makes exhaustive enumeration infeasi- ble even for moderately complex databases. To mitigate this issue, SafeQL introduces pruning mechanisms that restrict the number of refinements explored at each step. These strategies—type-based pruning and top퐾refinement pruning—jointly suppress unpromis- ing branches in the refinement tree, effectively reducing fan-out while preserving meaningful candidates. Algorithm 1: Best-first Search-based Refinement Input: 푞/* original query */ Input: 훼/* structure–embedding weight factor */ Input: refineQueue /* priority queue ordered by 훿(푞,푞 ′ ) */ 1 push (푞, 0) into refineQueue; 2 mark 푞 as visited; 3 Function searchBasedRefine(푞): 4 while refineQueue is not empty do 5푞 current ← arg min 푞 ′ ∈refineQueue 훿(푞,푞 ′ ); 6mark 푞 current as visited; 7tryExecute (푞 current ) 8return 푞 current ; /* nearest safe query found */ 9catchError 10R ← generateRefinements(휖(푞 current )); 11R pruned ← pruneRefinements(R); 12forall 푟 ∈ R pruned do 13푞 ′ ← applyRefinements(푞 current ,푟); 14if 푞 ′ not visited then 15훿 ← calculateDistance(푞,푞 ′ ,훼); 16push(푞 ′ ,훿) into refineQueue; SELECT ID, age FROM Users WHEREID= ′leo′ SELECT ID, age FROM Users WHEREuser= ′leo′ SELECT ID, age FROM Users WHEREname=′leo′ SELECT ID, age FROM Users WHEREname=′Leo′ ... SELECT ID, age FROM Users WHERE name =′Hana′ SELECT ID, age FROM Users WHERE name =′James′ Type-based pruned empty result → user (text) name (text) → user (text) ID (int) → user (text) age (int) ʹleoʹ→ʹLeoʹ (훿 = 0.13) ʹleoʹ→ʹJamesʹ (훿 = 0.49) ... ʹleoʹ→ʹHanaʹ (훿 = 0.27) unknown a�ribute Top-K pruned (K = 1) SELECT ID, age FROM Users WHEREage= ′leo′ Figure 7: Example of pruning in the safe refinement tree. 5.1 Type-based Refinement Pruning Type information provides a strong pruning signal because any type mismatch immediately yields an invalid query. In SQL, each operator and function has a fixed input signature; thus, if a substi- tuted attribute or value violates this signature, execution inevitably results in an error. SafeQL leverages the typing environmentΓto discard such candidates before semantic evaluation. Algorithm 2 presents this mechanism. It validates refinements underΓand prunes type-inconsistent candidates. In Figure 7,WHERE user = ‘leo’yieldsname,ID, andage, withΓ(name)= TEXTand Γ(ID)=Γ(age)= INT. Applying TypeCheck, onlynameis retained since‘leo’isTEXT. This prevents non-executable branches from entering the search. 5.2 Top-퐾 Refinement Pruning Even after type-based pruning, many refinements remain type- consistent but semantically distant. To improve efficiency, SafeQL ranks candidates by embedding similarity and retains only the top- 퐾per category. This is particularly beneficial for value refinements, where the out-degree is large and most database values are se- mantically unrelated to the query; in such cases, embedding-based similarity provides an effective signal for filtering out implausible candidates. A smaller퐾yields higher efficiency but risks over- pruning, while a larger퐾improves accuracy at increased search cost, a tradeoff we examine empirically in Section 7.4. Algorithm 3 summarizes this mechanism. SafeQL retrieves candi- dates, ranks them by semantic distance, and retains only the top-퐾. Examples are shown in Figure 7. This pruning suppresses unpromis- ing branches and focuses refinement on relevant candidates. Algorithm 2: Type-Based Refinement Pruning 1 Function pruneRefinements(R): 2 foreach 푟 ∈ R do 3switch 푟 do 4case Attribute Refinement:푎 1 ⊕푎 2 푟 −→ 푎 ′ 1 ⊕푎 2 do 5휏 1 , 휏 2 ←Γ(푎 ′ 1 ),Γ(푎 2 ); 6if typeCheck(⊕, 휏 1 , 휏 2 ) pass then 7add 푟 toR pruned ; 8case Value Refinement: 푎⊕ 푣 푟 −→ 푎⊕ 푣 ′ do 9휏 푎 , 휏 푣 ←Γ(푎),Γ(푣 ′ ); 10if typeCheck(휏 푎 , 휏 푣 ) pass then 11add 푟 toR pruned ; 12case Function Refinement: 푓 푟 −→ 푓 ′ do 13if typeCheck(푓 ′ ,Γ(푎 1 ), . . .) pass then 14add 푟 toR pruned ; Algorithm 3: Top-퐾 Refinement Pruning Input: 퐾/* number of nearest neighbors to explore */ 1 Function pruneRefinements(R): 2 foreach 푟 ∈ R do 3switch 푟 do 4case Relation Refinement: 푅 푟 −→ 푅 ′ do 5if|푥 ∈ 푅푒푙푠:∥푣 푅 −푣 푥 ∥ ≤ ∥푣 푅 −푣 푅 ′ ∥| ≤ 퐾 6add 푟 toR pruned ; 7case Attribute Refinement: 푎 푟 −→ 푎 ′ do 8if|푥 ∈ 퐴푡푠:∥푣 푎 − 푣 푥 ∥ ≤ ∥푣 푎 − 푣 푎 ′ ∥| ≤ 퐾 9add 푟 toR pruned ; 10case Value Refinement: 푣 푟 −→ 푣 ′ do 11if|푥 ∈ 푉푎푙푠:∥푣 푣 − 푣 푥 ∥ ≤ ∥푣 푣 − 푣 푣 ′ ∥| ≤ 퐾 12add 푟 toR pruned ; 13case Function Refinement: 푓 푟 −→ 푓 ′ do 14if|푥 ∈ 퐹푢푛푐:∥푣 푓 −푣 푥 ∥ ≤ ∥푣 푓 −푣 푓 ′ ∥| ≤ 퐾 15add 푟 toR pruned ; 6 SYSTEM ARCHITECTURE DESIGN Although the core of SafeQL lies in its search-based refinement algorithm, realizing it as a refinement framework for building prac- tical text-to-SQL systems requires addressing several system-level challenges. Unlike a conceptual refinement algorithm, an actual implementation must integrate deeply with the DBMS, eliminate redundant computation, remain robust against fundamentally in- correct queries, support syntactically complex SQL structures, and stay compatible with diverse Text-to-SQL systems. Consequently, SafeQL’s architecture is built around five key design considerations: • In-DBMS integration for precise error detection. • Systematic optimization for scalable refinement. • Hybrid refinement that complements search with regeneration. • Grammar extensions enabling broad SQL coverage. • Compatibility with diverse Text-to-SQL systems as a post- generation refinement layer. We describe each aspect below. 6.1 In-DBMS Integration External Text-to-SQL systems rely on error strings, which may mention identifiers (e.g., unknown relationR) but do not reliably indicate error locations; some DBMSs provide no positions, and even when they do (e.g., position 15), they refer only to character offsets and cannot distinguish multiple occurrences such asSELECT R.a FROM R . Locating the correct occurrence thus requires addi- tional parsing, which is ambiguous under aliasing, nested queries, or repeated references. SafeQL instead performs refinement inside the DBMS over the Abstract Syntax Tree (AST). When a query푞 푖 fails, the analyzer directly identifies the corresponding AST node. Modifications are restricted to these nodes; since the AST encodes attribute–relation links (e.g.,R.arefers toR), updates can be ap- plied without affecting unrelated occurrences, and each refined query is re-analyzed and validated. This enables precise and safer structurally grounded refinement beyond text-based approaches, despite slightly higher implementation complexity. 6.2 Systematic Optimization Executing refinement inside the DBMS introduces a new bottle- neck: repeated embedding and similarity computations across over- lapping query fragments. To eliminate such redundancy, SafeQL incorporates two in-database optimization layers: •Embedding Cache. Embeddings of relations, attributes, and values are precomputed and stored on disk, while frequently accessed items are cached in memory. This prevents repeated embedding computation across refinements. •Vector Index. To efficiently perform nearest-neighbor search (particularly for Top-퐾pruning), SafeQL builds an HNSW-based vector similarity index over cached embeddings. Since these structures are maintained in the DBMS, tuple and embedding updates are consistently reflected in the vector index via a vector extended relational system. Optimizing update main- tenance via incremental techniques [54,56] is left for future work. 6.3 Hybrid Refinement Strategy Even with precise in-database refinement, some queries are funda- mentally misgenerated. To handle these, SafeQL adopts a hybrid strategy combining search-based and regeneration-based refine- ment. When the number of refinement steps reaches a threshold (default 100) without producing a valid query, SafeQL triggers a re- generation loop. Such cases typically indicate that the initial query is too far from a valid structure for localized refinement to succeed, in which case an LLM generates a new candidate conditioned on the original input, the current query, and the DBMS error feed- back. This also addresses LLM-side errors, such as invalid or even non-parsable generations by reinitializing the search from a new candidate. The relative contribution of search-based refinement and hybrid fallback is analyzed in Table 3 and Subsection 7.2.1. This hybrid design ensures robustness to deeply misgenerated queries without undermining the efficiency of search-based refinement. 6.4 Grammar Extensions Practical Text-to-SQL systems must handle more complex SQL than the simplified grammar presented in Section 3. SafeQL therefore extends its support to standard SQL constructs appearing in bench- marks such as Bird [18] and Spider [57], including: •Nested subqueries: supports recursively scoped queries by re- fining subqueries in an inside-out manner. A subquery푞 푠 is recursively refined into푞 ′ 푠 (Line 3) and replaced (Line 5), after which refinement continues on the outer query (Line 6). The outputΛ(푞 ′ 푠 )is exposed to the DBMS analyzer (Line 4), enabling proper scope resolution for outer references. • Alias refinement: extends relation and attribute refinement to handle alias normalization (e.g., R.a). • Clause-level extensions: support clauses such asGROUP BY, ORDER BY, andLIMIT. For example,ORDER BY Otakes the form (O ::= 푎 | 푓(푎 1 ,푎 2 , . . .)) and is refined as: ORDER BY 푎 푟 −→ 푎 ′ , 푓(푎 1 , ...) 푟 −→ 푓(푎 ′ 1 , ...), 푓(푎 1 , ...) 푟 −→ 푓 ′ (푎 1 , ...). GROUP BY and LIMIT are handled analogously. Algorithm 4: Nested Subqery Refinement 1 Function recursiveRefine(푞): 2 if 푞 contains a subquery 푞 푠 then 3푞 ′ 푠 ← recursiveRefine(푞 푠 ); 4Λ(푞 ′ 푠 ) ←푎 1 ,푎 2 , . . .;/* 푞 ′ 푠 = SELECT 푎 1 ,푎 2 ,... */ 5replace 푞 푠 in 푞 with 푞 ′ 푠 ; 6 return searchBasedRefine(푞); 6.5 Compatibility with Text-to-SQL Systems SafeQL integrates with diverse Text-to-SQL systems as a post- generation refinement layer. It attaches to prompt-based systems without modifying prompts and applies to agent-based systems by refining their final queries to ensure safe execution in the DBMS. 6.6 Architecture Figure 8 summarizes how these components interact within the SafeQL architecture. When a query푞 푖 fails over the database퐷, the query refiner operates inside the DBMS to locate the error source 휖(푞 푖 )and generate refined candidates푞 ′ 푖 that correct the erroneous AST nodes (Section 6.1), while leveraging cached embeddings and vector indices for efficient similarity computation (Section 6.2). DBMS 휖(푞 푖 ) 푞 푖, Hybrid Refinement Manager Large Language Model Database (퐷) Query Executor Query Refiner Cache Manager Vector Index Manager 푞 푖 푞 푖+1 푞 푖 ′ 푞 푖 푞 safe regenera�on loop search loop Query Parser Figure 8: Architecture of the SafeQL framework. Within the search loop, the query refiner iteratively explores can- didate refinements until a valid query푞 safe is found. If unsuccessful until a threshold is reached, control moves to the regeneration loop, where an LLM regenerates a new query푞 푖+1 guided by the hybrid refinement manager (Section 6.3). Throughout the process, all inter- mediate queries are parsed and validated under the extended SQL grammar (Section 6.4), ensuring that even complex transformations remain syntactically correct and executable. Together, these mech- anisms elevate SafeQL from an algorithmic concept to a practical, database-native Text-to-SQL refinement framework. Implementation: SafeQL is implemented on top of PostgreSQL’s RawStmt structure, the canonical AST representation of SQL queries. It is packaged as a PostgreSQL [46] extension, using fastembed [43] with the bge-base-en-v1.5 model [51] for embedding computation and pgvecto.rs [45] for efficient vector indexing inside the DBMS. 7 EXPERIMENTAL EVALUATION In this section, we evaluate SafeQL in terms of two key dimensions: (1) accuracy, measured by execution accuracy, and (2) efficiency, measured by elapsed time and the number of LLM tokens consumed. Our goal is to demonstrate that SafeQL improves accuracy while reducing expensive LLM invocations across various settings. 7.1 Experimental Setup 7.1.1 Benchmarks. Bird[18] is a widely used cross-domain Text-to-SQL benchmark and our primary evaluation dataset. To align with our PostgreSQL- based implementation, we migrated the Bird to PostgreSQL. We evaluate both the full dev set (1,534 queries) and the mini dev set (500 queries), for main and detailed evaluation, respectively. Both dev sets span 11 databases and cover diverse queries, including joins, grouping, and subqueries. Spider[57] is another major cross-domain benchmark, consisting of 10,181 text-to-SQL pairs across 200 relational databases spanning more than 130 domains. For our experiments, we migrated Spider to PostgreSQL to verify SafeQL’s generalizability across different benchmarks and database environments. 7.1.2 Testbed systems. We integrate SafeQL with two representa- tive Text-to-SQL approaches: prompt-based and agent-based meth- ods. Specifically, we adopt DAIL-SQL and OpenSearch-SQL as the state-of-the-art systems for these approaches, respectively. DAIL-SQL[7] is a prompt-based system that optimizes question representation, example selection, and example organization for in- context learning. As the state-of-the-art prompt-based method on the Spider benchmark, it serves as a strong baseline for evaluating SafeQL’s refinement in prompting-based settings. OpenSearch-SQL[52] is an agent-based system that decomposes the Text-to-SQL process into four modules—Preprocessing, Extrac- tion, Generation, and Refinement—linked through a consistency alignment mechanism. It leverages dynamic few-shot learning with self-taught Query-CoT-SQL pairs and structured few-shot tem- plates, achieving state-of-the-art performance on the Bird bench- mark. We integrate SafeQL into OpenSearch-SQL to evaluate its refinement behavior under multi-agent coordination. 7.1.3 Comparison methods. To isolate the effect of refinement, we compare SafeQL with the refinement methods of five promi- nent Text-to-SQL systems—DIN-SQL, MAC-SQL, RED-SQL, CHESS- SQL, and OpenSearch-SQL—all of which adopt regeneration-based paradigms. •DIN-SQL [31] refines SQL without execution, assuming potential errors. Among its two prompt variants, GENERIC and GENTLE, we use GENTLE, which performed better in our experiments. • MAC-SQL [48] regenerates SQL using DBMS error messages. • RED-SQL [34] refines SQL using constraint-based violation reports rather than raw DBMS errors, providing data-aware feed- back that helps the LLM correct semantic inconsistencies. •CHESS-SQL [42] uses few-shot refinement examples combined with an execution error message. •OpenSearch-SQL [52] also uses few-shot examples, but provides more structured templates tailored to specific error messages. Table 1: Comparison of refinement methods. Refinement Methods Refinement Paradigm Use of Error Msg Use of Examples DIN-SQL Regeneration NoZero-shot MAC-SQLYesZero-shot RED-SQLYes † Zero-shot CHESS-SQLYesFew-shot OpenSearch-SQLYesFew-shot SafeQL (ours)Search + RegenerationYesFew-shot † RED-SQL uses constraint violation reports as extended error feedback. 7.1.4 Models. We use GPT-OSS-120B [1] as the primary model for all methods, as it provides strong performance among open- source LLMs while ensuring transparent and reproducible evalu- ation across methods. We also evaluate SafeQL with two comple- mentary model families: • commercial LLMs—GPT-4o [28] and GPT-3.5-turbo [27] • open source LLMs—Llama-3 [2] and Qwen-2.5 [44] This diversity allows us to assess SafeQL’s robustness and portabil- ity across model architectures and training paradigms. 7.1.5 Hardware and Software Environment. All experiments were conducted on a single server equipped with two AMD EPYC 7302 CPUs (3.0 GHz, 16 cores each), 2096 GB RAM (3200 MHz), and two NVIDIA A100 GPUs. LLM inference was served using vLLM [14]. 7.2 Main results We evaluate SafeQL on both the Bird and Spider benchmarks, mea- suring its impact on execution accuracy, token efficiency, and re- finement time. Across all settings, SafeQL consistently achieves higher accuracy while consuming far fewer LLM tokens than prior regeneration-based refinement methods. 7.2.1Bird results. Table 3 summarizes the results on the Bird full dev benchmark. SafeQL achieves the best overall performance, im- proving execution accuracy to 63.3% with DAIL-SQL in the prompt- based approach and 69.4% with OpenSearch-SQL in the agent-based approach—gains of +5.8% and +5.2%, respectively—while reducing execution errors by up to 87.4%, showing its robustness across both approaches. A deeper look at the regeneration-based methods highlights their inherent trade-offs: •DIN-SQL regenerates queries without execution feedback, and this often makes the queries worse, leading to accuracy loss. •MAC-SQL incorporates raw error messages but provides limited semantic guidance, resulting in modest gains. • RED-SQL improves accuracy with constraint-violation feedback, but at the cost of extremely high token usage and latency. •CHESS-SQL and OpenSearch-SQL leverage few-shot examples for stronger accuracy; among them, OpenSearch-SQL is the most balanced, requiring fewer tokens through a short prompt design while maintaining high performance. However, both still rely on full regeneration after every failure. In contrast, SafeQL(hybrid) employs search-based refinement that explores a structured space of error-directed transformations and invokes regeneration only when necessary. This approach enables SafeQL to reach valid queries with 1.8–15.1×fewer tokens and 1.9–29.6×far shorter refinement latency, achieving both higher accuracy and greater efficiency than existing regeneration-based methods. In addition, SafeQL(search), which relies solely on search- based refinement without issuing new SQL generations, resolves a large fraction of errors with no token overhead. As reflected inΔErr (%), search-based refinement already reduces a substantial portion of errors, leaving only 13%–17% of erroneous queries to proceed to hybrid fallback, while incurring a moderate runtime overhead of approximately 1.3–1.8× on average. 7.2.2Spider results. Table 4 shows the results on the Spider bench- mark in the prompt-based setting (DAIL-SQL), where prompting systems already achieve strong performance and agent-based sys- tems offer limited additional benefit. SafeQL achieves 91.7% execu- tion accuracy, improving the baseline by +4.6% and outperforming regeneration-based methods by 2.6–6.4%. SafeQL also demonstrates strong efficiency, requiring 5.1–96.5×fewer tokens and achieving 4.2–34.5×shorter refinement latency, while reducing execution errors by 77.4%. SafeQL(search) delivers most gains without ad- ditional tokens, indicating that search-based refinement remains effective and stable even in larger, diverse settings. 7.3 Accuracy analysis 7.3.1Model-wise results. Table 2 compares SafeQL across different model families on the Bird mini dev benchmark. The results show that SafeQL consistently improves execution accuracy regardless of model size or architecture. Open-source models such as Qwen and Llama exhibit substantial gains of +5–12%, narrowing the perfor- mance gap with larger proprietary models, while commercial LLMs (GPT-3.5-Turbo, GPT-4o) also improve by +4–9% in both prompt and agent-based settings. These results indicate that SafeQL’s search-based refinement is model-agnostic: it strengthens weaker models through explicit structural guidance while still providing measurable gains for strong systems. This consistency across model families highlights its gen- eral applicability and robustness across diverse LLM architectures. Table 2: Model-wise comparison of SafeQL performance on the Bird mini dev benchmark, whereΔEX denotes the accu- racy gain over the no-refinement baseline. Model Prompt-basedAgent-based EX (%)ΔEX (%)EX (%)ΔEX (%) Qwen-2.5-7B44.0 +9.647.2 +9.0 Qwen-2.5-72B60.2 +5.863.4 +5.2 Llama-3-8B37.4 +11.639.8 +10.2 Llama-3.3-70B 60.6 +5.663.6 +5.4 GPT-3.5-Turbo 52.4 +9.453.0 +8.8 GPT-4o61.8 +4.866.0 +4.6 Table 3: Comparison on the Bird full dev benchmark using GPT-OSS-120B, showing the absolute execution accuracy (EX), error reduction (ΔErr), additional LLM costs (ΔTokens), and elapsed time (ΔTime) for accuracy gains (ΔEX) via refinements. Refinements Prompt-based (DAIL-SQL)Agent-based (OpenSearch-SQL) EX (%)ΔEX (%)ΔErr (%)ΔTokens (퐾 )ΔTime (s) EX (%)ΔEX (%)ΔErr (%)ΔTokens (퐾 )ΔTime (s) No refinement57.5–64.2– DIN-SQL [31]57.2 −0.34.2 +7,748 +5,079 62.7 −1.611.0 +118,615 +62,104 MAC-SQL [48]59.3 +1.823.0 +3,865 +3,505 65.6 +1.419.7 +66,358 +42,004 RED-SQL [34]62.9 +5.444.2 +19,594 +20,110 66.7 +2.546.5 +411,469 +474,390 CHESS-SQL [42]62.6 +5.169.1 +6,653 +3,295 68.5 +4.380.3 +79,934 +35,573 OpenSearch-SQL [52] 62.0 +4.563.0 +3,799 +3,141 68.3 +4.169.3 +53,143 +30,880 SafeQL (search)62.5 +5.0 +70.9+0+1,24968.1 +3.9 +70.1+0+8,656 SafeQL (hybrid)63.3+5.8+83.0+1,305+1,653 69.4+5.2+87.4+28,779 +16,029 Table 4: Comparison on the Spider benchmark using GPT- OSS-120B under the DAIL-SQL prompt. Refinements Prompt-based EX (%)ΔEX (%)ΔErr (%)ΔTokens (퐾 )ΔTime (s) No refinement87.1– DIN-SQL85.3 −1.8 −3.2 +1, 055 +1194 MAC-SQL88.4 +1.3 −22.6 +396 +753 RED-SQL88.6 +1.5 −29.0 +7,528 +6,120 CHESS-SQL89.1 +2.0 −45.1 +1,279 +1,305 OpenSearch-SQL 89.1 +2.0 −48.7 +465 +1,050 SafeQL (search) 91.3 +4.2 -68.5+0+37 SafeQL (hybrid) 91.7+4.6-77.4+78+177 (a)Prompt-based ∎Occurrederrors∎Resolvederrors unknown rela�on unknown a�ribute unknown func�on empty result others Before Refinements A�er Refinements 훥퐸푟 81% 훥퐸푟 100% 훥퐸푟 80% 훥퐸푟 71% 훥퐸푟 40% (b)Agent-based unknown rela�on unknown a�ribute unknown func�on empty result others Before Refinements A�er Refinements 훥퐸푟 100% 훥퐸푟 100% 훥퐸푟 100% 훥퐸푟 84% 훥퐸푟 100% Figure 9: Error statistics of SafeQL. 7.3.2 Error analysis. Figure 9 compares the SafeQL’s error reduc- tion across the major error categories. These four representative forms of errors—unknown relation, unknown attribute, unknown function, and empty result—cover most of the execution failures observed in practice, and SafeQL consistently reduces all of them in both prompting- and agent-based settings. In the prompt-based setting, as shown in Figure 9(a), the majority of errors arise from unknown attribute or empty result. Because its extraction stage is less sophisticated than that of an agent-based set- ting, attribute mismatches and overly restrictive predicates appear more frequently. SafeQL resolves 81% of unknown attribute errors and 71% of empty result cases, demonstrating that search-based refinement substantially improves the reliability of prompt-driven generation. The remaining failures typically stem from severely misgenerated queries that cannot enter the safe query space, so no sequence of valid refinement steps can ever transform them into an executable query. In contrast, the agent-based setting, as shown in Figure 9(b) starts from a stronger baseline, as the LLM handles both extraction and generation, leading to a more balanced error distribution. Even so, SafeQL further eliminates nearly all remaining structural and functional errors—resolving 100% of unknown relation, attribute, and function errors—and reduces empty result errors by 84%. These results demonstrate that SafeQL operates orthogonally to the un- derlying Text-to-SQL systems and that the defined safe query space is not only theoretically sound but also practically effective in cap- turing nearly all executable refinements in real workloads. 7.4 Efficiency analysis Figure 10 presents efficiency ablation experiments on the Bird mini dev under the prompt-based setting, isolating the effects of SafeQL’s components: (1) pruning methods and (2) systematic optimizations. •Effect of pruning. Figure 10(a) shows that pruning substantially improves search efficiency without any loss of accuracy. Without pruning, refinement requires 1403 sec on Bird and 581 sec on Spider. Applying either type-based or top-퐾pruning individu- ally shortens this time, and combining both achieves the best performance—reducing the elapsed time to 647 sec and 177 sec, re- spectively. These results show that pruning strategies effectively bound the search space while preserving refinement quality. •Effect of systematic optimizations. Figure 10(b) evaluates the impact of systematic optimization on refinement latency. The baseline–lacking caching and indexing–requires over 20,000 sec on Bird and 9,000 sec on Spider. Introducing caching reduces this time to 1357 sec and 433 sec, while indexing alone yields only a modest improvement. When combined, the total time further decreases to 647 sec and 177 sec—amounting to over 30×over- all reduction—again with no degradation in execution accuracy. Among these optimizations, caching contributes the most by eliminating redundant embedding computations, whereas index- ing provides complementary acceleration for similarity lookups. Figure 11 shows how performance changes on the Bird mini dev benchmark, as we vary the two key parameters훼and퐾. We report results in a search-only setting to isolate the effect of these parameters. •Effect of휶. The higher훼increases the penalty for structural transformation during refinement, making the search space more compact and substantially reducing latency. However, this also introduces a trade-off: mid-range values (훼 ≈0.3–0.4) consis- tently provide the best execution accuracy, while larger훼values prioritize speed over precision. •Effect of푲. Raising퐾allows SafeQL to keep more candidates at each refinement step, effectively widening the search tree. This improves accuracy through broader exploration but also increases search time. In practice, accuracy saturates once퐾 ≥3, so setting퐾to at least 3 provides near-optimal performance while keeping runtime manageable. 1403 1216 964 647 581 337 254 177 100 1000 ∎Bird∎Spider Not pruned Elapsed Time (sec) Type pruned Top-K pruned Both pruned 20373 18725 1357 647 9129 9035 433 177 100 1000 10000 Baseline Elapsed Time (sec) +index +cache +both (a) Effect of Pruning methods(b) Effect of Systema�c op�miza�ons Figure 10: Efficiency ablation studies. Execu � on Accuracy (%) α= 58.4 59.2 60 60.2 60.2 60 59.7 58.6 57.2 57 59 61 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 2 780 621 517 379 243 189 176 157155 0 400 800 0.2 0.25 0.3 0.35 0.4 0.45 0.5 1 2 Elapsed Time (sec) α= (a) Execu�on accuracy while varying휶 Execu � on Accuracy (%) 퐾= 58.2 59.8 60.2 60.2 60.2 60.2 60.2 58 59 60 61 12345710 179 202 243 325 498 566 601 150 350 550 750 12345710 Elapsed Time (sec) 퐾= (c) Execu�on accuracy while varying푲 (b) Elapsed �me while varying휶 (c) Elapsed �me while varying푲 Execu�on accuracy Elapsed�me Figure 11: Effects of the parameters 훼 and 퐾. 7.5 System overhead Table 5 reports the system overhead of SafeQL on the Bird dev data- base; Spider is omitted due to its small size (<1 GB), where both the embedding cache and index are negligible. The reported primary storage includes all tuples and value embeddings, which are com- mon in Text-to-SQL systems [21] and are not introduced by SafeQL. SafeQL adds two auxiliary structures—an embedding cache and a vector index—both substantially smaller than the primary data, resulting in minimal storage overhead. Although index construc- tion is relatively expensive with a single thread (2472 sec), it can be significantly reduced via parallelism (295 sec). Since construction is performed once upfront and subsequent updates can be handled via incremental strategies [54,56], the overhead remains moderate in practice and can be further reduced with future optimizations. Table 5: SafeQL overhead on the Bird database (T = # threads). Primary storage SafeQL overhead Cache size Index size Index build time (1T / 16T) 8.04 GB686.6 MB375.2 MB2472 sec / 295 sec 8 RELATED WORK •LLM-based Text-to-SQL can be broadly categorized into prompt- based and agent-based approaches. Prompt-based approaches im- prove generation quality via prompt design. DAIL-SQL [7], MCS- SQL [15], PET-SQL [19], and CodexDB [47] enable in-context learning through few-shot prompting. Act-SQL [59] and CoT- SQL [41] enhance reasoning via Chain-of-Thought, while DTS- SQL [32] and DIN-SQL [31] uses a decomposition to breakdown complex questions. Agent-based approaches coordinate agents to generate queries and maintain consistency. C3 [5], CHESS [42], and CSC-SQL [40] select consistent outputs across candidates, OpenSearch-SQL [52] aligns intermediate states, and Alpha- SQL [16] applies Monte Carlo Tree Search over agent actions. •Search techniques in Text-to-SQL are used in decoding dur- ing generation, where greedy search selects the most probable token, as in SQLNet [53] and Seq2SQL [63], and beam search ex- plores multiple candidates, as in RAT-SQL [49], EditSQL [62], Sm- BoP [35], and ZeroNL2SQL [8]. At a higher level, Alpha-SQL [16] applies search via MCTS over agent actions to guide generation. These approaches perform search during generation, exploring multiple candidates while constructing SQL. However, while substantially enhancing generation quality and robustness, this also involves maintaining multiple candidates during decoding and generation, introducing additional computational overhead, which is further exacerbated in large models (e.g., LLMs), along with hallucination issues inherent to such models that may com- promise query safety. In contrast, SafeQL introduces search in the refinement stage within the DBMS, leveraging execution feed- back to ensure safety while avoiding unnecessary regeneration, enabling faster and more token-efficient refinement. 9 CONCLUSION AND DISCUSSION SafeQL introduces a novel search-based perspective on Text-to- SQL refinement, transforming database feedback into structured guidance for executable, intent-preserving queries. By coupling semantic search with in-database optimization, it achieves both reliability and efficiency, improving execution accuracy by up to 5.8% while reducing token usage by 15×over regeneration-based baselines. Rather than replacing existing generation paradigms, SafeQL complements them through structured, error-guided search process tightly coupled with database semantics. More broadly, this work relates to efforts to improve the safety of LLMs under formal correctness constraints, where hallucinations often produce invalid or non-executable results. Recent work in Automatic Program Repair (APR) similarly treats generation as a constraint-guided process integrated with type systems [22,23,61]. Likewise, SafeQL uses the DBMS analyzer as structured constraints to ground refinement in database semantics and ensure execution- safe query construction, suggesting broader integration of symbolic database reasoning with LLMs for more reliable database interfaces. ACKNOWLEDGMENTS This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)(No. RS-2024-00347471) and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. 2019-0-01267, GPU-based Ultrafast Multi- type Graph Database Engine SW). REFERENCES [1]Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al.2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925 (2025). [2] Meta AI. 2024. The Llama 3 Herd of Models. https://ai.meta.com/llama/ [3]Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Ka- plan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Sys- tems. 1877–1901. https://proceedings.neurips.c/paper_files/paper/2020/hash/ 1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html [4]Ziru Chen, Shijie Chen, Michael White, Raymond Mooney, Ali Payani, Jayanth Srinivasa, Yu Su, and Huan Sun. 2023. Text-to-SQL Error Correction with Lan- guage Models of Code. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 1359–1372. [5]Xuemei Dong, Chao Zhang, Yuhang Ge, Yuren Mao, Yunjun Gao, Jinshu Lin, Dongfang Lou, et al.2023. C3: Zero-shot text-to-sql with chatgpt. arXiv preprint arXiv:2307.07306 (2023). [6]Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2288–2292. https://doi.org/10.1145/3404835.3463098 [7] Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. Proc. VLDB Endow. 17, 5 (2024), 1132–1145.https: //doi.org/10.14778/3641204.3641221 [8]Zihui Gu, Ju Fan, Nan Tang, Songyue Zhang, Yuxin Zhang, Zui Chen, Lei Cao, Guoliang Li, Sam Madden, and Xiaoyong Du. 2023. Interleaving pre-trained language models and large language models for zero-shot nl2sql generation. arXiv preprint arXiv:2306.08891 (2023). [9]Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2019. Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation. https://doi.org/10.48550/arXiv.1905.08205 [10]Jaemin Hong and Sukyoung Ryu. 2003. Introduction to Programming Languages. (2003). [11]Radu Cristian Alexandru Iacob, Florin Brad, Elena-Simona Apostol, Ciprian- Octavian Truică, Ionel Alexandru Hosu, and Traian Rebedea. 2020. Neural approaches for natural language interfaces to databases: A survey. In proceedings of the 28th International Conference on Computational Linguistics. 381–395. [12]George Katsogiannis-Meimarakis and Georgia Koutrika. 2023. A survey on deep learning approaches for text-to-SQL. The VLDB Journal 32, 4 (2023), 905–936. [13]Hyeonji Kim, Byeong-Hoon So, Wook-Shin Han, and Hongrae Lee. 2020. Natural language to SQL: Where are we today? Proceedings of the VLDB Endowment 13, 10 (2020), 1737–1750. [14]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles. 611–626. [15]Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2025. MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation. In Proceedings of the 31st International Conference on Computational Linguistics. 337–353. https://aclanthology.org/2025.coling-main.24/ [16]Boyan Li, Jiayi Zhang, Ju Fan, Yanwei Xu, Chong Chen, Nan Tang, and Yuyu Luo. 2025. Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search. https://openreview.net/forum?id=kGg1ndttmI [17]Fei Li and Hosagrahar V. Jagadish. 2014. NaLIR: An Interactive Natural Language Interface for Querying Relational Databases. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. 709–712.https: //doi.org/10.1145/2588555.2594519 [18]Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al.2024. Can LLM Already Serve as a Database Interface? A Big Bench for Large-Scale Database Grounded Text-to- SQLs. Advances in Neural Information Processing Systems 36 (2024). [19]Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye, Ziyue Li, Rui Zhao, et al.2024. Pet-sql: A prompt-enhanced two-stage Text-to-SQL framework with cross-consistency. CoRR (2024). [20]Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2020. Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing.https: //doi.org/10.48550/arXiv.2012.12627 [21]Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, Ju Fan, Guoliang Li, Nan Tang, and Yuyu Luo. 2025. A survey of text-to-sql in the era of llms: Where are we, and where are we going? IEEE Transactions on Knowledge and Data Engineering (2025). [22]Niels Mündler, Jingxuan He, Hao Wang, Koushik Sen, Dawn Song, and Martin Vechev. 2025. Type-constrained code generation with language models. Proceed- ings of the ACM on Programming Languages 9, PLDI (2025), 601–626. [23] Shaan Nagy, Timothy Zhou, Nadia Polikarpova, and Loris D’Antoni. 2026. Chop- Chop: A Programmable Framework for Semantically Constraining the Output of Language Models. Proceedings of the ACM on Programming Languages 10, POPL (2026), 1905–1932. [24]Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, and Dragomir Radev. 2023. Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies. In Findings of the Association for Computational Linguistics: EMNLP 2023. 14935–14956. https: //doi.org/10.18653/v1/2023.findings-emnlp.996 [25] Narendra and Fukunaga. 1977. A branch and bound algorithm for feature subset selection. IEEE Transactions on computers 100, 9 (1977), 917–922. [26]Zheng Ning, Yuan Tian, Zheng Zhang, Tianyi Zhang, and Toby Jia-Jun Li. 2024. Insights into natural language database query errors: From attention misalign- ment to user handling strategies. ACM Transactions on Interactive Intelligent Systems 14, 4 (2024), 1–32. [27] OpenAI. 2023. GPT-3.5 Technical Report. https://openai.com [28] OpenAI. 2024. GPT-4o System Card. https://openai.com [29]Ana-Maria Popescu, Oren Etzioni, and Henry Kautz. [n.d.]. Towards a Theory of Natural Language Interfaces to Databases. ([n. d.]). [30]Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O. Arik. 2024. CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL. https://openreview.net/forum?id=CvGqMD5OtX [31]Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In- Context Learning of Text-to-SQL with Self-Correction. In Proceedings of the 37th International Conference on Neural Information Processing Systems. 36339–36348. [32]Mohammadreza Pourreza and Davood Rafiei. 2024. DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024. 8212–8220. [33]Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. https://doi.org/10.48550/arXiv.1908.10084 [34] Tonghui Ren, Chen Ke, Yuankai Fan, Yinan Jing, Zhenying He, Kai Zhang, and X. Sean Wang. 2025. The Power of Constraints in Natural Language to SQL Translation. Proc. VLDB Endow. 18, 7 (March 2025), 2097–2111. https://doi.org/ 10.14778/3734839.3734847 [35] Ohad Rubin and Jonathan Berant. 2021. SmBoP: Semi-autoregressive Bottom-up Semantic Parsing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies. 311–324. https://doi.org/10.18653/v1/2021.naacl-main.29 [36]Diptikalyan Saha, Avrilia Floratou, Karthik Sankaranarayanan, Umar Farooq Minhas, Ashish R. Mittal, and Fatma Özcan. 2016. ATHENA: An Ontology-Driven System for Natural Language Querying over Relational Data Stores. Proc. VLDB Endow. 9, 12 (2016), 1209–1220. https://doi.org/10.14778/2994509.2994536 [37]Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3715–3734. https://doi.org/10.18653/v1/2022.naacl-main.272 [38]Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. 2021. PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. 9895–9901. https://doi.org/10.18653/v1/2021.emnlp-main.779 [39]Jiawei Shen, Chengcheng Wan, Ruoyi Qiao, Jiazhen Zou, Hang Xu, Yuchen Shao, Yueling Zhang, Weikai Miao, and Geguang Pu. 2025. A Study of In-Context- Learning-Based Text-to-SQL Errors. arXiv preprint arXiv:2501.09310 (2025). [40]Lei Sheng and Shuai-Shuai Xu. 2025. CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning. https://doi.org/10.48550/arXiv.2505. 13271 [41]Chang-Yu Tai, Ziru Chen, Tianshu Zhang, Xiang Deng, and Huan Sun. 2023. Exploring Chain of Thought Style Prompting for Text-to-SQL. 5376–5393. https: //doi.org/10.18653/v1/2023.emnlp-main.327 [42]Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis. https://arxiv.org/abs/2405.16755v3 [43]Qdrant Team. 2024. FastEmbed: Lightweight and Efficient Text Embedding Library. https://github.com/qdrant/fastembed Python library for fast text embedding inference. [44] Qwen Team. 2024. Qwen2 Technical Report. https://qwen.ai [45] TensorChord Team. 2024. pgvecto.rs: High-performance Vector Search Extension for PostgreSQL. https://github.com/tensorchord/pgvecto.rs Rust-based vector indexing extension for PostgreSQL. [46]The PostgreSQL Global Development Group. 2024. PostgreSQL: The World’s Most Advanced Open Source Relational Database. https://w.postgresql.org/ Version 17. [47]Immanuel Trummer. 2022. CodexDB: Synthesizing code for query processing from natural language instructions using GPT-3 Codex. Proceedings of the VLDB Endowment 15, 11 (2022), 2921–2928. [48]Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Linzheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, et al.2025. Mac-sql: A multi-agent collaborative framework for text-to-sql. In Proceedings of the 31st International Conference on Computational Linguistics. 540–557. [49]Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 7567–7578. https://doi.org/10.18653/v1/2020.acl- main.677 [50]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems 35 (Dec. 2022), 24824–24837. [51]Jun Xiao, Chenglin Wang, Yuxin Huang, et al.2024. BGE-base-en-v1.5: English Text Embedding Model. https://huggingface.co/BAAI/bge-base-en-v1.5. Beijing Academy of Artificial Intelligence (BAAI). [52] Xiangjin Xie, Guangwei Xu, Lingyan Zhao, and Ruijie Guo. 2025. OpenSearch- SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Align- ment. Proc. ACM Manag. Data 3, 3 (2025), 194:1–194:24. https://doi.org/10.1145/ 3725331 [53] Xiaojun Xu, Chang Liu, and Dawn Song. 2017. SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning.https: //doi.org/10.48550/arXiv.1711.04436 [54]Yuming Xu, Hengyu Liang, Jin Li, Shuotao Xu, Qi Chen, Qianxi Zhang, Cheng Li, Ziyue Yang, Fan Yang, Yuqing Yang, et al.2023. Spfresh: Incremental in-place update for billion-scale vector search. In Proceedings of the 29th Symposium on Operating Systems Principles. 545–561. [55]Navid Yaghmazadeh, Yuepeng Wang, Isil Dillig, and Thomas Dillig. 2017. SQLizer: Query Synthesis from Natural Language. Proc. ACM Program. Lang. 1, OOPSLA (2017), 1–26. https://doi.org/10.1145/3133887 [56] Song Yu, Shengyuan Lin, Shufeng Gong, Yongqing Xie, Ruicheng Liu, Yijie Zhou, Ji Sun, Yanfeng Zhang, Guoliang Li, and Ge Yu. 2026. A Topology-Aware Localized Update Strategy for Graph-Based ANN Index. Proc. VLDB Endow. 19, 3 (2026), 495–508. [57]Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (Eds.). Association for Computational Linguistics, Brussels, Belgium, 3911–3921. https://doi.org/10. 18653/v1/D18-1425 [58] John M Zelle and Raymond J Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the national conference on artificial intelligence. 1050–1055. [59]Hanchong Zhang, Ruisheng Cao, Lu Chen, Hongshen Xu, and Kai Yu. [n.d.]. ACT- SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain- of-Thought. In The 2023 Conference on Empirical Methods in Natural Language Processing. [60]Kaizhong Zhang and Dennis Shasha. 1989. Simple Fast Algorithms for the Editing Distance between Trees and Related Problems. SIAM J. Comput. 18, 6 (Dec. 1989), 1245–1262. [61]Quanjun Zhang, Chunrong Fang, Yang Xie, YuXiang Ma, Weisong Sun, Yun Yang, and Zhenyu Chen. 2024. A systematic literature review on large language models for automated program repair. ACM Transactions on Software Engineering and Methodology (2024). [62]Rui Zhang, Tao Yu, Heyang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019. Editing-based SQL query generation for cross-domain context-dependent questions. In Pro- ceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 5338–5349. [63] Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. https: //doi.org/10.48550/arXiv.1709.00103 [64] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. 2022. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. https://openreview.net/forum?id=WZH7099tgfM

Instant research discovery