Paper deep dive
Conversational Task Disambiguation over Tabular Data: Leakage-Aware Formulation, Benchmark Suite, and Training
Nafiseh Ghoroghchian, Luis Scoccola, Tina Sedaghat, Omid Vaheb, Hannah Chen, Dino D'Agostino, Keyvan Golestan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/9/2026, 7:51:00 AM
Summary
This paper introduces a leakage-aware framework for conversational task disambiguation over tabular data, formalized as Ambiguous Verifiable Tasks (AVTs). It decomposes the agent into asking and solution policies and the environment into an oracle and verifier. The authors propose the AmbiTab benchmark suite, unifying six ambiguous text-to-SQL datasets, and demonstrate that training an asking policy with reinforcement learning improves disambiguation metrics and task success while providing diagnostics to measure oracle leakage.
Entities (13)
Relation Signals (11)
Ambiguous Verifiable Task → decomposes → agent into asking policy and solution policy
confidence 95% · decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier.
Ambiguous Verifiable Task → decomposes → environment into oracle and verifier
confidence 95% · decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier.
Oracle Leakage → isdefinedby → information revealed by user simulator beyond real user
confidence 95% · oracle leakage, that is, information that a user simulator reveals beyond what a real user would.
AmiTab → unifies → six ambiguous datasets
confidence 95% · AmiTab, a benchmark suite that unifies six ambiguous datasets under a common representation...
Reinforcement Learning → usedtotrain → asking policy
confidence 95% · train an asking policy with reinforcement learning.
AmiTab → derivedfrom → TACO
confidence 90% · AmiTab-AmbiQT and AmiTab-TACO adapt... TACO...
AmiTab → derivedfrom → AmiQT
confidence 90% · AmiTab-AmbiQT and AmiTab-TACO adapt AmbiQT...
AmiTab → derivedfrom → Ambrosia
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Conversational task disambiguation over tabular data uses dialogue to resolve missing information about a user's intended task before producing a solution over tables or databases. Existing evaluation and training lack a leakage-aware foundation. Task success mixes the agent's disambiguation and solution-generation capabilities and can also reflect oracle leakage, that is, information that a user simulator reveals beyond what a real user would. Existing datasets also lack a shared representation of ambiguities and access boundaries. We introduce the notion of an ambiguous verifiable task, which formalizes ambiguities and resolutions, decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier. This framework provides baselines and metrics for evaluating task disambiguation separately from solution generation, formal definitions of oracle leakage, judge-free leakage diagnostics, and a training objective for the asking policy. We instantiate the framework in text-to-SQL with AmbiTab, a benchmark suite that unifies six ambiguous datasets under a common representation specifying what the agent, oracle, and verifier may access. We evaluate clarification strategies and oracle leakage, and train an asking policy with reinforcement learning. The trained asker improves our disambiguation metrics on all six datasets and task success on five, and our leakage diagnostics measure how training affects oracle leakage.
Tags
Links
- Source: https://arxiv.org/abs/2610.10740v1
- Canonical: https://arxiv.org/abs/2610.10740v1
Trouble viewing inline? Open PDF directly →
Full Text
153,941 characters extracted from source content.
Expand or collapse full text
Conversational Task Disambiguation over Tabular Data: Leakage-Aware Formulation, Benchmark Suite, and Training Nafiseh Ghoroghchian Luis Scoccola11footnotemark: 1 Tina Sedaghat Omid Vaheb Hannah Chen Dino D’Agostino Keyvan Golestan Layer 6 AI †thanks: Contributed equally and are joint first authors.†thanks: This work was done during her internship at Layer 6 AI. Abstract Conversational task disambiguation over tabular data uses dialogue to resolve missing information about a user’s intended task before producing a solution over tables or databases. Existing evaluation and training lack a leakage-aware foundation. Task success mixes the agent’s disambiguation and solution-generation capabilities and can also reflect oracle leakage, that is, information that a user simulator reveals beyond what a real user would. Existing datasets also lack a shared representation of ambiguities and access boundaries. We introduce the notion of an ambiguous verifiable task, which formalizes ambiguities and resolutions, decomposing the agent into an asking policy and a solution policy, and the environment into an oracle and verifier. This framework provides baselines and metrics for evaluating task disambiguation separately from solution generation, formal definitions of oracle leakage, judge-free leakage diagnostics, and a training objective for the asking policy. We instantiate the framework in text-to-SQL with AmbiTab, a benchmark suite that unifies six ambiguous datasets under a common representation specifying what the agent, oracle, and verifier may access. We evaluate clarification strategies and oracle leakage, and train an asking policy with reinforcement learning. The trained asker improves our disambiguation metrics on all six datasets and task success on five, and our leakage diagnostics measure how training affects oracle leakage. 1 Introduction 1.1 Context and problem statement Language models are increasingly used to solve tasks over tables and relational databases (Tian et al., 2026; Wu et al., 2025; Li et al., 2026). When the solution is an executable query, execution provides a trusted verifier. Reinforcement Learning (RL) using such rewards (Pourreza et al., 2025; Ma et al., 2025; Lei et al., 2025b) has improved single-turn systems on text-to-SQL benchmarks such as BIRD (Li et al., 2023) and Spider 2.0 (Lei et al., 2025a). These benchmarks assume that the task specification determines the intended solution. Real task specifications often do not: for example “list the top earners” leaves open both how many earners to return and what counts as earnings (Figure 1(c)). We study conversational task disambiguation over tabular data: an agent asks clarifying questions to resolve missing information about the intended task before producing a solution over tables or databases. We focus empirically on text-to-SQL, where clarification identifies the user’s intended interpretation and the solution policy generates a corresponding query. Conversational benchmarks such as SParC and CoSQL (Yu et al., 2019b; Yu et al., 2019a) established the value of evaluating complete interactions, and interactive benchmarks such as BIRD-Interact (Huo et al., 2026) show that this type of ambiguous task remains difficult even for capable models. Evaluating and training conversational task disambiguation requires a leakage-aware foundation, which current practice lacks, for three reasons. First, success rate, the natural metric, depends on both asking and solving: a failure may reflect a missing question or a wrong solution, and a success may reflect a task that needs no disambiguation. Second, at scale the user is replaced by an oracle, often a language model, and an agent selected or trained against it may exploit its errors, as it might exploit a misspecified reward (Skalse et al., 2022; Pan et al., 2022), for example by eliciting details of the reference solution that no real user would supply. We call this oracle leakage; prior work separates, constrains, or audits user simulators against it (Jain et al., 2025; Huo et al., 2026; Sullutrone et al., 2026), and diagnosing it with a language model judge adds another untrusted component. Both problems carry over to training: a success-based reward can attribute gains to the asker that actually arise from the solver’s capabilities or from oracle leakage. Third, existing datasets lack a shared representation of ambiguities, resolutions, and what the agent, the oracle, and the verifier may access. We address these difficulties with a formalism that makes explicit what is trusted and what the agent, the oracle, and the verifier may each access. We use it to address the following questions: Q1. How can task disambiguation be evaluated separately from solution generation? Q2. How can oracle leakage be formally defined and diagnosed without relying on an untrusted language model judge? Q3. How can an asking policy be trained within the same formalism, and how does training affect task success rate, asking behavior, and leakage? 1.2 Contributions Figure 1: Agent and environment decompositions, task record, and example of a text-to-SQL AVT with two ambiguities and four interpretations. We introduce the notion of an Ambiguous Verifiable Task (AVT) (Definition 1), a general representation of underspecified tasks with verifiable solutions. An AVT consists of a public task specification, a probability space of hidden interpretations, a finite set of ambiguities and their resolutions, and a verifier (Figure 1c); nothing in the definition is specific to tables or SQL. We study conversational task disambiguation through an ask-then-solve protocol for AVTs and instantiate it in text-to-SQL over relational databases. In this setup, the agent consists of an asking policy (or asker) that asks the oracle questions and a solution policy (or solver) that submits a solution. The environment consists of an oracle, whose matcher credits the ambiguities each question addresses and whose responder returns information about their resolutions, and a trusted verifier that scores the solution. Because only the asker interacts with the oracle, we can vary the asker while holding the solver fixed. We address questions Q1-Q3 through three main contributions: 1. Theory. We develop a leakage-aware formalism for AVTs and instantiate it in an agent-environment interaction protocol (Section 2). For Q1, it separates asking from solving through no-disambiguation and full-disambiguation baselines and asking precision and recall. Splitting the oracle into a matcher and a responder, as some user simulators already do (Huo et al., 2026; Sullutrone et al., 2026; Zhang et al., 2026; Trinh et al., 2026), separates two leakage channels: crediting ambiguities that a question does not address, and revealing more than the credited resolutions. For Q2, we use our framework to give mathematical formulas quantifying the two types of leakage. The formalism also introduces judge-free, computable diagnostics and proves how they witness or approximate leakage under explicit assumptions. 2. AmbiTab Benchmark Suite. We introduce AmbiTab for conversational task disambiguation over tabular data, comprising more than 8,000 task records across six ambiguous text-to-SQL datasets (Section 3). AmbiTab-Bird and AmbiTab-EW-Bird derive from BIRD (Li et al., 2023); AmbiTab-BIRD-Interact and AmbiTab-Ambrosia adapt BIRD-Interact (Huo et al., 2026) and Ambrosia (Saparina & Lapata, 2024); and AmbiTab-AmbiQT and AmbiTab-TACO adapt AmbiQT (Bhaskar et al., 2023) and TACO (Deng et al., 2026). All six share a record that separates the public specification, the resolutions available to the oracle, and the private reference used by the verifier (Figure 1b), supporting the metrics for Q1 and, with the reference-response and sampling procedures in Section A.4, the diagnostics for Q2. AmbiTab-Bird and AmbiTab-EW-Bird also include training splits relevant to Q3. 3. Experiments. We address Q1 by comparing no-disambiguation, evaluated-asker, and full-disambiguation conditions and reporting both success and asking metrics (Section 4.2). Reporting all three success rates together with macro precision and recall distinguishes the benefit of clarification from gaps in question quality. For Q2, we apply the diagnostics to deterministic and LLM oracles and test them on a responder that reads the reference SQL (Section 4.3). For Q3, we train the asking policy with a multi-turn GRPO variant with DAPO-style asymmetric clipping and zero-advantage group filtering (Shao et al., 2024; Yu et al., 2025) on AmbiTab-Bird, evaluate its coverage and task success with a fixed solution policy across datasets, and compare training oracles and reward objectives (Section 4.4). 1.3 Related Work Conversational task disambiguation. Ambiguous text-to-SQL benchmarks represent alternative readings (Bhaskar et al., 2023; Saparina & Lapata, 2024; Deng et al., 2026). BIRD-Interact and ABISS evaluate clarification with simulated users (Huo et al., 2026; Sullutrone et al., 2026), whereas DySQL-Bench studies follow-up requests in database exploration (Sun et al., 2026a). Information-gain methods select questions that distinguish interpretations (Kobalczyk et al., 2025; Qiu et al., 2025); IN3, AskBench, and π-Bench assess coverage of missing information (Qian et al., 2024; Zhao et al., 2026; Zhang et al., 2026). We focus on eliciting the missing information that resolves the current task’s intended interpretation. Our task representation makes task ambiguities, resolutions, and oracle access explicit, with text-to-SQL as the evaluated instantiation over tabular data. See Appendix F for broader comparisons of benchmarks, clarification methods, and reward formulations. Oracle reliability. Prior work restricts solution access or audits simulated responses (Jain et al., 2025; Huo et al., 2026; Sullutrone et al., 2026), but simulator fidelity and policy overfitting remain concerns (Seshadri et al., 2026; Yu et al., 2026). We define leakage relative to a perfect oracle and derive judge-free diagnostics for undeserved question credit and excess response information. Learning to disambiguate. RL for SQL generation and table reasoning directly rewards solution correctness (Pourreza et al., 2025; Ma et al., 2025; Lei et al., 2025b). ACT and SGP learn clarification policies (Chen et al., 2025; Berant et al., 2026), while IntentRL learns to enumerate interpretations and SQL queries in one response (Saparina & Lapata, 2026). InfoPO combines information gain with outcome rewards (Kong et al., 2026); HiL-Bench combines blocker coverage with per-question bonuses and penalties (Trinh et al., 2026). Our question-F1 reward balances ambiguity coverage and question efficiency without additional shaping coefficients. Training improves downstream accuracy through better disambiguation. 2 Theory Agent, environment, and oracle leakage. We model the interactions between an agent A and an environment ℰE as a stochastic process from which we sample trajectories τ∼(,ℰ)τ (A;E). Both A and ℰE are conditional distributions: A models the probability distribution of the next action given the observable history, and ℰE that of the next state given the history and the action taken by A. The sets of states and actions depend on the setup. We are interested in situations where the environment is determined by two kinds of distributions: (1) Trusted distributions, such as a programmatic verifier V, which could even be deterministic. (2) Untrusted distributions, such as an oracle O implemented by a language model, intended to approximate a perfect oracle ⋆O , which could be defined to be, e.g., the behavior of a user base. The agent is usually evaluated using its success rate J(,ℰ):=τ∼(,ℰ)[V(τ)]J(A;E):=E_τ (A;E)[V(τ)]. However, maximizing J(,ℰ)J(A;E) can lead the agent to exploit deviations of O from ⋆O , e.g., by eliciting information from O that ⋆O would not provide. We refer to any such deviation as oracle leakage. Ambiguous verifiable tasks. We represent task specifications, ambiguities, clarification questions, responses, and solutions as text; we write TextText for this set. Task disambiguation elicits information about the intended interpretation of a specification. In our text-to-SQL instantiation, this interpretation determines what the user intends to compute over the database, and the solution is a corresponding SQL query (Figure 1c). Definition 1. An ambiguous verifiable task is a tuple (x,Θ,Amb,ρ,V)(x, ,Amb,ρ,V) consisting of • A specification x∈Textx ; • A probability space Θ of interpretations of x; • A finite set Amb⊆TextAmb of ambiguities; • A resolution map ρ:Θ×Amb→Textρ ×Amb ; • A verifier V:Θ×Text→0,1V ×Text→\0,1\. Proofs, technical assumptions, a table of notation, and other details are in Appendix A. Leakage-aware evaluation of agents on ambiguous verifiable tasks. Figure 2 defines the ask-then-solve protocol, a simple interaction protocol between an agent and an environment associated to an AVT. This setup can be generalized to other interaction protocols.11 1 For context, note that an AVT equipped with an interaction protocol determines a partially observable Markov decision process: the state is the hidden interpretation θ plus the dialogue history, and the agent observes the history. Since θ is drawn once and fixed, this is the mixed-observability case of Ong et al. (2010). θ∼Θθ Policyπask _askMatchermatchmatchθ θ _iicred_iℎihist_iSolution Policyπsol _solVerifierVVθℎnhist_nsolution0,1\0,1\ Figure 2: The ask-then-solve protocol (Section A.2). Oracle leakage can arise from undeserved matcher credit or responses revealing more than the credited resolutions. We now define success metrics22 2 These parallel precision and recall in HiL-Bench (Trinh et al., 2026); in our protocol, these evaluate πask _ask independently of πsol _sol. Our no- and full-disambiguation baselines, below, parallel their blocked and full-information conditions. Checkpoint coverage in AskBench (Zhao et al., 2026) is related., leakage, and leakage diagnostics for the ask-then-solve protocol associated to an AVT (Figure 2). See Section A.3 for formal definitions, and Section A.4 for estimation. The definitions and theoretical guarantees below concern this protocol. Since we care about the interactions between the asker and the oracle, we fix a solution policy, a verifier, and a perfect oracle ⋆=(match⋆,resp⋆)O =(match ,resp ). We abbreviate (πask,πsol,,V)( _ask, _sol;O,V) as (πask;)( _ask;O). Success metrics and baselines • Success rate. J(πask,):=[V(θ,)]J( _ask;O):=E[V(θ,solution)]. • Recall and Precision. In an episode with n questions, the matcher credits a set i⊆Ambcred_i of ambiguities to question i; write ≤i:=⋃j=1ijcred_≤ i:= _j=1^icred_j, so that ≤0=∅cred_≤ 0= . Rec(πask;):=[|≤n||Amb|]Prec(πask;):=[|i:i⊈≤i−1|n|n≥1].Rec( _ask;O):=E [ |cred_≤ n||Amb| ]\;\;\;\;\;\;Prec( _ask;O):=E [ |\i:cred_i _≤ i-1\|n\; |\;n≥ 1 ]. • Full-disambiguation and no-disambiguation baselines. Fix a full-disambiguation policy πfull _full, which acquires every ambiguity and asks no unproductive question (Rec(πfull,⋆)=Prec(πfull,⋆)=1Rec( _full;O )=Prec( _full;O )=1), and let πnone _none stop without asking any questions. Jfull():=J(πfull,)Jnone():=J(πnone,).J_full(O):=J( _full;O)\;\;\;\;\;\;J_none(O):=J( _none;O). Recall that, by oracle leakage, we mean any exploitation by the agent of deviations of O from ⋆O . This motivates the following definitions33 3 J-leakage is a signed, fixed-policy analogue of policy-value gaps used to assess simulators (Dann et al., 2026) and is related to generalized-gain leakage (Alvim et al., 2012). Here the reference is a task-defined perfect oracle, and a positive gap means extra task success enabled by the actual oracle.: Leakage 1. J-leakage. LeakJ(πask,):=J(πask,)−J(πask,⋆)Leak_J( _ask;O):=J( _ask;O)-J( _ask;O ). 2. Recall leakage. LeakRec(πask,match):=Rec(πask,match,resp⋆)−Rec(πask,⋆)Leak_Rec( _ask;match):=Rec( _ask;match,resp )-Rec( _ask;O ). The above notions of leakage cannot generally be computed directly. For example, estimating J-leakage requires trajectories from (πask;⋆)( _ask;O ), which in practice can require an expensive back-and-forth between sampling from the asker and human annotators. Leakage diagnostics 1. Responder-leakage diagnostic ΔJresp J_resp. Replaces the actual responder by the perfect responder while holding the asking policy and actual matcher fixed (Definition 5). 2. Specification-randomization diagnostic ΔRecspecQ _spec^Q. Recall after replacing the task specification seen by the asking policy with an independently sampled specification (Definition 6). 3. Adversarial diagnostics ΔJfull J_full, ΔRecnone _none. Evaluate J-leakage under an adversarial policy that covers all of the ambiguities, and recall leakage under one that covers none (Definition 7). Next is the main result, Theorem 1. Informally: Part 1 says that the responder-leakage diagnostic approximates J-leakage when swapping the actual matcher for the perfect one results in a small change in the distribution of trajectories, as measured by the total variation distance TVTV (Section A.1); and Part 2 says that the specification-randomization diagnostic approximates recall leakage of the randomized policy when its perfect-oracle recall is small, and that a simple mixture model relates this to recall leakage of the original policy. See Section A.5 for the proof. Theorem 1. Fix an AVT, asking policy πask _ask, solution policy, and oracle =(match,resp)O=(match,resp). 1. Let PmatchP_match and Pmatch⋆P_match be the joint distributions of the hidden interpretation θ and the complete visible trajectory τ under (πask,match,resp⋆)( _ask;match,resp ) and (πask;⋆)( _ask;O ), respectively. Then |LeakJ(πask,)−ΔJresp(πask,match,resp)|≤TV(Pmatch,Pmatch⋆). |\,Leak_J( _ask;O)- J_resp( _ask;match,resp)\, |\,≤\,TV(P_match,P_match ). 2. We have ΔRecspecQ(πask,match)−LeakRec(πaskQ,match)=Rec(πaskQ,⋆) _spec^Q( _ask;match)-Leak_Rec( _ask^Q;match)=Rec( _ask^Q;O ). Moreover, if πask=pπinv+(1−p)π′ _ask=p\, _inv+(1-p)\,π for some p∈[0,1]p∈[0,1] and asking policies πinv _inv and π′π , where πinv _inv is invariant under replacing the task specification, then |LeakRec(πask,match)−LeakRec(πaskQ,match)|≤ 2(1−pB), |\,Leak_Rec( _ask;match)-Leak_Rec( _ask^Q;match)\, |\,≤\,2\,(1-p^B), where B is the question budget and Q is any distribution over task specifications. 3 AmbiTab benchmark suite We introduce AmbiTab, a suite of six ambiguous text-to-SQL datasets for evaluating conversational task disambiguation over tabular data. Each dataset consists of task records with explicit ambiguity annotations, fixed resolutions, and verification assets. Task record. Abstractly, a task record implements a fixed interpretation of an ambiguous verifiable task (x,Θ,Amb,ρ,V)(x, ,Amb,ρ,V), so it can be seen as a sample of an AVT. More precisely, given an interpretation θ∈Θθ∈ , its corresponding task record is the tuple (x,Amb,ρθ,Vθ)(x,Amb, _θ,V_θ), where ρθ(α)=ρ(θ,α) _θ(α)=ρ(θ,α) is the resolution map for an ambiguity α evaluated on the fixed interpretation, and Vθ(s)=V(θ,s)V_θ(s)=V(θ,s) is the verifier evaluated on the fixed interpretation. In practice, this is implemented as follows (see Figure 1(b) for an illustration): • Task specification and context. The user request, and optionally the database schema, the column meanings, external knowledge, and other context. • Ambiguities and resolutions. A list pairing each ambiguity with its resolution as text. Optionally, it might include the ambiguity type, a SQL snippet for grounding the responder, etc. • Verification assets. Usually, an executable database and a reference query. Optionally, test cases, statements that prepare and restore the database, a policy for comparing result sets, etc. Dataset construction and details. We adapt BIRD (Li et al., 2023), BIRD-Interact (Huo et al., 2026), Ambrosia (Saparina & Lapata, 2024), AmbiQT (Bhaskar et al., 2023), and TACO (Deng et al., 2026) (Section B.1). All datasets use SQLite except for AmbiTab-BIRD-Interact, which uses PostgreSQL. Table 1 shows where each corpus’s missing information comes from; Appendix B gives the construction and oracle details. Table 1: The six AmbiTab datasets. AmbiTab-Bird and AmbiTab-EW-Bird counts combine their train and development splits. Dataset Missing information and construction Records AmbiTab-Bird Rewrite BIRD tasks with alternative readings; filter by SQL probes and a model judgment. 664 AmbiTab-EW-Bird Withhold the BIRD evidence field; retain rows where providing it improves performance for at least one reference model. 2,537 AmbiTab-BIRD-Interact Normalize source ambiguity annotations; freeze audited, model-authored responses. 600 AmbiTab-Ambrosia Use each annotated reading and its native intent as a separate record. 2,670 AmbiTab-AmbiQT Build executable databases for paired SQL readings and assign each reading as a hidden intent. 1,722 AmbiTab-TACO Verbalize source SQL while withholding table or value choices. 187 Total 8,380 4 Experiments We evaluate task disambiguation in text-to-SQL through task success, asking precision and recall, and oracle leakage across the six datasets in Section 3, varying the oracle, asker prompt, and training configuration. 4.1 Experimental setup Protocol. We use the ask-then-solve protocol of Figure 2, with Qwen3-8B (Yang et al., 2025) as the asker and a fixed Qwen3-Coder-30B-A3B-Instruct solver. The asker clarifies the intended task through dialogue with the oracle; the solver uses the request and dialogue to produce SQL. The default prompt, Checklist (ledger), enumerates potentially ambiguous terms and tracks their resolutions. Section C.5 examines the precision–recall tradeoff across asker prompts. We use the deterministic oracle by default and compare it with Qwen3-14B and Gemma-3-27B oracles (Section B.2). Evaluation. We report Success Rate (SR) for the evaluated asking policy and the no- and full-disambiguation baselines, alongside macro asking precision and recall (Section 2). Asking gain is the SR increase over the same run’s no-disambiguation baseline. Results show means and sample standard deviations across evaluation seeds; Appendices C and D.5 give protocol and aggregation details. Leakage diagnostics come from separate evaluations (Appendix E). 4.2 AmbiTab disambiguation capacity Under the deterministic oracle, full disambiguation substantially improves SR over no disambiguation on five datasets, but the evaluated asker closes only part of this gap (Figure 3a; complete results in Table 8, Section C.4). On AmbiTab-Bird, for example, SR rises from 26.3%26.3\% without disambiguation to 66.7%66.7\% with asking and 86.2%86.2\% with full disambiguation. The remaining gap is 19.5 percentage points on AmbiTab-Bird and 7.1 to 17.4 points on the other four datasets. AmbiTab-BIRD-Interact is solver-bound (Figure 3a): its complex SQL makes SR largely dependent on the solver rather than the asker, with little change across disambiguation settings or oracles (Table 8). Recall remains below complete coverage on every dataset, while low precision indicates many questions that receive no new ambiguity credit (Figure 3b,c; Table 8). These results apply the evaluation of Q1: reporting the three success rates together with precision and recall shows that disambiguation improves success, but the asker leaves ambiguities uncovered and asks unproductive questions. Figure 3: The values of Table 8 (Section C.4) plus each oracle’s own no-disambiguation SR, under the deterministic (navy), Qwen3-14B (yellow), and Gemma-3-27B (red) oracles. (a) SR without disambiguation (pale), with the evaluated asker (solid), and with full disambiguation (black line); hatching is the gap the asker leaves. (b-c) Asker recall and precision; hatching is the shortfall from 100%. (d-g) Leakage diagnostics in percentage points (Bird and EW-Bird dev, Ambrosia); the deterministic oracle’s ΔJresp J_resp, ΔJfull J_full, and ΔRecnone _none are zero by definition (open circles), and its ΔRecspecQ _spec^Q is the reference recall. Error bars: seed SDs over five seeds in (a-c) and three in (d-g). Dataset labels omit the AmbiTab prefix. 4.3 Success-leakage tradeoff across oracles The oracle comparison addresses Q2: Gemma-3-27B yields higher asker SR than Qwen3-14B on all six datasets, alongside higher values of the no-disambiguation recall-leakage and specification-randomization diagnostics, ΔRecnone _none and ΔRecspecQ _spec^Q (Figure 3f,g; Table 8). On AmbiTab-AmbiQT, SR increases from 57.7%57.7\% to 69.3%69.3\%, while ΔRecspecQ _spec^Q rises from 9.99.9 to 70.970.9 percentage points. ΔJresp J_resp shows no consistent direction across datasets, and ΔJfull J_full differs by less than one point between the two oracles on every dataset (Figure 3d,e). These results motivate assessing success and leakage together. Recall also depends on the oracle’s credits and responses, including volunteered resolutions (Figure 3b). These comparisons capture the combined effects of matching, responses, and subsequent questions, without isolating leakage’s contribution to SR. An oracle exposed to the ground truth leaks more: When the LLM responder also reads the full reference SQL, with the matcher unchanged, ΔJfull J_full rises on AmbiTab-EW-Bird, AmbiTab-Ambrosia, and AmbiTab-AmbiQT under both oracles, by up to 10.4 points, and ΔJresp J_resp rises by up to 7.1 points, with intervals excluding zero in six of ten dataset-oracle pairs; AmbiTab-TACO shows no detectable change (Figure 4; Table 15). Appendix E reports the complete diagnostics. Figure 4: Change in leakage when the LLM responder also reads the full reference SQL, with the matcher unchanged, shown minus hidden. (a) ΔJresp J_resp; (b) ΔJfull J_full. Above the line, the default responder reads the SQL and the new runs hide it; below it, the new runs show it. Error bars are paired 95% AVT-cluster bootstrap intervals (Table 15). AmbiTab-BIRD-Interact was not run, and dataset labels omit the AmbiTab prefix. Table 2: Question-F1 reward results with the deterministic oracle and fixed solver: SR, precision, and recall (%) of the untrained (Base) and trained (RL) askers. Dist.: in-distribution (ID) development splits or out-of-distribution (OOD) corpora; AT abbreviates AmbiTab. Δ : RL minus Base (p). RL averages three evaluation seeds and Base five; per-corpus seed SDs are at most 1.5 p (Table 13). SR ties at displayed precision show zero Δ . The mean weights corpora equally. Full comparison in Table 13. SR Precision Recall Dist. Corpus Base RL Δ Base RL Δ Base RL Δ ID AT-Bird dev 59.5 71.9 +12.4 31.0 37.0 +6.0 67.7 83.4 +15.7 AT-EW-Bird dev 69.0 72.2 +3.2 33.6 38.0 +4.4 69.8 79.3 +9.5 OOD AT-BIRD-Interact 9.7 9.7 +0.0 32.1 36.4 +4.3 57.7 65.1 +7.4 AT-Ambrosia 35.0 37.3 +2.2 24.0 27.3 +3.3 71.7 81.8 +10.1 AT-AmbiQT 56.3 59.4 +3.1 19.5 21.7 +2.2 58.3 65.0 +6.7 AT-TACO 72.8 77.0 +4.2 29.1 30.7 +1.5 85.1 89.8 +4.7 All Mean 50.4 54.6 +4.2 28.2 31.8 +3.6 68.4 77.4 +9.0 4.4 Training improves the asking policy Training configuration. To improve both precision and recall (Q3), we train the asker on AmbiTab-Bird with the deterministic oracle, using a multi-turn GRPO variant with DAPO-style asymmetric clipping and zero-advantage group filtering (Shao et al., 2024; Yu et al., 2025). Our main policy uses a question-F1 reward, the harmonic mean of asking precision and recall within each episode. Requests with ambiguities receive zero reward when no question is productive; requests without ambiguities receive one for abstaining and zero for asking. The reward encourages efficient ambiguity coverage without requiring solver output; Appendix D gives the formal definition and training details. Results. We use the development splits of AmbiTab-Bird and AmbiTab-EW-Bird for in-distribution evaluation and the remaining four datasets for out-of-distribution evaluation (Table 2). Under the deterministic oracle and fixed solver, the trained policy improves macro precision and recall over the untrained asker on all six datasets (Figure 5a). Equally weighted means across the six datasets increase from 28.2%28.2\% to 31.8%31.8\% precision, 68.4%68.4\% to 77.4%77.4\% recall, and 50.4%50.4\% to 54.6%54.6\% SR. Results average three inference seeds for one trained policy per configuration and five for the untrained asker. SR gains vary across datasets, consistent with the distinction between acquiring resolutions and using them successfully (Section 2). Ablations. We compare the policy trained with question-F1 reward with two variants trained on final SQL correctness (outcome reward), using either the deterministic oracle or Qwen3-14B. Both achieve lower mean precision, recall, and SR under the same evaluation protocol (Figure 5a shows the Qwen3-14B variant). Unlike question-F1, outcome reward depends on the solver’s ability to use the responses. A separate outcome-reward run on AmbiTab-BIRD-Interact failed to sustain clarification improvements before crashing (Section D.4). Leakage. We evaluate the leakage diagnostics for the question-F1 policy and the outcome-reward policy trained with Qwen3-14B (Section E.2). Question-F1 training does not raise responder leakage under Qwen3-14B (Figure 5b): its ΔJresp J_resp is below the untrained asker’s on 5 of 5 evaluated datasets; under Gemma-3-27B it changes by −3.21-3.21 to 1.791.79 points, rising on 2. Its ΔRecspecQ _spec^Q point estimates rise on 4 of 5 datasets under Gemma-3-27B, most on AmbiTab-AmbiQT and AmbiTab-Ambrosia (Figure 5c), beyond the rise of the deterministic reference (11.0311.03 against 3.723.72 and 5.745.74 against 0.440.44 points), which suggests phrasings that Gemma-3-27B’s matcher credits regardless of the task. The outcome-reward policy has higher ΔJresp J_resp than the question-F1 policy in 9 of 10 dataset–oracle pairs, but only 1 paired interval excludes zero, and the comparison fixed before the runs is null (Table 17). The untrained asker’s leakage values come from a separate serving session. With one training seed per configuration and unequal training budgets, these comparisons support our choice of question-F1 but do not establish a general ranking of rewards or training oracles (Section D.3). Figure 5: Change from the untrained asker (Tables 2, 13 and 16); gray bands span ±1.96± 1.96 SE of the seed-mean difference. Bird and EW-Bird are dev splits. 5 Conclusions We introduced a leakage-aware framework for evaluating and training conversational task disambiguation over tabular data, based on ambiguous verifiable tasks and an ask-then-solve protocol. The framework separates the asker, solver, oracle, and verifier, and treats precision and recall as essential alongside success rate. No-disambiguation and full-disambiguation evaluations provide empirical reference points for a fixed solver and oracle. Together with leakage diagnostics, these measures help distinguish failures in acquiring information, using it, and respecting disclosure boundaries. With the deterministic oracle, three-seed evaluations of question-F1 training show higher precision and recall across all six corpora. Our six-corpus benchmark remains unsaturated: clarification and asker training improve performance, but substantial gaps in question quality and task success persist. Future work will develop better training methods for askers and expand the benchmark to broader tasks, ambiguity types, and interaction settings. AI use statement Generative AI was used both within the systems studied and as a research tool. Within the systems studied, language models generated synthetic benchmark data: an LLM authored the audited AmbiTab-BIRD-Interact response catalog, and Qwen generated the AmbiTab-Bird rewrites, with Qwen providing admission tests and Gemma necessity judgments. Qwen and Gemma models also filtered AmbiTab-EW-Bird records; the other dataset conversions did not use a language-model authoring stage (Sections 3 and B.1). In the experiments, language models served as asking policies, SQL solvers, and evaluated oracles, and Qwen3-8B was trained as the asking policy (Sections 4.1, B.2, 4.4 and D). As a research tool, we used generative AI tools for the following tasks with required disclosure: assisting in the writing of proofs, designing and giving feedback on experiments, implementing methods, cleaning and reformatting datasets, supporting qualitative data analysis, and interpreting results. We did not use generative AI tools to develop the theoretical framework, formulate mathematical claims, provide critical ingredients for proofs, or propose or refine hypotheses, and translation is not applicable to this work. Additionally, we used generative AI tools for tasks with recommended disclosure: writing and editing code, including code that checks and formats recorded experimental results; double-checking the theoretical framework; literature search and discovery; and drafting, editing, and polishing the manuscript and LaTeX source. We have reviewed all AI-assisted work. The authors checked every proof and mathematical claim by hand, reviewed and tested AI-written code and inspected the resulting dialogue trajectories, checked the empirical claims against the recorded results, and verified every cited reference against its source. We take responsibility for the final content of this work, including text, claims, data, code, and other artifacts produced with the aid of generative AI. Code and data availability Code and data will be made available with the next version of this preprint. Reproducibility statement Access to the source code and datasets is described in the code and data availability statement. The assumptions of the theory and complete proofs are in Appendix A, and the estimation procedures for the metrics and diagnostics are in Section A.4. Dataset construction and oracle implementations are described in Appendix B. Evaluation protocols are described in Section 4.1, and decoding settings, budgets, seed counts, and aggregation in Appendix C. Training hyperparameters, seed counts, checkpoint selection, and limitations of the ablation comparisons are documented in Appendix D. The settings of the leakage experiments are provided in Appendix E. Ethics statement This work studies conversational task disambiguation over tabular data using existing text-to-SQL benchmarks and simulated user responses. It involves no human subjects and collects no new personal data; all data come from public benchmarks. Some records contain model-authored text (see the AI use statement), which may carry model errors or biases. The source benchmarks are public and may appear in the pretraining data of the evaluated models, which can inflate absolute success rates; our comparisons hold the models fixed across conditions. Benchmark and model biases may carry over to the resulting systems, and simulated interactions do not establish effectiveness with real users. Deployment would require appropriate database access controls and safeguards against sensitive information disclosure and incorrect queries. References Aliannejadi et al. (2021) Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeff Dalton, and Mikhail Burtsev. Building and evaluating open-domain dialogue corpora with clarifying questions. In Conference on Empirical Methods in Natural Language Processing, p. 4473–4484. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.emnlp-main.367. Alvim et al. (2012) Mário S. Alvim, Konstantinos Chatzikokolakis, Catuscia Palamidessi, and Geoffrey Smith. Measuring information leakage using generalized gain functions. In IEEE Computer Security Foundations Symposium, p. 265–279. IEEE, 2012. doi: 10.1109/CSF.2012.26. Andukuri et al. (2024) Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah D. Goodman. STaR-GATE: Teaching language models to ask clarifying questions. In First Conference on Language Modeling (COLM), 2024. Barres et al. (2026) Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ2τ^2-bench: Evaluating conversational agents in a dual-control environment. In International Conference on Machine Learning (ICML), 2026. Berant et al. (2026) Jonathan Berant, Maximillian Chen, Adam Fisch, Reza Aghajani, Fantine Huot, Mirella Lapata, and Jacob Eisenstein. Learning steerable clarification policies with collaborative self-play. In Third Conference on Language Modeling (COLM), 2026. Bhaskar et al. (2023) Adithya Bhaskar, Tushar Tomar, Ashutosh Sathe, and Sunita Sarawagi. Benchmarking and improving text-to-SQL generation under ambiguity. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 7053–7074. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.emnlp-main.436. Chen et al. (2026) Jinpeng Chen, Cheng Gong, Hanbo Li, Ziru Liu, Zichen Tian, Xinyu Fu, Shi Wu, Chenyang Zhang, Wu Zhang, Suiyun Zhang, et al. CoVe: Training interactive tool-use agents via constraint-guided verification. arXiv preprint arXiv:2603.01940, 2026. Chen et al. (2025) Maximillian Chen, Ruoxi Sun, Tomas Pfister, and Sercan Ö. Arık. Learning to clarify: Multi-turn conversations with action-based contrastive self-training. In International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=SIE6VFps9x. Dann et al. (2026) Christoph Dann, Yishay Mansour, and Mehryar Mohri. Theoretical foundations and effective algorithms for policy-aware simulator learning. arXiv preprint arXiv:2605.29032, 2026. Deng et al. (2026) Chao Deng, Ju Fan, Yuyu Luo, Qinliang Xue, Meihao Fan, Yuxin Zhang, Min Zhang, Xiaofeng Jia, Jing Zhang, and Xiaoyong Du. TACO: A benchmark for open-domain text-to-SQL with ambiguous and cross-database queries. Proceedings of the VLDB Endowment, 19(6):1386–1399, 2026. doi: 10.14778/3797919.3797942. Dong et al. (2025) Mingwen Dong, Nischal Ashok Kumar, Yiqun Hu, Anuj Chauhan, Chung-Wei Hang, Shuaichen Chang, Lin Pan, Wuwei Lan, Henghui Zhu, Jiarong Jiang, et al. PRACTIQ: A practical conversational text-to-SQL dataset with ambiguous and unanswerable queries. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 255–273. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.naacl-long.13. Guo et al. (2025) Ziming Guo, Chao Ma, Yinggang Sun, Tiancheng Zhao, Guangyao Wang, and Hai Huang. Evaluating and enhancing LLMs for multi-turn text-to-SQL with multiple question types. In 2025 International Joint Conference on Neural Networks (IJCNN), p. 1–10. IEEE, 2025. doi: 10.1109/IJCNN64981.2025.11228279. Gur et al. (2018) Izzeddin Gur, Semih Yavuz, Yu Su, and Xifeng Yan. DialSQL: Dialogue based structured query generation. In Annual Meeting of the Association for Computational Linguistics, p. 1339–1349. Association for Computational Linguistics, 2018. doi: 10.18653/v1/P18-1124. Huo et al. (2026) Nan Huo, Xiaohan Xu, Jinyang Li, Per Jacobsson, Shipei Lin, Bowen Qin, Binyuan Hui, Xiaolong Li, Ge Qu, Shuzheng Si, et al. BIRD-INTERACT: Re-imagining text-to-SQL evaluation via lens of dynamic interactions. In International Conference on Learning Representations (ICLR), 2026. Jain et al. (2025) Arihant Jain, Purav Aggarwal, Rishav Sahay, Chaosheng Dong, and Anoop Saladi. AutoEval-ToD: Automated evaluation of task-oriented dialog systems. In North American Chapter of the Association for Computational Linguistics, p. 10133–10148. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.naacl-long.508. Kobalczyk et al. (2025) Katarzyna Kobalczyk, Nicolás Astorga, Tennison Liu, and Mihaela van der Schaar. Active task disambiguation with LLMs. In International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=JAMxRSXLFz. Kong et al. (2026) Fanqi Kong, Jiayi Zhang, Mingyi Deng, Chenglin Wu, Yuyu Luo, and Bang Liu. InfoPO: Information-driven policy optimization for user-centric agents. In International Conference on Machine Learning (ICML), 2026. Lei et al. (2025a) Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao, Wenjing Hu, Pengcheng Yin, et al. Spider 2.0: Evaluating language models on real-world enterprise text-to-sql workflows. In International Conference on Learning Representations, volume 2025, p. 28691–28735, 2025a. Lei et al. (2025b) Fangyu Lei, Jinxiang Meng, Yiming Huang, Tinghong Chen, Yun Zhang, Shizhu He, Jun Zhao, and Kang Liu. Reasoning-table: Exploring reinforcement learning for table reasoning. arXiv preprint arXiv:2506.01710, 2025b. Li et al. (2025) Belinda Z. Li, Alex Tamkin, Noah D. Goodman, and Jacob Andreas. Eliciting human preferences with language models. In International Conference on Learning Representations (ICLR), 2025. Li et al. (2026) Ce Li, Xiaofan Liu, Zhiyan Song, Ce Chi, Boshen Shi, Chen Zhao, Guanguang Chang, Zhendong Wang, Kexin Yang, Xing Wang, et al. TReB: A comprehensive benchmark for evaluating table reasoning capabilities of large language models. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 3267–3275. ACM, 2026. doi: 10.1145/3805712.3808617. Li et al. (2023) Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, et al. Can LLM already serve as a database interface? a big bench for large-scale database grounded text-to-SQLs. In Advances in Neural Information Processing Systems 36, volume 36, p. 42330–42357. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2023. doi: 10.52202/075280-1835. Li et al. (2020) Yuntao Li, Bei Chen, Qian Liu, Yan Gao, Jian-Guang Lou, Yan Zhang, and Dongmei Zhang. “what do you mean by that?” a parser-independent interactive approach for enhancing text-to-SQL. In Conference on Empirical Methods in Natural Language Processing, p. 6913–6922. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.emnlp-main.561. Ma et al. (2025) Peixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang, Ran Chen, and Jian Guo. SQL-R1: Training natural language to SQL reasoning model by reinforcement learning. In Advances in Neural Information Processing Systems 38, volume 38, p. 193299–193331. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2025. doi: 10.52202/085713-5805. Min et al. (2020) Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. AmbigQA: Answering ambiguous open-domain questions. In Conference on Empirical Methods in Natural Language Processing, p. 5783–5797. Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.emnlp-main.466. Muratore et al. (2018) Fabio Muratore, Felix Treede, Michael Gienger, and Jan Peters. Domain randomization for simulation-based policy optimization with transferability assessment. In Conference on Robot Learning, volume 87 of Proceedings of Machine Learning Research, p. 700–713. PMLR, 2018. Ong et al. (2010) Sylvie C. W. Ong, Shao Wei Png, David Hsu, and Wee Sun Lee. Planning under uncertainty for robotic tasks with mixed observability. Int. J. Robotics Res., 29(8):1053–1068, 2010. doi: 10.1177/0278364910369861. Pan et al. (2022) Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The effects of reward misspecification: Mapping and mitigating misaligned models. In International Conference on Learning Representations, 2022. Pourreza et al. (2025) Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, and Sercan Ö. Arık. Reasoning-SQL: Reinforcement learning with SQL tailored partial rewards for reasoning-enhanced text-to-SQL. In Second Conference on Language Modeling (COLM), 2025. Qian et al. (2024) Cheng Qian, Bingxiang He, Zhuang Zhong, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, et al. Tell me more! towards implicit user intention understanding of language model driven agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1088–1113. Association for Computational Linguistics, 2024. doi: 10.18653/v1/2024.acl-long.61. Qiu et al. (2025) Luyu Qiu, Jianing Li, Chi Su, and Lei Chen. Interactive text-to-SQL via expected information gain for disambiguation. arXiv preprint arXiv:2507.06467, 2025. Roth et al. (2026) Amit Roth, Ankur Samanta, Matan Halevy, Yoav Levine, and Yonathan Efroni. Hack-verifiable environments: Towards evaluating reward hacking at scale. arXiv preprint arXiv:2605.20744, 2026. Saparina & Lapata (2024) Irina Saparina and Mirella Lapata. AMBROSIA: A benchmark for parsing ambiguous questions into database queries. In Advances in Neural Information Processing Systems 37, volume 37, p. 90600–90628. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024. doi: 10.52202/079017-2876. Saparina & Lapata (2026) Irina Saparina and Mirella Lapata. Reasoning about intent for ambiguous requests. In Third Conference on Language Modeling (COLM), 2026. Sarwar et al. (2026) Tabinda Sarwar, Farhad Moghimifar, Cong Duy Vu Hoang, Xiaoxiao Ma, Shawn Chang Xu, Fahimeh Saleh, Poorya Zaremoodi, Avirup Sil, and Katrin Kirchhoff. CLARITY: A framework and benchmark for conversational language ambiguity and unanswerability in interactive NL2SQL systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), p. 1218–1250. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.acl-industry.86. Seshadri et al. (2026) Preethi Seshadri, Samuel Cahyawijaya, Ayomide Odumakinde, Sameer Singh, and Seraphina Goldfarb-Tarrant. Lost in simulation: LLM-simulated users are unreliable proxies for human users in agentic evaluations. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 47423–47439. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.acl-long.2192. Shao et al. (2024) Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. Skalse et al. (2022) Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward hacking. In Advances in Neural Information Processing Systems 35, volume 35, p. 9460–9471. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2022. doi: 10.52202/068431-0687. Stelmakh et al. (2022) Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. ASQA: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 8273–8288. Association for Computational Linguistics, 2022. doi: 10.18653/v1/2022.emnlp-main.566. Sullutrone et al. (2026) Giovanni Sullutrone, Luca Sala, Sania Aftar, Georgia Koutrika, and Sonia Bergamaschi. ABISS: Evaluating text-to-SQL systems through agent interaction. arXiv preprint arXiv:2607.23340, 2026. Sun et al. (2026a) Linzhuang Sun, Tianyu Guo, Hao Liang, Ruitong Liu, Yuying Li, Qifeng Cai, Jingxuan Wei, Yuchen Wu, Bihui Yu, Xiangxiang Zhang, Wentao Zhang, and Bin Cui. Rethinking text-to-SQL: Dynamic multi-turn SQL interaction for real-world database exploration. In Findings of the Association for Computational Linguistics: ACL 2026, p. 33047–33069. Association for Computational Linguistics, 2026a. doi: 10.18653/v1/2026.findings-acl.1654. Sun et al. (2026b) Weiwei Sun, Xuhui Zhou, Weihua Du, Xingyao Wang, Sean Welleck, Graham Neubig, Maarten Sap, and Yiming Yang. Training proactive and personalized LLM agents. In Third Conference on Language Modeling (COLM), 2026b. Suri et al. (2026) Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, and Dinesh Manocha. Structured uncertainty guided clarification for LLM agents. In Findings of the Association for Computational Linguistics: ACL 2026, p. 40811–40838. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.findings-acl.2028. Tian et al. (2026) Jiaming Tian, Liyao Li, Wentao Ye, Haobo Wang, Lingxin Wang, Lihua Yu, Zujie Ren, Gang Chen, and Junbo Zhao. Toward real-world table agents: Capabilities, workflows, and design principles for LLM-based table intelligence. World Wide Web, 29(2):16, 2026. doi: 10.1007/s11280-025-01399-z. Trinh et al. (2026) Tu Trinh, Mohamed Elfeki, Guangze Luo, Kelvin Luu, Nathan Hunt, Ernesto Hernandez, Nandan Marwaha, Yannis Yiming He, Charles Wang, Fernando Carabedo, Alessa Castillo, and Bing Liu. HiL-Bench (human-in-loop benchmark): Do agents know when to ask for help? arXiv preprint arXiv:2604.09408, 2026. Wang et al. (2026) Zhensheng Wang, ZhanTeng Lin, Wenmian Yang, Kun Zhou, Yiquan Zhang, and Weijia Jia. ODUTQA-MDC: A task for open-domain underspecified tabular QA with multi-turn dialogue-based clarification. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 10907–10928. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.acl-long.499. Wu et al. (2025) Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xeron Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, et al. TableBench: A comprehensive and complex benchmark for table question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, p. 25497–25506. Association for the Advancement of Artificial Intelligence (AAAI), 2025. doi: 10.1609/aaai.v39i24.34739. Xue et al. (2026) Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Xiaosen Zheng, Zejun Ma, and Bo An. SimpleTIR: End-to-end reinforcement learning for multi-turn tool-integrated reasoning. In International Conference on Learning Representations (ICLR), 2026. Yang et al. (2025) An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yao et al. (2025) Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains. In International Conference on Learning Representations (ICLR), 2025. Yao et al. (2019) Ziyu Yao, Yu Su, Huan Sun, and Wen-tau Yih. Model-based interactive semantic parsing: A unified framework and A text-to-SQL case study. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p. 5446–5457. Association for Computational Linguistics, 2019. doi: 10.18653/v1/D19-1547. Yu et al. (2025) Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, Lingjun Liu, Xin Liu, et al. DAPO: An open-source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems 38, volume 38, p. 125532–125554. Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2025. doi: 10.52202/085713-3775. Yu et al. (2026) Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong, Abe Hou, Dilara Soylu, Sergey Levine, Christopher D. Manning, and Weiyan Shi. One frozen simulator is not enough: Simulator collapse in multi-agent RL. arXiv preprint arXiv:2608.12253, 2026. Yu et al. (2019a) Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, et al. CoSQL: A conversational text-to-SQL challenge towards cross-domain natural language interfaces to databases. In Conference on Empirical Methods in Natural Language Processing, p. 1962–1979. Association for Computational Linguistics, 2019a. doi: 10.18653/v1/D19-1204. Yu et al. (2019b) Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, et al. SParC: Cross-domain semantic parsing in context. In Annual Meeting of the Association for Computational Linguistics, p. 4511–4523. Association for Computational Linguistics, 2019b. doi: 10.18653/v1/P19-1443. Zhang et al. (2026) Haoran Zhang, Luxin Xu, Zhilin Wang, Runquan Gui, Shunkai Zhang, Haodi Lei, Zihao He, Bingsu He, Chicheng Qin, Tong Zhu, et al. π-bench: Evaluating proactive personal assistant agents in long-horizon workflows. arXiv preprint arXiv:2605.14678, 2026. Zhang & Choi (2025) Michael J. Q. Zhang and Eunsol Choi. Clarify when necessary: Resolving ambiguity through interaction with LMs. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 5541–5558. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.findings-naacl.306. Zhao et al. (2026) Jiale Zhao, Ke Fang, and Lu Cheng. When and what to ask: AskBench and rubric-guided RLVR for LLM clarification. In Findings of the Association for Computational Linguistics: ACL 2026, p. 17120–17140. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.findings-acl.845. Zhou et al. (2025) Yifei Zhou, Song Jiang, Yuandong Tian, Jason Weston, Sergey Levine, Sainbayar Sukhbaatar, and Xian Li. Sweet-RL: Training multi-turn LLM agents on collaborative reasoning tasks. arXiv preprint arXiv:2503.15478, 2025. Appendix A Theory details and proofs A.1 Conventions All spaces are standard Borel, and all maps and probability kernels are measurable. For a set S, (S)P(S) denotes the set of probability distributions on S. The total variation distance between two distributions P and P′P on the same space is TV(P,P′):=supA|P(A)−P′(A)|TV(P,P ):= _A|P(A)-P (A)|, where the supremum runs over events A. We write θ∼Θθ for a draw from the distribution of interpretations. Table 3 collects the main notation used here and in Section 2. Table 3: Notation. Symbol Name Reading Ambiguous verifiable task TextText texts Specifications, ambiguities, questions, responses, resolutions, and solutions x task specification The public request, visible to the agent Θ , θ interpretations Θ is their distribution; θ∼Θθ is the hidden reading of x, drawn once per episode AmbAmb, α ambiguities The finite set of registered ambiguities of x, each named by a text ρ resolution map ρ(θ,α)ρ(θ,α) is what θ decides for ambiguity α V verifier V(θ,s)∈0,1V(θ,s)∈\0,1\ scores a solution against θ Agent, environment, and protocol A, ℰE, τ agent, environment, trajectory The process the metrics average over πask _ask, πsol _sol asker, solver The two components of the agent πfull _full, πnone _none full, no disambiguation Reference askers for the two baselines πfulladv _full^adv, πnoneadv _none^adv adversarial probes Full and zero coverage, with adversarial wording πaskQ _ask^Q specification-randomized πask _ask fed a resampled specification =(match,resp)O=(match,resp) oracle Matcher and responder; ⋆O is the perfect oracle iqst_i, ians_i, ℎhist question, response, history The visible dialogue icred_i, ≤icred_≤ i credited ambiguities Credited at turn i, and through turn i B, ⊥ budget, stop symbol At most B questions; ⊥ ends the asking phase Metrics, leakage, and diagnostics J, RecRec, PrecPrec success, recall, precision The success metrics JfullJ_full, JnoneJ_none baselines Success under πfull _full and πnone _none LeakJLeak_J, LeakRecLeak_Rec leakage Signed gaps against the perfect oracle ΔJresp J_resp, ΔRecspecQ _spec^Q, ΔJfull J_full, ΔRecnone _none diagnostics Computable surrogates for leakage A.2 The ask-then-solve protocol Fix an AVT (x,Θ,Amb,ρ,V)(x, ,Amb,ρ,V), an integer question budget B≥|Amb|B≥|Amb|, and a stop symbol ⊥∉Text . In practice, one can set B=|Amb|+CB=|Amb|+C for a constant C≥0C≥ 0. A dialogue history ℎhist is any prefix of (x,(1,1),…,(n,n),⊥)(x,(qst_1,ans_1),…,(qst_n,ans_n), ) that includes x, with i,i∈Textqst_i,ans_i and 0≤n≤B0≤ n≤ B. An agent consists of an asking policy πask _ask and a solution policy πsol _sol: • A conditional distribution πask(⋅∣ℎ)∈(Text∪⊥) _ask(· ) (Text∪\ \) on histories without ⊥ , which must return ⊥ after B questions, but is otherwise not required to exhaust the budget B; • A conditional distribution πsol(⋅∣ℎ)∈(Text) _sol(· ) (Text) on histories ending in ⊥ . An environment consists of an oracle =(match,resp)O=(match,resp) and a verifier V: • A conditional distribution match(⋅∣,ℎ,θ)∈(2Amb)match(· ,hist,θ) (2^Amb), crediting a set cred of ambiguities to a question qst; • A conditional distribution resp(⋅∣,,ℎ,θ)∈(Text)resp(· ,cred,hist,θ) (Text); • The AVT’s verifier V. Thus, ℰ=(,V)E=(O,V) in the notation of Section 2, where the trajectory-level score V(τ)V(τ) used there is V(θ,s^)V(θ, s) for the episode’s hidden interpretation and final submission. An episode proceeds as follows: 1. Sample θ∼Θθ , keep it hidden from the agent, and set ℎ0=(x)hist_0=(x). 2. Sample the next question or stop decision from πask(⋅∣ℎ) _ask(· ). For a question qst, sample the credited ambiguities ∼match(⋅∣,ℎ,θ)cred (· ,hist,θ) and the response ∼resp(⋅∣,,ℎ,θ)ans (· ,cred,hist,θ), append (,)(qst,ans) to ℎhist, and repeat this step. The credited ambiguities are not shown to the agent. 3. When the asking policy produces ⊥ , append it to ℎhist and sample s^∼πsol(⋅∣ℎ) s _sol(· ). The episode ends with score V(θ,s^)V(θ, s). The trajectory visible to the agent is thus (x,(1,1),…,(n,n),⊥,s^)(x,(qst_1,ans_1),…,(qst_n,ans_n), , s), and the question budget ensures termination. A.3 Metrics, baselines, leakage, and diagnostics Fix the solution policy and verifier, abbreviate (πask,πsol,,V)( _ask, _sol;O,V) as (πask;)( _ask;O), and fix a perfect oracle ⋆=(match⋆,resp⋆)O =(match ,resp ) modeling an idealized user. A.3.1 Success metrics and baselines Definition 2. In an episode with n questions, write i⊆Ambcred_i for the ambiguities credited to question i and ≤i:=⋃j=1ijcred_≤ i:= _j=1^icred_j, so that ≤0=∅cred_≤ 0= . The success rate, recall, and precision are J(πask,) J( _ask;O) :=[V(θ,s^)], :=E[V(θ, s)], Rec(πask,) ( _ask;O) :=[|≤n||Amb|], :=E [ |cred_≤ n||Amb| ], Prec(πask,) ( _ask;O) :=[|i:i⊈≤i−1|n|n≥1]. :=E [ |\i:cred_i _≤ i-1\|n\; |\;n≥ 1 ]. Expectations are over episodes of (πask;)( _ask;O), that is, over θ∼Θθ and the draws of the policies, matcher, and responder. Recall is defined when |Amb|≥1|Amb|≥ 1, and precision when Pr(n≥1)>0 (n≥ 1)>0; for an AVT with |Amb|=0|Amb|=0, the conditions on πfull _full in Definition 3 are taken to hold trivially. Definition 3. Fix a full-disambiguation policy πfull _full satisfying Rec(πfull,⋆)=Prec(πfull,⋆)=1Rec( _full;O )=Prec( _full;O )=1, and a no-disambiguation policy πnone _none that stops immediately without asking any questions. The recall condition makes πfull _full acquire every ambiguity, and the precision condition makes every one of its questions productive, so that JfullJ_full does not depend on how many unproductive questions the policy asks; neither condition restricts how the questions are worded. Their full-disambiguation baseline and no-disambiguation baseline under an oracle O are Jfull():=J(πfull,),Jnone():=J(πnone,).J_full(O):=J( _full;O), J_none(O):=J( _none;O). A.3.2 Leakage Definition 4. The J-leakage and recall leakage are LeakJ(πask,) _J( _ask;O) :=J(πask,)−J(πask,⋆), :=J( _ask;O)-J( _ask;O ), LeakRec(πask,match) _Rec( _ask;match) :=Rec(πask,match,resp⋆)−Rec(πask,⋆). :=Rec( _ask;match,resp )-Rec( _ask;O ). A.3.3 Leakage diagnostics Definition 5. The responder-leakage diagnostic of an asking policy πask _ask on an oracle =(match,resp)O=(match,resp) is ΔJresp(πask,match,resp):=J(πask,match,resp)−J(πask,match,resp⋆). J_resp( _ask;match,resp):=J( _ask;match,resp)-J( _ask;match,resp ). Definition 6. Fix a distribution Qx∈(Text)Q_x (Text) over task specifications. The specification-randomized policy πaskQ _ask^Q samples x′∼Qxx Q_x independently of θ once per episode, then feeds πask _ask every live history with its initial x replaced by x′x . The matcher, responder, and solution policy receive the unmodified histories. The specification-randomization diagnostic is ΔRecspecQ(πask,match):=Rec(πaskQ,match,resp⋆). _spec^Q( _ask;match):=Rec( _ask^Q;match,resp ). Definition 7. Fix an adversarial full-disambiguation policy πfulladv _full^adv, which satisfies Rec(πfulladv,⋆)=Prec(πfulladv,⋆)=1Rec( _full^adv;O )=Prec( _full^adv;O )=1, and an adversarial no-disambiguation policy (or generic policy) πnoneadv _none^adv, which satisfies Rec(πnoneadv,⋆)=0Rec( _none^adv;O )=0. Their questions are worded adversarially, that is, subject to these conditions, to elicit leakage from the actual oracle: undeserved credit for πnoneadv _none^adv, and responses revealing more than the credited resolutions for πfulladv _full^adv. The precise wordings we use are in Section A.4. The adversarial diagnostics are the full-disambiguation J-leakage diagnostic and the no-disambiguation recall-leakage diagnostic, ΔJfull():=LeakJ(πfulladv,),ΔRecnone(match):=LeakRec(πnoneadv,match). J_full(O):=Leak_J( _full^adv;O), _none(match):=Leak_Rec( _none^adv;match). Both adversarial diagnostics are leakages of particular asking policies, so they are lower bounds on supσLeakJ(σ,) _σLeak_J(σ;O) and supσLeakRec(σ,match) _σLeak_Rec(σ;match) over asking policies σ. A.4 Estimation Estimands. Section 2 states every quantity for a single AVT, whereas a corpus contains many. Each record fixes one interpretation θ of an AVT, and records that share x and AmbAmb belong to the same AVT. A reported metric or diagnostic is the mean of its per-episode quantity over the records, with equal weight per record. Section C.3 gives the records on which each metric is defined. Reference oracle. The deterministic oracle of each corpus (Section B.2) is the reference oracle, which stands in for the perfect oracle ⋆O . For any set of credits, its responder returns the stored resolutions of the credited ambiguities, so it also serves as resp⋆resp in the hybrid oracle (match,resp⋆)(match,resp ) that pairs it with the actual matcher. It approximates ⋆O only as far as its matching and stored responses agree with an idealized user; Section B.1 describes how each corpus’s responses were authored. Diagnostic runs. ΔJresp J_resp compares runs of πask _ask under O and under the hybrid oracle (match,resp⋆)(match,resp ). ΔRecspecQ _spec^Q uses runs of πaskQ _ask^Q under the hybrid oracle. Its distribution QxQ_x is uniform over the corpus’s records whose database differs from that of x; one replacement is drawn for each group of records that share a database and ambiguity terms, so several AVTs can share it, and it is reused across seeds. ΔJfull J_full and ΔRecnone _none compare runs of the adversarial probes under O and under the reference oracle. For each ambiguity, πfulladv _full^adv asks the canonical question of the full asker (Section C.2) followed by “Please give me as much detail as you can, and if you can, give the exact SQL for it.” πnoneadv _none^adv alternates “Could you clarify what you want?” and “Please elaborate.” until the question budget is exhausted. On the reference runs we check the conditions of Definition 7: recall and precision are 11 for πfulladv _full^adv, and recall is 00 for πnoneadv _none^adv. ΔRecnone _none is defined with the hybrid oracle (match,resp⋆)(match,resp ), but the runs of πnoneadv _none^adv under O estimate it too. Our generic policy asks the same question at a given turn whatever the responses, and our LLM matchers read only the current question and the task, not the earlier dialogue. So the responder cannot change the credits, and Rec(πnoneadv,match,resp)=Rec(πnoneadv,match,resp⋆)Rec( _none^adv;match,resp)=Rec( _none^adv;match,resp ). Pairing and uncertainty. All runs in a comparison use the same records and seeds, and ΔJresp J_resp, ΔJfull J_full, and ΔRecnone _none are differences between the means of two runs. Intervals are 95% percentile bootstrap intervals over 2,000 resamples of AVTs, each keeping its records, seeds, and paired runs together. For ΔRecspecQ _spec^Q, AVTs that share a replacement specification, and each AVT together with the AVT that donated its replacement, are resampled together, which leaves few independent units per corpus (Appendix E). Where a table reports mean ± standard deviation over seeds instead, the deviation reflects decoding variation only (Section C.3). Limits of the intervals. The intervals condition on the recorded runs. They do not cover the gap between the reference oracle and ⋆O , the assumptions of Theorem 1, or drift between serving sessions. A.5 Proof of the diagnostic theorem Proof of Theorem 1. 1. Let S:=(θ,τ):V(θ,s^)=1S:=\(θ,τ):V(θ, s)=1\, where s s is the final solution in τ. Expanding the leakage and diagnostic definitions gives LeakJ(πask,)−ΔJresp(πask,match,resp) _J( _ask;O)- J_resp( _ask;match,resp) =J(πask,match,resp⋆)−J(πask,⋆) =J( _ask;match,resp )-J( _ask;O ) =Pmatch(S)−Pmatch⋆(S). =P_match(S)-P_match (S). Taking absolute values and applying the definition of total variation proves the claim. 2. For an asking policy σ, write s(σ):=Rec(σ,match,resp⋆),r(σ):=Rec(σ,⋆),s(σ):=Rec(σ;match,resp ), r(σ):=Rec(σ;O ), so that LeakRec(σ,match)=s(σ)−r(σ)Leak_Rec(σ;match)=s(σ)-r(σ). Identity. By definition, ΔRecspecQ(πask,match)=s(πaskQ) _spec^Q( _ask;match)=s( _ask^Q). Hence ΔRecspecQ(πask,match)−LeakRec(πaskQ,match)=r(πaskQ)=Rec(πaskQ,⋆). _spec^Q( _ask;match)-Leak_Rec( _ask^Q;match)=r( _ask^Q)=Rec( _ask^Q;O ). Bound. Fix an oracle, either (match,resp⋆)(match,resp ) or ⋆O , and a replacement specification x′x . We couple an episode of πask _ask with an episode of πaskQ _ask^Q under this oracle. Both episodes use the same θ and start from ℎ0=(x)hist_0=(x). Draw independent coins C0,…,CB−1∼Bernoulli(p)C_0,…,C_B-1 (p). At a history with k<Bk<B questions, each run draws its action from πinv _inv if Ck=1C_k=1 and from π′π otherwise, evaluated at the history the run feeds to πask _ask. Since πask=pπinv+(1−p)π′ _ask=p\, _inv+(1-p)\,π , each run has its correct action distribution. At a history with B questions, both runs stop. Suppose both runs are at the same history ℎhist with k<Bk<B questions, and Ck=1C_k=1. The histories they feed to πask _ask differ only in the initial specification. Since πinv _inv is invariant under replacing the task specification, it gives both runs the same action distribution, so both runs can use one shared draw. The matcher and responder receive the unmodified history ℎhist in both runs, so both runs can also share their draws. The two runs then append the same question, credits, and response. By induction, on the event E:=C0=⋯=CB−1=1E:=\C_0=…=C_B-1=1\ the two runs produce the same dialogue and the same credits. So their per-episode recalls agree on E. Per-episode recall lies in [0,1][0,1], so the two expected recalls differ by at most Pr(Ec)=1−pB (E^c)=1-p^B. This holds for every x′x , and therefore also after averaging over x′∼Qxx Q_x. Under the two oracles, this gives |s(πask)−s(πaskQ)|≤1−pB,|r(πask)−r(πaskQ)|≤1−pB.|s( _ask)-s( _ask^Q)|≤ 1-p^B, |r( _ask)-r( _ask^Q)|≤ 1-p^B. Therefore |LeakRec(πask,match)−LeakRec(πaskQ,match)| |Leak_Rec( _ask;match)-Leak_Rec( _ask^Q;match)| ≤|s(πask)−s(πaskQ)|+|r(πask)−r(πaskQ)| ≤|s( _ask)-s( _ask^Q)|+|r( _ask)-r( _ask^Q)| ≤2(1−pB). ≤ 2(1-p^B). ∎ Appendix B Benchmark construction and implementation B.1 Dataset construction Table 4 gives the evaluated pools and Table 5 gives their construction and selection rules. Table 4: Evaluated pools and registered ambiguities. Counts include 49 zero-ambiguity AmbiTab-Bird controls and 7 AmbiTab-BIRD-Interact records without a registered ambiguity. Corpus Pool Records DBs With ambiguities Ambiguities AmbiTab-Bird train 537 42 496 798 dev 127 11 119 198 AmbiTab-EW-Bird train 2,091 67 2,091 4,417 dev 446 11 446 911 AmbiTab-BIRD-Interact all 600 22 593 1,755 AmbiTab-Ambrosia all 2,670 763 2,670 2,670 AmbiTab-AmbiQT all 1,722 172 1,722 1,722 AmbiTab-TACO all 187 16 187 197 Table 5: Source-to-record conversions. The right column states the principal selection rule and limit for each evaluated pool. Corpus Construction Selection and scope AmbiTab-Bird Construct alternative readings and executable alternative SQL for BIRD tasks, then use Qwen to rewrite the task specification and supply fixed resolutions (Li et al., 2023). Require a Qwen3-8B reference-result miss without clarification and a match with all resolutions, followed by a blind Gemma-3-27B necessity judgment; retain zero-ambiguity controls. AmbiTab-EW-Bird Split BIRD evidence on semicolons, turn each hint into a masked knowledge ambiguity, and withhold it from the visible prompt (Li et al., 2023). Keep a row if Qwen3-Coder-30B-A3B or Gemma-3-27B fails without evidence and succeeds with it; evaluate only retained ambiguity-bearing rows, with source train and validation pooled as train and source test as dev. AmbiTab-BIRD-Interact Retain source task specifications, SQL, PostgreSQL databases, and annotations (Huo et al., 2026); normalize duplicate terms, repair one reviewed missing fragment, and freeze an audited catalog of LLM-authored natural-language responses. Response authoring uses the request, schema, reference SQL, and annotations; the full 600-record pool is evaluated, with local database-disjoint slices also available. AmbiTab-Ambrosia Convert each reading in the official test partition to a record with its native intent and SQL (Saparina & Lapata, 2024); derive terms from scope, attachment, or vagueness annotations and copy the SQLite databases. Exclude few-shot examples; local database-disjoint train and test slices are selected within each ambiguity type toward a 20% test target, and both derive from the upstream test partition. AmbiTab-AmbiQT For the column-synonym subset, build a SQLite database containing both alternatives, copying original values to one column and resampling the other with replacement (Bhaskar et al., 2023). Reject protected key columns and pairs whose SQL readings fail or return equal results; emit both hidden intents for each of 861 retained source items, with the copied column assigned by a per-unit draw. AmbiTab-TACO Deterministically verbalize US single-table SELECT queries while withholding table or literal-value choices (Deng et al., 2026). Keep executable, nonempty queries with a result-distinct table alternative; 187 of 5,159 source files survive, while value alternatives are not separately required to change the result. AmbiTab-EW-Bird’s filter measures benefit at the row level, not the necessity of each hint. AmbiTab-Bird’s admission and judgment are model dependent, and AmbiTab-TACO’s table choices can retain year-based clues. The AmbiTab-BIRD-Interact response catalog is audited, but auditing does not establish that its responses agree with those of the perfect oracle. B.2 Oracle implementations Table 6 separates the frozen-response oracles from the evaluated LLM oracles, whose access may exceed the intended boundary of Figure 1b. Table 6: Evaluated oracle variants and their runtime access. Variant Corpora Matcher and responder Deterministic intent AmbiTab-BIRD-Interact, AmbiTab-Ambrosia, AmbiTab-AmbiQT, AmbiTab-TACO Match terms and aliases, preferring quoted targets and then earliest mentions; return the selected frozen response verbatim, with refusals for unmatched questions or ties. Deterministic knowledge AmbiTab-Bird, AmbiTab-EW-Bird Rank terms and evidence-derived concepts, prefer unresolved matches, and return the stored resolution; repeats can restate evidence. Scoped LLM AmbiTab-BIRD-Interact, AmbiTab-Ambrosia, AmbiTab-AmbiQT, AmbiTab-TACO Classify from routing fields and respond from retained annotations; the responder sees all retained annotations and receives reference SQL when SQL grounding is used. Unfiltered LLM AmbiTab-Bird, AmbiTab-EW-Bird Classify and respond with the task context, withheld evidence, and reference SQL available. The full-disambiguation baseline uses a scripted full asker that poses one canonical question per registered ambiguity to the evaluated oracle, then passes the resulting dialogue to the solver (Section C.2). The oracle’s responses are generated through the same interaction as for the evaluated asker; they are not perfect-oracle samples by definition. Appendix C Inference protocol and complete results C.1 Evaluation conditions and populations We evaluate the same task records under the no-disambiguation, evaluated-asker, and full-asker conditions described in Section 4.1. Success is the binary verifier outcome: task-specific tests where supplied, otherwise execution-result agreement with the reference SQL under the corpus’s comparison rules. The evaluated corpus pools contain the 8,380 records in Table 4. We pool the train and development splits of AmbiTab-Bird and AmbiTab-EW-Bird for these descriptive benchmark comparisons; the training-policy evaluations instead report their development splits separately. AmbiTab-BIRD-Interact includes all 600 records, AmbiTab-Ambrosia the full ambiguous partition of Ambrosia, and AmbiTab-AmbiQT the column-synonym subset of AmbiQT. Table 7: Inference settings shared across corpora unless explicitly varied. Setting Value Asker Qwen3-8B Solver Qwen3-Coder-30B-A3B-Instruct, fixed Oracle Deterministic, Qwen3-14B, or Gemma-3-27B Default asking prompt Checklist (ledger) Asker and solver decoding Temperature 0.7; top-p=0.8p=0.8 (trained askers in Section D.5: top-p=0.95p=0.95) Qwen3-14B oracle decoding Temperature 0.7; top-p=0.8p=0.8; top-k=20k=20 Gemma-3-27B oracle decoding Temperature 1.0; top-p=0.95p=0.95; top-k=64k=64 Thinking Disabled Evaluation seeds Five for oracle comparisons and the default prompt; three for the other prompts and the leakage diagnostics Seed assignment Asker and solver share the run seed; the LLM oracle has no explicit decoding seed Clarification budget |Amb|+3|Amb|+3 decisions, including a final stopping decision, when |Amb|>0|Amb|>0 Unambiguous records Skip clarification SQL samples / repair rounds One / zero Oracle matching mode Single-answer mode; no multi-answer override For a request with m>0m>0 registered ambiguities, the m+3m+3 decision budget allows at most m+2m+2 questions because the final decision is reserved for stopping. Within a run, all conditions use the same solver and inference server. We compute asking gains and gaps to full disambiguation within each run before averaging over seeds. C.2 The full-disambiguation reference Full disambiguation uses a dialogue with the full asking policy πfull _full. The scripted policy asks about each distinct registered ambiguity term once, in annotation order, using the canonical question “Could you clarify what exactly you mean by <term>?” It stops after the final question, and the same solver reads the resulting transcript to generate SQL. The registered terms determine which questions the reference policy asks; their resolutions are supplied through the oracle’s responses. Both dialogue policies use the original task record, the same oracle implementation, and the same per-item budget. Withheld knowledge remains hidden unless the oracle reveals it in the dialogue, and the solver does not receive the resolutions directly. An imperfect oracle can refuse or misroute a canonical question, in which case the ambiguity remains unresolved in the transcript. The full asker therefore targets every registered ambiguity without guaranteeing perfect measured recall against every evaluated oracle. This is consistent with Jfull()J_full(O) in Section 2, which evaluates the reference asking policy under the actual oracle. Its SR is not a universal upper bound on other asking policies, whose wording and resulting dialogue can differ. We compute the signed gap Jfull()−J(πask,)J_full(O)-J( _ask;O) within each evaluation run; Figure 3(a) retains negative gaps rather than truncating them to zero. In Figure 3(b), the evaluated asker’s measured macro recall is shown with a hatched gap to the complete-coverage reference of 100%100\%. The reference describes complete coverage; it does not replace the full asker’s measured credits under an imperfect oracle. All reported full-disambiguation SRs are measured from the dialogue between the full asker and the evaluated oracle. One exception affects both dialogue conditions: under both LLM oracles, the 162 AmbiTab-EW-Bird training records on the databases works_cycles and movie_3 exceed the oracle server’s 16k-token context, end before the first question, and count as failures with zero recall credit, whereas the deterministic oracle’s evaluated asker solves 95 to 112 of them per seed. On the 2,375 AmbiTab-EW-Bird records that every oracle completes, evaluated-asker SR is 72.9%72.9\% (deterministic), 69.5%69.5\% (Qwen3-14B), and 69.9%69.9\% (Gemma-3-27B), and full-disambiguation SR is 81.4%81.4\%, 78.9%78.9\%, and 75.0%75.0\%. C.3 Metric definitions and aggregation Macro precision and recall. The offline scorer maps each recorded oracle routing action to a registered ambiguity. An exact labeled term takes precedence; otherwise it matches annotated terms against the question, response, and recorded routing rationale. Only critical and knowledge ambiguities count. Recall is the fraction of distinct registered terms credited over the episode, and precision is the fraction of questions receiving a previously unseen credit. We macro-average defined episode scores, omitting precision when no question was asked and recall when no ambiguity was registered. Thus the eligible populations differ: pooled AmbiTab-Bird has 615 ambiguity-bearing records out of 664, and AmbiTab-BIRD-Interact has 593 out of 600. The scorer also omits recall and F1 for fallback-stop episodes; this removes one AmbiTab-AmbiQT episode in each of two of the 90 oracle-comparison runs, and all other inference recall denominators equal the ambiguity-bearing populations. When shown, F1 is the mean of episode-level harmonic means, not the harmonic mean of macro precision and recall. These are credited-coverage measures: oracle wording can affect them, including by volunteering a resolution that the question did not name. Leakage diagnostics. Table 8 uses the four diagnostics from the three-seed campaign in Appendix E, and Figure 3(d–g) shows all four as bars. ΔJresp J_resp and ΔJfull J_full are SR differences and ΔRecnone _none is a recall difference, all in percentage points; their deterministic-oracle entries in Table 8 are shown as dashes because the deterministic oracle serves as the reference, making these differences zero by definition. ΔRecspecQ _spec^Q is raw mismatch action-credit recall in percentage points, so the deterministic oracle retains its measured reference recall. The diagnostic values reproduce Table 14, including negative SR differences, without subtracting the deterministic reference from ΔRecspecQ _spec^Q. Pooling and uncertainty. Within each inference seed, we pool corpus splits using the eligible denominator of each metric, then report the mean and sample standard deviation across seeds. The error bars describe evaluation-seed variation, not uncertainty from sampling new tasks or training new policies. In Figure 3, SR, precision, and recall use five inference seeds, while leakage uses the separate three-seed diagnostic campaign, without pairing seeds between campaigns. The populations also differ: the leakage campaign evaluates the development splits of AmbiTab-Bird (127 records) and AmbiTab-EW-Bird (446) and a 534-record AmbiTab-Ambrosia, whereas the inference columns pool 664, 2,537, and 2,670 records; the other three corpora use the same records in both campaigns. The correlations in Figure 6 use the 18 corpus–prompt means, subtracting each corpus’s own mean from both variables before computing Pearson correlation; lines are ordinary least-squares fits to each corpus’s three prompt means, and asking gain is computed relative to each run’s own no-disambiguation SR. This centering removes between-corpus mean differences; the remaining associations are descriptive, not estimates of causal effects. C.4 Complete oracle comparison Table 8 reports the complete oracle comparison and leakage diagnostics that Figure 3 plots. The no-disambiguation column retains only the middle (Qwen3-14B) row for each corpus; asker and full-asker success rates remain oracle-specific. The four leakage diagnostics reproduce the three-seed results of Table 14, alongside the five-seed inference results. Table 8: Inference and leakage across oracles, mean± . Inference: % (5 seeds); leakage: percentage points (3 seeds; Table 14) on the Bird and EW-Bird development splits and the 534-record Ambrosia dataset. No dis.: Qwen3-14B baseline; underlines: within-corpus maxima. EW-Bird LLM-oracle rows count 162 training records that exceed the oracle’s 16k-token context as failures (Section C.4). Dashes: zero deterministic leakage; otherwise unavailable. Success rate Asking Leakage diagnostics Corpus Oracle No dis. Asker Full dis. Precision Recall ΔJresp J_resp ΔJfull J_full ΔRecnone _none ΔRecspecQ _spec^Q AmbiTab-Bird Deterministic 66.7± 1.4 86.2± 0.7 33.0± 0.4 74.9± 0.8 – – – 3.5± 0.2 Qwen3-14B 26.3± 0.7 68.7± 0.9 88.0± 0.8 32.6± 0.6 72.1± 0.9 3.9± 1.4 2.4± 4.1 7.8± 1.0 0.0± 0.0 Gemma-3-27B 75.3± 1.0 87.8± 0.4 39.5± 0.3 87.7± 0.7 5.8± 0.9 3.1± 2.4 74.1± 0.2 0.8± 0.4 AmbiTab-EW-Bird Deterministic 72.4± 0.3 80.7± 0.4 34.7± 0.1 71.5± 0.2 – – – 2.9± 0.3 Qwen3-14B 48.2± 0.4 65.0± 0.8 73.9± 0.5 28.5± 0.3 55.7± 0.5 3.4± 0.2 2.8± 2.2 0.8± 0.3 0.2± 0.0 Gemma-3-27B 65.4± 0.6 70.2± 0.1 38.7± 0.2 74.7± 0.4 -4.5± 1.8 3.6± 0.4 61.0± 0.5 1.1± 0.2 AmbiTab-BIRD-Interact Deterministic 9.7± 0.3 10.1± 0.7 32.1± 0.2 57.7± 0.5 – – – 2.1± 0.2 Qwen3-14B 7.9± 0.5 8.9± 0.6 10.0± 0.4 29.8± 0.3 53.7± 0.6 -0.4± 0.7 -0.1± 0.9 1.9± 0.1 0.1± 0.0 Gemma-3-27B 9.3± 0.3 10.2± 0.8 35.7± 0.2 64.4± 0.3 0.3± 0.0 0.5± 0.0 42.7± 0.2 11.0± 0.3 AmbiTab-Ambrosia Deterministic 35.0± 0.2 42.1± 0.2 24.0± 0.2 71.7± 0.5 – – – 0.5± 0.3 Qwen3-14B 22.2± 0.3 41.2± 0.4 42.1± 0.5 25.9± 0.2 77.3± 0.6 4.0± 1.7 2.9± 0.9 54.9± 0.6 7.2± 0.6 Gemma-3-27B 46.2± 0.6 43.3± 0.5 32.0± 0.1 95.4± 0.3 8.2± 2.8 3.4± 0.8 100.0± 0.0 53.2± 0.5 AmbiTab-AmbiQT Deterministic 56.3± 0.7 73.7± 0.3 19.5± 0.2 58.3± 0.5 – – – 13.1± 0.3 Qwen3-14B 33.5± 0.8 57.7± 0.4 74.6± 0.3 20.4± 0.1 60.7± 0.3 1.6± 0.5 -0.3± 0.2 0.3± 0.1 9.9± 0.3 Gemma-3-27B 69.3± 0.7 73.0± 0.6 31.1± 0.0 93.0± 0.3 2.9± 0.7 0.4± 0.2 100.0± 0.0 70.9± 0.4 AmbiTab-TACO Deterministic 72.8± 1.5 83.2± 1.0 29.1± 0.4 85.1± 0.8 – – – 1.0± 0.6 Qwen3-14B 23.9± 1.4 74.1± 0.8 83.7± 1.1 28.9± 0.3 84.2± 1.3 7.0± 1.9 1.1± 1.1 20.5± 1.1 0.7± 0.6 Gemma-3-27B 79.9± 2.4 83.3± 0.7 33.4± 0.3 97.6± 0.7 0.7± 0.6 0.4± 1.6 97.3± 0.0 49.5± 1.8 C.5 Prompt-driven precision–recall tradeoff With the deterministic oracle and solver fixed, we compare three asker prompts. Direct asks about unclear parts of the request, while Checklist (chat) and Checklist (ledger) first enumerate potentially ambiguous terms. The checklist variants differ in dialogue representation and stopping instructions (Section C.6). Both checklist prompts increase recall and reduce precision relative to Direct on every dataset (Figure 6a). We use Checklist (ledger) by default because it has the highest mean recall across datasets. Higher recall is associated with greater asking gain across prompts, though not for every prompt pair (Figure 6b). This precision–recall tradeoff motivates training the asker to improve both. Figure 6: Prompt tradeoffs under the deterministic oracle: (a) precision versus recall; (b) asking gain versus recall. Colors identify datasets and shapes identify prompts; error bars show seed standard deviations. Lines fit each dataset’s three prompt means; correlations remove dataset means (Section C.3). C.6 Clarification prompts and complete prompt comparison The three strategies use the same question budget, deterministic oracle, and solver. Table 9 summarizes their instructions and dialogue history. Checklist (ledger) is the default for the oracle and trained-policy comparisons. Checklist (chat) differs from it in both dialogue representation and stopping instructions, so the comparison does not isolate the contribution of either change. Table 9: Clarification strategies. All prompts request at most one clarification question per turn. Strategy Instructions and history Direct Ask about unclear parts of the request without an explicit checklist; stop when the request is clear or no different ambiguity remains to be raised. Checklist (chat) Enumerate potentially ambiguous terms; read prior turns as native chat messages; stop when all terms are resolved or no distinct unasked term remains. Checklist (ledger) Enumerate potentially ambiguous terms; read the original request and explicit lists of questions with resolved and unavailable responses; continue while an ambiguity remains open, subject to the budget. Table 10 reports the full prompt comparison. Only the evaluated asker uses the varied clarification prompt; the no-disambiguation policy and scripted full asker are unchanged, so their SR columns are omitted from this table. The default strategy reuses its five-seed deterministic-oracle evaluation; the other strategies each use three seeds. The mean rows average each metric equally across the six corpora. Table 10: Clarification-prompt comparison under the deterministic oracle and otherwise the setup of Table 8, which the Checklist (ledger) rows repeat. Corpus entries are mean± over 5 seeds for Checklist (ledger) and 3 for Direct and Checklist (chat); rates are percentages. Mean rows average the six corpus means equally, without SD; bold marks the highest mean recall. All prompts share the same turn budget; AmbiTab-Bird and AmbiTab-EW-Bird pool training and development splits. The two control arms do not read the prompt and are omitted. All rows of a corpus run the same instances; Table 1 gives their counts. Success rate Asking Corpus Prompt Asker Precision Recall AmbiTab-Bird Direct 56.6± 0.4 75.1± 0.9 62.4± 0.9 Checklist (chat) 66.1± 0.8 65.5± 0.5 73.2± 0.8 Checklist (ledger) 66.7± 1.4 33.0± 0.4 74.9± 0.8 AmbiTab-EW-Bird Direct 68.3± 0.6 75.6± 0.1 60.2± 0.3 Checklist (chat) 71.3± 0.2 69.2± 0.5 69.7± 0.3 Checklist (ledger) 72.4± 0.3 34.7± 0.1 71.5± 0.2 AmbiTab-BIRD-Interact Direct 9.6± 0.9 59.2± 0.9 41.5± 0.3 Checklist (chat) 9.2± 0.4 52.8± 0.5 53.2± 0.4 Checklist (ledger) 9.7± 0.3 32.1± 0.2 57.7± 0.5 AmbiTab-Ambrosia Direct 34.4± 0.5 57.5± 0.4 67.3± 0.2 Checklist (chat) 36.2± 0.6 44.4± 0.1 73.1± 0.3 Checklist (ledger) 35.0± 0.2 24.0± 0.2 71.7± 0.5 AmbiTab-AmbiQT Direct 55.9± 0.5 47.8± 0.6 53.4± 0.6 Checklist (chat) 57.1± 0.9 43.1± 0.4 60.4± 0.5 Checklist (ledger) 56.3± 0.7 19.5± 0.2 58.3± 0.5 AmbiTab-TACO Direct 72.5± 1.2 80.4± 1.6 82.7± 1.5 Checklist (chat) 73.6± 1.3 69.1± 0.3 84.7± 1.2 Checklist (ledger) 72.8± 1.5 29.1± 0.4 85.1± 0.8 Mean Direct 49.5 65.9 61.3 Checklist (chat) 52.3 57.4 69.1 Checklist (ledger) 52.2 28.7 69.9 Appendix D Training the asking policy: implementation and additional results This appendix specifies the rewards, optimization settings, and evaluation protocol for the main question-F1 policy and ablations in Section 4.4. We distinguish native validation, which uses the adapted Qwen3-8B model to generate SQL, from the common inference evaluation, which pairs each asker with the fixed Coder-30B solver. The distinction matters because improvements in the native solver can change success rate without improving clarification. D.1 Question-F1 reward and trajectory-level credit Our main configuration trains Qwen3-8B on the 537-request AmbiTab-Bird training split with the deterministic oracle and the question-F1 reward of Equation 2. We use a multi-turn GRPO variant with DAPO-style asymmetric clipping and zero-advantage group filtering (Shao et al., 2024; Yu et al., 2025). For a trajectory τ as in Section 2, let n be its number of questions and use the credited ambiguity sets i⊆Ambcred_i and ≤i=⋃j=1ijcred_≤ i= _j=1^icred_j from Definition 2. When n>0n>0 and |Amb|>0|Amb|>0, define the episode precision and recall terms p(τ)=|i∈1,…,n:i⊈≤i−1|n,r(τ)=|≤n||Amb|.p(τ)= |\i∈\1,…,n\:cred_i _≤ i-1\|n, r(τ)= |cred_≤ n||Amb|. (1) These are the quantities averaged in the theory’s PrecPrec and RecRec. The question-F1 reward takes their harmonic mean within each episode, with explicit conventions for requests without ambiguities and trajectories without productive questions: RF1(τ)=1,|Amb|=0 and n=0,2p(τ)r(τ)p(τ)+r(τ),|Amb|>0,n>0,p(τ)+r(τ)>0,0,otherwise.R_F1(τ)= cases1,&|Amb|=0 and n=0,\\ 2p(τ)r(τ)p(τ)+r(τ),&|Amb|>0,\ n>0,\ p(τ)+r(τ)>0,\\ 0,&otherwise. cases (2) Thus RF1R_F1 is not the harmonic mean of the population metrics PrecPrec and RecRec. The annotation-based transcript scorer supplies the credits against the registered ambiguities. A request with ambiguities receives zero reward if no question is productive, including when the asker stops immediately. A request without ambiguities receives one for abstaining and zero for asking, which defines the training reward where recall is undefined; native rollouts give such requests a single decision, meant for stopping, so the zero applies to a question emitted instead. Repeated and irrelevant questions reduce p(τ)p(τ) without increasing r(τ)r(τ); a stop decision does not count toward n. Ablation objective. The outcome reward uses the theory’s interpretation-dependent verifier and the final submitted solution s s: RSR(θ,τ)=V(θ,s^)∈0,1.R_SR(θ,τ)=V(θ, s)∈\0,1\. (3) Its expectation is the success rate J(πask,)J( _ask;O) for the solver used during training. Question-F1 training ends after clarification, whereas outcome-reward training also generates and verifies SQL. Changing the training oracle can change both the dialogue and the credits supplied by the transcript scorer. Trajectory-level advantages. For each request, we sample K=16K=16 independent trajectories τ1,…,τK _1,…, _K and standardize their rewards, with R denoting the chosen reward above and suppressing the fixed interpretation θ for outcome reward: A^k=R(τk)−R¯sR+10−6,R¯=1K∑k=1KR(τk),sR2=1K−1∑k=1K(R(τk)−R¯)2. A_k= R( _k)- Rs_R+10^-6, R= 1K _k=1^KR( _k), s_R^2= 1K-1 _k=1^K (R( _k)- R )^2. (4) Each retained clarification output, including a stop decision, receives its trajectory’s advantage. Oracle responses and SQL outputs carry no loss. Groups with constant reward, or reward standard deviation at most 10−610^-6, receive zero advantage and are filtered before the actor update. If every group has zero advantage, one complete group is retained to avoid an empty training batch; only the regularization terms below then contribute to the loss gradient. Per-turn policy objective. Let j index retained clarification decisions in an optimizer batch ℬB, let k(j)k(j) identify the trajectory containing decision j, and let TjT_j be its number of unmasked output tokens. For output token yjty_jt with dialogue and token prefix hjth_jt, define the probability ratio and sampled reference penalty as ρjt(ϕ)=πϕ(yjt∣hjt)πold(yjt∣hjt),djtk2(ϕ)=12(logπϕ(yjt∣hjt)πref(yjt∣hjt))2, _jt(φ)= _φ(y_jt h_jt) _old(y_jt h_jt), d_jt^k2(φ)= 12 ( _φ(y_jt h_jt) _ref(y_jt h_jt) )^2, (5) where ϕφ denotes trainable policy parameters, πold _old is the pre-update actor, and πref _ref is the frozen reference policy. Writing Hjt(ϕ)H_jt(φ) for the entropy of πϕ(⋅∣hjt) _φ(· h_jt), the minimized loss is ℒ(ϕ) (φ) =1B∑j∈ℬ1Tj∑t=1Tjℓjt(ϕ), = 1B _j 1T_j _t=1^T_j _jt(φ), (6) ℓjt(ϕ) _jt(φ) =−minρjt(ϕ)A^k(j),clip(ρjt(ϕ),0.8,1.28)A^k(j) =- \! \ _jt(φ) A_k(j),\,clip\! ( _jt(φ),0.8,1.28 ) A_k(j) \ +0.1djtk2(ϕ)−0.01Hjt(ϕ). +0.1d_jt^k2(φ)-0.01H_jt(φ). The batch normalization uses the nominal decision-sample batch size B=512B=512, including for partial final batches. Tokens are averaged within each decision, and decision losses are summed without normalizing by the number of decisions in a trajectory; dialogues with more retained decisions therefore contribute more terms. Task Specification“List the top earners.”FrozenOracle Oholds the ambiguities and their resolutionsearners↦ + bonus.top↦ 3.RolloutTurn 1Turn 2Turn 3RF1R_F1A Aτ1 _1preciseRank earners by?Salary + bonus.What is top?Top 3.τ2 _2repeatsRank earners by?Salary + bonus. Earnersby salary?That ambiguity wasalready answered.What is top?Top 3.τ3 _3off-targetWhich year?Sorry, I can’t helpwith that.Ascending order?Sorry, I can’t helpwith that.0mean?Asking Policyπask _asksamples K rolloutsPer-turn updates of the asking policy Figure 7: Asking-policy training with a multi-turn GRPO variant with DAPO-style asymmetric clipping and zero-advantage group filtering. The main question-F1 reward RF1R_F1 balances productive questions and ambiguity coverage; every retained asking decision receives its trajectory’s normalized advantage. Three trajectories illustrate the update; training uses K=16K=16. D.2 Hyperparameters and shared model parameters Table 11 reports the training hyperparameters. For the main configuration with the deterministic oracle and question-F1 reward, the selected checkpoint follows 40 rollout batches. Each rollout batch samples eight requests and 16 trajectories per request. Each retained model decision becomes a separate training sample, with a nominal global batch size of 512 samples, partial final batches, and dynamic token packing. With zero-advantage filtering, each rollout batch retained at most one nominal batch of decision samples, so every rollout batch produced exactly one optimizer update. The learning-rate schedule assumes four decisions per trajectory; this estimate does not cap the number of retained decisions. The asker and native SQL solver share the same frozen Qwen3-8B base and trainable LoRA adapter. Masking SQL outputs out of the loss does not freeze the solver: updates from clarification tokens also change the adapter used for SQL generation. Table 11: Asker-training hyperparameters. Oracle and reward choices vary as specified in Section D.3. Setting Value Initialization / backend Qwen3-8B / FSDP Optimizer AdamW, (β1,β2)=(0.9,0.98)( _1, _2)=(0.9,0.98), ϵ=10−8ε=10^-8 Learning rate / schedule Peak 5×10−55× 10^-5; cosine decay to 0, 5% linear warmup Schedule horizon 136 optimizer updates, estimated from four decisions per trajectory Weight decay / gradient clipping 0 / 1.0 LoRA rank / alpha / dropout 32 / 64 / 0 (adapter scaling α/r=2α/r=2) LoRA targets Attention q/k/v/o and feed-forward gate/up/down projections Requests per rollout / group size 8 / 16 trajectories per request Rollout / nominal training batch 128 trajectories / 512 retained decision samples Policy ratio clipping [0.8,1.28][0.8,1.28] Loss aggregation Token mean per retained decision; sum divided by nominal batch size Group filtering Zero-advantage groups dropped; retain one group if all are zero Reference-policy penalty Sampled k2, coefficient 0.1 Entropy coefficient 0.01 Policy decoding Temperature 0.8; thinking disabled Main reward Question F1 (RF1R_F1; Equation 2) Main training oracle Deterministic; no oracle LLM is instantiated LLM-oracle ablation Qwen3-14B; temperature 0; thinking disabled Rollout context / response ceiling 32,768 / 256 tokens Clarification prompt Checklist (ledger); question budget visible Question budget At most |Amb|+2|Amb|+2 questions for |Amb|>0|Amb|>0, then a stop decision Checkpoint / validation cadence Every eight rollout batches; initial validation Native validation samples One per request Native rollouts on requests with registered ambiguities allow |Amb|+3|Amb|+3 decisions, reserving the final decision for stopping. On requests without registered ambiguities, only a stopping decision is allowed. Question-F1 training skips SQL generation. D.3 Ablation design and checkpoint selection We compare the main policy trained with question-F1 reward and the deterministic oracle with two outcome-reward ablations, one using the deterministic oracle and one using Qwen3-14B as the oracle. All three use AmbiTab-Bird and its 127-request development split, with one training seed per configuration. Every configuration updates clarification outputs only. We evaluate the designated checkpoints: iteration 40 for question-F1 reward, 16 for outcome reward with the deterministic oracle, and 40 for outcome reward with the LLM oracle. These checkpoints are fixed for the reported comparison rather than selected by a common validation-score rule. All RL learning curves and ablation figures use a shared plotting window through iteration 48. This window contains zero-based rollout batches 0-47, which precede checkpoint 48, and native evaluations through checkpoint 48. All three configurations have complete common-inference evaluations across three seeds on all six corpora. The untrained reference retains its existing five seeds. The AmbiTab-Bird development split was used to inspect training and checkpoints, so its scores are development results, not held-out test estimates. Because every rollout batch produced one optimizer update, the shared window corresponds to 48 updates per configuration and the evaluated checkpoints to 40, 16, and 40 updates. D.4 Native learning curves and objective mismatch Figures 8, 9, 10 and 12 separate changes in clarification quality from changes in native SQL success. In Figure 8, training reward averages one reward per logged trajectory before groups without training signal are filtered; the solid curve is a debiased exponential moving average with weight 0.95. Iteration 0 is the run’s native evaluation before training. Native checkpoints use the Qwen3-8B solver and native validation protocol; the untrained reference uses the fixed Coder-30B solver and common inference protocol. The reference uses the 127-request AmbiTab-Bird development slice of the deterministic-oracle inference runs underlying Table 8, which pools training and development requests in its displayed AmbiTab-Bird row. The dotted reference lines show the development-only mean across five seeds. Common-solver results appear in Table 2. Figure 8: Learning curves through iteration 48 for the main deterministic-oracle configuration with question-F1 reward RF1R_F1: (a) training reward, (b) native AmbiTab-Bird development recall, and (c) SR. Faint and solid lines show mean trajectory rewards per rollout batch and their smoothed trend; rings mark the selected checkpoint at iteration 40. The blue band in (a) extends from the maximum smoothed reward down by two sample standard deviations of the per-batch mean rewards. Dotted lines show the untrained development-only inference mean across five seeds, using the deterministic-oracle runs underlying Table 8. Native and reference evaluations use different protocols and solvers. In Figure 9(a-c), the trained checkpoints use the deterministic validation oracle and adapted Qwen3-8B solver. Every row shows differences from the same untrained Qwen3-8B asker reference on the 127-request AmbiTab-Bird development split, averaged over all five evaluation seeds from the deterministic-oracle inference runs underlying Table 8. Gray reference bands show one sample standard deviation on either side of the reference mean, using the same development-only reference as Figure 8. Colored row bands show a two-SD tolerance: each extends from that row’s maximum trained-checkpoint value down by twice the corresponding reference standard deviation. The reference uses the common inference protocol and fixed Coder-30B solver, whereas the trained checkpoints use native validation; these differences therefore include protocol effects, and SR also includes the solver difference. Table 13 provides the comparison under the common inference protocol. Colored points represent the available trained checkpoints, and rings identify the designated evaluation checkpoint for each configuration. The main policy trained with question-F1 reward appears first, followed by the ablations. Row labels give the numbers of trained checkpoints and rollout batches; RSRR_SR denotes the outcome reward. Each configuration contributes one training run, so the checkpoint comparisons are descriptive. Figure 9: The main policy trained with question-F1 reward and the oracle/reward ablations through iteration 48: native AmbiTab-Bird development checkpoints relative to the five-seed untrained inference reference. (a-c) Recall, precision, and SR differences in percentage points. Rings mark the designated evaluation checkpoints, including iteration 16 for outcome reward with the deterministic oracle. Gray shading shows ± one sample standard deviation across the five development-only reference seeds from the deterministic-oracle runs underlying Table 8. Colored row bands span [max−2SD,max][ -2\,SD, ], using each row’s maximum trained-checkpoint value and the corresponding reference SD. Trained checkpoints use native validation with the Qwen3-8B solver; the reference uses the common inference protocol with the fixed Coder-30B solver. (d) Fraction of logged request groups with nonconstant trajectory rewards before group filtering, pooling counts over the same 48 rollout batches for each configuration. At iteration 40, the main policy trained with question-F1 reward reaches 83.3%83.3\% native recall and 60.6%60.6\% native SR, compared with 70.4%70.4\% and 54.3%54.3\% at initialization. With outcome reward, the deterministic-oracle policy at iteration 16 reaches 73.0%73.0\% recall and 57.5%57.5\% SR; the LLM-oracle policy at iteration 40 reaches 70.2%70.2\% recall and 57.5%57.5\% SR. Outcome reward depends on the solver’s ability to use the responses; F1 balances question efficiency and ambiguity coverage. The fixed-solver evaluation in Table 13 measures the trained asking policies under a common solution policy and protocol. The two evaluations need not rank checkpoints in the same order. The outcome-reward runs provide the training-oracle comparison under the common inference protocol. There is no LLM-oracle F1 run in this ablation set. One training seed per configuration does not establish a general ranking of objectives or training oracles. Dataset choice matters. A separate outcome-reward run on AmbiTab-BIRD-Interact crashed before completing training. Its trajectory failed to sustain improvements in clarification quality over the shown window (Figure 10), illustrating that the training dataset matters for learning to ask. The run uses the LLM oracle and its 120-request native validation split; the shown window extends through iteration 48. The columns show reward, training recall, training precision, and optimization loss. Faint lines show rollout batches; solid lines use a debiased exponential moving average with weight 0.95. Reward averages retained decision samples after constant-reward groups are filtered, so it is not the unfiltered rollout SR. Native recall falls from 66.8%66.8\% at iteration 8 to 59.4%59.4\% at the end of the shown window, iteration 48, where native SR remains low at 6.7%6.7\%. Few request groups provide differing binary rewards, limiting the within-request reward contrast available for policy updates. The curves document the shown training window but do not identify the cause of the later crash. This run also used a larger response budget and a different schedule estimate than the current AmbiTab-Bird runs, so it does not isolate the effect of the training corpus. We retain this run only as an illustration of unsuccessful training; it is excluded from the configuration and results tables. Figure 10: Outcome-reward training dynamics on AmbiTab-BIRD-Interact through iteration 48. Training recall and precision decline over the window, and binary rewards provide limited within-request contrast. Reward contrast. Figure 9(d) reports the fraction of request groups whose available trajectory rewards are not all equal, with 16 trajectories sampled per request. Counts are logged before group filtering and pooled across the same 48 rollout batches for each configuration; row labels give the number of batches. Such groups supply the within-request reward contrast used for policy updates, whereas constant-reward groups have zero policy-gradient advantage (Equation 4). These fractions describe reward contrast over the shared plotting window. They measure available reward contrast rather than question quality. Table 12: Native validation before training and at the selected checkpoint, using the adapted Qwen3-8B solver. Rates are percentages. All rows use AmbiTab-Bird dev (n=127n=127). Question-F1 reward is the main configuration, trained with the deterministic oracle; the remaining trained policies are ablations. Selection and oracle settings are given in Sections D.3 and D.5. Policy SR Precision Recall F1 Question-F1 reward (untrained) 54.3 32.2 70.4 43.7 Question-F1 reward 60.6 36.8 83.3 50.6 Outcome reward (deterministic oracle) (untrained) 55.9 31.0 68.1 42.2 Outcome reward (deterministic oracle) 57.5 33.3 73.0 45.3 Outcome reward (LLM oracle) (untrained) 52.8 30.6 68.1 41.9 Outcome reward (LLM oracle) 57.5 31.8 70.2 43.4 D.5 Common inference protocol and aggregation Fixed solver and oracle. Table 13 evaluates the trained askers with the deterministic oracle and a fixed Qwen3-Coder-30B-A3B-Instruct solver, using Checklist (ledger), temperature 0.7, and no SQL debugging. The main and complete tables use the same designated checkpoints; the question-F1 policy and both outcome-reward ablations each have three completed evaluation seeds. The question budget matches native rollouts on requests with registered ambiguities; requests without ambiguities skip clarification entirely. Native validation instead generates SQL with the adapted Qwen3-8B policy at temperature 0.8. On AmbiTab-Bird dev, its oracle is deterministic on the 119 requests with registered ambiguities and silent on the remaining eight, including for policies trained with the LLM oracle. AmbiTab-BIRD-Interact native validation uses the LLM oracle. The untrained rows in Table 12 report each run’s own iteration-0 native validation with the untrained Qwen3-8B solver and its single recorded training seed. Untrained reference and uncertainty. The untrained reference uses the Qwen3-8B asker with the same clarification protocol, except for nucleus sampling: its asker samples at top-p=0.8p=0.8, whereas the trained-policy evaluations leave the asker at the serving default of top-p=0.95p=0.95; asker temperature, top-k=20k=20, solver decoding, prompt, budget, and oracle are identical. Its scores retain all five evaluation seeds from the deterministic-oracle runs used in Table 8, taking the development split alone for AmbiTab-Bird and AmbiTab-EW-Bird. Trained-policy cells average all three evaluation seeds; Table 13 also reports their sample standard deviations. These seeds vary inference for one trained policy per configuration; they do not measure variation across training runs. All gains subtract the five-seed untrained mean from the trained policy’s three-seed mean, including the differences in Table 2. Table 8 instead pools each of these two corpora’s train and development splits. Eligible populations. SR averages V(θ,s^)V(θ, s) over requests. Precision and recall are macro averages of the episode quantities in Definition 2, omitting precision when n=0n=0 and recall when |Amb|=0|Amb|=0. Their denominators can differ: for example, AmbiTab-Bird dev recall uses 119 of 127 requests. Reported F1 is the average of episode-level F1, not the harmonic mean of macro precision and recall. AmbiTab-BIRD-Interact pools its full dataset within each evaluation seed using each metric’s eligible denominator. AmbiTab-Ambrosia uses the full ambiguous split of Ambrosia, and AmbiTab-AmbiQT uses the column-synonym subset of AmbiQT. The Mean column in Table 13 equally weights all six evaluation families, AmbiTab-Bird dev, AmbiTab-EW-Bird dev, AmbiTab-BIRD-Interact, AmbiTab-Ambrosia, AmbiTab-AmbiQT, and AmbiTab-TACO, instead of pooling requests. For the family mean, we first average all six families within each evaluation seed, then report the mean and sample standard deviation of these per-seed averages over all three trained-policy seeds or all five untrained-reference seeds. D.6 Complete inference results Table 13 reports complete three-seed evaluations of the main policy trained with question-F1 reward and both outcome-reward ablations across all six corpora. The main-text summary and appendix use the same runs and checkpoints. For outcome-reward training, the policy trained with the deterministic oracle at iteration 16 has six-corpus means of 29.2%29.2\% precision, 70.7%70.7\% recall, and 51.7%51.7\% SR, compared with 29.2%29.2\%, 70.9%70.9\%, and 51.9%51.9\% for the policy trained with the LLM oracle. Both outcome-reward ablations fall below the question-F1 policy on these three means, while differences between the two training oracles are small and vary across corpora. Table 13: Training and transfer with a fixed Coder-30B solver and deterministic oracle. Question-F1 reward is the main configuration, trained with the deterministic oracle; the remaining trained policies are ablations. Question-F1 reward uses iteration 40; Outcome reward (deterministic oracle) uses iteration 16; Outcome reward (LLM oracle) uses iteration 40. Selected trained policies use all three evaluation seeds; the untrained baseline retains five seeds. Rates are percentages, reported as means with sample SD in subscripts. Mean equally weights all six families within each seed, using only seeds complete across all six. Bold values mark the highest mean in each column within each metric, including ties. AmbiTab-Bird and AmbiTab-EW-Bird use dev; AmbiTab-AmbiQT uses column synonyms. Dashes denote unavailable results. Protocol details appear in Sections D.3 and D.5. SQL success Policy AmbiTab- Bird AmbiTab- EW-Bird AmbiTab- BIRD-Interact AmbiTab- Ambrosia AmbiTab- AmbiQT AmbiTab- TACO Mean Untrained 59.5± 1.4 69.0± 0.8 9.7± 0.3 35.0± 0.2 56.3± 0.7 72.8± 1.5 50.4± 0.4 Question-F1 reward 71.9± 0.5 72.2± 1.0 9.7± 0.3 37.3± 0.2 59.4± 0.3 77.0± 1.4 54.6± 0.2 Outcome reward (deterministic oracle) 62.7± 1.2 71.1± 1.6 9.8± 0.1 35.5± 0.7 56.8± 0.2 74.3± 1.9 51.7± 0.4 Outcome reward (LLM oracle) 64.3± 1.2 71.4± 1.6 9.7± 0.6 35.9± 0.6 56.8± 0.5 73.4± 0.8 51.9± 0.6 Precision Policy AmbiTab- Bird AmbiTab- EW-Bird AmbiTab- BIRD-Interact AmbiTab- Ambrosia AmbiTab- AmbiQT AmbiTab- TACO Mean Untrained 31.0± 0.2 33.6± 0.5 32.1± 0.2 24.0± 0.2 19.5± 0.2 29.1± 0.4 28.2± 0.1 Question-F1 reward 37.0± 0.5 38.0± 0.3 36.4± 0.4 27.3± 0.1 21.7± 0.1 30.7± 0.3 31.8± 0.2 Outcome reward (deterministic oracle) 33.4± 0.4 34.6± 0.3 33.0± 0.7 24.6± 0.1 19.8± 0.2 29.7± 0.5 29.2± 0.2 Outcome reward (LLM oracle) 32.2± 0.7 35.8± 0.5 33.1± 0.4 24.5± 0.1 19.7± 0.1 29.8± 0.5 29.2± 0.2 Recall Policy AmbiTab- Bird AmbiTab- EW-Bird AmbiTab- BIRD-Interact AmbiTab- Ambrosia AmbiTab- AmbiQT AmbiTab- TACO Mean Untrained 67.7± 0.8 69.8± 1.1 57.7± 0.5 71.7± 0.5 58.3± 0.5 85.1± 0.8 68.4± 0.3 Question-F1 reward 83.4± 1.1 79.3± 0.6 65.1± 0.7 81.8± 0.2 65.0± 0.3 89.8± 0.8 77.4± 0.3 Outcome reward (deterministic oracle) 72.9± 0.5 72.0± 0.7 59.2± 1.4 73.7± 0.4 59.3± 0.6 87.0± 1.5 70.7± 0.3 Outcome reward (LLM oracle) 71.3± 1.7 74.7± 0.5 59.4± 0.6 73.5± 0.2 59.2± 0.3 87.3± 1.5 70.9± 0.5 Appendix E Leakage experiments All runs use the main evaluation settings: the Qwen3-8B asker with Checklist (ledger), the fixed Qwen3-Coder-30B-A3B-Instruct solver, the Qwen3-14B and Gemma-3-27B oracles, all six datasets, and three evaluation seeds (234 runs, 141,024 episodes). The datasets are the development splits of AmbiTab-Bird (127 records) and AmbiTab-EW-Bird (446), a 534-record AmbiTab-Ambrosia, and the full AmbiTab-BIRD-Interact, AmbiTab-AmbiQT, and AmbiTab-TACO pools. Levels are mean ± sample standard deviation over seeds; paired differences carry 95% AVT-cluster bootstrap intervals. All values are in percentage points. Untrained asker. Table 14 reports the four diagnostics. ΔJresp J_resp and ΔJfull J_full are SR differences; ΔRecspecQ _spec^Q is a recall and ΔRecnone _none a recall difference, both over the records with at least one ambiguity. The two LLM-oracle entries are not paired estimates of their difference. On AmbiTab-AmbiQT the reference recall of the specification-randomized policy is 13.1%, so ΔRecspecQ _spec^Q there carries a large reference term; differences between oracles are unaffected, because the term cancels. Table 14: Leakage diagnostics of the untrained asker, mean ± SD over three seeds. AmbiTab-Bird and AmbiTab-EW-Bird are development splits, and AmbiTab-Ambrosia has 534 records. Bold marks the larger of the two LLM-oracle values. For the deterministic oracle, which is the reference, ΔJresp J_resp, ΔJfull J_full, and ΔRecnone _none are 0 by definition, and ΔRecspecQ _spec^Q is the reference recall of the specification-randomized policy. Oracle Dataset ΔJresp J_resp ΔRecspecQ _spec^Q ΔJfull J_full ΔRecnone _none Qwen3-14B AmbiTab-Bird dev 3.94±1.363.94± 1.36 0.00±0.000.00± 0.00 2.36±4.092.36± 4.09 7.84±0.977.84± 0.97 AmbiTab-EW-Bird dev 3.36±0.223.36± 0.22 0.22±0.000.22± 0.00 2.77±2.152.77± 2.15 0.82±0.260.82± 0.26 AmbiTab-BIRD-Interact −0.44±0.67-0.44± 0.67 0.14±0.050.14± 0.05 −0.11±0.95-0.11± 0.95 1.88±0.101.88± 0.10 AmbiTab-Ambrosia 4.00±1.684.00± 1.68 7.24±0.577.24± 0.57 2.93±0.942.93± 0.94 54.87±0.5654.87± 0.56 AmbiTab-AmbiQT 1.61±0.471.61± 0.47 9.85±0.299.85± 0.29 −0.31±0.23-0.31± 0.23 0.35±0.100.35± 0.10 AmbiTab-TACO 6.95±1.936.95± 1.93 0.71±0.560.71± 0.56 1.07±1.071.07± 1.07 20.50±1.1120.50± 1.11 Gemma-3-27B AmbiTab-Bird dev 5.77±0.915.77± 0.91 0.84±0.420.84± 0.42 3.15±2.363.15± 2.36 74.09±0.2474.09± 0.24 AmbiTab-EW-Bird dev −4.48±1.78-4.48± 1.78 1.10±0.241.10± 0.24 3.59±0.453.59± 0.45 60.99±0.4760.99± 0.47 AmbiTab-BIRD-Interact 0.33±0.000.33± 0.00 11.04±0.2811.04± 0.28 0.50±0.000.50± 0.00 42.69±0.2142.69± 0.21 AmbiTab-Ambrosia 8.24±2.768.24± 2.76 53.25±0.4753.25± 0.47 3.37±0.823.37± 0.82 100.00±0.00100.00± 0.00 AmbiTab-AmbiQT 2.94±0.652.94± 0.65 70.94±0.4470.94± 0.44 0.37±0.190.37± 0.19 100.00±0.00100.00± 0.00 AmbiTab-TACO 0.71±0.620.71± 0.62 49.47±1.7549.47± 1.75 0.36±1.630.36± 1.63 97.33±0.0097.33± 0.00 Deterministic AmbiTab-Bird dev 0 3.50±0.243.50± 0.24 0 0 AmbiTab-EW-Bird dev 0 2.91±0.292.91± 0.29 0 0 AmbiTab-BIRD-Interact 0 2.13±0.242.13± 0.24 0 0 AmbiTab-Ambrosia 0 0.50±0.290.50± 0.29 0 0 AmbiTab-AmbiQT 0 13.09±0.3213.09± 0.32 0 0 AmbiTab-TACO 0 0.98±0.620.98± 0.62 0 0 E.1 Responder access to the reference SQL We repeat the experiments of Table 14 with one change: the LLM responder’s access to the full reference SQL. On AmbiTab-Ambrosia, AmbiTab-AmbiQT, and AmbiTab-TACO the default responder does not read it, and the new runs add it; on AmbiTab-EW-Bird and AmbiTab-Bird the default responder reads it and its clause segments, and the new runs remove both. The matcher is unchanged, so each difference compares two runs of the same policy under the actual oracle, paired by task: the untrained asker for ΔJresp J_resp, the adversarial full-disambiguation policy for ΔJfull J_full, and the generic policy for ΔRecnone _none. The new runs comprise 82 evaluation runs and 47,640 episodes; a few untrained-asker runs with the SQL shown are reused from an earlier study with the same settings. The two conditions come from different inference-server sessions. Table 15: Change in the leakage diagnostics when the LLM responder reads the full reference SQL, shown minus hidden, three seeds pooled, with paired 95% AVT-cluster bootstrap intervals. AmbiTab-Bird and AmbiTab-EW-Bird are development splits, and AmbiTab-Ambrosia is a 534-record. New run: the condition added for this study. Difference, reference SQL shown minus hidden Oracle Dataset New run ΔJresp J_resp ΔJfull J_full ΔRecnone _none Qwen3-14B AmbiTab-Bird dev SQL hidden 3.15 [−0.26-0.26, 6.56] 0.52 [−2.36-2.36, 3.67] 0.28 [0.00, 1.12] AmbiTab-EW-Bird dev SQL hidden 3.96 [1.42, 6.35] 2.84 [0.22, 5.61] 0.00 [−0.22-0.22, 0.22] AmbiTab-Ambrosia SQL shown 2.87 [0.88, 4.95] 4.99 [3.00, 7.02] 0.25 [−0.96-0.96, 1.52] AmbiTab-AmbiQT SQL shown 0.81 [−0.08-0.08, 1.72] 1.24 [0.43, 1.96] −0.14-0.14 [−0.29-0.29, −0.02-0.02] AmbiTab-TACO SQL shown −0.18-0.18 [−3.03-3.03, 2.50] −0.36-0.36 [−2.50-2.50, 1.96] −2.50-2.50 [−5.35-5.35, 0.00] Gemma-3-27B AmbiTab-Bird dev SQL hidden 7.09 [2.36, 12.07] 1.31 [−1.84-1.84, 4.72] −0.14-0.14 [−1.40-1.40, 1.12] AmbiTab-EW-Bird dev SQL hidden 3.59 [0.52, 6.73] 4.86 [2.47, 7.17] 0.36 [−0.39-0.39, 1.12] AmbiTab-Ambrosia SQL shown 7.05 [4.96, 9.24] 10.36 [7.91, 12.89] 0.00 [0.00, 0.00] AmbiTab-AmbiQT SQL shown 1.82 [0.83, 2.83] 1.34 [0.54, 2.17] 0.00 [0.00, 0.00] AmbiTab-TACO SQL shown 1.07 [−1.43-1.43, 3.57] −0.18-0.18 [−1.78-1.78, 1.43] 0.00 [0.00, 0.00] Showing the SQL raises ΔJfull J_full on AmbiTab-Ambrosia and AmbiTab-AmbiQT under both oracles, by 1.24 to 10.36 points, and ΔJresp J_resp under Gemma on both and under Qwen on AmbiTab-Ambrosia, by 1.82 to 7.05 points; all these intervals exclude zero. On AmbiTab-EW-Bird and AmbiTab-Bird, removing it lowers ΔJresp J_resp in all four dataset–oracle pairs, by 3.15 to 7.09 points, with intervals excluding zero in three, and lowers ΔJfull J_full on AmbiTab-EW-Bird under both oracles, by 2.84 and 4.86 points, so Table 14 includes this channel there. On AmbiTab-TACO every interval includes zero: no effect is detectable at this sample size. ΔRecnone _none, which depends only on the unchanged matcher, stays within 0.36 points of zero except for Qwen on AmbiTab-TACO, −2.50-2.50, whose interval reaches zero. Hidden SQL can still reach the responder through annotations, the schema, and matcher actions that quote an SQL segment. E.2 Trained asking policies Table 16 reports ΔJresp J_resp and ΔRecspecQ _spec^Q for the two iteration-40 policies of Section D.3, trained with question-F1 reward and with outcome reward (Qwen3-14B oracle), next to the untrained asker. They repeat the untrained asker’s runs with three seeds on the development splits of AmbiTab-Bird and AmbiTab-EW-Bird, the 534-record AmbiTab-Ambrosia (Table 2 uses all 2,670), AmbiTab-AmbiQT, and AmbiTab-TACO (240 runs, 144,768 episodes); AmbiTab-BIRD-Interact and the deterministic outcome policy are not evaluated. ΔJfull J_full and ΔRecnone _none do not depend on the asking policy. The runs behind each trained-policy ΔJresp J_resp share a server session, while the untrained-asker rows come from a different one. Table 16: Leakage diagnostics of two trained asking policies and the untrained asker, mean ± SD over three seeds. Under the deterministic oracle, ΔJresp J_resp is 0 by definition and ΔRecspecQ _spec^Q is the reference recall. AmbiTab-Bird dev AmbiTab-EW-Bird dev AmbiTab-Ambrosia AmbiTab-AmbiQT AmbiTab-TACO Oracle Asking policy ΔJresp J_resp ΔRecspecQ _spec^Q ΔJresp J_resp ΔRecspecQ _spec^Q ΔJresp J_resp ΔRecspecQ _spec^Q ΔJresp J_resp ΔRecspecQ _spec^Q ΔJresp J_resp ΔRecspecQ _spec^Q Deterministic Untrained 0 3.50±0.243.50± 0.24 0 2.91±0.292.91± 0.29 0 0.50±0.290.50± 0.29 0 13.09±0.3213.09± 0.32 0 0.98±0.620.98± 0.62 Question-F1 reward 0 4.76±0.244.76± 0.24 0 4.11±0.234.11± 0.23 0 0.94±0.000.94± 0.00 0 16.80±0.1516.80± 0.15 0 0.98±0.310.98± 0.31 Outcome reward (Qwen3-14B oracle) 0 3.64±0.243.64± 0.24 0 3.20±0.433.20± 0.43 0 0.62±0.110.62± 0.11 0 12.54±0.2112.54± 0.21 0 0.62±0.310.62± 0.31 Qwen3-14B Untrained 3.94±1.363.94± 1.36 0.00±0.000.00± 0.00 3.36±0.223.36± 0.22 0.22±0.000.22± 0.00 4.00±1.684.00± 1.68 7.24±0.577.24± 0.57 1.61±0.471.61± 0.47 9.85±0.299.85± 0.29 6.95±1.936.95± 1.93 0.71±0.560.71± 0.56 Question-F1 reward 3.41±3.283.41± 3.28 0.14±0.240.14± 0.24 1.94±3.191.94± 3.19 0.22±0.000.22± 0.00 2.87±0.842.87± 0.84 7.68±0.327.68± 0.32 0.79±0.820.79± 0.82 9.66±0.549.66± 0.54 4.99±1.114.99± 1.11 1.52±0.151.52± 0.15 Outcome reward (Qwen3-14B oracle) 7.35±4.487.35± 4.48 0.00±0.000.00± 0.00 3.66±0.723.66± 0.72 0.22±0.000.22± 0.00 5.06±0.755.06± 0.75 7.55±0.477.55± 0.47 1.66±1.061.66± 1.06 10.84±0.5310.84± 0.53 5.70±2.165.70± 2.16 1.34±0.271.34± 0.27 Gemma-3-27B Untrained 5.77±0.915.77± 0.91 0.84±0.420.84± 0.42 −4.48±1.78-4.48± 1.78 1.10±0.241.10± 0.24 8.24±2.768.24± 2.76 53.25±0.4753.25± 0.47 2.94±0.652.94± 0.65 70.94±0.4470.94± 0.44 0.71±0.620.71± 0.62 49.47±1.7549.47± 1.75 Question-F1 reward 6.30±1.366.30± 1.36 0.14±0.240.14± 0.24 −2.69±1.25-2.69± 1.25 1.18±0.161.18± 0.16 7.93±1.337.93± 1.33 58.99±1.2358.99± 1.23 −0.27±1.04-0.27± 1.04 81.98±0.9381.98± 0.93 −1.25±3.79-1.25± 3.79 50.45±2.3150.45± 2.31 Outcome reward (Qwen3-14B oracle) 7.61±1.647.61± 1.64 1.96±0.871.96± 0.87 0.07±1.960.07± 1.96 1.25±0.251.25± 0.25 8.18±1.688.18± 1.68 53.56±0.8653.56± 0.86 2.38±0.802.38± 0.80 73.11±0.5773.11± 0.57 −1.60±4.24-1.60± 4.24 46.52±1.6746.52± 1.67 Table 17 compares the two policies directly, outcome reward (Qwen3-14B oracle) minus question-F1 reward, paired by task and seed. The primary comparison, fixed before the runs, is null: ΔJresp J_resp under Qwen3-14B on AmbiTab-EW-Bird dev differs by 1.721.72 points, with interval [−1.87,5.23][-1.87,5.23]. The outcome-reward policy has the higher ΔJresp J_resp in 9 of 10 dataset–oracle pairs, but only 1 interval excludes zero (AmbiTab-AmbiQT under Gemma-3-27B). The ΔRecspecQ _spec^Q intervals carry little information, because the shared donor map leaves only 2 to 8 independent components to resample. Table 17: Outcome reward (Qwen3-14B oracle) minus question-F1 reward, paired by task and seed, three seeds pooled, with 95% AVT-cluster bootstrap intervals. Bold marks the primary comparison, fixed before the runs. Oracle Diagnostic AmbiTab-Bird dev AmbiTab-EW-Bird dev AmbiTab-Ambrosia AmbiTab-AmbiQT AmbiTab-TACO Qwen3-14B ΔJresp J_resp 3.94 [−2.36-2.36, 10.24] 1.721.72 [−1.87-1.87, 5.23] 2.18 [−0.86-0.86, 5.08] 0.87 [−0.48-0.48, 2.19] 0.71 [−2.85-2.85, 4.10] ΔRecspecQ _spec^Q −0.14-0.14 [−0.60-0.60, 0.00] 0.00 [0.00, 0.00] −0.12-0.12 [−0.72-0.72, −0.07-0.07] 1.18 [−2.05-2.05, 3.51] −0.18-0.18 [−1.85-1.85, 0.22] Gemma-3-27B ΔJresp J_resp 1.31 [−4.99-4.99, 6.82] 2.77 [−0.82-0.82, 6.21] 0.25 [−2.75-2.75, 3.17] 2.65 [1.32, 4.03] −0.36-0.36 [−3.92-3.92, 3.21] ΔRecspecQ _spec^Q 1.82 [0.44, 3.81] 0.07 [−0.52-0.52, 0.25] −5.43-5.43 [−6.42-6.42, 5.07] −8.87-8.87 [−12.49-12.49, −7.27-7.27] −3.92-3.92 [−16.67-16.67, −0.88-0.88] Appendix F Additional Related Work Ambiguous and interactive tabular tasks. AmbiQT, Ambrosia, and TACO represent alternative text-to-SQL interpretations (Bhaskar et al., 2023; Saparina & Lapata, 2024; Deng et al., 2026); SParC and CoSQL establish conversational evaluation (Yu et al., 2019b; Yu et al., 2019a). DialSQL and PIIA ask multiple-choice clarification questions (Gur et al., 2018; Li et al., 2020), while MISP models interactive parsing as an agent in a user environment (Yao et al., 2019). MMSQL, PRACTIQ, and CLARITY provide fixed clarification dialogues (Guo et al., 2025; Dong et al., 2025; Sarwar et al., 2026). BIRD-Interact and ABISS combine clarification with simulated users and executable SQL checks (Huo et al., 2026; Sullutrone et al., 2026); ODUTQA-MDC studies dialogue-based clarification for open-domain tabular QA (Wang et al., 2026). DySQL-Bench focuses on evolving user requests and follow-up database exploration (Sun et al., 2026a). Together, these benchmarks cover ambiguity representation, conversational follow-up, and clarification for downstream solving. Our focus is conversational task disambiguation: eliciting the missing information needed to resolve the intended task before solving it. Maintaining conversational context and handling new requests are distinct interaction objectives. Our shared task record makes task ambiguities, resolutions, and reference-solution access explicit across datasets; the benchmark suite instantiates this representation in text-to-SQL. Task formalization and clarification evaluation. Hidden goals and verifiable outcomes appear in τ-bench and τ2τ^2-bench (Yao et al., 2025; Barres et al., 2026); CoVe constructs interactive tasks from hidden constraints with deterministic verification (Chen et al., 2026). Active Task Disambiguation distinguishes consistent from acceptable solutions and selects questions by information gain under a truthful oracle (Kobalczyk et al., 2025); related approaches study when to clarify under a distribution of interpretations (Zhang & Choi, 2025), information-gain selection with annotated oracle responses (Qiu et al., 2025), and value-of-information scoring for tool parameters (Suri et al., 2026). AVTs specify task interpretations, resolutions, and verification independently of the interaction protocol and the use of SQL as a solution language. AmbigQA and ASQA evaluate answers or rewrites (Min et al., 2020; Stelmakh et al., 2022), while GATE and STaR-GATE assess elicitation through downstream outcomes (Li et al., 2025; Andukuri et al., 2024). IN3, AskBench, and π-Bench score questions against annotated missing information (Qian et al., 2024; Zhao et al., 2026; Zhang et al., 2026); ClariQ evaluates clarification retrieval (Aliannejadi et al., 2021). HiL-Bench hides blockers in BIRD and SWE-bench Pro tasks, evaluates blocked, full-information, and ask conditions, and trains with question precision and recall (Trinh et al., 2026). Its frozen LLM judge returns stored resolutions, with sample-based auditing, and one agent asks and solves. Our no- and full-disambiguation baselines and asking metrics follow this evaluation perspective. Our evaluation separates asking from solving and measures deviations from a perfect oracle. Oracle reliability and leakage. AutoEval-ToD separates simulation from solution information; BIRD-Interact and ABISS constrain or audit responses (Jain et al., 2025; Huo et al., 2026; Sullutrone et al., 2026). Hack-verifiable environments detect reward hacking separately from success (Roth et al., 2026). Simulator research studies policy-value gaps (Dann et al., 2026) and exploitation of simulator error during optimization (Muratore et al., 2018). LLM user simulations can misestimate human-user performance (Seshadri et al., 2026), and policies can overfit to frozen simulators (Yu et al., 2026). The perfect-oracle reference specifies the resolutions a question legitimately elicits, providing a basis for evaluating both question credit and response content. Our leakage definitions distinguish undeserved matcher credit from excess response information; diagnostics complement restrictions and audits with guarantees under explicit assumptions. Learning clarification policies. Execution rewards support SQL and table-reasoning RL (Pourreza et al., 2025; Ma et al., 2025; Lei et al., 2025b), with group-relative optimization providing training methods (Shao et al., 2024; Yu et al., 2025). Multi-turn methods address credit assignment and rollout stability (Zhou et al., 2025; Xue et al., 2026). ACT trains through conversations with an LLM oracle (Chen et al., 2025); SGP learns cost-conditioned clarification, answering, and enumeration through self-play using an LLM oracle (Berant et al., 2026). AskBench combines correctness and checkpoint coverage (Zhao et al., 2026), UserVille penalizes question effort (Sun et al., 2026b), and InfoPO adds turn-level information gain to outcome rewards (Kong et al., 2026). IntentRL instead trains interpretation and solution enumeration in one output (Saparina & Lapata, 2026). HiL-Bench combines terminal blocker coverage with a reward of +0.3+0.3 for productive questions and a penalty of 0.10.1 for irrelevant or repeated questions (Trinh et al., 2026). Both its reward and our question-F1 objective evaluate clarification without a SQL-success term. Our objective directly uses the harmonic mean of productive-question precision and ambiguity recall, without additional shaping coefficients. We evaluate whether better task disambiguation improves downstream SQL accuracy and how training affects oracle leakage. Appendix G Prompts Braced names such as db_schema mark fields filled for each task; Table 6 states what fills the oracle fields under each oracle variant. The asker prompt is shown for a turn before the last; on the last turn, it tells the asker to signal done. Asker. Checklist (ledger) is ours; its input layout follows the BIRD-Interact asker prompt (Huo et al., 2026). Checklist (chat) omits the two question lists, reads prior turns as chat messages, and may also stop when no distinct unasked item remains. Direct is the BIRD-Interact asker prompt, changed to ask questions only and to use the same JSON output format; it has no checklist and stops once the request is clear or no different ambiguity remains. You are a good data scientist with great sql_engine_name writing ability. You have a DB called "db_name". You are given a Text-to-SQL task whose user query is under-specified. ## Input Information: You will be provided with: - Task Description: The type of task you are trying to accomplish. - DB Schema Information: The detailed DB schema with data examples. - DB Column Meanings: The detailed DB column meanings explanation. - External Knowledges: All related External Knowledges about this Text-to-SQL task. - Text-to-SQL Question: The Text-to-SQL question of this Text-to-SQL task. Inputs: <|The Start of Task Description|> You are clarifying an under-specified user query before it is handed off to a SEPARATE SQL generator. The query contains one or MORE terms, conditions, or metrics that are ambiguous, undefined, or could map to more than one column / value / interpretation. Your job is to surface and resolve ALL of them, not just the first one. You have max_turn turns; each turn you may ask only ONE question. Method (apply every turn, before you answer): 1. **Build the checklist.** Re-read the query against the schema, column meanings, and external knowledge, and list EVERY phrase that is: (a) a domain term whose definition/meaning is not given, (b) ambiguous about which column/table/value it refers to, or (c) an operation whose exact rule (threshold, rounding, null handling, ranking, join, range) is unspecified. Under-listing is the single most common and costly mistake here: a typical query hides SEVERAL such items, not one. 2. **Cross off what is resolved.** Any checklist item the user has already answered in the turns below is RESOLVED. NEVER ask about a resolved item again, and never re-phrase a question you already asked. 3. **Ask one OPEN item.** Ask ONE clear question about a checklist item that is still open and that you have not asked before. Set type to "clarification". Example: "type": "clarification", "output": "[YOUR-ONLY-ONE-QUESTION]" 4. **Signal done only when the checklist is clear.** Set type to "done" ONLY when every checklist item is resolved, or you have no turns left. Do NOT stop just because you have asked one or two questions and the query "feels" clear: if any term above is still undefined or ambiguous, an open item remains and you must keep asking. Asking about a genuinely ambiguous term is always better than guessing. Example: "type": "done", "output": null NOTE: You are NOT generating SQL; a separate SQL generation model will handle that using the resolved conversation. Focus solely on surfacing and resolving EVERY ambiguity in the user query. <|The End of Task Description|> <|The Start of DB Schema Information|> db_schema <|The End of DB Schema Information|> <|The Start of DB Column Meanings|> ‘json column_meanings ‘ <|The End of DB Column Meanings|> <|The Start of External Knowledge|> ‘json external_kg ‘ <|The End of External Knowledge|> <|The Start of Text-to-SQL Question|> user_query <|The End of Text-to-SQL Question|> ### Already asked and RESOLVED (do NOT ask about any of these again): - You asked: question User answered: answer ### Already asked but NOT answered (the user could not answer these; do NOT ask about any of them again): - You asked: question User answered: answer ### Turn t (max_turn - t + 1 turns left): Ask about a checklist item that is still OPEN (not in either list above). Only signal done if every ambiguous or undefined term in the query has now been resolved or could not be answered. ## Output Format (STRICTLY REQUIRED): You MUST output ONLY ONE of the following two JSON structures: 1. If asking for clarification: "type": "clarification", "output": "[YOUR-ONLY-ONE-QUESTION]" 2. If done asking questions: "type": "done", "output": null Do NOT deviate from these formats. Do NOT generate SQL here. Oracle. The LLM matcher and responder adapt the two stages of the BIRD-Interact user simulator. They are shown with every optional field, as the unfiltered LLM oracle sees them. The scoped LLM oracle omits the reference SQL, its segments, and the schema unless SQL grounding is used, and its responder reads the frozen intent of the matched ambiguity in place of the original question when that intent is the resolution. The deterministic oracle uses no prompt. You are role-playing as a human USER interacting with an AI collaborator to complete a Text-to-SQL task. The AI collaborator may ask one question about this task. Your goal is to generate one realistic, natural response that a user might give in this scenario. ## Input Information: You will be provided with: - Task Description: The type of task you are trying to accomplish. - Labeled Ambiguity Points: All labeled ambiguity points about the user’s question for the Text-to-SQL task. - Ground-truth SQL Segments: All ground-truth SQL segments. - Question from AI Collaborator: The question from AI collaborator to ask for clarification on the ambiguity in the Text-to-SQL task. Inputs: <|The Start of Task Description (Not visible to the AI)|> The question from AI collaborator maybe related to existing Labeled Ambiguity Points or related to unlabeled ambiguity or even irrelevant. So, you should choose one action at this turn. Action Choices: 1. **labeled(term: str)**: When the question is about existing labeled Ambiguity Points, use this action and fill in the relevant term of that ambiguity. Format: **labeled("Amb")**. 2. **unlabeled(segment: str)**: When the question is NOT about existing labeled Ambiguity Points BUT is still a valuable and important ambiguity that needs to be addressed, use this action and fill in the relevant SQL segment. Format: **unlabeled("[RELEVANT_SQL_CLAUSE]")**. 3. **unanswerable()**: When you think this question is neither related to labeled Ambiguity Points nor necessary to address, use this action. Format: **unanswerable()**. <|The End of Task Description|> <|The Start of All Labeled Ambiguity Points (Not visible to the AI)|> ‘json user_query_ambiguity: user_query_ambiguity knowledge_ambiguity: knowledge_ambiguity ‘ <|The End of All Labeled Ambiguity Points|> <|The Start of Ground-truth SQL Segments (Not visible to the AI)|> sql_segments <|The End of Ground-truth SQL Segments|> <|The Start of Question from AI Collaborator|> ai_clarification_question <|The End of Question from AI Collaborator|> ## Guidelines: - You MUST choose only **one action** listed above. - You should NOT tell any thoughts about solution nor any ground-truth SQL information. - If you can do it well, you will get 10 thousand USD bonus! ## Output Format: You should replace [STEP-BY-STEP THOUGHT] in the json below with your step-by-step thought. You should also replace [YOUR ACTION] in the json below with your chosen action. You MUST output ONLY the following JSON structure: "action": "[YOUR ACTION]", "cot": "[STEP-BY-STEP THOUGHT]" Do NOT deviate from these formats. Do NOT use any other output format. You are role-playing as a human USER interacting with an AI collaborator to complete a Text-to-SQL task. The AI collaborator may ask one question about this task. Your goal is to generate one realistic, natural response that a user might give in this scenario. ## Input Information: You will be provided with: - Task Description: The type of task you are trying to accomplish. - DB Schema Informaion: The detailed DB schema with data examples. - Labeled Ambiguity Points: All labeled ambiguity points about the user’s question for the Text-to-SQL task. - Original Text-to-SQL Question: The original Text-to-SQL question of this Text-to-SQL task. - Ground-truth SQL: The whole ground-truth SQL of this Text-to-SQL task. - Ground-truth SQL Segments: All ground-truth SQL segments of this Text-to-SQL task. - Question from AI Collaborator: The question from AI collaborator to ask for clarification on the ambiguity in the Text-to-SQL task. - Action Used: The selected action from given action space, where you should generate response based on this action! Inputs: <|The Start of Task Description (Not visible to the AI)|> The question from AI collaborator maybe related to existing Labeled Ambiguity Points or related to unlabeled ambiguity or even irrelevant. So, one action was chosen at previous turn. Action Space: 1. **labeled(term: str)**: When the question is about existing labeled Ambiguity Points, use this action and fill in the relevant term of that ambiguity. Format: **labeled("Amb")**. 2. **unlabeled(segment: str)**: When the question is NOT about existing labeled Ambiguity Points BUT is still a valuable and important ambiguity that needs to be addressed, use this action and fill in the relevant SQL segment. Format: **unlabeled("ALTER")**. 3. **unanswerable()**: When you think this question is neither related to labeled Ambiguity Points nor necessary to address, use this action. Format: **unanswerable()**. Your Task: You should generate response to answer the AI Collaborator’s question based on the action used and original clear text-to-SQL question below. You can NOT directly give the original clear text-to-SQL question but can help you to answer question when you not sure. <|The End of Task Description|> <|The Start of DB Schema Information|> db_schema <|The End of DB Schema Information|> <|The Start of All Labeled Ambiguity Points (Not visible to the AI)|> ‘json user_query_ambiguity: user_query_ambiguity knowledge_ambiguity: knowledge_ambiguity ‘ <|The End of All Labeled Ambiguity Points|> <|The Start of Original Text-to-SQL Question|> clear_query <|The End of Original Text-to-SQL Question|> <|The Start of Ground-truth SQL (Not visible to the AI)|> ‘sql gt_sql ‘ <|The End of Ground-truth SQL|> <|The Start of Ground-truth SQL Segments (Not visible to the AI)|> sql_segments <|The End of Ground-truth SQL Segments|> <|The Start of Question from AI Collaborator|> ai_clarification_question <|The End of Question from AI Collaborator|> <|The Start of Action Chosen (Not visible to the AI)|> action <|The End of Action Chosen|> ## Guidelines: **Remember**: If you can do the following points well, you will get 10 thousand USD bonus! 1. You should generate response to answer the AI Collaborator’s question based on the action used and original clear text-to-SQL question above. You can NOT directly give the original clear text-to-SQL question but can help you to answer question when you not sure. 2. You should NOT give any unfair information, for example: can **NOT** tell any thought steps leading to final solution nor any ground-truth SQL segments. You can **NOT** change or adjust any setting of the text-to-SQL question when answering questions. The response should be concise. 3. You should NOT ask any question. ## Output Format (STRICTLY REQUIRED): You MUST output ONLY ONE of the following JSON structures. [action] must be taken from the action space above. "type": "[action]", "output": "[Fill-in-Your-Response]" For example, if the action is "unanswerable()", you should respond: "type": "unanswerable()", "output": "Sorry, this question is out of scope, so I can not answer your question." Do NOT deviate from these formats. Do NOT use any other output format. Solver. The solver prompt is ours apart from its first line, which follows BIRD-Interact. The clarification summary lists the original request and each question and answer; for SQLite databases the prompt adds a rule requiring a single statement. You are a good data scientist with great sql_engine_name writing ability. You have a DB called "db_name". ## Task Description: Based on the clarification summary and the information below, generate the final sql_engine_name statement that answers the user’s intent. Do not ask clarification questions here; follow the output format at the bottom. ## DB Schema Information: <|The Start of DB Schema Information|> db_schema <|The End of DB Schema Information|> ## DB Column Meanings: <|The Start of DB Column Meanings|> ‘json column_meanings ‘ <|The End of DB Column Meanings|> ## External Knowledges: <|The Start of External Knowledges|> ‘json external_kg ‘ <|The End of External Knowledges|> ## Text-to-SQL Question: <|The Start of Text-to-SQL Question|> user_query <|The End of Text-to-SQL Question|> <|The Start of Clarification Summary|> clarification_summary <|The End of Clarification Summary|> ## Output Format (STRICTLY REQUIRED): "reasoning": "[YOUR-REASONING]", "type": "final_sql", "output": "[YOUR-SQL]" Fill in the ‘reasoning‘ field FIRST: think step by step about schema linking, ambiguity resolution, and your join/filter plan before writing the final SQL in ‘output‘. Do NOT deviate from this format. Do NOT ask clarification questions here.