Paper deep dive
Querying Structured Data Through Natural Language Using Language Models
Hontan Valentin-Micu, Bunea Andrei-Alexandru, Tantaroudas Nikolaos Dimitrios, Popovici Dan-Matei
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/10/2026, 2:05:41 AM
Summary
The paper presents an open-source methodology for querying structured, non-textual datasets using natural language by fine-tuning a compact LLM (DeepSeek R1-Distill-8B) with QLoRA. The approach uses synthetic data generation to create question-answer pairs, enabling the model to generate executable queries for specialized databases, specifically demonstrated on accessibility data in Durangaldea, Spain.
Entities (5)
Relation Signals (3)
DeepSeek R1-Distill-8B â finetunedwith â QLoRA
confidence 100% · We fine-tune a compact modelâDeepSeek R1-Distill-8Bâusing QLoRA
German Aerospace Center (DLR) â createddataset â Durangaldea
confidence 95% · This dataset was collected and curated by the German Aerospace Center (DLR)
DeepSeek R1-Distill-8B â queries â Durangaldea
confidence 90% · We evaluate our approach on a dataset describing accessibility to essential services across Durangaldea
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper presents an open source methodology for allowing users to query structured non textual datasets through natural language Unlike Retrieval Augmented Generation RAG which struggles with numerical and highly structured information our approach trains an LLM to generate executable queries To support this capability we introduce a principled pipeline for synthetic training data generation producing diverse question answer pairs that capture both user intent and the semantics of the underlying dataset We fine tune a compact model DeepSeek R1 Distill 8B using QLoRA with 4 bit quantization making the system suitable for deployment on commodity hardware We evaluate our approach on a dataset describing accessibility to essential services across Durangaldea Spain The fine tuned model achieves high accuracy across monolingual multilingual and unseen location scenarios demonstrating both robust generalization and reliable query generation Our results highlight that small domain specific models can achieve high precision for this task without relying on large proprietary LLMs making this methodology suitable for resource constrained environments and adaptable to broader multi dataset systems We evaluate our approach on a dataset describing accessibility to essential services across Durangaldea Spain The fine tuned model achieves high accuracy across monolingual multilingual and unseen location scenarios demonstrating both robust generalization and reliable query generation Our results highlight that small domain specific models can achieve high precision for this task without relying on large proprietary LLMs making this methodology suitable for resource constrained environments and adaptable to broader multi dataset systems.
Tags
Links
- Source: https://arxiv.org/abs/2604.03057v1
- Canonical: https://arxiv.org/abs/2604.03057v1
Trouble viewing inline? Open PDF directly â
Full Text
44,314 characters extracted from source content.
Expand or collapse full text
Querying Structured Data Through Natural Language Using Language Models Valentin-Micu Hontan 1 , Andrei-Alexandru Bunea 1 , Nikolaos Dimitrios Tantaroudas 2 , and Dan-Matei Popovici 1 1 National University of Science and Technology POLITEHNICA Bucharest valentin.micu@stud.acs.upb.ro, andrei.bunea@stud.acs.pub.ro, matei.popovici@upb.ro 2 Institute of Communication and Computer Systems (ICCS), Athens, Greece nikolaos.tantaroudas@iccs.gr Abstract. This paper presents an open-source methodology for allow- ing users to query structured, non-textual datasets through natural lan- guage. Unlike Retrieval-Augmented Generation (RAG), which struggles with numerical and highly structured information, our approach trains an LLM to generate executable queries. To support this capability, we introduce a principled pipeline for synthetic training-data generation, producing diverse questionâanswer pairs that capture both user intent and the semantics of the underlying dataset. We fine-tune a compact modelâDeepSeek R1-Distill-8Bâusing QLoRA with 4-bit quantization, making the system suitable for deployment on commodity hardware. We evaluate our approach on a dataset describing accessibility to essen- tial services across Durangaldea, Spain. The fine-tuned model achieves high accuracy across monolingual, multilingual, and unseen-location sce- narios, demonstrating both robust generalization and reliable query gen- eration. Our results highlight that small, domain-specific models can achieve high-precision for this task without relying on large proprietary LLMs, making this methodology suitable for resource-constrained envi- ronments and adaptable to broader multi-dataset systems. Keywords: Natural Language querying· tool-using LLMs· model fine- tuning· Large Language Models 1 Introduction The reasoning capabilities of language models are inherently constrained by the patterns and information from the data they have seen during training. To over- come this constraint, a substantial body of research focuses on augmenting lan- guage models with external knowledge sources. A widely adopted such method is Retrieval-Augmented Generation (RAG) [11]. In a typical industrial RAG pipeline, when the model is prompted with a query, an external retrieval system searches a text corpus and returns the top- k most relevant documents. These retrieved fragments are then injected into the modelâs context â often combined with a system prompt that provides arXiv:2604.03057v1 [cs.CL] 3 Apr 2026 2Hontan et al. instructions on how the information should be used - enabling the language model to produce an accurate, data-backed answer. Similarity search in a RAG system operates by representing text fragments as vector embeddings in an n-dimensional space. A user query is encoded into the same space, and the system identifies the embeddings in the database that are most similar to it. Similarity is typically computed using the â 2 (Euclidean) distance, after which the plain-text associated with the nearest embeddings is retrieved. RAG has been successfully deployed in a wide range of applications, and most production-grade LLM-powered systems rely on variants of this approach. However, RAG is limited when a system must access specialized information that is not naturally expressed as unstructured textâfor example, numerical data or time-series. Consider the following query: Enumerate small towns that have good access to hospitals (in minutes by car) or to supermarkets (in minutes by bike). Such questions require precise, structured, and often updated information that cannot be reliably retrieved through text alone. Given a dataset capable of pro- viding the required information, we would like a system that can understand its structure, extract and reason about the relevant data, and use it in order to formulate a response. This capability is already feasible for LLMs with hundreds of billions of pa- rameters and context windows on the order of 100k tokens, as supported by many recent state-of-the-art commercial models. In such settings, the extensive context window allows developers to supply detailed descriptions of the dataset schema, access methods, API interfaces, and related metadata directly within the prompt. Data access formats have also been standardized through frame- works such as Anthropicâs Model Context Protocol (MCP) [2], an open protocol that enables AI systems to securely interact with external tools, data sources, and services in a structured and interoperable manner. However, the practicality and deployment of such systems is constrained by their reliance on extremely large language models. This limitation is twofold. First, many approaches depend on closed-source LLMsâsuch as those provided by OpenAI, Anthropic or xAIâs Grok, or Googleâs Geminiâwhich raises con- cerns regarding operational cost, data privacy, and control over the underlying infrastructure. Second, running open-source models at the hundred-billionâ to trillion-parameter scale is generally infeasible for most organizations due to pro- hibitive hardware requirements. Alternative strategies exist, such as generating SQL queries directly from natural language [19, 6], but these methods are syntax or query-specific and tend to achieve strong performance only when paired with very large prediction models. In this paper, we propose a fully open-source methodology for building LLM- based systems that can answer natural-language questions using specialized, non-textual data. Our approach introduces the following key contributions: (i) Title Suppressed Due to Excessive Length3 it relies on comparatively small modelsâincluding those in the 7-8B parameter rangeâwhile achieving high accuracy and (i) supporting a workflow that can be almost entirely automated. Our approach is based on model fine-tuning, which may require specialized hardware, but remains (i) cost-effective given the relatively small model sizes we target. We employ larger models, such as GPT-4, only during the training phase, not at deployment. Once trained, our system operates independently of any large or proprietary model. We apply our methodology to a dataset describing accessibility conditions in the Durangaldea region of northern Spain. This dataset was collected and curated by the German Aerospace Center (DLR) as part of the FUTURAL (Future Rural) project, which aims to bring data-driven intelligence to rural communities. Our system is currently being piloted to support stakeholders in exploring the dataset and extracting actionable insights, thereby contributing to Quality of Life assessments in the Durangaldea region. We further argue that our approach can be integrated into a multi-dataset system and, when paired with recent highly efficient million-parameter architectures such as Tiny Recursive Models [10], can be deployed on resource-constrained devices, including laptops. The remainder of this paper is organized as follows: Section 2 introduces the problem setting and outlines the overall methodology. Section 3 describes the construction of the dataset we used for training and evaluation. Section 4 illustrates the model fine-tuning procedure. Section 5 reports the evaluation metrics and experimental results. Section 6 analyzes the performance of the deployed application. Section 7 discusses limitations of the current approach and directions for future research. Section 8 reviews related work, and Section 9 concludes the paper. 2 Problem Setting and Methodology The DLR dataset we employ contains more than 100,000 records detailing geo- graphic coordinates and travel times (walking, cycling, and driving) to hospitals, supermarkets, and pharmacies across Durangaldea, Biscay (Basque Country, northern Spain), covering over 240 kmÂČ. It was developed for an ongoing study on how accessibility to essential services has changed in rural Durangaldea, ex- amining the relationship between demographic dynamics, settlement patterns, and walkability. Our goal is to build a natural-language interface to the dataset, enabling ac- cessibility experts and other stakeholders to query it interactively. We aimed for a scalable, easily deployable solution that generalizes beyond the Durangaldea data, decouples the model from the dataset (which may evolve), and remains agnostic to query structure and syntax, assuming only that data access is avail- able through a known query language or API. Additionally, we only considered a fully open, non-proprietary approach that does not rely on closed-source lan- guage models. Retrieval-Augmented Generation proved inadequate for numerical informa- tion, as embeddings do not reliably encode structured quantitative data. We 4Hontan et al. Fig. 1. Overview of the model inference and query generation process therefore explored a system in which the model generates executable queries, retrieves the corresponding data, and then reasons over the results. The overall functionality of our system is outlined in Figure 1. Instead of producing natural- language output directly, the model first generates a query based on the user question; the query is then executed; the returned data is fed back into the (user) prompt for answer generation. This approach, inspired by tool-use prompting methods such as those proposed in [17], requires an additional, detailed, system prompt describing the dataset schema and query syntax. While effective with very large models, our experiments showed that smaller models often produce syntactically incorrect or semantically irrelevant queries, limiting accuracy. Early query-generation approaches such as [15] rely on large training datasets to teach models when and how to generate queries. In such work, an external factual dataset is instrumented by removing selected pieces of knowledge and re- placing them with queryâanswer examples (e.g., âWhat is the current population of Berlin?â). During training, the model learns to recognize when information is missing and a query is required. After such a query is generated, text generation is paused, the query is executed, and the returned result is inserted into the context before generation resumes. This approach depends on the availability of high-quality training data con- taining diverse and accurate questionâanswer pairs. However, such datasets rarely exist for specialized or domain-specific databases, including the one considered in our work. To address this limitation, we propose a method for synthetic dataset creation, focusing on broad coverage of dataset values to support gen- eralization, while ensuring rich linguistic variation. Building an expert-level data retriever with LLMs is supported by various commercial solutions; however, our solution diverges from these approaches and is guided by two key constraints: (i) it must run on consumer-grade hard- ware. In this work we focus on the NVIDIA GeForce RTX 3090, a 24-GB Am- pere GPU suitable for inference and light training which is significantly more Title Suppressed Due to Excessive Length5 affordable than datacenter-class GPUs such as the A100; and (i) it must rely solely on open-source models. We use DeepSeek R1-Distill-8B [8], a compact distilled variant of the DeepSeek-R1 family. Distillation transfers reasoning ca- pabilities from a large model into a smaller one; in this case, the architecture is based on the 8-B Llama-3.1 Instruct model [1], fine-tuned on reasoning-oriented data. The resulting model is lightweight enough for deployment on the RTX 3090 while retaining strong problem-solving performance. We fine-tune our model through the LoRA (Low-Rank Adaptation) [4] adapter framework. LoRA introduces trainable low-rank matrices into selected weight layers of a frozen base model, allowing the system to learn domain-specific tasks without modifying the original model parameters. Adapters operate as lightweight, modular extensions that can be loaded or unloaded dynamically, enabling efficient specialization while preserving the underlying modelâs gen- eral linguistic and reasoning capabilities. This design makes training resource- efficientâonly a small number of additional parameters are trained. At the same time deployment remains lightweight, since adapters require minimal memory and can be swapped in almost instant time during inference. A central advantage of this architecture is its extensibility: a single base model can be paired with multiple adapters, each tailored to a different dataset or domain, thereby supporting multi-dataset systems without the need to duplicate the full model. We integrate the fine-tuned model into an application pipeline that combines geolocation services and a user-facing interface, enabling natural- language queries to be mapped onto structured queries and executed over the underlying dataset. 3 Dataset creation Our objective was to construct a training dataset containing questionâanswer pairs that accurately capture both user intent and the information present in the underlying data. An additional challenge was that the dataset did not ref- erence street addresses; instead, all locations were specified through geographic coordinates in the WGS-84 [9] coordinate system. Our dataset-generation procedure proceeds as follows. For each table T avail- able in the database, we consider all projections Ï A 1 ,...,A i (T) that capture se- mantically meaningful information. For every such projection, we construct a set of template questions designed to capture the types of information expressible from that attribute subset. Table 1 illustrates an example: a question instanti- ated from a template derived from a projection of the Hospitals table containing the attributes Location and Distance. For each identified projection, we use state of the art models (Gemini 2.5 Pro and DeepSeek R1) to generate templates that convey distinct semantic in- terpretations. We additionally include relevant superprojectionsâprojections p j whose attribute sets strictly contain those of p i whenever they give rise to ques- tions that reflect meaningfully different informational aspects. For instance, the 6Hontan et al. ProjectionÏ Location,Drive_Dist (Hospitals) Template What is the nearest location_type from location? QuestionWhat is the nearest hospital from Durango? Table 1. Generating question from projections question Is the nearest hospital closer by bike or on foot? is generated from a template p which is a superprojection of Ï Location (Hospitals). To ensure soundness, we manually inspect generated templates and filter- out invalid or duplicate templates. When instantiating each question template, we used the Overpass API [12] to extract a representative geographical sample consisting of 358 unique locations from the Durangaldea region. For every instantiated question, we also generated a corresponding correct answer, which includes the associated database query used to compute it, as illustrated in Figure 2. QuestionWhat is the nearest hospital from Durango by drive? Correct answer with query Theclosesthospitalyoucanfindis <API>get_closest_distance_time(category="hospital", mode="drive", location="Abadiño, Durango", metric_to_extract="distance") -> "distance": 0.402, "time": 0.537</API> 0.402km away. Fig. 2. Question-answer pair for model fine-tuning During inference, the model generates a call whenever it deems necessary. Once the call is produced, token generation is paused, the database query is invoked, and the returned value is inserted into the context. When generation resumes, the model has access to the complete callâresponse pair and can use it to formulate its final answer. For this reason, the training data must contain both the question and the resulting answer, ensuring the model learns how to incorporate the query output into a coherent and correct final response. Finally, to ensure sufficient linguistic variability, we employed state-of-the-art models (Gemini 2.5 Pro and DeepSeek R1) to generate syntactic paraphrases of the questions while preserving all location-specific information from the dataset. This step enables evaluation of the modelâs robustness to paraphrasing and its ability to retain the parameters required for accurate querying. The complete dataset thus created contains 44849 question answer pairs. 4 Model fine-tuning We fine-tuned the DeepSeek R1-Distill-8B model, which exhibits reasoning abil- ities exceeding those of typical models of comparable size. Effective query gener- Title Suppressed Due to Excessive Length7 Fig. 3. Evaluation metrics during fine-tuning. ation requires structured reasoning: recognizing when a query is necessary, iden- tifying the required arguments, and mapping question semantics to the appro- priate query. The modelâs strong latent reasoning capacity makes it well suited for learning these decision patterns [3]. Moreover, the model displays low vari- ance in output structure relative to similarly sized alternatives, improving query generation reliability. The model was fine-tuned on a single NVIDIA RTX 3090 GPU using the Transformer Reinforcement Learning library, specifically the SFTTrainer compo- nent, which provides an optimized framework for supervised fine-tuning of large language models. To reduce memory consumption and enable efficient training on commodity hardware, we employed 4-bit quantization [5], which compresses model weights while preserving core representational capacity. Fine-tuning was performed using QLoRA [4] adapters, allowing us to update only a small set of low-rank adaptation parameters: approximately 10M parameters were trained, rather than the full model. Training was done with a batch size of 32 examples and a cosine learning-rate scheduler, chosen for its smooth decay properties that support stable convergence during fine-tuning. The model was trained for four epochs, each requiring sev- eral hours of compute time. To mitigate overfitting, we periodically evaluated performance on a held-out validation set and saved checkpoints corresponding to the lowest validation loss. In addition, we implemented an Early Stopping cri- terion that halted training when no further improvement was observed, ensuring computational efficiency and preventing unnecessary parameter updates. Figure 3 illustrates the evolution of the evaluation loss (left) and the evalua- tion runtime (right) across training steps. The loss curve shows a clear downward trend during the early stages of training, dropping from approximately 0.031 to 0.024 by step 1500. This indicates that the model rapidly learns the task struc- ture during the initial epochs. After this point, the loss stabilizes, with only minor fluctuations; the slight increase observed at step 2500 suggests the on- set of overfitting, consistent with the Early Stopping criterion applied during training. 8Hontan et al. The evaluation runtime remains effectively constant throughout training, with variations of less than two seconds across all checkpoints. This stability reflects the fixed computational footprint of QLoRA-based fine-tuning: because only a small set of adapter parameters is updated, the computational cost of forward passes remains unchanged. The absence of runtime drift confirms that neither model degradation nor GPU resource saturation occurred during train- ing, further validating the reliability of the training setup. The final model is publicly available 3 , allowing anyone to reproduce the results. 5 Evaluation 5.1 Metrics We evaluated system performance using two standard NLP metrics: ROUGE- L and BLEU-4. These metrics provide complementary assessments of structural alignment and syntactic precision. ROUGE-L utilizes the Longest Common Sub- sequence (LCS) to quantify the overlap between reference (R) and generated (G) tokens. By focusing on sequence order rather than strict contiguity, ROUGE-L effectively captures the preservation of key informational unitsâsuch as query arguments and factual componentsâeven in the presence of paraphrasing. The metric is illustrated in Equation awhere LCS len is the length of the longest common subsequence, and |R| and |G| represent the respective lengths of the reference and generated sequences. BLEU-4 measures similarity via contiguous n-gram overlaps (1- to 4-grams), making it highly sensitive to syntactic accuracy. This sensitivity is critical for query generation, where minor deviations in punc- tuation or token order can significantly degrade output validity. As shown in Equation b, the score is calculated using the geometric mean of modified n-gram precisions p n . We assign equal weights (w n = 1/4) to each n-gram and apply a Brevity Penalty (BP) to penalize insufficiently long outputs. While BLEU-4 is less tolerant of paraphrasing than ROUGE-L, it serves as a rigorous proxy for the modelâs adherence to required syntactic structures. ROUGE-L = 2 LCS len |R|+|G| (a) BLEU-4 = BP Ă exp P 4 n=1 w n logp n (b) In practice, both metrics are computed for each generated response and then averaged across the evaluation set, providing different perspectives on semantic fidelity (ROUGE-L) and syntactic accuracy (BLEU-4). Apart from BLEU-4 and ROUGE we also look at the percentage of exact matches (EM), namely the proportion of answers in which the generated query matches the reference exactly (Equation 1), with no missing parameters, incor- rect values, misspellings or wrong parameter order. EMA = N exact N total Ă 100% (1) 3 https://huggingface.co/valy3124/durangaldea-assistantFinalPD Title Suppressed Due to Excessive Length9 Case / SubsetSizeExact Match (%)BLEU-4ROUGE-L Unseen Locations50089.00.990.98 Semantic Variants50094.20.990.99 Multilingual queries Spanish10081.00.970.95 Catalan10086.00.980.96 Basque10024.00.810.71 Galician10093.00.990.98 French10064.00.950.91 Full dataset evaluation330080.00.960.93 Full dataset evaluation (excluding multilingual cases) 280085.00.970.95 Table 2. Model accuracy 5.2 Evaluation results For evaluation, we reserved 2,800 queries from the training dataset as a held-out monolingual test set. In addition, we generated 500 multilingual queries using the same procedure described in Section 3. This evaluation design serves two purposes. First, it allows us to assess the modelâs ability to generalize across lan- guages and dialects. Although DeepSeek is pretrained on a multilingual corpus, our fine-tuning was performed exclusively in English; therefore, testing multi- lingual generalization provides insight into how much of the original linguistic capability is retained after adaptation. Second, multilingual support is of prac- tical importance, as real users in the Durangaldea region may interact with the application in different spoken languages. We also evaluated the model on entirely unseen geographic locations, isolat- ing this scenario to measure its ability to extrapolate beyond the spatial distribu- tion encountered during training. This is essential for determining the robustness of location-dependent reasoning and query formation. Finally, we analyzed semantic variations of questions whose locations were seen during training and found that the model exhibits best performance on these. The complete set of evaluation results is shown in Table 2. Our model shows strong generalization to unseen locations, achieving an Ex- act Match of 89% and near-perfect BLEU-4 and ROUGE-L scores. This indicates that the model reliably handles novel geographic inputs and correctly forms API calls even when encountering regions not present during fine-tuning. For multilingual queries, performance varies substantially by language. Accu- racy remains high for Spanish, Catalan, Galician, and French, reflecting the mul- tilingual prior knowledge inherited from the DeepSeek pretraining. In contrast, performance on Basque is markedly lower, which is consistent with Basqueâs relatively limited representation in most large-scale multilingual corpora. This suggests that while the fine-tuned model preserves substantial multilingual capa- bility, its robustness depends strongly on the linguistic coverage of the underlying foundation model. 10Hontan et al. ModelAccuracy (%) Syntax errs (%) Location errs (%) Other (%) Ours94.2005.8 DeepSeek R197.200.22.6 GPT497.200.42.4 Gemini9802.00 Grok97003 Table 3. Accuracy against state-of-the-art models When evaluating the full dataset, including both monolingual and multilin- gual scenarios, the model achieves an Exact Match of 80%. Restricting evaluation to monolingual English queries increases performance to 85%, confirming that multilingual variability is the main source of accuracy degradation. BLEU-4 and ROUGE-L remain consistently high across settings, indicating that even when the model does not achieve exact matches, its predictions typically preserve the correct query structure and essential content. We evaluated the performance of our proposed model against several state- of-the-art (SOTA) large language models (LLMs). To ensure a fair comparison, the SOTA models were evaluated in a zero-shot setting (no additional training or fine-tuning), using structured prompts that detailed the API schema, dataset characteristics, and query generation logic. Comparative analysis was conducted using the Semantic Variants dataset subset, as this fragment represents the peak performance benchmark for our ar- chitecture. As summarized in Table 3, our model achieved a 94.2% exact match (EM) rate. Although this slightly trails the SOTA benchmark of 97.0%, the re- sults demonstrate that our solution maintains high accuracy and remains com- petitive within the current landscape of large-scale generative models. 6 Application performance The fine-tuned model was integrated into a web application that supports both free-form natural-language queries and on-click generation of predefined ques- tion types relevant to typical user needs. The system interfaces with Google Maps and relies on the Overpass API to convert street-level user inputs into ge- ographic coordinates, enabling seamless interaction with the underlying dataset. Two representative screenshots are shown in Figure 4. Model inference was performed on the same NVIDIA RTX 3090 GPU used during training. The averaged runtime breakdown in Figure 5 indicates that model inference is the primary source of latency, contributing roughly three seconds per request. In contrast, database lookup, backend logic, and communi- cation overhead remain comparatively small. Backend processing time also includes the applicationâs guardrail mecha- nisms, which filter and validate user inputs before forwarding them to the model. These guardrails ensure that the system rejects out-of-scope or unsafe queries, thereby reducing hallucinations, preventing unsupported queries, and enforcing Title Suppressed Due to Excessive Length11 Fig. 4. Web application for model queries domain constraints. Although this contributes modestly to backend latency, it plays a critical role in maintaining reliability and user safety. The dominance of inference time suggests that end-to-end performance is chiefly limited by the modelâs computational footprint. Fig. 5. Application performance 7 Limitations & Future work Applicability to broader scenarios. The application developed in this work is currently being integrated into the FUTURAL Metasearch platform, which provides unified access to heterogeneous agricultural data originating from mul- tiple smart services. Our broader objective, however, is to establish a general methodology for connecting arbitrary queryable datasets to natural-language interfaces with minimal engineering and fine-tuning effort. A principal challenge in achieving this objective is the lack of pre-existing training data suitable for supervised fine-tuning. The synthetic data generation pipeline introduced in Sec- tion 3 is a systematic solution to this problem: although not fully automated, 12Hontan et al. it follows a principled procedure that can be adapted to datasets with similar structural characteristics. Nevertheless, this methodology does not scale gracefully when a large num- ber of datasets, each encoding distinct domain knowledge, are combined. In such settings, the space of admissible queries may grow exponentially, making manual template definition and projection-based question generation impracti- cal. For these more complex, multi-domain scenarios, an approach grounded in ontology-based domain abstraction, coupled with LLM-driven topic and query- space generation, may offer significant advantages. Exploring such strategies is a promising direction which we consider for future research. Application scalability. Our application is designed to process individual requests efficiently, but it does not yet scale to highly concurrent usage sce- narios. This limitation is part of our design: the system was developed with emphasis on simplicity and minimal hardware requirements rather than large- scale deployment. Nevertheless, several strategies can be employed to ensure scalability as demand grows. Some of these mechanisms are already being ex- plored for the FUTURAL Metasearch platform. When latency constraints allow, multiple queries can be grouped into a single forward pass through the model, a technique known as batching. Batching significantly improves GPU utilization and inference throughput, particularly for smaller models or bursty workloads. Additional optimizationsâsuch as more aggressive quantization, low-rank adap- tation methods, pruning, or using a smaller distilled model variantâcan reduce inference time and memory consumption. These optimizations directly increase per-server capacity. Moreover, many user queries are expected to exhibit a rela- tively simple and repetitive structure, often involving distances, travel times, or nearest-location requests. As a result, numerous queriesâor paraphrased vari- ants of the same underlying questionâcan be served directly from a response cache rather than invoking the model for each request. Graceful error recovery Our current architecture lacks automated recov- ery mechanisms for handling malformed or logically incorrect queries, relying instead on standard error reporting. To enhance system robustness, future iter- ations will explore iterative error correction strategies. This approach involves implementing a feedback loop where execution-time errors trigger a re-prompting sequence, enabling the model to refine and regenerate queries based on specific failure diagnostics. 8 Related work The paper that introduces the means to connect a LLM with external APIs is Toolformer [15]. Toolformer is a self-supervised framework that enables lan- guage models to learn how and when to call external toolsâsuch as calculators, search engines, or APIsâusing only minimal human annotation. The key idea is to let a pretrained model generate candidate tool calls within unlabeled text, execute those calls, and retain only the examples where the returned results measurably improve the modelâs ability to predict the original text. These fil- Title Suppressed Due to Excessive Length13 tered examples form an augmented training set that teaches the model to invoke tools autonomously, integrate tool outputs into its reasoning, and produce more accurate final answers. This approach allows Toolformer to extend a modelâs capabilities without requiring large-scale manual datasets or task-specific engi- neering. Toolformer cannot operate directly on a database for which no training dataset exists because its learning process depends on having text in which the model can propose, execute, and verify tool calls. The method works only when the model can generate candidate queries in context, run them on the tool, and then measure whether the toolâs output improves its ability to predict the sur- rounding text. If no dataset contains references to the databaseâs contentsâno examples of the entities, relationships, or queries associated with it. Gorilla [13] is a large language model designed specifically for reliable API invocation at scale. It combines LLM generation with retrieval-augmented ground- ing, using API documentation and signatures to guide the model toward valid, executable calls. Rather than memorizing APIs, Gorilla retrieves relevant func- tion specifications at inference time and conditions the model on them, enabling accurate argument selection and reducing hallucinated or invalid calls. This ap- proach allows Gorilla to generalize to thousands of real-world APIs, making it one of the first LLMs explicitly optimized for large-scale, real-world tool use. In contrast, our method is intentionally lightweight and domain-specific, fo- cusing on a single structured dataset with a fixed schema and a well-defined space of admissible queries. Rather than relying on large-scale API documentation or retrieval systems, the approach uses a synthetic dataset-generation pipeline to produce training examples tailored directly to the underlying database. This en- ables effective fine-tuning even when no pre-existing training corpus exists. Be- cause the model is optimized for a narrow set of tasks, it avoids the complexity and overhead associated with multi-tool reasoning, leading to higher reliability within its domain. Overall, Gorilla and our method occupy different points in the design space of tool-using language models. Gorilla excels when broad coverage across tools and domains is essential, while our approach is better suited for targeted, high- precision applications with minimal hardware and engineering overhead. The simplicity and reliability of the latter make it a practical solution for real-world systems where the domain is well defined and the cost of large-scale retrieval infrastructures cannot be justified. ReAct [18] is a framework in which a language model interleaves natural- language reasoning steps with actions, such as search queries, tool calls, or en- vironment interactions. Instead of producing a final answer directly, the model generates a sequence of thoughtâactionâobservation triplets: it reasons in text, issues a tool call, observes the toolâs output, and continues reasoning. This itera- tive loop enables the model to solve complex tasks that require external informa- tion, multi-step planning, or verification. Compared with our approach, ReAct is general-purpose and interactive, whereas our method is domain-specific and single-step. ReAct is designed for environments where the model must repeatedly 14Hontan et al. query external tools and refine its reasoning, relying on dynamic feedback from the environment. By contrast, our system requires only one structured API call per query, generated directly from the natural-language input without multi-step planning. Moreover, ReAct assumes access to a rich tool ecosystem and depends on the modelâs ability to reason and act iteratively, which increases complex- ity and computational cost. Our approach focuses on simplicity, low hardware requirements, and high reliability within a fixed domain, achieved through a synthetic training pipeline tailored to the datasetâs structure. ReAct and Gorilla belong to a broader line of research exploring how language models interact with external tools. Earlier efforts such as ToolAlpaca [16] focus on training models to use tools through supervised demonstrations, but they depend heavily on manually cu- rated instructionâtool pairs and therefore require substantial annotation effort. Confucius [7] extends this idea by introducing iterative tool learning, where the model refines tool-use behaviors across multiple steps, but its multi-round inter- action paradigm is computationally expensive and less suitable for low-resource environments. GPT4Tools [17] demonstrates strong tool-use performance by leveraging a very large base model (GPT-4), yet its reliance on massive pro- prietary training data makes the approach difficult to reproduce in constrained academic or deployment settings. Finally, CREATOR [14] explores disentan- gling tool creation from tool usage, enabling language models to propose new tools entirely from descriptions; however, its scope is far broader than required for single-dataset applications and introduces significant model and system com- plexity. In contrast to these general-purpose and often resource-intensive approaches, our method targets a specific structured dataset, requires minimal hardware, and avoids the need for large curated corpora or iterative interaction loops. By relying on a principled but lightweight synthetic dataset generation pipeline, our system achieves high reliability in a focused domain without the overhead associated with multi-tool or multi-step reasoning frameworks. Finally, we mention Model Context Protocol (MCP) [2], which has emerged as the de-facto standard for integrating external tools with large language mod- els. MCP provides a uniform mechanism through which servers expose tools, each accompanied by a formally defined JSON schema that the client loads into the modelâs working context. In principle, this offers a clean, typed interface for tool invocation. In practice, however, MCP places a substantial cognitive and computational burden on the model: a single MCP server can expose dozens of tools, collectively amounting to tens of thousands of tokens of schemas and de- scriptions that must be loaded into the context window before any task-specific reasoning begins. This leads to context saturation, competition between global tool definitions and task-relevant information, and a rapid degradation of tool- call accuracyâparticularly in multi-step workflows, where small per-call error rates compound exponentially. These limitations highlight a core challenge: MCP exposes not only tools, but too much tool metadata, forcing the model to reason, plan, and select actions while navigating an unnecessarily large tool universe. Title Suppressed Due to Excessive Length15 By contrast, our approach operates in a constrained, dataset-specific setting, avoiding the overhead of global schema loading and eliminating the need for the model to sift through irrelevant tools. Instead of exposing a large, heterogeneous action space, we train the model to produce a single, domain-specific API call, supported by a carefully constructed synthetic dataset. This results in signifi- cantly reduced hardware requirements, and higher reliability within the targeted domain. 9 Conclusion This paper presents a fully open-source, natural language question-answering (QA) framework optimized for accessibility data within the Durangaldea region of Spain. Our results demonstrate that the proposed system achieves perfor- mance metrics comparable to state-of-the-art (SOTA) large language models, while at the same time running on commodity hardware. It successfully bridges the gap between specialized local datasets and high-fidelity generative responses. Beyond the specific implementation, we propose a generalized methodology for the rapid deployment of QA systems across diverse tabular or relational datasets. By leveraging high levels of automation, this workflow significantly reduces the requirement for human expert intervention during the dataset preparation and model alignment phases. While the current architecture is optimized for a single- domain dataset, future research will focus on extending this system to multi- dataset environments, addressing the challenges of cross-domain query gener- ation and data fusion to provide a more holistic analytical tool for arbitrary collections of datasets. Acknowledgments. This paper has been funded by the by the European Union, via the FUTURAL project - Empowering the FUTure through innovative Smart Solutions for rURAL areas (HORIZON EUROPE) Project ID 101083958. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Executive Agency. Neither the Eu- ropean Union nor the granting authority can be held responsible for them. References 1. The llama 3 herd of models. Tech. rep., Meta (2024), https://ai.meta.com/research/publications/the-llama-3-herd-of-models/ 2. Anthropic:Introducingthemodelcontextprotocol. https://w.anthropic.com/news/model-context-protocol (Nov 2024), accessed: 2025-03-30 3. DeepSeek-AI: Deepseek r1: Incentivizing reasoning capability in large language models via reinforcement learning. arXiv preprint arXiv:2501.04652 (2025), https://arxiv.org/abs/2501.04652 4. Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: efficient fine- tuning of quantized llms. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. NIPS â23, Curran Associates Inc., Red Hook, NY, USA (2023) 16Hontan et al. 5. Frantar, E., Ashkboos, S., Hoefler, T., Alistarh, D.: Gptq: Accurate post- training quantization for generative pretrained transformers. arXiv preprint arXiv:2210.17323 (2022), https://arxiv.org/abs/2210.17323 6. Gao, D., Wang, H., Li, Y., Sun, X., Qian, Y., Ding, B., Zhou, J.: Text- to-sql empowered by large language models: A benchmark evaluation (2023), https://arxiv.org/abs/2308.15363 7. Gao, S., Shi, Z., Zhu, M., Fang, B., Xin, X., Ren, P., Chen, Z., Ma, J., Ren, Z.: Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum (2023), https://arxiv.org/abs/2308.14034 8. Guo, D., et al.: Deepseek-r1: Incentivizing reasoning capability in llms. arXiv preprint arXiv:2501.12948 (2025), https://arxiv.org/abs/2501.12948 9. Imagery, N., Agency, M.: Department of defense world geodetic system 1984: Its definition and relationships with local geodetic systems. Tech. Rep. TR8350.2, NIMA (2000) 10. Jolicoeur-Martineau, A.: Less is more: Recursive reasoning with tiny networks (2025), https://arxiv.org/abs/2510.04871 11. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., KĂŒttler, H., Lewis, M., Yih, W.t., RocktĂ€schel, T., Riedel, S., Kiela, D.: Retrieval-augmented generation for knowledge-intensive nlp tasks. In: Larochelle, H., Ranzato, M., Had- sell, R., Balcan, M.F., Lin, H.T. (eds.) Advances in Neural Information Processing Systems. vol. 33, p. 9459â9474. Curran Associates, Inc. (2020) 12. Olbricht, R.: The overpass api. https://wiki.openstreetmap.org/wiki/Overpass_API (2012), accessed: 2025-12-11 13. Patil, S.G., Zhang, T., Wang, X., Gonzalez, J.E.: Gorilla: Large language model connected with massive apis. In: Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C. (eds.) Advances in Neural Information Processing Systems. vol. 37, p. 126544â126565. Curran Associates, Inc. (2024). https://doi.org/10.52202/079017-4020 14. Qian, C., Han, C., Fung, Y.R., Qin, Y., Liu, Z., Ji, H.: Creator: Tool creation for disentangling abstract and concrete reasoning of large language models (2024), https://arxiv.org/abs/2305.14318 15. Schick, T., Dwivedi-Yu, J., DessĂŹ, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach them- selves to use tools. In: NeurIPS (or arXiv preprint arXiv:2302.04761) (2023), https://arxiv.org/abs/2302.04761 16. Tang, Q., Deng, Z., Lin, H., Han, X., Liang, Q., Cao, B., Sun, L.: Toolalpaca: Generalized tool learning for language models with 3000 simulated cases (2023), https://arxiv.org/abs/2306.05301 17. Yang, R., Song, L., Li, Y., Zhao, S., Ge, Y., Li, X., Shan, Y.: Gpt4tools: Teaching large language model to use tools via self-instruction (2023), https://arxiv.org/abs/2305.18752 18. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: React: Synergizing reasoning and acting in language models. In: The eleventh international conference on learning representations (2022) 19. Zeng, Y., Gao, Y., Guo, J., Chen, B., Liu, Q., Lou, J.G., Teng, F., Zhang, D.: Rec- parser: A recursive semantic parsing framework for text-to-sql task. In: Bessiere, C. (ed.) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. p. 3644â3650. International Joint Conferences on Arti- ficial Intelligence Organization (7 2020). https://doi.org/10.24963/ijcai.2020/504, https://doi.org/10.24963/ijcai.2020/504, main track