Paper deep dive

Data Agents: Agentic Data Systems

Guoliang Li, Peiyao Zhou, Xuanhe Zhou, Ji Sun, Yuyu Luo, Ju Fan

Year: 2026Venue: arXiv preprintArea: cs.DBType: PreprintEmbeddings: 94

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/27/2026, 3:48:42 AM

Summary

The paper introduces 'Data Agents', a new paradigm for data systems that autonomously manage, process, and analyze data with minimal human intervention. It addresses limitations of traditional systems (manual pipelines, limited semantics, reactive processing) by proposing a framework with six components: semantic data organization, semantic operators, agentic pipeline orchestration, feedback-driven refinement, memory management, and proactive adaptation. The system utilizes LLMs, vector indexes, and multi-agent strategies to execute complex data tasks from natural language queries.

Entities (16)

Data Agent · concept · 98%DATAAGENT · system · 95%Large Language Model · technology · 95%Agentic Pipeline Orchestration · component · 92%Feedback-Driven Refinement · component · 92%Memory Management · component · 92%Proactive Adaptation · component · 92%Semantic Data Organization · component · 92%Semantic Operators · component · 92%Data Analytics Agent · specialized-agent · 90%Data Science Agent · specialized-agent · 90%Semantic Catalog · data-structure · 90%Model Context Protocol · interfaceprotocol · 88%AWS Glue · industrial-system · 85%LangChain · llm-orchestration-stack · 85%PostgreSQL · database-system · 85%

Relation Signals (13)

Data Agent → comprisescomponent → Proactive Adaptation

confidence 95% · Our Data Agent system includes six components: ... and proactive adaptation.

Data Agent → comprisescomponent → Semantic Data Organization

confidence 95% · Our Data Agent system includes six components: semantic data organization...

Data Agent → comprisescomponent → Semantic Operators

confidence 95% · Our Data Agent system includes six components: ... semantic operators...

Data Agent → comprisescomponent → Agentic Pipeline Orchestration

confidence 95% · Our Data Agent system includes six components: ... agentic pipeline orchestration...

Data Agent → comprisescomponent → Memory Management

confidence 95% · Our Data Agent system includes six components: ... memory management...

Data Agent → comprisescomponent → Feedback-Driven Refinement

confidence 95% · Our Data Agent system includes six components: ... feedback-driven refinement...

Data Agent → proposessolutionto → Limitations of Traditional Data Systems

confidence 95% · To address these challenges, we propose a new paradigm called the Data Agent... Traditional data systems face profound limitations

Data Agent → enablesspecializedagent →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Traditional data systems face profound limitations in the AI era, relying on human-crafted pipelines, lacking semantic understanding of heterogeneous data, and operating through rigid, reactive processing. To address these challenges, we propose a new paradigm called the Data Agent, designed to manage, process, and analyze data with minimal human intervention. Data agents autonomously execute a wide range of data-related tasks, transforming traditional data systems by shifting from manual design to autonomous orchestration, from literal manipulation to semantic interpretation, and from reactive to proactive processing. Our Data Agent system includes six components: semantic data organization, semantic operators, agentic pipeline orchestration and optimization, feedback-driven refinement, memory management, and proactive adaptation. Building on this foundation, we also develop two specialized agents: the data analytics agent and the data science agent. Experiments on real benchmarks demonstrate significant performance gains of our data agent over state-of-the-art methods. We identify open challenges to guide future research in building fully autonomous data systems.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

91,791 characters extracted from source content.

Expand or collapse full text

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 0, NO. 0, JANUARY 20251 Data Agents: Agentic Data Systems Guoliang Li, Peiyao Zhou, Xuanhe Zhou, Ji Sun, Yuyu Luo, Ju Fan Abstract—Traditional data systems face profound limitations in the AI era, relying on human-crafted pipelines, lacking semantic understanding of heterogeneous data, and operating through rigid, reactive processing. To address these challenges, we propose a new paradigm called the Data Agent, designed to manage, process, and analyze data with minimal human intervention. Data agents autonomously execute a wide range of data-related tasks, transforming traditional data systems by shifting from manual design to autonomous orchestration, from literal manipulation to semantic interpretation, and from reactive to proactive processing. Our Data Agent system includes six components: semantic data organization, semantic operators, agentic pipeline orchestration and optimization, feedback-driven refinement, memory management, and proactive adaptation. Building on this foundation, we also develop two specialized agents: the data analytics agent and the data science agent. Experiments on real benchmarks demonstrate significant per- formance gains of our data agent over state-of-the-art methods. We identify open challenges to guide future research in building fully autonomous data systems. Index Terms—Data Agent, Agentic Data Systems, Data Ana- lytics Agent, Data Science Agent, Large Language Model 1. INTRODUCTION Data systems that encompass the collection, storage, pro- cessing, management, and analysis of data have played vital roles in numerous data-related applications [1], [66], [25], [68]. However, traditional data systems, including conven- tional relational databases (e.g., PostgreSQL and MySQL), data warehouses, and manually orchestrated data pipelines built with tools (e.g., Airflow), face limitations in the AI era when supporting autonomous and intelligent applications [47], [21]. Although modern cloud providers offer services that automate data infrastructure scaling and execution (e.g., AWS Glue for ETL, SageMaker for ML, and Bedrock for LLM integration), these platforms still largely require humans to define the processing logic, configure pipelines, and specify data schemas. In other words, they automate execution but lack semantic data understanding and intent-driven autonomous orchestration. Limitation 1: Manually-Designed Pipelines. Most current data systems depend on manually-designed pipelines that are tailored to specific tasks, such as building data science pipelines [5], [17], [61], [13] or facilitating interactive data analytics [55], [18], [12]. These designs result in high adapta- tion costs when transitioning to new domains and use cases, significantly restricting the scalability and flexibility of the data system. For example, traditional data cleaning pipelines usually consist of several manual steps: identifying relevant Guoliang Li, Peiyao Zhou, and Ji Sun are with the Department of Computer Science, Tsinghua University, Beijing, China. E-mail: liguo- liang@tsinghua.edu.cn; Xuanhe Zhou is with Shanghai Jiao Tong University, Shanghai, China. Yuyu Luo is with The Hong Kong University of Science and Technology (Guangzhou), China. Ju Fan is with Renmin University of China, Beijing, China. Corresponding author: Guoliang Li data, applying transformation rules (such as normalizing nu- merical values and standardizing date formats), validating data consistency, and managing exceptions or missing values [38], [37], [4], [58], [19]. Each step requires hand-coded rules and deep domain knowledge, which are intricately connected to the specific dataset or task. Consequently, when migrating a pipeline designed for financial transaction data to healthcare patient records, substantial manual modifications are neces- sary to account for differences in data schema, formats, and domain-specific rules. This high adaptation cost greatly limits the scalability and flexibility of such systems. Limitation 2: Limited Semantics Capabilities. Existing data systems struggle to grasp the complexity and semantics of real- world heterogeneous data, which includes both structured and unstructured data. They operate under a query-time complete- ness assumption that treats the stored dataset as the complete and authoritative source of truth. Consequently, these systems fall short in managing diverse, noisy, or semantically rich data environments that require adaptive understanding and reasoning rather than fixed logic. For example, analyzing data from a data lake requires interpreting complex nested struc- tures (e.g., multi-row/column headers and merged cells) and semantically dense entries such as product descriptions [49]. Similarly, financial reports demand reasoning over hetero- geneous structures, inconsistent terminology, and context- dependent content [51]. Current systems often cannot align semantically equivalent domain concepts or infer the intent behind aggregated records, making effective heterogeneous data processing highly challenging. Limitation 3: Reactive Processing. Traditional data systems are limited to reacting to predefined queries or reports and lack the ability to proactively identify trends, anomalies, and necessary updates. This proactive capability is crucial for enabling organizations to anticipate changes and respond more strategically. For instance, in a reactive analytics scenario, a company might run reports at the end of each month to analyze sales data. If the analysis reveals that certain products have consistently underperformed, the company would respond by adjusting future orders or launching targeted promotions. However, by the time these insights are gathered and acted upon, the company may have already incurred losses from overstocking or missed opportunities to capitalize on market trends. Transitioning from reactive processing to proactive processing presents a significant challenge. Paradigm Shift. To overcome these limitations, we propose a new paradigm called the Data Agent, which autonomously and proactively discovers, acquires, manages, processes, and analyzes data. It executes data tasks to achieve a closed-loop system, moving from data to decision-making with minimal human intervention. Data agents transform traditional data systems by shifting from manual design to autonomous arXiv:2609.24137v1 [cs.DB] 21 Sep 2026 (NL) Requests Context-Aware Responses Data Tools Tree-Structured Semantic Catalog Tagging Unified Data View Agentic Expert Intent Agent Planning AgentValidation Agent ... Semantic Data Organization Agentic Pipeline Orchestration, Optimization, Execution Traditional & Semantic Operators Data Agent Engines LLMs embeddings management Proactive Adaptation Proactive Task Adaptation Proactive Know. Adaptation ExpertsMeta Tool API Memory User Feedback Loop Management Feedback Agent Memory Management Memory Extraction op ... Op ... Cross-Check Pipeline Errors Ground Truth Feedback Negative Message interfaces preprocess Memory Agent Current Pipeline analyticsvisualization Confidence Score op MCP Server op MCP Server op MCP Server ... ... op Op Short-Term Long-Term User B Short-Term Long-Term User C Short-Term Long-Term User A intermediate information history metadata ... Episodic Mem Semantic Mem Cross-User Memory NL Request Semantic Relation A2A API SQL op op op op op op MCP Server MCP Server LLM Calls op op literal operators Task GraphPipeline Generation Dry Run Evaluation Dependencies Constraints Memory Forgetting Memory Retrieval Memory Consolidation filter relation project relation group relation join relation filter semantic extract semantic group semantic ... Dataset Configuration NL Query Submission Result Review Data Processing and Profiling Feedback-Looped Execution Memory Retrieval and Update Fig. 1. Data Agent Architecture. orchestration, from literal manipulation to semantic interpre- tation, and from reactive processing to proactive processing. Data agents have the potential to revolutionize a wide range of data-related tasks, from data preparation to analysis and decision-making. For example, when a company detects a revenue drop at a retail chain, traditional data systems sim- ply report that “sales have declined”. In contrast, a Data Agent proactively generates hypotheses, such as competitor promotions, severe weather, or declining delivery ratings. Then it automatically orchestrates a semantic pipeline to retrieve external evidence from sources such as weather APIs, re- view platforms, and competitors’ websites. Afterward, it au- tonomously conducts deep semantic understanding, integrated analysis, and reasoning, ultimately generating decision-making recommendations for timely actions. Note that unlike general AI agents that still require significant human oversight and intervention for designing and coding data-related tasks [29], [26], [8], [30], Data Agents operate with a high level of autonomy for performing end-to-end data tasks with minimal human intervention, creating a self-driven process from data collection to decision making. To better position the Data Agent paradigm beyond existing data automation systems and general-purpose agent frame- works, we compare it with industrial platforms (e.g., AWS Glue), generic LLM orchestration stacks (e.g. LangChain), workflow compilers (e.g. AutoML) and existing NL2SQL systems in Table I. Specifically, we identify four categories of existing systems: (i) Industrial systems (e.g., AWS Glue), which provide ETL, cataloging, and scheduling support but still require human-defined workflows; (i) LLM orchestration stacks (e.g., LangChain and AutoGen), which support tool calling and RAG but lack native semantic data organization, data-centric optimization, and specialized data operators; (i) Workflow compilers (e.g., AutoML, Apache Airflow), which compile human-defined DAGs but require manual pipeline specifications; and (iv) NL2SQL systems (e.g., DIN-SQL [39], DAIL-SQL [7]), which focus primarily on structured schema mapping via NL-to-SQL translation, lacking multi-step tool orchestration, feedback loops, and memory. The Data Agent paradigm is a fundamental architectural shift tailored specif- ically for data systems, integrating semantic understanding, autonomous end-to-end orchestration, data-centric memory, and proactive adaptation into a unified framework. Challenges. There are numerous challenges involved in tran- sitioning from manual design to autonomous orchestration, from literal manipulation to semantic interpretation, and from reactive processing to proactive processing. Challenge 1. Autonomous Pipeline Orchestration. Directly using LLMs to orchestrate a pipeline composed of relational and semantic operators for tasks described in natural language is impractical, because LLMs can produce hallucinations and lack domain-specific knowledge and up-to-date information. There are several challenges in orchestrating high-quality pipelines while minimizing LLM costs and latency. First, understanding the task, environment, underlying data, and domain knowledge presents a significant challenge. Second, evaluating the quality of the pipeline and iteratively improving it is challenging. Third, the abundance of tools and operators makes it costly to generate pipelines from a large number of possible operator combinations. Challenge 2. Semantic Interpretation. Using LLMs directly for processing complex tasks like semantic interpretation and 2 TABLE I COMPARISON OF DATA AGENTS WITH TYPICAL AUTOMATION PARADIGMS. DimensionIndustrial SystemsLLM Orch. StacksWorkflow CompilersNL2SQL SystemsData Agents Core Functionality ETL / Catalog / Scheduling Tool calling / RAG / Agents DAG Compilation NL-to-SQLAutonomous semantic analytics pipeline orchestration Autonomous End-to-end Orchestration # Human-defined workflows # Human-defined logic # Human-defined DAG NL-to-pipelines; Feedback-driven Semantic- Aware Processing #G# Without data-centric optimization ## SQL operations only Semantic data organization Proactive Adaptation #### Evolving memory and data profiles Data-Centric Memory # Execution state only G# Generic conversational # DAG runtime state only G# Query-session context only Working, Episodic, Semantic memory understanding is expensive. Lowering the costs associated with employing LLMs for processing semantic operations is a significant challenge. Challenge 3. Proactive Processing. Proactive systems must be adaptive, continually updating their processes and models based on new data or changing conditions. Continuously predicting, learning, and adjusting to ensure that the system remains relevant and effective over time is challenging. Our Contributions. To address these challenges, we propose a data agent framework. First, the Semantic Data Organization component manages and links structured, unstructured, and semi-structured data. It builds semantic catalogs and indexes to facilitate the semantic retrieval of heterogeneous data. Second, the Semantic and Generative Operator component enriches traditional literal operators with and LLM-driven reasoning and generation to support semantic-aware processing. Third, the Agentic Pipeline Orchestration and Optimization compo- nent adopts a multi-agent strategy to interpret a given task described in natural language, decompose the task, orches- trate a pipeline, and then optimize and execute the pipeline. Fourth, the Feedback-loop Management component evaluates the pipeline’s quality (e.g., correctness and confidence) and delivers ground-truth based feedback for performance refine- ment. Fifth, the Memory Management component maintains short-term memory for infrequent and temporary information, such as task context and execution logs, and long-term mem- ory for retaining frequent information, like historical data, metadata, and orchestration feedback. It also designs a cross- user memory sharing mechanism to enhance user experience like cold-start. Sixth, the Proactive Management component continuously predicts, learns, and adjusts the system to ensure that it remains relevant and effective over time. The data agent provides the essential foundation for au- tonomous data processing. Specialized data agents, such as data analytics agents and data science agents, depend heavily on this foundational layer to function effectively. We explore how to design data science and data analytics agents by leveraging the capabilities of the data agent. We make the following contributions. (1) We propose a system called DATAAGENT, exploring a new paradigm for data systems that can autonomously handle data- related tasks with minimal human intervention. (2) We propose effective techniques for designing a data agent, including semantic data organization, semantic oper- ators, agentic pipeline orchestration, feedback loop manage- ment, short-term and long-term memory management, and proactive adaptation. (3) We explore effective techniques for designing data analyt- ics agents and data science agents. (4) We have conducted extensive experiments using real benchmarks, and the results showed that our data agent sig- nificantly outperformed state-of-the-art techniques. (5) We outline open problems to guide future directions. The remainder of the paper is organized as follows. Sec- tion 2 presents the overall DATAAGENT framework. Section 3 and Section 4 elaborate two key components, namely memory management and feedback management. Section 5 introduces the data analytics agent and data science agent. Section 6 evaluates our system, and Section 7 discusses open problems. 2. DATA AGENT FRAMEWORK This section presents a data agent framework (see Figure 1) intended to autonomously manage data-related tasks. Given a dataset, a task expressed in natural language (e.g., “report average spending of customers who spent over $1,000 on food last month”), and a predefined library of operators, including relational operators (such as filter, project, group by, and join) and semantic operators (like semantic filter, semantic extraction, and semantic grouping), the data agent autonomously orchestrates an executable pipeline composed of these operators. It selects and sequences the appropriate operators to match the intent of the natural language query, optimizes the pipeline for efficiency and correctness, and then executes it to produce the final result. Semantic Data Organization. It organizes heterogeneous data and provides a unified semantic layer to support seman- tic access across diverse data sources, including structured, unstructured, and semi-structured data. First, to effectively 3 identify semantically relevant data, DATAAGENT constructs a unified semantic catalog across heterogeneous data sources using a tree-structured hierarchy. Each node in the tree rep- resents a semantic label, such as basketball or NBA, while the edges between parent and child nodes denote category- subcategory relationships. Each leaf node connects to all semantically related nodes within that category. Second, to unify diverse data, DATAAGENT generates embeddings for heterogeneous datasets and links data points that are highly similar in embedding space. Third, to efficiently identify relevant data, DATAAGENT builds vector indexes on these embeddings and maintains a semantic catalog that maps data to semantically related categories. The semantic catalog is materialized in the shared memory layer, enabling all agents to access a consistent, structured view of data semantics rather than relying on isolated representations. By utilizing this approach, DATAAGENT can efficiently identify relevant data for a given task using the semantic catalog and vector indexes. Specifically, for a given query, it utilizes sparse embeddings of the keywords along with dense embeddings of the entire query to search for relevant semantic categories within the semantic catalog. Using these semantic categories, it then identifies the pertinent data based on semantic indexes. Relational and Semantic Operators. Traditional relational operators (e.g., filter, join, and group-by) are effective for structured data but lack the semantic capabilities needed for semantic matching, entity recognition, and intent-level oper- ations, especially over unstructured and semi-structured data. To address this limitation, DATAAGENT incorporates a set of semantic operators, such as semantic filtering, semantic group- ing, semantic joining, and semantic extraction, and integrates them with conventional relational operators. These semantic operators can be implemented through LLM invocation, LLM- generated programs, or embedding-based matching, depending on the task requirement and execution cost. For unstructured and semi-structured data, DATAAGENT directly applies semantic operators to perform content under- standing and transformation. For structured data, DATAAGENT can invoke existing system APIs (e.g., Spark and Flink) or issue SQL queries to database engines (e.g., PostgreSQL, ClickHouse, and DuckDB). These connections are established through Model Context Protocol (MCP) interfaces, which allow DATAAGENT to natively integrate existing data sys- tems while also supporting user-defined drivers and custom operators. Thus, the atomic operators in DATAAGENT include semantic operators, relational operators, system APIs, and SQL queries. Given an NL task, the goal of DATAAGENT is to autonomously translate the user intent into an executable pipeline composed of these atomic operators. To reduce latency and cost while maintaining accuracy, DATAAGENT executes semantic operators using a cascad- ing strategy. First, it performs approximate nearest neighbor (ANN) retrieval over vector representations to obtain an initial candidate set. Second, it applies threshold-based pruning with predefined upper and lower confidence bounds to discard clearly irrelevant candidates and accept clearly relevant ones. Third, for the remaining ambiguous candidates, it invokes a lightweight LLM to conduct coarse-grained semantic filtering or constraint checking. Finally, a stronger LLM performs rig- orous verification on the minimized subset and produces struc- tured outputs, such as filtered rows, semantic join mappings, or extracted entities. This progressive design confines expensive LLM calls to the most ambiguous cases, thereby achieving a practical trade-off between effectiveness and efficiency. Agentic Pipeline Orchestration, Optimization and Exe- cution. We design a multi-agent strategy to automatically generate a pipeline using the aforementioned operators, opti- mize pipelines based on multi-goal optimization (e.g., improve the quality, reduce the cost and latency). In this approach, each agent specializes in a particular sub-task, such as task understanding, task decomposition, data profile extraction, query planning, reasoning, and pipeline validation. These agents are coordinated through Agent-to-Agent (A2A) proto- cols, where agents exchange structured messages (e.g., JSON objects encoding task graphs, execution states, and operator signatures). For example, during task understanding and de- composition, high-level goals are parsed and transformed into executable sub-flows. During pipeline optimization, an agent optimizes the pipeline for various objectives like latency, cost, interpretability, and accuracy. When it comes to executing pipeline actions, the agent assembles reusable semantic op- erators, along with necessary tools or data sources, instead of functioning as monolithic black boxes. To illustrate how DATAAGENT generates a pipeline, consider the NL query “Which product experienced the largest decline in sales, and why?”, DATAAGENT first uses the Semantic Data Organiza- tion component to retrieve relevant tables, reviews, catalogs, and indexes built offline, while also obtaining useful historical knowledge from memory. The orchestration component then generates an executable pipeline consisting of three steps: (1) an SQL generation operator over the sales table to identify the product with the largest monthly sales decline; (2) a semantic filtering operator over the reviews table to retain negative reviews related to the identified product; and (3) a semantic extraction and grouping operator to extract reported issues from the filtered reviews and group them by cause with counts. The generated pipeline combines structured analysis over relational data with semantic reasoning over textual evidence. Its accuracy is evaluated against task-specific reference outputs (e.g., returned records, numerical values, rankings, aggregated statistics). For pipeline orchestration, DATAAGENT first converts the user’s NL request into a structured task graph where nodes represent sub-goals, matched to candidate atomic operators (semantic, relational, API, SQL), and edges capture the po- tential execution and data dependencies (e.g., involving the same data sources). Upon the task graph, a planning agent decides an operator sequence by evaluating logical coher- ence and resource constraints (e.g., memory usage, batch size compatibility, execution locality). For non-deterministic operators like LLM calls, a validation agent further refines the plan by conducting low-cost dry runs (on sampled data or in simulated environments) for multiple candidate execution sequences in parallel, and dynamically selects the one with 4 the most promising accuracy and latency. Through this co- operative process, DATAAGENT autonomously synthesizes an optimized, executable pipeline. Note that if a pipeline is incor- rect, DATAAGENT uses feedback mechanisms to identify and correct errors, thereby improving its quality. The activation of different components is primarily event-driven. The execution of each node in the task graph triggers the activation of specific semantic or relational operators. Furthermore, events such as a low confidence score or a pipeline execution error trigger the Feedback Loop Management component (Section 4), which in turn reactivates the planning or validation agents to refine the pipeline. For pipeline optimization, we design a cost-based approach. DATAAGENT estimates the cardinality of semantic operators using the semantic catalog, and applies standard techniques to estimate the cardinality of relational operators. Based on these estimates, DATAAGENT uses a dynamic programming method to select the pipeline with the lowest expected cost. For pipeline execution, we adopt a cascading strategy for semantic operators. Given a semantic operator, DATAAGENT uses embedding models and lightweight LLMs to filter out irrelevant data. For each model, we define an upper threshold (items with similarity above this threshold are accepted as true matches) and a lower threshold (items with similarity below this threshold are rejected). DATAAGENT prunes items below the lower threshold and verifies items whose similarities fall between the two thresholds using larger LLMs. Feedback Loop Management. Crucially, this feedback loop component provides effective mechanisms to enhance pipeline quality. First, DATAAGENT verifies pipeline correct- ness through cross-check validations, which involve generat- ing multiple pipelines and merging them using a majority- voting validation approach. For instance, it runs these multiple pipelines (each pipeline returns a set of results) and collects the results from each pipeline. Next, it ranks the results based on the frequency (number of pipelines that produce each result). For each pipeline, it calculates a “coverage score” which is the sum of the frequencies of the results this pipeline produces. At last, it selects the pipeline with the highest coverage score. Second, DATAAGENT assigns confidence scores to each pipeline and offers feedback for those with low scores. For example, it provides negative feedback for pipelines with a low “coverage score”. Third, DATAAGENT triggers warning signals and initiates corrective sub-flows upon detecting issues such as pipeline errors or conflicts. Fourth, DATAAGENT injects data with ground truth and provides negative feedback for pipelines that make errors on this ground truth. Fifth, DATAAGENT meticulously generates a data sample, uses an LLM to compute responses for it, and then evaluates the quality of the pipeline based on the LLM-generated responses. To achieve this, LLMs are used to generate data that satisfies the question and then execute the pipeline on the generated data. To prevent infinite loops, the system implements internal mechanisms including confidence thresholds and a maximum retry limit. The user acts as an optional, high-level validator: the system invokes user confirmation only when its internal confidence remains below a critical threshold after the maxi- mum number of retries. Short-Term and Long-Term Memory Management. Mem- ory Management (M) is strictly an internal component of the overall DATAAGENT architecture. The user interacts with the DATAAGENT interface, and the DATAAGENT internally queries and updates the M without requiring the user to act as an intermediary. To ensure continuity and adaptability, the DATAAGENT uses short-term and long-term memory to store semantic knowledge, contextual metadata, and prior decisions. This enables it to reuse modeling templates, adapt to distribution shifts, and produce traceable, versioned artifacts. Short-term memory holds transient information – such as task context, execution logs, ratings and feedback, and temporal patterns – so only immediately relevant data is kept, with irrelevant or outdated items quickly discarded. Long-term memory stores persistent, frequently used information like historical data and metadata. To manage long-term memory efficiently, DATAAGENT uses a hierarchical structure that layers memory by access frequency and relevance, and builds vector indexes to improve retrieval. It also employs efficient promotion mechanisms that extract useful items from short- term memory and persist them in long-term memory. Finally, it applies conflict-detection techniques to identify and resolve inconsistencies across memories, improving reliability. Proactive Adaptation. Our system supports proactive adapta- tion to handle evolving tasks, shifting user intents, and concept drift in dynamic environments. Rather than reacting only after performance degrades, the system continuously monitors data agents using a set of predefined performance metrics, including query success rate, pipeline execution latency, result plausibil- ity scores, user feedback signals. When significant deviations are detected (e.g., a sudden drop in accuracy or an increase in ambiguous queries), the system triggers an automated root- cause analysis: specialized diagnostic agents examine recent interactions, data distributions, and agent behavior to hypoth- esize potential causes—such as outdated knowledge, schema changes, or emerging user goals. These hypotheses are then verified through targeted validation (e.g., testing alternative pipelines on recent data or checking consistency with updated domain facts). Once confirmed, the system proactively fixes them, which may include updating memory priors, adjusting operator preferences, or replanning the pipeline – often before the user even notices a problem. This closed-loop, anticipatory approach ensures the system remains robust, relevant, and responsive in the face of continuous change. Workflow. DATAAGENT shifts data systems from static pipelines to goal-driven, autonomous, adaptive processes. (1) The Semantic Data Organization component discovers and links relevant structured, unstructured, and semi-structured data; (2) the Semantic Data Operators component enriches data processing operations with LLM-based reasoning and generation; (3) the Agentic Pipeline Orchestration, Optimiza- tion and Execution component interprets user intent, decom- poses the task, composes a tailored execution plan, optimizes and executes the pipeline; (4) the Feedback Loop Man- agement component monitors pipeline correctness, verifies results, provides feedback, and adjusts future behavior; (5) 5 TABLE I EVENT-DRIVEN ACTIVATION RULES OF SYSTEM COMPONENTS EventTriggered ActionActivated Component User query receivedRetrieve relevant historical memory and prior task contextMemory Management Query context initializedPerform semantic data profiling over available sourcesSemantic Data Organization Query, memory, and profiles readyGenerate, optimize, and execute the analytical pipelineAgentic Pipeline Orchestration Pipeline execution completedVerify execution correctness and detect runtime or semantic errorsFeedback Loop Management Execution feedback availableStore useful intermediate plans and observations into short-term memoryMemory Management Data missing error detectedRefine semantic profiles and update source understandingSemantic Data Organization Updated profiles availableRefine and regenerate the pipeline with improved groundingAgentic Pipeline Orchestration Pipeline runtime or logic error detectedRepair the pipeline structure and retry executionAgentic Pipeline Orchestration Short-term memory updatedConsolidate stable insights into long-term memoryMemory Management Data sources updatedRefresh semantic catalogs, statistics, and retrieval indexesSemantic Data Organization Task completedAnticipate likely follow-up analytical requirementsProactive Adaptation the Memory Management component maintains both short- term and long-term memory to store context and historical information; and (6) the Proactive Adaptation component anticipates new requirements by continuously learning from interactions. Specialized data agents, such as the data science agent and the data analytics agent, are built on top of the general data agent to provide capabilities tailored to specific data processing. These components are activated by specific events and perform corresponding actions (see Table I). To support this process, we explicitly define two primary communication mechanisms among components. (1) Agent- to-Agent (A2A) protocols: Agents (e.g., planning agent, exe- cution agent, validation agent) in our system exchange JSON- formatted messages with predefined fields (e.g., task_id, input, output, status). These messages encode task graphs, execution states, and operator signatures, enabling coordinated multi-agent workflows. (2) Shared Memory and Semantic Catalog: The shared memory layer acts as a common data pool, where agents publish their intermediate results (e.g., extracted entities, filtered subsets) also in structured formats (e.g., DataFrames) rather than plain text, allowing downstream operators to process them programmatically. 3. MEMORY MANAGEMENT IN DATA AGENTS The memory management component serves as the data agent’s persistent knowledge repository, storing relevant data, past interactions, and learned insights accumulated throughout the workflow. It not only retains contextual information for continuity across tasks but also retrieves and supplies pertinent memories to inform and refine pipeline generation, enabling the data agent to adapt, improve, and make more informed decisions over time. A. Data Agent Memory Categories The data agent achieves a balance between immediate responsiveness (via working memory) and persistent, scalable intelligence (via long-term memory). This architecture not only enhances coherence and personalization but also ensures factual accuracy and task efficiency over extended interactions. Short-term Memory. The working memory (usually short- term memory) serves as the model’s contextual workspace, temporarily storing current conversation history, user interac- tions, and intermediate reasoning processes (such as chain- of-thought). It acts as the “mental scratchpad” or “cognitive workbench” for immediate reasoning tasks. However, it has limited capacity (constrained by context window length), volatile nature (cleared at session termination), dynamic and temporary storage. For example, in continuous dialogues, the LLM’s ability to reference previously mentioned information relies on working memory. Long-term Memory. To overcome the limitations of working memory, the agent leverages long-term memory, which resides externally (e.g., in vector databases or structured knowledge stores) and is selectively retrieved into working memory when needed. Long-term memory is categorized into three key types. (i) Episodic memory enables personalized, context-aware in- teractions by capturing user-specific experiences, preferences, and events tied to particular individuals or sessions. It records what happened from the user’s perspective (such as choices, feedback, or intentions) and uses this history to tailor future responses. For example, if a user says, “I prefer PostgreSQL over other databases”, this preference is stored as an episodic memory and used to guide database-related recommendations, ensuring continuity and personalization across interactions. (i) Semantic memory stores general, objective knowledge about the world—facts, concepts, domain rules, and common- sense reasoning that are not tied to any individual user. Its role is to ground the agent’s responses in accurate, verifiable information. For instance, when asked about “PostgreSQL’s query rewrite rules”, the data agent retrieves authoritative documentation from an external knowledge base via semantic memory, ensuring factual correctness rather than relying solely on internal model parameters. (i) Procedural memory encodes how to do things: it preserves reusable skills, workflows, and step-by-step procedures for completing specific tasks. This memory type allows the agent to automate complex operations consistently and efficiently. For example, upon receiving a request for data analysis, the agent can invoke a stored procedural memory to automatically generate a standardized analytical report template, including data cleaning, visualization, and statistical summary steps, based on best practices or previously validated pipelines. These long-term memories equip the data agent with the ability to remember personal context, reason with factual accuracy, and execute tasks reliably. In our implementation, each long-term memory entry is stored as a structured tuple with metadata (e.g., type, timestamp, importance score, em- beddings), and conflict resolution is handled through a graph- based memory algorithm (Algorithm 1) with specific reso- lution rules (recency-based, confidence-based, confirmation- based) as detailed in Section 3-B. The version graph maintains full traceability of memory states, enabling the system to re- 6 construct the chronological evolution of any stored preference. Transferring from Working Memory to Long-term Mem- ory. To transfer information from working memory to long- term memory, data agents follow a structured workflow. First, they identify what is worth remembering—filtering high-value, user-relevant, or task-critical content from the transient context (called memory extraction). Next, this information is encoded and structured into a standardized format, often enriched with metadata (e.g., memory type, timestamp, importance score) and converted into embeddings for semantic retrieval. The structured memory is then inserted in a persistent backend (memory insertion), such as a vector database or knowledge graph, ensuring durability across sessions. In practice, memory tuples stored in long-term memory may conflict with one another—either due to evolving user preferences, corrected facts, or contradictory experiences. To maintain consistency, the system employs memory update operators that detect and resolve such conflicts. Additionally, memories can become outdated or irrelevant over time, necessitating controlled for- getting. The system can automatically delete or archive such entries based on triggers like API deprecation notices, lack of recent use, or explicit user feedback. This memory deletion mechanism prevents the agent from relying on obsolete knowl- edge, keeping its behavior accurate and up to date. Finally, the system enables future retrieval (memory retrieval) by querying this long-term store when relevant – pulling pertinent memories back into working memory to inform reasoning, personalize responses, or guide actions in new interactions. B. Data Agent Memory Operators The data agentic memory system encompasses a full life- cycle of memory management, including: (1) Memory extraction: identifying and distilling salient information from interactions or observations; (2) Memory insertion, update, and deletion (i.e., forgetting): persisting new memories, revising conflicting or outdated ones, and removing obsolete entries; (3) Memory retrieval: retrieving relevant memories based on semantic similarity, user context, or descriptions; (4) Memory consolidation: periodically summarizing, com- pressing, or reorganizing memories to improve coherence, reduce redundancy, and enhance utility. These capabilities enable the data agent to learn continu- ously, maintain consistent knowledge, and adapt over time. 1) Memory Extraction: The memory extraction phase iden- tifies, distills, and evaluates valuable information from raw interaction logs or working memory to determine what should be preserved in long-term memory. This process is carried out through three specialized operators. The summarization and condensation operator compresses lengthy dialogues or documents into concise, high-fidelity summaries that retain essential facts, decisions, and user intentions—dramatically reducing storage overhead while pre- serving meaning. The structured extraction operator precisely extracts entities (people, projects), attributes (preferences, specifications), events (decisions, changes) from unstructured text and structures them. Given the working memory content, DATAAGENT invokes LLM function calling capabilities or pre-trained named entity recognition (NER) models, which output key-value pairs or knowledge triples for enriching episodic metadata or building semantic knowledge graphs. The value assessment operator predicts long-term potential value of information before storage, dynamically determining storage priority and duration. Given working memory content, DATAAGENT uses rule-based (high score for “my preference” mentions) combined with learned metrics (information en- tropy, relevance to core objectives, access frequency predic- tion), which outputs quantitative importance score for guiding storage, retrieval, and forgetting strategies. For example, dur- ing the execution of queries with a particular pattern, if the semantic filter is preferred more frequently and consistently achieves a higher positive feedback rate than the traditional filter, the system assigns a higher importance score to semantic filters based on both the recurrence and effectiveness of their filtered results. This memory is therefore retained for a longer period and prioritized during retrieval for similar queries that are frequent and effective. In contrast, the traditional filter receives a lower score and is discarded or set to expire quickly. 2) Memory Insertion/Update/Deletion: The memory man- agement system uses insertion, update, and deletion (also called “forgetting”) operators to maintain the agent’s memory as accurate, relevant, and efficient. Notably, the system often preserves version history, allowing past states to be audited or restored if needed. First, Insertion adds new memories to long-term storage. Second, Update manages changes and resolves conflicts between new and existing information. When conflicts arise (e.g., multiple values for the same key), the system uses rule-based strategies: (i) Recency-based selection (“prefer the latest”) for time-sensitive data, (i) Confidence- based selection using LLM-derived or retrieval scores, and (i) Explicit confirmation by invoking an LLM when am- biguity persists. Conflicting entries are not overwritten but retained as parallel versions linked via a version graph, en- suring traceability. Conflicts are dynamically resolved during retrieval by selecting the most relevant version based on the query context, ensuring consistency and auditability. Third, Deletion removes or archives outdated, irrelevant, or low-value memories, similar to how humans naturally forget unimportant details. This prevents memory bloat and maintains system per- formance. Deletion follows three strategies: (i) Value-driven: Periodically purges memories with importance scores below a threshold, (i) Time-driven: Removes memories after a set duration unless marked as high-value, and (i) Instruction- driven: Deletes memories in response to explicit user requests. Together, these operators ensure the agent’s memory remains accurate, adaptive, and efficient—retaining critical information while discarding the unnecessary. 3) Memory Retrieval/Search: The memory retrieval op- erator is responsible for finding the most relevant memory segments from long-term storage to support the current task. To maximize recall and precision, DATAAGENT uses a multi- stage retrieval pipeline that combines semantic, textual, and metadata-based signals. (i) Semantic Retrieval: It computes vector embeddings for the current query and retrieves the top- k memory entries with the highest semantic similarity (e.g., 7 Algorithm 1: Memory Insertion&Conflict Resolution Input: Extracted new memory M new , Vector Index I, Memory Version Graph G = (V, E) Output: Updated Memory Graph G and Index I 1 v new ← ComputeEmbedding(M new ); // Step 1: Locate relevant nodes 2 Anchors← VectorSearch(I, v new , top-k); 3 Conf lictN odes←∅; // Step 2: Detect Conflicts 4 foreach node∈ Anchors do // Find related memories or leaves 5RelatedN odes← TraverseGraph(G, node); 6foreach r ∈ RelatedN odes do 7isconf lict← LLMJudgeConflict(M new , r); 8if isconf lict then 9Conf lictN odes← Conf lictN odes∪r; // Step 3: Resolve conflict 10 AddNode(G, M new ); 11 foreach target∈ Conf lictN odes do 12action← ApplyRule(M new , target); 13if action == “Overwrite” then 14Deactivate(target); 15AddEdge(G, target→ M new ); 16 UpdateIndex(I, M new ); 17 return G,I; using cosine similarity in a vector database). This captures meaning even when wording differs. (i) Textual Retrieval: It performs keyword- or lexical-based matching (e.g., BM25) to find entries with the strongest textual overlap with the query. This helps catch precise matches that embedding models might smooth over. (i) Metadata Filtering: It narrows the candidate pool using structured metadata – such as time range (e.g., only memories from the last 10 minutes), memory type (e.g., only procedural or user-specific episodic memories), or importance score (e.g., exclude low-priority entries). After finding relevant memory segments, DATAAGENT leverages the memory version graph (constructed during in- sertion) to search for relevant ones. Starting from the anchor nodes, it traverses adjacent and preceding nodes along the directed edges, and returns an ordered list of entries with timestamps. This allows the data agent to reconstruct the chronological evolution of a state or preference, resolving historical conflicts by tracing the active version paths. Finally, DATAAGENT fuses and reranks all candidates using a weighted scoring function that balances: semantic similarity, textual match strength, temporal recency, importance score (user preferences or verified facts rank higher). The result is a refined list of the top-k most relevant memory segments, ordered by overall relevance to the current context. 4) Memory Consolidation: The memory consolidation op- erator periodically refines stored memories, transforming frag- mented experiences into richer, higher-level knowledge. It achieves this by compressing, abstracting, and linking raw memory entries, turning isolated events into meaningful pat- terns or generalized insights. For example, it may recognize recurring themes and create a meta-memory like: “User is an advanced Python learner focused on performance opti- mization,” which can guide more proactive and context-aware recommendations. Consolidation builds connections between previously unrelated memories, weaving them into a coherent knowledge network. By transforming episodic fragments into structured, interconnected knowledge, consolidation enables the data agent to reason more deeply, anticipate user needs, and deliver increasingly intelligent responses over time. C. Data Agent Memory Storage To enable efficient and semantically aware access to both recent context and historical knowledge, DATAAGENT uses a unified memory storage architecture with distinct handling for working and long-term memory. Working memory that captures the immediate conversation history and task context is typically kept in a lightweight, short-lived buffer (e.g., in- session cache or token-limited prompt context). It is not per- sisted but periodically distilled into long-term memory. Long- term memory is persistently stored in a vector database. Each memory entry is encoded into a dense vector and enriched with structured metadata, including: Memory type (e.g., episodic, semantic, or procedural), Source (e.g., user statement, external knowledge base), Timestamp, and Importance score (based on relevance, user intent, or usage patterns). This design allows the agent to perform semantic search across large-scale memory collections. During inference, it retrieves the most relevant information – whether from recent interactions (via working memory) or accumulated experience (via long-term memory) – to ground its reasoning, personalize responses, and maintain coherent, context-aware behavior over time. 4. FEEDBACK MANAGEMENT The critical challenge of pipeline orchestration lies in veri- fying the correctness of the generated pipeline. Since ground- truth pipelines are often unavailable and errors can be subtle, we introduce an error detection and feedback mechanism to iteratively improve pipeline quality. Our approach identifies and addresses two key types of pipeline errors. (1) Syntactic errors: Invalid operator usage, incorrect pa- rameter formats, or defective pipeline structures. (2) Semantic errors: Logically incorrect transformations that produce results inconsistent with the user’s intent (e.g., filtering on the wrong attribute). Syntactic Error Feedback. To detect syntactic errors in automatically generated data pipelines (e.g., invalid operator usage, incorrect parameter formats, defective structure), we employ a multi-layered validation strategy. First, schema- aware validation ensures all referenced columns exist and have compatible types. Second, operator signature checking verifies that each operator is called with the correct arguments, types, and required parameters. Third, pipeline grammar validation parses the pipeline as a structured program to confirm that it follows valid sequencing and compositional rules (e.g., no cy- cles, proper operator order). Additionally, a static dry run using metadata or mock data can catch mismatches and runtime- like issues without running the full execution. Optionally, 8 an LLM-guided self-check can provide heuristic feedback on ambiguous cases. In this way, these methods enable robust, au- tomated detection of syntactic flaws before pipeline execution, improving reliability and reducing downstream failures. Semantic Error Feedback. We employ three verification strategies to detect and correct semantic errors in pipelines. (i) Cross-Validation via Multiple Pipeline Generation. To enhance robustness and reduce the risk of generating a single flawed interpretation, DATAAGENT generates multiple diverse candidate pipelines for the same natural language (NL) query – each representing a plausible way to translate the user’s intent into executable operations. These candidates are then executed on the actual dataset, and their outputs are systematically compared using both quantitative and qualitative criteria (e.g., result cardinality, data distribution, logical consistency, and alignment with expected semantics). Pipelines that produce consistent, coherent, and semantically plausible results, espe- cially those that converge with other high-quality candidates, are assigned higher confidence scores and prioritized. In contrast, pipelines yielding outlier results (e.g., empty outputs, implausible aggregates, or values inconsistent with the query’s scope) are flagged as potentially erroneous and are either discarded or sent back for refinement through feedback loops. This cross-validation approach acts as a built-in consistency check, mimicking how multiple experts might independently solve the same problem and converge on a reliable answer. (i) Synthetic Data Consistency Checking. To proactively catch semantic or logical errors, DATAAGENT generates a small, controlled set of synthetic data that embodies key charac- teristics of the real dataset (such as schema structure, value ranges, and edge cases) but is simple enough to reason about manually or by LLMs. It then executes two parallel processes on this synthetic data: (1) the actual generated pipeline, and (2) a reference interpretation of the user’s intent, derived either from the LLM’s step-by-step reasoning trace or a human-aligned expected behavior. By comparing the outputs of both, it assesses whether the pipeline faithfully implements the intended logic. If the results match closely – in both content and structure – it gains confidence in the pipeline’s correctness. However, if there is a mismatch (e.g., the pipeline returns all users while the expected result filters only high- value ones), it signals a semantic or logical flaw, such as misinterpreting a filter condition, applying an operation in the wrong order, or conflating similar concepts. This technique acts as a lightweight, interpretable “unit test” for pipeline validation, enabling early error detection without requiring full-scale execution or ground-truth labels. (i) Pipeline-to-NL Consistency Evaluation. To verify that the generated pipeline truly reflects the user’s original intent, we employ a Pipeline-to-Natural-Language (Pipeline2NL) consis- tency check. Specifically, we automatically translate the exe- cuted pipeline back into a clear, human-readable natural lan- guage description. For example: “Filter customers who spent over $1,000 on food last month, then compute their average spending”. This reconstructed statement is then compared to the original user query (e.g., “What’s the average spending of customers who spent more than $1,000 on food last month?”) using semantic similarity metrics such as embedding-based cosine similarity or LLM-powered entailment scoring. A high similarity score indicates strong alignment between what the user asked for and what the pipeline actually does. Conversely, a low similarity score raises a red flag: it suggests a potential misinterpretation or implementation drift, such as filtering on the wrong category (e.g., “total spending” instead of “food spending”), omitting a time constraint (“last month”), or performing an incorrect aggregation. By closing the loop from code back to language, this technique provides an interpretable, intent-aware validation layer that complements execution-based checks and helps catch subtle semantic errors that might otherwise go unnoticed. 5. DATA AGENT VARIANTS The data agent establishes the groundwork for autonomous data processing. Specialized data agents, like the data analytics agent and data science agent, depend on this foundation to deliver these essential capabilities. Beyond this, such data agents should also specify task requirements, design feedback mechanisms, and provide proactive specifications. A. Data Analytics Agent It supports data analytics on heterogeneous data, includ- ing both structured and unstructured data. Unlike structured queries (where schema, types, and relationships are prede- fined), unstructured data is often free-form and diverse in format. The challenge lies not only in the absence of structure but also in the ambiguity, heterogeneity, and lack of clear semantic anchors. Traditional data pipelines struggle to handle such irregular inputs at scale [1], [66]. This represents a funda- mental shift: from retrieval-based document systems handling unstructured data and syntactic analytics on structured data to semantic analytics across heterogeneous data. The data analytics agent not only locates relevant content, but also interprets, reasons, and synthesizes insights, advancing data analytics toward an agent-driven paradigm. Basic Idea. The data analytics agent must access all data sources, build a semantic catalog for heterogeneous data, define data analytics-related semantic operators, and integrate with the APIs, tools, and SQL interfaces of the underlying systems. The data analytics agent then creates a pipeline for the specified task, optimizes it, and executes it. Agentic Pipeline Orchestration. The data analytics agent starts by parsing the user query into a high-level semantic plan. It identifies the query intent (e.g., root cause analysis of a decline in economic indicators), selects relevant data (e.g., knowledge of possible causes and relevant data sources), chooses pertinent semantic operators (e.g., semantic filtering of relevant data and semantic grouping for the causes), and orchestrates a semantic pipeline. The pipeline orchestration proceeds iteratively, with LLMs assisting in interpretation, where data retriever extracts relevant context from surround- ing documents, and pipeline validator ensures correctness and consistency. When misalignment is detected, such as a pipeline error or low confidence level, the system reroutes the pipeline and updates its confidence scores, emulating human 9 troubleshooting behavior. Moreover, to support such dynamic capability, the data analytics agent leverages its unified oper- ator library, planning engine, and feedback-driven execution layer. These components interact with vector indexes, LLM APIs, and memory modules, enabling continual refinement and learning from past executions. We also store crucial context information and historical data in short-term and long-term memory, which are used to guide pipeline generation and provide feedback for improving pipeline quality. Feedback Mechanism. Providing feedback on pipeline per- formance is crucial for enhancing its quality, as generated pipelines might be incorrect. We develop several effective techniques to ensure high-quality feedback. Firstly, we utilize the pipeline execution capability to provide feedback; if a pipeline fails to execute, we can issue negative feedback. Secondly, we employ a GAN-based technique that converts the pipeline into a textual description, evaluates this description against the task description, and uses the degree of matching to provide feedback. Thirdly, we search for relevant historical tasks/pipelines and assess the similarity between this historical information and the current task and pipeline. If any inconsis- tencies are found, we provide negative feedback. Agentic Pipeline Optimization. Given a semantic pipeline with relational operators (or sub-pipeline as SQL) and se- mantic operators, the data analytics agent optimizes the pipeline [22]. This includes pushing down low-cost operators and reordering semantic operators for enhanced efficiency. These optimization techniques are highly dependent on the semantic cardinality and cost estimation of these semantic operators. To achieve this, we use the semantic catalog to assist in semantic estimation. Specifically, for a given semantic op- erator, we identify relevant catalog labels, fetch some samples under these labels, use the samples to estimate cardinality/cost, and then calculate the overall cardinality/cost. Pipeline Execution. Give an optimized pipeline, we develop efficient methods for its execution. At the operator level, we design three effective techniques. First, we implement batching techniques to group multiple data items together, allowing us to invoke the LLMs more efficiently and reduce costs. Second, we utilize embedding-based techniques to filter out data items with low similarity to the query. Third, we use cascading techniques with LLMs to enhance performance by employing smaller LLMs for pruning to bypass larger LLMs. At the pipeline level, we use a vectorized execution model to improve the KV cache hit rate, thereby reducing the cost of using LLMs. We also group the data points based on their shared prefixes in order to enhance the KV caching capabilities. Visualization Agent. It transforms structured query results into clear, informative, and contextually appropriate visual representations [40], [32], [34]. It autonomously selects the most suitable chart type – such as a bar chart for categorical comparisons, a line chart for temporal trends, or a pie chart for part-to-whole relationships. It intelligently designs the visual encoding: choosing appropriate coordinate systems, mapping data fields to visual channels (x/y position, color, size), and generating meaningful legends, labels, and titles that enhance interpretability [30]. The agent also considers best practices in perceptual effectiveness and avoids misleading visualiza- tions [54], [33]. By combining data semantics, task context, and visualization theory, it ensures that each visualization communicates insights accurately and intuitively to the user. B. Data Science Agent Unlike existing methods that manually address isolated data science steps such as data discovery [40], data cleaning [58], data integration, and model selection [11], [27], [10], data- science agent is capable of autonomously coordinating the entire workflow. Specifically, given an online data science task described in natural language, this stage autonomously decomposes the task into a pipeline composed of sub-tasks that align with data agent capabilities. It then selects the appropriate agent for each sub-task and dynamically refines the pipeline to ensure both efficiency and robustness. Foundational Agents for Data Science. We develop an agent for each specific functionality in data science, including a data discovery agent, data standardization agent, data cleaning agent, model selection agent, parameter selection agent, model training agent, and model evaluation agent. We outline the functionality and description of each agent, which will be utilized in multi-agent pipeline orchestration. We connect common data-science tools with external ecosystems: machine learning libraries, feature engineering packages, PyData toolk- its, databases, visualization platforms, and tuning frameworks. By integrating these various tools into a unified process, the data science agent enables an interpretable, adaptive, and con- tinually evolving data science workflow that balances automa- tion without human intervention. Notice that our data science agent is designed to be extensible, allowing integration of new data agents through agent specification. Once integrated, the new agent can seamlessly collaborate with existing agents within our multi-agent pipeline orchestration framework. Multi-Agent Pipeline Orchestration. Given an online task, we start by using LLMs to decompose the task into a pipeline of multiple sub-tasks based on the foundational agent profiles. Each sub-task is assigned to an appropriate data agent [41]. We refine the pipeline iteratively with additional adjustments, such as merging or further decomposing sub-tasks. The pipeline is executed in a parallel bottom-up manner according to its topological order. To ensure robustness, we dynamically refine the pipeline based on intermediate results, including: (i) modifying sub-tasks at the agent level, and (i) global-level re-planning, where complete intermediate results are stored in our data catalog to avoid redundant computation. Once all sub-tasks are executed, the data agent delivers the final result. Feedback Mechanism. If a pipeline encounters errors, we provide feedback to help LLMs correct those mistakes [20]. To achieve this, we design several feedback mechanisms to provide effective feedback. First, we autonomously generate a data sample and use LLMs to directly generate an answer for it. This answer serves as the ground truth for evaluating the pipeline (or a sub-agent). If the pipeline’s answer significantly differs from the LLM-generated answer, feedback is provided to the agent with detailed descriptions of which data is missing and which data should not be included in the answer. 10 TABLE I DABSTEP LEADERBOARD RESULTS. AgentEasy Acc.Hard Acc. DATAAGENT94.4450.79 DS-PlaVer87.5045.24 Amity DA Agent v0.180.5641.01 Mphasis-IZI-Agents80.5628.04 Claude 4 Sonnet ReACT Baseline81.9419.84 Open Data Scientist84.7216.40 o4-mini Reasoning Prompt Baseline76.3914.55 Claude 3.7 Sonnet ReACT Baseline75.0013.76 o3-mini Reasoning Prompt Baseline72.2213.76 Gemini 2.5 Pro Reasoning Prompt66.6712.70 GPT 4.1 ReACT Baseline80.5612.43 o1 Reasoning Prompt Baseline69.4411.11 This strategy can also serve as feedback for the foundational agents mentioned above. For example, in the case of a data standardization agent, we can use LLMs to standardize a sample and then use the results from this sample to generate feedback. Second, we can inject some samples that include ground-truth data. If the pipeline fails to process these samples correctly, we can provide feedback with detailed descriptions. Third, we can use users’ ratings, such as upvotes or downvotes, as feedback, which can be used to enhance the pipeline quality. 6. EXPERIMENTAL EVALUATION This section comprehensively evaluates our system DATAA- GENT on two challenging benchmarks that jointly cover a broad spectrum of data-centric tasks: (i) DABStep, a multi- step data-analysis agent benchmark with 450 questions span- ning easy and hard settings, and (i) Spider 2.0-Lite, a text- to-SQL benchmark with 547 examples designed for realistic SQL query generation across BigQuery, Snowflake, and SQLite databases. Notably, DABStep evaluates multi-step reasoning over heterogeneous inputs, including unstructured documents, lookup tables, and schema-related metadata, with its hard split further stressing semantic grounding and cross-document reasoning under high-noise, multi-source settings. Spider 2.0- Lite evaluates SQL generation under schema-level noise and realistic database complexity across multiple database engines, thereby testing the agent’s ability to generalize across typical SQL dialects and noisy schema environments. For DABStep, we follow the official evaluation on the val- idated split and adopt each competitor’s public configuration as listed on the leaderboard. DATAAGENT uses Qwen 3 as the backbone LLM and the same input documents/questions as all baselines. For fairness, we keep default decoding and tool settings unless prescribed by the leaderboard entries; no task-specific fine-tuning is applied. Accuracy is reported for both Easy and Hard levels, with emphasis on the latter. For Spider 2.0-Lite, we follow the leaderboard protocol, where all methods are ranked by execution accuracy. We compare against a broad set of representative baselines, including (1) specialized systems developed by well-known research orga- nizations, such as Snowflake ReForce, DeepSeek LinkAlign, RSL-SQL+DeepSeek, and Claude-based Spider-Agents from Anthropic; and (2) prompt-style and agentic methods built on leading LLM families, including Qwen 3, Gemini 2.5, Claude 3.7/4, GPT o4-mini/o3-mini/o1/4.1, and DeepSeek-V3. TABLE IV SPIDER 2.0-LITE LEADERBOARD RESULTS. MethodScore DATAAGENT44.50 ReForce + o3 (Snowflake)37.84 ReForce + Qwen3 (Snowflake + Qwen)35.60 Autolink + DeepSeek-R134.92 RSL-SQL + o3 (DeepSeek + HUST VLR)33.09 LinkAlign + DeepSeek-R1 (DeepSeek)33.09 RSL-SQL + DeepSeek-R1 (DeepSeek + HUST VLR)30.53 ReForce + o1-preview (Snowflake)30.35 Spider-Agent + Claude-3.7-Sonnet28.52 Spider-Agent + Claude-4-Sonnet27.79 A. Overall Accuracy Results on DABStep. Table I summarizes the results. DATAAGENT achieves the best performance on both metrics, reaching 94.44% (easy) and 50.79% (hard). Compared to the second-place system (DS-PlaVer; Gemini 2.5-Pro), DATAA- GENT improves by +6.94 points on the easy split (94.44 vs. 87.50) and +5.55 points on the hard split (50.79 vs. 45.24). Notably, Claude- and OpenAI-based systems perform signif- icantly lower on the hard tasks, underscoring the importance of robust multi-step reasoning and cross-document grounding. Results on Spider 2.0-Lite. Table IV shows the Spider 2.0- Lite leaderboard. DATAAGENT again leads with a score of 44.5, surpassing Snowflake’s ReForce (+o3 at 37.84) by +6.66 points and outperforming both Qwen3-enhanced and Claude- based Spider-Agents (35.6 and 28.5). The consistent advantage of DATAAGENT across both benchmarks demonstrates its strong ability to generalize from multi-step document reason- ing (DABStep) to realistic text-to-SQL tasks (Spider 2.0-Lite), reinforcing its role as a data agent framework. B. Ablation Study To further analyze the contribution of each core compo- nent, we conduct an ablation study on both DABStep and Spider 2.0-Lite. We categorize queries based on characteristics such as reasoning depth and data source usage complexity. For w/o Semantic Organization, we remove structured schema profiling and unstructured retrieval augmentation, exposing the planner only to raw source interfaces without semantic catalogs or indexed background knowledge. For w/o Feedback Loop, we disable semantic validators and intermediate verifi- cation modules, retaining only runtime exception signals from downstream execution engines. For w/o Memory Management, we disable memory operations and only retain the context with incremental actions and feedback. Figure 2 shows that all three components consistently con- tribute to performance across both DABStep and Spider 2.0- Lite, while their impact varies by query characteristics. Without Semantic Organization. Removing Semantic Orga- nization leads to a performance drop, especially on categories that require cross-source semantic alignment or retrieval over large, noisy data spaces, such as multiple data statistics in DABStep and multi-table queries in Spider 2.0-Lite. This confirms that semantic catalogs and indexes are critical for mapping high-level user intents to relevant data in large and noisy data sources. 11 Simple Query Single Data Statistic Multiple Data Statistic Matching with Complex Logic Query Categories 0 20 40 60 80 100 61.1 62.5 63.9 93.5 29.1 66.3 82.5 98.3 18.2 27.0 29.1 47.0 18.2 19.7 25.2 34.0 (a) DABStep Data Agent w/o Semantic Data Organization w/o Feedback Loop w/o Memory Management Aggregation Query Multi-table Query Nested Query Recursive Data Analysis Query Categories 0 20 40 60 80 100 40.0 50.0 55.0 75.0 41.0 43.6 46.2 59.0 19.6 19.6 23.5 31.4 8.0 12.0 20.0 28.0 (b) Spider-2.0-Lite Data Agent w/o Semantic Data Organization w/o Feedback Loop w/o Memory Management Accuracy Fig. 2. Ablation Study Across Query Categories. Simple Query Single Data Statistic Multiple Data Statistic Matching with Complex Logic Query Categories 0 1 2 3 4 5 6 7 8 Average #Steps 1.38 1.13 1.60 2.63 4.25 4.00 8.22 6.00 (a) DABStep Data Agent Qwen3 ReACT Baseline Aggregation Query Multi-table Query Nested Query Recursive Data Analysis Query Categories 0 2 4 6 8 10 4.98 4.28 4.81 4.30 5.48 5.82 5.12 9.55 (b) Spider-2.0-Lite Data Agent ReFoRCE Fig. 3. Comparison of Iteration Steps. Without Feedback Loop Management. Disabling Feedback Loop causes more noticeable degradation on queries with deeper reasoning chains (e.g., complex logic matching, recur- sive data analysis). This suggests the feedback mechanisms are particularly effective at detecting latent logical deviations that are increasingly difficult for LLMs to find during planning as query complexity and reasoning depth grow. Without Memory Management. The effect of removing Memory Management is more noticeable on all but the simplest queries. As query complexity increases, multi-turn pipeline orchestration introduces more intermediate decisions and feedback signals, making the context lengthy and diluting the LLM’s attention. The memory manager preserves the most relevant intermediate decisions and feedback signals during multi-turn pipeline orchestration, and also enables the reuse of dataset-specific knowledge and previously successful pipeline patterns, thereby keeping the planning context compact and allowing the agent to solve complex tasks more efficiently. Additionally, the proactive capabilities of DATAAGENT are partially validated through the ablation study: the w/o Memory Management variant, which disables the evolving memory and pattern reuse, shows consistent performance degrada- tion across both benchmarks, confirming the memory-driven proactive adaptation contributes meaningfully to system effec- tiveness. More advanced proactive capabilities, such as fully autonomous problem identification and self-initiated corrective workflows without human prompts, remain open challenges. Overall, the ablation results align well with the intended responsibilities of the three modules, showing that the gains of DATAAGENT do not come from a single component but from their complementary interaction. C. Cost and Efficiency Analysis We analyze the token and efficiency benefits brought by DATAAGENT from end-to-end latency and token consumption. Execution Time Reduction. Table V reports the average execution time per query on DABStep and Spider 2.0-Lite. DATAAGENT consistently reduces the total execution time compared with the corresponding baselines, achieving a 2.32× speedup on DABStep and a 1.45× speedup on Spider 2.0-Lite. The latency reduction mainly comes from the complementary effects of the three core components. First, the Semantic Data Organization component improves data understanding TABLE V END-TO-END EXECUTION TIME COMPARISON. DatasetData AgentBaselineSpeedup DABStep2.8 min6.5 min (Qwen3 ReACT Baseline) 2.32× Spider 2.0-Lite5.1 min7.4 min (ReFoRCE)1.45× TABLE VI TOKEN USAGE COMPARISON ON DABSTEP (REACT IS QWEN3 REACT). Category Input Tokens (k)Output Tokens (k) DATAAGENTReACTDATAAGENTReACT Simple Query16.933.30.912.93 Single Data Statistic17.0120.30.852.56 Multiple data Statistic18.0110.21.103.67 Matching with Complex Logic27.6178.52.9413.68 TABLE VII TOKEN USAGE COMPARISON ON SPIDER 2.0-LITE. Category Input Tokens (k)Output Tokens (k) DATAAGENTReFoRCEDATAAGENTReFoRCE Aggregation Query10.871.22.9811.32 Multi-table Query18.588.44.796.24 Nested Query 20.464.74.828.13 Recursive Data Analysis 18.552.86.7613.21 and source grounding, allowing the planner to identify relevant tables, documents, and attributes earlier and thus reducing unnecessary trial-and-error exploration. Second, the Feedback Loop detects semantic deviations and logical inconsistencies during orchestration, preventing erroneous plans from propa- gating into expensive downstream execution failures. Third, the Memory Management component accumulates reusable dataset-specific procedures, successful workflow patterns, and historical corrections, enabling the system to reuse previously effective orchestration strategies instead of repeatedly explor- ing similar search spaces from scratch. Iteration Step Reduction. To better understand where the efficiency gain comes from, Figure 3 further compares the average number of reasoning-and-execution steps across query categories. Here, a step denotes one complete orchestration or execution iteration in the orchestration pipeline, including intermediate verification and potential local repair. This metric captures the overall interaction complexity of the full data analytics workflow. We have two key observations. First, DATAAGENT consistently takes fewer steps across nearly all query categories, showing that the architecture reduces unnecessary exploration and repair cycles throughout the full pipeline. Second, as query difficulty increases, the step growth of DATAAGENT is flatter than the baselines, especially on categories involving multiple data sources, nested logic, or complex analytical goals. This indicates that the combined effects of semantic data understanding, early feedback-based correction, and reusable memory effectively control the expan- sion of orchestration horizons in complex scenarios. LLM Cost Reduction. Tables VI and VII further compare token usage. DATAAGENT uses fewer tokens, and the token growth remains more stable as task difficulty increases. On DABStep, DATAAGENT reduces input tokens from 33.3k (by Qwen3 ReACT Baseline) to 16.9k on simple queries, and from 178.5k to 27.6k on matching tasks with complex logic, achieving up to a 6.5× reduction. This difference is mainly because the baseline approach tends to operate over larger raw contexts and rely on repeated exploration during multi-step 12 reasoning, leading to substantial token consumption. In con- trast, DATAAGENT leverages semantic data organization and memory to focus on relevant data and reuse prior knowledge, resulting in more compact and efficient context usage. 7. RELATED WORK AND OPEN PROBLEMS We classify the capabilities of data agents into five levels (L1–L5) according to their degree of autonomy and learning ability. L1 designates the data agent as a human-assisted copi- lot, with humans taking the lead while the agent assists users through prompt-based tool invocation and RAG [60], [6], [56], [2], [16], [36], [43], [42]. L2 facilitates task-oriented agent automation, where users specify high-level intents, and the agent independently manages task decomposition, planning, orchestration, reasoning, and decision-making [45], [15], [35], [23], [3], [59], [9], [67], [49], [62], [31], [24]. L3 makes a shift to an agent-led workflow and positions the data agent as the lead, continuously enhancing its capabilities through iterative human feedback, continuous learning, and collaboration with multiple agents. This allows the system to adapt, generalize across tasks, and improve with experience [65], [53], [63], [13], [64], [44], [52], [28], [51], [48], [46], [50], [57], [14]. L4 proactively initiates tasks, autonomously detects events, and adaptively addresses the events with complex collabo- ration and short-term and long-term memory management techniques. L5 represents a fully autonomous agent capable of operating independently, performing tasks from start to finish without human intervention. Although this paper proposes a data agent system and develops key techniques to enhance result quality and reduce LLM token costs, several open challenges remain. Self-Evolving Data Agents. Self-evolving data agents aim to improve their ability to discover, query, analyze, and manage data through accumulated experience. However, achieving reliable and sustained improvement raises several open chal- lenges. First, reliable feedback and credit assignment remain difficult. A successful execution does not necessarily imply a correct analysis: an agent may generate executable SQL while using inappropriate joins, misleading aggregations, or unsup- ported assumptions. Feedback must distinguish execution suc- cess from semantic correctness and identify which decisions contributed to an outcome. Second, safe and stable evolution requires balancing adaptation with preservation of established capabilities. Updates to prompts, memories, tools, or policies may improve one workload while degrading another. Deter- mining when to update, how to validate changes, and when to roll back remains challenging, particularly under changing schemas and data distributions. Third, memory quality and transferability are unresolved. Agents must determine which experiences to retain, reconcile contradictory observations, and avoid treating context-specific solutions as general rules. Effective reuse requires tracking provenance, applicability conditions, and expiration, while respecting privacy and access boundaries. Advanced Proactive Capabilities. DATAAGENT supports ba- sic proactive capabilities, including adapting semantic data catalogs to new data sources, adjusting vector indexes to shifting query distributions, and leveraging long-term memory to reuse effective pipeline patterns. However, more advanced proactive behaviors—such as autonomously detecting events, launching tasks without user prompts, and executing fully self-directed workflows—remain significant research hurdles. Addressing these challenges will require breakthroughs in con- tinuous monitoring, autonomous goal formulation, and robust self-healing mechanisms to enable our data agents to operate independently and adaptively in dynamic environments. Data Security and Privacy Issue. Deploying data agents in industrial settings poses some challenges. (1) Data Secu- rity and Privacy: Robust measures like data masking, local model deployment, and strict access control are needed to protect sensitive information such as personally identifiable information (PII) and prevent data leakage during LLM calls or memory sharing. (2) Regulatory Compliance: Industries must adhere to regulations like GDPR, requiring systems to ensure auditability and traceability. (3) Domain Knowledge Integra- tion: Many tasks depend on specialized domain knowledge not captured during LLM training, making efficient integration and updating of this knowledge critical. (4) Robustness under Extreme Conditions: Adversarial inputs and distribution shifts demand self-healing mechanisms and enhanced resilience. Multimodal Data Agents. Analysts regularly work across diverse modalities, including textual reports, structured tables, time-series logs, visual charts, source code, videos, audios, and knowledge graphs. A key challenge is the semantic and structural misalignment between these modalities. 8. CONCLUSION We propose a data agent paradigm to revolutionize data sys- tems by autonomously and proactively handling data-related tasks with semantic capabilities. Our data agent system is designed with components for semantic data organization, semantic operators, agentic pipeline orchestration, feedback mechanisms, memory management, and proactive adaptation. We develop a data analytics agent and a data science agent. Experimental results demonstrate that our data agents signifi- cantly outperform state-of-the-art methods. ACKNOWLEDGMENT This paper was supported by NSF of China (62525202, 62232009),NationalKeyR&DProgramofChina (2023YFB4503600), and Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China(JYB2025XDXM509). REFERENCES [1] B. Abu-Salih, P. Wongthongtham, D. Zhu, K. Y. Chan, and A. Rudra. Introduction to big data technology. In Social Big Data Analytics: Practices, Techniques, and Applications, pages 15–59. Springer, 2021. [2] F. Biester, M. Abdelaal, and D. D. Gaudio. Llmclean: Context-aware tabular data cleaning via llm-generated ofds. In ADBIS, 2024. [3] L. Cao. Tablemaster: A recipe to advance table understanding with language models. CoRR, abs/2501.19378, 2025. [4] C. Chai, N. Tang, J. Fan, and Y. Luo. Demystifying artificial intelligence for data preparation. In SIGMOD, pages 13–20. ACM, 2023. [5] D. Chen, Y. Huang, Z. Ma, H. Chen, X. Pan, C. Ge, D. Gao, Y. Xie, Z. Liu, J. Gao, et al. Data-juicer: A one-stop data processing system for large language models. In SIGMOD, pages 120–134, 2024. [6] S. Dharwada, H. Devrani, J. R. Haritsa, and H. Doraiswamy. Query rewriting via llms. CoRR, abs/2502.12918, 2025. 13 14 [7] D. Gao, H. Wang, Y. Li, X. Sun, Y. Qian, B. Ding, and J. Zhou. Text- to-sql empowered by large language models: A benchmark evaluation. Proc. VLDB Endow., 17(5):1132–1145, 2024. [8] H. Gao, J. Geng, W. Hua, M. Hu, and et al. A survey of self-evolving agents: On path to artificial super intelligence. CoRR, 2507.21046, 2025. [9] Y. Ge, Y. Liu, Y. Mao, and Y. Gao.Text-to-pipeline: Bridging natural language and data preparation pipelines.arXiv preprint arXiv:2505.15874, 2025. [10] S. Guo, C. Deng, Y. Wen, H. Chen, Y. Chang, and J. Wang. Ds-agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453, 2024. [11] M. M. Hassan, A. Knipper, and S. K. K. Santu. Chatgpt as your personal data scientist. arXiv preprint arXiv:2305.13657, 2023. [12] H. Herodotou, H. Lim, G. Luo, N. Borisov, L. Dong, F. B. Cetin, and S. Babu. A self-tuning system for big data analytics. In CIDR, 2011. [13] S. Hong, Y. Lin, B. Liu, and et al. Data interpreter: An LLM agent for data science. In ACL, pages 19796–19821, 2025. [14] R. Hu, Y. Luo, G. Li, S. Wu, and Y. Luo. Opensql: Data-efficient text-to- sql for open-source llms via synthesized intermediate supervision. Proc. VLDB Endow., 19(7):1628–1642, 2026. [15] X. Hu, Z. Zhao, S. Wei, Z. Chai, and et al. Infiagent-dabench: Evaluating agents on data analysis tasks. In ICML. OpenReview.net, 2024. [16] Q. Huang and T. Zhao. Leveraging large language models for entity matching. CoRR, abs/2405.20624, 2024. [17] T. Klaeger, S. Gottschall, and L. Oehm. Data science on industrial data— challenges in brown field applications. Challenges, 12(1), 2021. [18] M. Kunjir and S. Babu. Black or white? how to develop an autotuner for memory-based analytics. In SIGMOD, 2020. [19] H. Lan, T. Wang, Z. Bao, G. Li, D. Ji, G. Lee, F. Luo, Z. Huang, H. Qiu, and G. Hua. Agenticscholar: Agentic data management with pipeline orchestration for scholarly corpora. Proc. ACM Manag. Data, 4(3):131:1–131:28, 2026. [20] B. Li, C. Chen, Z. Xue, Y. Mei, and Y. Luo. Deepeye-sql: A software- engineering-inspired text-to-sql framework. CoRR, 2510.17586, 2025. [21] B. Li, Y. Luo, C. Chai, G. Li, and N. Tang. The dawn of natural language to SQL: are we fully ready? [experiment, analysis & benchmark]. Proc. VLDB Endow., 17(11):3318–3331, 2024. [22] B. Li, Y. Peng, Y. Xie, S. Lu, Y. Zhu, X. Mu, X. Liu, and Y. Luo. Deepeye: A steerable self-driving data agent system. In SIGMOD, 2026. [23] B. Li, J. Zhang, J. Fan, Y. Xu, C. Chen, N. Tang, and Y. Luo. Alpha- sql: Zero-shot text-to-sql using monte carlo tree search. arXiv preprint arXiv:2502.17248, 2025. [24] G. Li, J. Sun, J. Pan, J. Wang, Y. Xie, R. Liu, and W. Nie. Gaussdb- vector: A large-scale persistent real-time vector database for LLM applications. Proc. VLDB Endow., 18(12):4951–4963, 2025. [25] G. Li, J. Wang, C. Zhang, and J. Wang. Data+ ai: Llm4data and data4llm. In SIGMOD, pages 837–843, 2025. [26] X. Li. A review of prominent paradigms for LLM-based agents: Tool use, planning (including RAG), and feedback learning. In ACL, 2025. [27] Z. Li, Q. Zang, D. Ma, J. Guo, T. Zheng, M. Liu, X. Niu, Y. Wang, J. Yang, J. Liu, et al.Autokaggle: A multi-agent framework for autonomous data science competitions. arXiv arXiv:2410.20424, 2024. [28] C. Liu, M. Russo, M. Cafarella, L. Cao, P. B. Chen, Z. Chen, M. Franklin, T. Kraska, S. Madden, R. Shahout, et al. Palimpzest: Optimizing ai-powered analytics with declarative query processing. In CIDR, page 2, 2025. [29] J. Luo, W. Zhang, Y. Yuan, and et al.Large language model agent: A survey on methodology, applications and challenges. CoRR, abs/2503.21460, 2025. [30] T. Luo, C. Huang, L. Shen, B. Li, S. Shen, W. Zeng, N. Tang, and Y. Luo.nvbench 2.0: Resolving ambiguity in text-to-visualization through stepwise reasoning. In NeuIPS, 2025. [31] Y. Luo, G. Li, J. Fan, and N. Tang. Data agents: Levels, state of the art, and open problems. In Companion of the International Conference on Management of Data, SIGMOD Companion 2026, Bengaluru, India, 31 May 2026 - 5 June 2026, pages 571–579. ACM, 2026. [32] Y. Luo, X. Qin, N. Tang, and G. Li. Deepeye: Towards automatic data visualization. In ICDE, pages 101–112, 2018. [33] Y. Luo, N. Tang, G. Li, J. Tang, C. Chai, and X. Qin. Natural language to visualization by neural machine translation. IEEE Trans. Vis. Comput. Graph., 28(1):217–226, 2022. [34] Y. Luo, Y. Zhou, N. Tang, G. Li, C. Chai, and L. Shen. Learned data- aware image representations of line charts for similarity search. Proc. ACM Manag. Data, 1(1):88:1–88:29, 2023. [35] G. Ouyang, J. Chen, Z. Nie, Y. Gui, Y. Wan, H. Zhang, and D. Chen. nvagent: Automated data visualization from natural language via collab- orative agent workflow. In ACL, 2025. [36] M. Parciak, B. Vandevoort, F. Neven, L. M. Peeters, and S. Vansum- meren. Schema matching with large language models: an experimental study. In VLDB Workshops. VLDB.org, 2024. [37] G. Penedo, H. Kydl ́ ıcek, L. B. Allal, and et al. The fineweb datasets: Decanting the web for the finest text data at scale. In NeurIPS, 2024. [38] G. Penedo, Q. Malartic, D. Hesslow, R. Cojocaru, et al. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116, 2023. [39] M. Pourreza and D. Rafiei. DIN-SQL: decomposed in-context learning of text-to-sql with self-correction. In NeurIPS, 2023. [40] X. Qin, Y. Luo, N. Tang, and G. Li. Making data visualization more efficient and effective: a survey. VLDB J., 29(1):93–117, 2020. [41] J. Ruan, Z. Xu, Y. Peng, F. Ren, Z. Yu, X. Liang, J. Xiang, Y. Chen, B. Liu, C. Wu, Y. Luo, and J. Zhang. Aorchestra: Automating sub-agent creation for agentic orchestration, 2026. [42] S. Sadruddin, J. D’Souza, E. Poupaki, A. Watkins, H. Babaei Giglou, A. Rula, B. Karasulu, S. Auer, A. Mackus, and E. Kessels. Llms4schemadiscovery: A human-in-the-loop workflow for scientific schema mining with large language models. In ESWC, 2025. [43] Y. Sui, M. Zhou, M. Zhou, S. Han, and D. Zhang. Table meets LLM: can large language models understand structured table data? A benchmark and empirical study. In WSDM, pages 645–654. ACM, 2024. [44] J. Sun, G. Li, P. Zhou, and et al. Agenticdata: An agentic data analytics system for heterogeneous data. CoRR, abs/2508.05002, 2025. [45] M. Sun, R. Han, B. Jiang, and et al. LAMBDA: A large model based data agent. CoRR, abs/2407.17535, 2024. [46] Z. Sun, J. Wang, X. Zhao, J. Wang, and G. Li.Data agent: A holistic architecture for orchestrating data+ ai ecosystems. IEEE Data Engineering Bulletin, 2025. [47] Z. Sun, S. Zhong, D. Wen, J. Han, G. Li, Y. Yan, P. Zhang, Y. Su, X. Qi, B. Sun, C. Yang, T. Fang, and H. Ruan. Agenticdatabench: A comprehensive benchmark for data agents. VLDB, 2027. [48] Z. Sun, X. Zhou, G. Li, X. Yu, J. Feng, and Y. Zhang. R-bot: An llm-based query rewrite system. Proc. VLDB Endow., 2025. [49] Z. Tang, B. Niu, X. Zhou, B. Li, W. Zhou, J. Wang, G. Li, X. Zhang, and F. Wu. St-raptor: Llm-powered semi-structured table question answering. Proc. ACM Manag. Data, 2026. [50] Z. Tang, W. Wang, Z. Zhou, Y. Jiao, B. Xu, B. Niu, D. Zhou, X. Zhou, G. Li, Y. He, W. Zhou, Y. Song, C. Tan, X. Yang, C. Liu, B. Wang, C. He, X. Wang, and F. Wu. Llm/agent-as-data-analyst: A survey, 2025. [51] J. Wang and G. Li.Aop: Automated and interactive llm pipeline orchestration for answering complex queries. CIDR, 2025. [52] J. Wang, G. Li, and J. Feng. idatalake: An llm-powered analytics system on data lakes. IEEE Data Eng. Bull., 49(1):57–69, 2025. [53] J. Wang, Y. Li, J. Wu, S. Xu, and G. Li. Unify: A system for unstructured data analytics. Proc. VLDB Endow., 2025. [54] Y. Xie, Z. Zhang, Y. Wu, S. Lu, J. Zhang, Z. Yu, J. Wang, S. Hong, B. Liu, C. Wu, and Y. Luo. Visjudge-bench: Aesthetics and quality assessment of visualizations. CoRR, abs/2510.22373, 2025. [55] G. Xiong, J. Bao, and W. Zhao. Interactive-kbqa: Multi-turn interactions for knowledge base question answering with llms, 2024. [56] H. Xiu, L. Zhang, T. Zhang, J. Yang, and J. Chen. Query performance explanation through large language model for htap systems.arXiv preprint arXiv:2412.01709, 2024. [57] S. Xu, J. Wang, and G. Li. Bridging the gap: Cardinality estimation for semantic queries on unstructured data. SIGMOD, 2026. [58] C. Yang, Y. Luo, C. Cui, J. Fan, C. Chai, and N. Tang. Data imputation with limited data redundancy using data lakes. VLDB, 2025. [59] P. Yu, G. Chen, and J. Wang. Table-critic: A multi-agent framework for collaborative criticism and refinement in table reasoning. In ACL, 2025. [60] J. Zhang, G. Li, and J. Su. SAGE: A framework of precise retrieval for RAG. In ICDE, pages 1388–1401. IEEE, 2025. [61] S. Zhang, J. Fan, M. Fan, G. Li, and X. Du. Deepanalyze: Agentic large language models for autonomous data science. CoRR, 2510.16872, 2025. [62] Y. Zhang, M. Fan, J. Fan, M. Yi, Y. Luo, G. Li, B. Wu, and W. Zhou. Reward-sql: Boosting text-to-sql via stepwise execution-aware reasoning and process-supervised rewards. Proc. ACM Manag. Data, 4(3):228:1– 228:27, 2026. [63] Z. Zhang, Z. Liang, Y. Wu, T. Lin, Y. Luo, and N. Tang. Datamosaic: Explainable and verifiable multi-modal data analytics through extract- reason-verify. CoRR, abs/2504.10036, 2025. [64] X. Zhao, X. Zhou, and G. Li. Chat2data: An interactive data analysis system with rag, vector databases and llms.Proc. VLDB Endow., 17(12):4481–4484, 2024. [65] W. Zhou, J. Sun, X. Zhou, and G. Li. Gaussmaster: An llm-based database copilot system. CoRR, abs/2506.23322, 2025. [66] X. Zhou, J. He, W. Zhou, H. Chen, Z. Tang, H. Zhao, X. Tong, G. Li, Y. Chen, J. Zhou, et al.A survey of llm× data.arXiv preprint arXiv:2505.18458, 2025. [67] X. Zhou, G. Li, Z. Sun, Z. Liu, and et al. D-bot: Database diagnosis system using large language models. VLDB, 17(10):2514–2527, 2024. [68] Y. Zhu and et al. A survey of data agents: Emerging paradigm or overstated hype? CoRR, abs/2510.23587, 2025.

Instant research discovery