Paper deep dive
Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners
Shani C Spivak, Aditi Krishna, Mahsan Nourani, Melanie Tory
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/29/2026, 12:25:17 PM
Summary
This paper introduces 'vibe analysis,' a new workflow where data visualization (Vis) practitioners use Large Language Models (LLMs) to iteratively create and refine visualizations via natural language prompts, analogous to 'vibe coding.' Based on semi-structured interviews with 18 members of the Data Visualization Society, the study finds that practitioners use LLMs for both creative (ideation, design) and technical (wrangling, coding) tasks. Key challenges include LLM hallucinations, gaps in Vis-specific knowledge, and the need for chart verification. The authors propose future research into Vis-specific LLM guidance and integrations with low-code tools.
Entities (8)
Relation Signals (6)
Large Language Models → enables → Vibe Analysis
confidence 95% · Our findings show that Vis designers actively use LLMs for both creative and technical aspects of the visualization process. A new visualization workflow is emerging, a process we call vibe analysis
Vibe Analysis → isanalogousto → Vibe Coding
confidence 95% · vibe analysis is analogous to vibe coding, a programming paradigm where developers primarily generate code via interaction with code-generating LLMs
Vibe Analysis → involves → Natural Language Prompts
confidence 92% · users analyze data and generate visualizations by interacting with LLMs through natural language (NL) prompts
Shani C. Spivak → affiliatedwith → Northeastern University
confidence 90% · Shani C. Spivak Affiliation: Northeastern University
Large Language Models → posesriskof → Hallucination
confidence 90% · they come with challenges and limitations, including hallucinations, deceptive behavior, and errors
Data Visualization Society → providedparticipantsfor → Vibe Analysis
confidence 90% · we conducted semi-structured interviews with members of the Data Visualization Society
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are enticing in their promise to support data visualization (Vis) through faster and simpler workflows for data prep, analysis, and visualization creation. Yet LLMs are notoriously error-prone and not built for data visualization tasks. Few studies have explored LLM adoption among Vis practitioners. To fill this gap, we conducted semi-structured interviews with members of the Data Visualization Society, a global community of data visualization designers. Our findings show that Vis designers actively use LLMs for both creative and technical aspects of the visualization process. A new visualization workflow is emerging, a process we call vibe analysis, analogous to vibe coding. Some key challenges raised by participants parallel those of vibe coding, while others are Vis-specific, like gaps in Vis knowledge and chart verification. This work opens up opportunities for research combining LLM-mediated work with Vis tools that incorporate data visualization guidance, constraints, and best practices.
Tags
Links
- Source: https://arxiv.org/abs/2609.31922v1
- Canonical: https://arxiv.org/abs/2609.31922v1
Trouble viewing inline? Open PDF directly →
Full Text
72,667 characters extracted from source content.
Expand or collapse full text
X 8 /June FEATURE ARTICLE Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners Journal: IEEE Computer Graphics & Applications Shani C. Spivak Affiliation: Northeastern University, Boston, MA 02125, USA Aditi Krishna Affiliation: Northeastern University, Boston, MA 02125, USA Mahsan Nourani Affiliation: Northeastern University, Portland, ME 04101, USA Melanie Tory Affiliation: Northeastern University, Oakland, CA 94613, USA 2026 Abstract Large language models (LLMs) are enticing in their promise to support data visualization (Vis) through faster and simpler workflows for data prep, analysis, and visualization creation. Yet LLMs are notoriously error-prone and not built for data visualization tasks. Few studies have explored LLM adoption among Vis practitioners. To fill this gap, we conducted semi-structured interviews with members of the Data Visualization Society, a global community of data visualization designers. Our findings show that Vis designers actively use LLMs for both creative and technical aspects of the visualization process. A new visualization workflow is emerging, a process we call vibe analysis, analogous to vibe coding. Some key challenges raised by participants parallel those of vibe coding, while others are Vis-specific, like gaps in Vis knowledge and chart verification. This work opens up opportunities for research combining LLM-mediated work with Vis tools that incorporate data visualization guidance, constraints, and best practices. Conversational AI agents powered by Large Language Models (LLMs)11 1 For brevity, we use the term LLM in lieu of conversational AI agent throughout this paper. have the potential to enable a major leap forward in data visualization workflows. Visualization work can be complex; practitioners must balance various design goals with limitations related to data quality, access, computation, and tool knowledge. Moreover, professionals across many roles are increasingly performing data analytics tasks even when data work is not their core responsibility. LLMs can support this work by offering design guidance or partially automating complex data pipelines. We present an interview study that explores whether LLMs are living up to this promise. Although LLMs have potential to bridge the gap in skill and speed demanded of data visualization (Vis) tools and workers, they come with challenges and limitations, including hallucinations, deceptive behavior, and errors [1]. Additionally, LLM use has been shown to impact how people approach, process, and remember task-related information [2, 3]. In Vis work, these impacts also relate to audience perception (how people interpret visualizations, data or topic biases, design issues, or viewer assumptions). Data visualizations are often perceived as objective and factual, even when they are subjective, misleading, or incorrect [4]. Given the challenges of effectively detecting LLM errors, overreliance on LLMs in Vis work can be risky; undetected errors can lead users to costly misunderstandings and misinformed decisions. So how are data visualization practitioners using LLMs? Understanding current practices and challenges around the use of LLMs for data work is a critical first step to building better interaction tools, guidance, and governance. To understand how visualization practitioners are using LLMs, we first conducted a survey advertised via the Data Visualization Society (DVS), a cross-functional, tool-agnostic community for sharing Vis design ideas and practices. We followed this with semi-structured interviews with a subset of participants to collect examples of LLM use, verification practices, and insights into participants’ trust in and reliance on these tools. We find that visualization practitioners already use LLMs across much of their work, in creative and surprising ways. Based on an analysis and synthesis of our interviews, we introduce the process of vibe analysis, which we define as an iterative analytic approach where users analyze data and generate visualizations by interacting with LLMs through natural language (NL) prompts rather than direct manual creation. Vibe analysis is analogous to vibe coding [5], a programming paradigm where developers primarily generate code via interaction with code-generating LLMs through NL prompts, but with several data visualization-specific differences discussed in our findings. Our findings open up opportunities for future work on LLMs to better support Vis work through research into Vis-specific guidance for LLMs, Vis-specific LLM training for users, better integrations of LLM and no-code or low-code Vis tools, and adoption of a combination of LLM and Vis tools in ways that trade off the strengths and weaknesses of each. Our work contributes: 1) An introduction and in-depth description of vibe analysis, a conceptual model of a process by which visualization practitioners engage with LLMs for data work; 2) Information about how Vis practitioners are using LLMs, challenges and benefits in this process, and opportunities to facilitate this work; and 3) A call to action to the visualization community to think critically about where and how we apply LLMs and how we learn or train others to use them to present data. Related Work User Preferences and Adoption of LLMs Beyond user perceptions for data visualizations [4], the impacts of LLM use on people who use them [3], and what data workers need from their tools [6, 7], previous work has also centered on user expectations and preferences for interactions with analytic chatbots. For instance, Qian et al. [2] studied LLM adoption among practitioners at a large technology company, characterizing participants’ perceptions, integration strategies, and reported usage scenarios. They showed how LLMs allow high-level insights from the start, reversing a previous bottom-up approach (i.e., data prep first, then insights). They also found that LLMs enabled automated insight discovery and data generation, though they cautioned that some of these changes could result in a shallower understanding. Closest to our work is an interview study by Schetinger et al. [8] with 21 experts (across visualization, machine learning, art, and art history) to examine implications of generative models for visualization, identifying opportunities like rapid prototyping, while highlighting concerns about bias amplification and misleading visualizations. In contrast to their work, our study focuses specifically on visualization practitioners; moreover, our findings reveal additional insights about user trust, verification practices, and challenges in using LLMs for Vis. Although these studies contribute to a general understanding of data practitioner needs, none identify the approach we describe as vibe analysis. Furthermore, we survey practitioners across countries, sectors, and roles and consider all Vis work, allowing for a more comprehensive view of adoption trends, user practices, and roadblocks. Vibe Coding The way our participants described LLM use closely matches a collaborative, iterative process for software development known as vibe coding. In February of 2025, Andrej Karpathy introduced the term vibe coding in a tweet, describing it as a way to abstract away from the programming language itself to develop software essentially through conversation with an LLM. Hailed for enabling the offloading of much of the code-writing behind software development, making development more accessible to non-coders, vibe coding is often described as opening up new creative possibilities to people who would not otherwise have the opportunity to code [5]. Chou et al. [9] explored how software developers actually engage in vibe coding, revealing a variety of behaviors and degrees of reliance. They highlight how practitioners review, refine and correct, always contending with the stochastic nature of generation, what they describe as “rolling the dice”. It can be expected, then, that other creative practices that depend on coding may benefit from a similar approach, but to our knowledge, no such process definition exists for data analysis or visualization. Visualization activities differ from coding in that Vis work also involves domain-specific tools, rules, heuristics, and constraints, many of which are learned by practitioners but are not necessarily part of model training or common knowledge to lay users of LLMs. Still, just as LLMs have seeped into the software development chain [10, 11], they are making their impact in Vis. While vibe coding has been explored across industry and academia [5, 9, 12, 13, 14, 15], to our knowledge, our work is the first interview study to provide a sector and role-agnostic view of how LLMs are being used in Vis processes (not just one part of the Vis pipeline or for a certain type of Vis). We offer a view of LLM adoption, including practices and limitations that can be used to improve interaction. Our work introduces vibe analysis as a practice and describes its defining characteristics, benefits and challenges. Methodology Our goal was to characterize how LLMs are used by practitioners in their Vis work. Specifically, we wanted to understand current practices and challenges in the use of LLMs for visualization (RQ1) and in how practitioners verify LLM responses in this context (RQ2). To address these questions, we designed a semi-structured interview for Vis professionals. To find these practitioners, we first developed a screening survey and distributed it via the Data Visualization Society (DVS). Survey participants could opt-in to be contacted for a paid interview. The study was approved by our Institutional Review Board and informed consent was obtained. Survey questions, interview protocol, and other study materials are available in the supplemental material. Participants & Interview Protocol We manually reviewed responses to the screener survey to validate participants and ensure representation across diverse data work needs and tasks. This led to recruiting and interviewing 18 Vis practitioners (4 Female, 13 Male, 1 Prefer not to say) across 8 countries. Interviews were conducted between September and October, 2025. See supplemental material for more information about interview participants. Interview Design and Protocol Interview questions built on the questions asked in the screening survey. We aimed to gather in-depth information about participants’ use of LLMs in their Vis work through examples, questions about challenges, strategies, verification processes and concerns. We divided questions into three groups: 1) Questions about Vis Expertise (e.g., As related to data visualization, which of the following describe your role in the past year? Which tools do you tend to use for data visualization?), 2) Questions about LLM use to contextualize their comfort level and general use (e.g., For what part of the data visualization process do you use Large Language Models (LLMs)?), and 3) Demographic Questions (e.g., age range, gender). We included questions about trust and verification because we were interested in practitioners’ perspective on these, but had no guiding framework for this and all qualitative coding was bottom-up. Please see Supplemental Material for the full survey and interview guide. Two researchers on our team carried out the interviews virtually over Microsoft Teams. Average interview duration was 28m (SD: 8m). Sessions were recorded and transcribed. We anonymized all transcriptions before analysis. Researchers also took notes of the most important points in the discussion, which were used in one case where the recording failed. Participants who completed the interview received a $30\$30 (USD) gift card. Qualitative Analysis For each transcript, we applied inductive thematic coding [16]. First, two researchers independently conducted a round of open coding, followed by a session to reconcile and consolidate themes. The coding process included a mix of in vivo codes and researcher-denoted concepts. New codes were developed throughout transcript review. Finally, both researchers independently performed closed coding and then results were merged. There were some instances where only one coder noted a finding, but otherwise any differences were resolved through discussion. Several themes were identified as part of focused coding and additional review, which we discuss next in the Findings Section. Crisan et al. [6] inspired us to examine where in the Vis pipeline participants were using LLMs. However, for Vis pipeline components, we adopted naming conventions and groupings of higher and lower order processes based on descriptions provided by our participants; this ensures that the pipeline components presented here directly reflect the results of our interviews. Findings Our participants described an iterative process of working with LLMs to accomplish data tasks across various stages of the visualization pipeline. Below, we characterize what we call vibe analysis (see Figure 2 for a conceptual model). In the subsections below, we first describe vibe analysis and then discuss themes around the changing nature of visualization work, strategies employed by participants, and challenges they brought up. The scope of Vis work described here reflects what participants described in interviews and framing from related research [6, 17], but this is inherently non-comprehensive, as other aspects of data analysis and visualization are also likely to be impacted by LLM use. Throughout the findings, ratios indicate the number of interviewees out of 18, unless otherwise noted. These values are descriptive indicators, but should not be taken as estimates of prevalence among visualization practitioners more broadly. Information about specific LLMs participants mentioned using is provided in Figure 1 and more detailed information about which tools participants use is in Table 3.1 of the supplemental material. Figure 1: LLM tools our participants reported using Figure 2: Conceptual model of vibe analysis with portions inspired by the data storytelling lifecycle [17] and the vibe coding cycle [5]. This is an author-developed synthesis informed by the interview analysis and these existing models. Vibe Analysis One of our participants described their use of LLMs as: P9: “[…] comparable to me taking somebody’s data, mocking it up really quick and sending them a screenshot. […] another use case is not vibe coding, but vibe modeling [when] I want to see what a fake dataset will look like.” Like this participant, many others described working with LLMs in loosely organized sessions, prompting LLMs with goals rather than specific directions. Most participants (11/18) used anthropomorphic language in relation to LLMs, and indicated a kind of collaborative design process that differs from more traditional Vis tool use in which data is organized and visualized according to the tool-specific processes (e.g., Tableau’s interface or Excel’s chart options). Practitioners used vibe analysis to quickly get a workable overview or chart(s), often using LLMs to brainstorm (ideate), wrangle, explore, and visualize data. Vibe analysis is related to vibe coding, a coding approach that relies on LLMs, allowing programmers to generate working code by providing natural language descriptions rather than manually writing it [5]. We use vibe analysis as an umbrella term for the full process of LLM-mediated visualization work. Like vibe coding, vibe analysis involves an iterative process of completing data tasks and creating visualizations, where much of the work is done through NL prompts to an LLM rather than direct specification. Humans shift from direct code-writing / pipeline specification to articulating goals, curating content, and evaluating quality. Figure 2, adapted from Ge at al. [5], shows a conceptual model of vibe analysis that developed as a result of our coding process. As in any visual analytics process, the practitioner works through a series of analytics tasks (green box, bottom left) including planning & ideation, data collection & prep, exploration, review & refinement, and finishing touches. Any of these activities might be supported by, or primarily conducted through, an LLM assistant. The iterative vibe analytics loop (center of figure) involves human specification of NL instructions to an LLM, code creation by the LLM, and review of results and/or code by the analyst. This may all happen within the LLM’s UI, may be orchestrated via more than one LLM, or may occur in concert with traditional Vis tools (e.g., Tableau, Power BI, Excel) and separate code environments (e.g., Observable, Svelte). Various external factors (bottom right) contextualize this process and may influence how it unfolds. The central cycle in orange arrows includes the cyclical and iterative actions represented by the other orange arrows—the user providing instructions to the LLM and reviewing project components, the LLM providing feedback and questions to the user and producing code, and the project’s interaction environment providing both context for the LLM and a space to review and iterate on results. In comparison to the vibe coding conceptual model [5], our vibe analytics conceptual model includes the Vis pipeline, and updates project context to more closely mirror Vis-specific constraints like data access and format restrictions and architecture needs. We also incorporate external factors noted in Riche et al.’s data storytelling process [17] because all of these factors came up in interviews; this includes deadlines, LLM costs, data restrictions, and applicable law and policy. Components of the vibe analytics process, as observed in our interviews, are described in detail in the sections below. Visualization Goals Practitioners noted use of LLMs throughout the Vis development process, illustrated by activities in the green box of Figure 2. While the phases and analytics tasks align with existing approaches to Vis work, participants described a shift in the way they work as a result of using LLMs. In Table 1, we break down aspects of the Vis pipeline grouped by stage (planning and ideation, data collection and prep, exploration, review and refine, and finishing touches), and detail these higher order processes with the percent of participants who mentioned using LLMs for the underlying tasks. Table 1: Parts of the Vis pipeline for which participants mentioned using LLMs. We distinguish between processes which are mostly technical (in blue) and those which are mostly creative (in orange). Participants are ordered from the left with those using LLMs across more processes to those using them for fewer on the right. Higher-order processes Lower-order processes Count of participants using LLMs P4 P7 P14 P6 P10 P11 P17 P5 P3 P9 P15 P12 P16 P18 P1 P2 P8 P13 Planning & Ideation Planning 2/18 ✓ ✓ Ideation 11/18 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Data Collection & Prep Definitions 1/18 ✓ Data Collection 4/18 ✓ ✓ ✓ ✓ Data Generation 1/18 ✓ Data Labeling 2/18 ✓ ✓ Data Wrangling 13/18 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Examples 2/18 ✓ ✓ Exploration Exploration 12/18 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Experimentation 3/18 ✓ ✓ ✓ Prototyping 5/18 ✓ ✓ ✓ ✓ ✓ Analysis/Modeling 5/18 ✓ ✓ ✓ ✓ ✓ Visualization Design 17/18 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Insights 3/18 ✓ ✓ ✓ Review & Refine Feedback 4/18 ✓ ✓ ✓ ✓ Programming 8/18 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Debugging 6/18 ✓ ✓ ✓ ✓ ✓ ✓ Reformatting 2/18 ✓ ✓ Verification 4/18 ✓ ✓ ✓ ✓ Finishing Touches Polish 3/18 ✓ ✓ ✓ Language/Storytelling 6/18 ✓ ✓ ✓ ✓ ✓ ✓ Website Support 1/18 ✓ Count of processes where LLMs are used 9 9 9 8 8 8 8 7 6 6 6 5 5 5 4 4 4 4 We found that most interviewees used LLMs for visualization design (17/18), data wrangling (13/18), exploration (12/18), and ideation (11/18). Some participants reported using LLMs for analysis (5/18), insight discovery (5/18), programming (8/18), debugging or troubleshooting support (6/18), and language or storytelling support (6/18). These findings demonstrate that Vis practitioners use LLMs for both creative and functional aspects of the visualization pipeline, which was somewhat surprising to us; we anticipated that practitioners would prefer to offload technical and repetitive tasks to LLMs, basically vibe coding, while handling creative tasks themselves. Interestingly, use of LLMs for finishing touches (like reformatting or polish) were less common than for earlier stages of visualization work. In addition, LLMs were also underutilized for some repetitive activities, such as planning, data labeling, data generation, and reformatting. We delve further into each stage below. Planning & Ideation: At this stage, LLMs were used for preparatory tasks including organizing and brainstorming in the beginning of the pipeline. Overall, 11/18 of participants noted using LLMs in early phases, mostly for creative ideation but sometimes for exploring how to approach analysis and design. For example, P4 described using LLMs during the creative process: “[…] I might ask for suggestions on which chart types might suit a dataset or how to make a visualization more engaging […]”. P6 described prompting the LLM with starter information and asking for advice on an analysis plan: “I write a small prompt […] maybe 10 lines just stating what do I want and […] I ask the chat to improve this small prompt to make it into a plan for some like visualization or code change.” Some participants did raise concerns about the potential for a LLM to warp or constrain creativity (P9: “Never ideation. I don’t want it to corrupt my brain.”, P1: “I never use [it] for giving me ideas about […] charts because I know they are giving four basic charts.”). Data Collection & Prep: LLMs were used to guide or manage data tasks including exploring data definitions and examples, data collection or generation, data labeling, and data formatting. 14/18 of participants mentioned using LLMs for data collection and prep. Most participants described using LLMs for data wrangling tasks, such as data cleaning, processing, enrichment, and prep (P1: “Yes, very much a time saving thing […] cleaning is one of the most important parts”). Some participants also noted using LLMs in data collection (4/18) and synthetic data generation (1/18). Other tasks included asking for definitions, examples, and data labels. Exploration: This includes using LLMs to conduct exploratory data analysis, statistical analysis and other data modeling, along with experimentation and prototyping. Other key LLM tasks at this stage include insight generation and visualization design. This stage had the highest rate of LLM use at 17/18. Many participants noted a preference for using LLMs for more complex visualizations (P13: “I have tried to use LLMs to create data visualizations. I thought it would be easy here for me for example to make like a map because you know a bar chart I can do just as quickly in myself;” P7: “[…] to build more complex data visualization and I mean like dashboards or web-based interface when you need to navigate […] to build a product or a final output.”). Also common was exploration to identify trends and better understand the data. Some participants noted offloading much of the analytic work to LLMs (P3: “the analysis is being taken care of by the LLM.”), while others used it for rapid prototyping and other experimentation (P10: “I create rough outlines of things using LLM-based sketches for moving towards production.”). Review & Refine: In one of the last stages of the pipeline, participants noted use of LLMs for feedback, reformatting, programming and debugging support. This is also where those who use LLMs for verification do so. Overall, 12/18 of participants noted using LLMs in this stage, with support in programming (8/18) as the most frequently noted. For instance, P5 reported using AI to get, “[…] code snippets, maybe to brush up my code or even get a much better code snippet that helps me […]”. Others mentioned using LLMs to seek feedback on visualization design (4/18), for debugging or troubleshooting (6/18), and for checking its own or another model’s work (4/18). Finishing Touches: In these last steps, LLMs are used to improve language and storytelling, add polish to an otherwise finished product, or support website integration. Only 8/18 of participants noted using LLMs for finishing touches, with language and storytelling support as the most common (6/18). This usage reflects both language support for non-native English speakers (P7: “I also find it useful to kind of write the text in English and pass through an LLM to improve fluency”) and for storytelling (P17: ‘So I believe that with the use of [LLMs] I can improve my visual[s] and also improve my storytelling.”). We speculate that lower usage of LLMs in applying finishing touches might be due to the lack of consistency and precision. P13 described a challenge with getting the LLM to appropriately apply finishing touches: “I asked it to shade, you know, the following states, and it did it brilliantly, except it forgot one. And so then when I asked it to add that one, it added that one plus the Eastern seaboard. And then I’m like, well, wait, what happened? And then it like uncolored the Eastern seaboard and put in all the Canadian provinces.” P13 eventually had to give up, “And so despite many attempts to use the [LLM] to make this map, I ended up having to hand do it with dots […]. So it just didn’t look as nice, but it was at least accurate.” This example illustrates an important opportunity to improve LLM-supported vis tools to maintain a constrained and consistent behavior. It also highlights a concern about the stochastic nature of LLMs – what [9] et al. describe as rolling the dice – using prompt after prompt of similar intent to try to get to a good outcome knowing they’l get something new each time. The Interaction Environment The vibe cycle, model, and constraint elements in Figure 2 bear many similarities to vibe coding. We distinguish between constraints (which are project-based specifications and project team domain knowledge related to the interaction environment and model) and external factors (which include policy, time and other factors affecting, but not specific to a project, like data privacy regulations and organizational licenses). Recognizing the role of data sources, tools, and schemas as first class citizens in Vis, we draw together a set of constraints that may apply to any given Vis project. Constraint 1: Data, APIs. Data sources and APIs impact project requirements and constrain analytic options. P14: “The software that is created is using R and we connect it with the models through the API” P15: “I’ve used it for pulling data from APIs” Constraint 2: Data Schemas define how data is related, stored, and organized, therefore constraining analytic options. P14: “…send the model the structure of the database, not the database, just the schema” Constraint 3: Domain Knowledge. we adopt Ge et al.’s [5] description of domain knowledge, as documentation, specifications, and best practices. P18: “The transfer of information in hand to the machine gets difficult sometimes […] For example, if you would ask a fellow lab mate [to] explain them the same thing, it’s much easier because they would be much more aware of the context that’s around it.” Constraint 4: Architecture where vibe coding lists execute environment, we instead opt for architecture to include the various applications (Excel, Tableau, Power BI), coding languages and environments that may be integrated. P1: “Copilot because I have 365 subscription. So for Excel thing copilot I found is pretty much they can work with me for five hours, no questions.” The Model We adopt Ge et al.’s [5] descriptions of vibe coding components: 1) Tools: Definitions and signatures of callable tools (compilers, testing frameworks, version control); 2) Memory: Historical interaction memory (multi-turn dialogue context, prior decision records); and 3) Tasks: Current tasks (pending actions, task queue, execution status). External Factors Participants described many factors that influenced the vibe analytics process (see Figure 2, bottom right), mirroring the external factors identified by Riche et al. [17] for data storytelling. As Riche et al. noted, target audience considerations impact each step of the process, but are distinct from analytic and creative processes. Our interviews suggest external factors also impact constraints and the model; participants sometimes noted legal and cost considerations to their use of specific models or versions. External Factor 1: Legal. May include copyright or data use restrictions, among other Vis-specific legal implications like privacy protections. P13: “We have an internal copilot that only has our data […] So that’s what I’m using. Both because of the privacy issues, but also just frankly, because we’re restricted, I can’t use anything else here.” External Factor 2: Ethics. Ethical considerations related to the data being shared or the potential for misuse or misinterpretation. Misleading charts are a concern for the Vis community [18]. P9: “I think the parts that worry me are when you start asking it things […] about people where […] the whole thing is built on a dataset of racist humans and so the model itself will be racist in ways that no one will catch and it’l just […] keep kind of propagating that in ways that are harder to identify.” External Factor 3: Setting. What is the context of the work? Is it part of a presentation or interactive exhibit? Setting impacts design choices. P10: “You can find lots of examples on my blog where I have shared both snapshots and interactive versions of what I’ve created with LLMs.” P15: “I’m mostly a static person. I’m mostly building out [of] Illustrator or Excel, and it’s allowed me to expand on that into interactive pieces, which I just haven’t been able to build.” External Factor 4: Time. Will the audience be guided synchronously or will they view a project asynchronously? This may impact how features are presented and annotated. P18: “Let’s say if there’s a lot of variables included in […] multimodal data that we have collected, then we would love to have help from LLM there. What is the best way to represent it so that people can get the […] simplicity and the also the interaction between different modalities that we are trying to use?” External Factor 5: Cost. May include costs to access data, licensing, or proprietary Vis tools. P18: “So I think most of the newer models […] have a similarity in terms of intelligence […] especially in the free tier. We don’t have a paid version of any of these LLMs, so […] if the one free tier expires then […] we move to the other LLM.” External Factor 6: Medium. Choice of medium (e.g. scrollytelling site22 2 Refers to a website where visualizations, text, and media updates as the user scrolls., dashboard, static chart) has impacts across the Vis process. P4: “I found Claude useful when I need more narrative supports like writing […] descriptions […] because it’s better at longer structured text and I use Gemini for design related brainstorming.” Vibe Analysis and the Changing Nature of Data Work Our interviews indicate that LLMs are changing the way in which visualization and analytics tasks are accomplished by data practitioners. Overall, our interviewees tended to use vis tools when they had specific details to visualize or needed simple charts they could build quickly themselves, and used LLMs for more complex, vague, or text-heavy tasks. 5/18 of participants explicitly noted a shift in the kind of work they did or how they did it. Examples include shifting from using vis tools to coding (P2: “I would say it helped me improve my skills and save my time and it changed […] some working habits like shifted from a freely developed software to coding languages.”) or from coding to prompting (P6: “I can do more in less time, but the task has moved away from coding skills to more prompting, to creation, to select[ing] what is useful from the elements.”). Changes in how work is done with AI mean that practitioners are abstracting away many fine-grained details of design and development, representing a shift in how visualizations are crafted. Participants described many benefits to using LLMs in data work, including increased efficiency (14/18), insight discovery (11/18), and expanded capabilities (9/18). Efficiency was often described in terms of taking over or helping to speed up parts of a process. For example, P6 used LLMs to bootstrap code writing, “I don’t do anything from scratch any longer,” and P11 used LLMs for brainstorming, “Previously this ideation process could take months […] Now the same thing can be done in 15 days or one month.” Supporting insight discovery was described as complementary to human practices, often generating ideas that users may not have identified independently (P13: “There was one framing that [the LLM] did where I was like, I don’t know if I would have come up with [that] on my own.”). Expanded capabilities allowed users to code in unfamiliar languages, use tools without training, or develop products they could not otherwise, such as P15 being able to build interactive tools that they were unable to create before. Similar skill expansion has been observed in the use of vibe coding, enabling non-developers to build working applications [5]. This is largely beneficial, but could lead to technical debt (see the Challenges Section). Participants’ perception of the impact of LLMs on their jobs and the nature of work echo sentiments shared by developer communities and tech sector leadership [10, 11]. The fear of being replaced was frequently noted (P5: “So yeah, I actually worry that one day this language model would be good enough to take over my job”). Additionally, 5/18 of participants felt compelled to use AI for work, either because their employers told them as much, or because they would fall behind otherwise (P4: “But I think people who don’t learn to use them, they would be at a really great disadvantage.” P5: “It doesn’t seem practical in this day and time to try to get information and visualize it without the use of the language model”). This corroborates recent reporting that firms are pushing their employees to use AI in their work [11]. Vis Practitioners’ Strategies Changing Prompting and Interactions with LLMs (Pipeline Phase: all). Participants commonly noted error-handing strategies through a change in their interactions with the model. Prominent themes included giving the LLM more context or guidance (9/18), starting over (8/18), breaking tasks down into steps or chunks (6/18), and refining the prompt (6/18). About 4/18 ask the LLM to check its own work and 3/18 mentioned they ask the LLM for explanations. Error Prevention (Pipeline Phase: all). Practitioners reported strategizing to prevent potential LLM errors, for example by having a plan before getting started (4/18), using a Git repository for version control to enable easy, stepwise corrections (2/18)33 3 We did not ask about Github, it came up organically in only 2 of the 18 interviews., being very specific in prompting (5/18), describing the data (4/18) or providing a sample (1/18), and asking for pseudocode or code instead of trying to get directly to a visualization (1/18). Others opt to only use LLMs for repetitive tasks (e.g. providing the LLM with their own code and guidance on how to repeat a process) (3/18) or just for rough sketches or starting points (1/18) (P12: “So not trying to get it absolutely right through the LLM, I think [is] something I found [to be] key in my work with it. Let the results be as good as they are and then pick up […] and keep going on from there.”). More strategies are shown in Table 2, along with the challenges or concerns they may help mitigate. These instances showed participants adapting to LLM pitfalls. While many of these strategies could be considered best practices, their necessity puts the onus on users. Applications that explicitly support these strategies could help mitigate errors and improve user interactions. We explore this further in the Discussion. Table 2: Strategies for handling LLM responses mapped to applicable challenges. Column codes are expanded below. Strategy Common pitfalls Societal impacts Model quirks System limits User concerns CP1 CP2 CP3 CP4 CP5 SI1 SI2 SI3 SI4 SI5 SI6 SI7 MQ1 MQ2 SL1 SL2 SL3 UC1 UC2 UC3 UC4 UC5 Ask for alternatives ∙ Ask for code instead of chart ∙ ∙ ∙ Ask for explanations ∙ ∙ ∙ ∙ Ask for pseudocode ∙ ∙ ∙ Be specific ∙ ∙ ∙ ∙ Break things down ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ Debug/troubleshoot ∙ ∙ Describe data structure ∙ Git repo version control ∙ Give a data sample ∙ Give more guidance ∙ ∙ ∙ ∙ ∙ ∙ ∙ ∙ Have a plan ∙ ∙ ∙ ∙ ∙ Limit expectations ∙ ∙ LLM does the checking ∙ ∙ ∙ ∙ Only use for repetitive tasks ∙ ∙ ∙ ∙ ∙ ∙ ∙ Refine the prompt ∙ ∙ ∙ ∙ ∙ ∙ Slow down ∙ Start again ∙ ∙ ∙ ∙ ∙ ∙ Take a break ∙ Use multiple models ∙ ∙ ∙ Common pitfalls CP1 Errors and hallucinations CP2 Lack of consistency CP3 Misinterpretation CP4 LLM does not know VIS well CP5 Time/effort to correct or finalize LLM quirks MQ1 Rabbit holes MQ2 Verbosity Societal impacts SI1 Bias SI2 Data leak/privacy/cyber SI3 Deepfakes SI4 Environmental impact SI5 Epistemic uncertainty/unknown training data SI6 Lack of trust SI7 Misinformation System limits SL1 Number of tokens SL2 Single point of failure SL3 Some data topics trigger guardrails User concerns UC1 Do not really learn to code UC2 Lack of user skill/training UC3 Limiting/directing thought process UC4 Overreliance UC5 Overtrust Trust Calibration (Pipeline Phase: all). Most participants mentioned they do not fully trust the models they use, especially when dealing with sensitive data. Some discussed checking and reviewing LLM outputs. Many described a complete lack of trust (P11: “Yeah, so my base rule is […] never trust LLM output. So […] I check it with the data.” P12: “But experience tells me in general not to trust, so I always check.”) or limited trust (P1:“No, I never completely trusted them 100%.”). Frequently, some form of validation was mentioned as a kind of trust lever (P16: “the only way that I could trust the final results was to review the code, […] check it line by line to see if the logic is correct.”). One participant also described a lack of trust in the companies behind LLMs. This cautious skepticism and mindful trust manifested in how participants described whether and how they showcase and share LLM outcomes. 4/18 said they only use LLM results internally. Another 4/18 reported openly sharing results, while the majority (10/18) fell in-between; i.e., sharing LLM outputs after modifying or verifying them. Handling Sensitive Data (Pipeline Phase: Data Collection & Prep). With sensitive data, 8/18 of participants said they would not use LLMs at all. About 5/18 said they use masking techniques or remove personally identifiable information before using LLMs and 4/18 said they would only share a database schema or description. Other approaches included local or quarantined models (4/18) or using proxy data (3/18). Two participants mentioned relying on LLM privacy settings. Result Verification (Pipeline Phase: Review & Refine). Many participants described personal experimentation and strategies to check the model’s response. These included looking for visible errors (8/18) (P13: “what I’ve done, just in the past anyway with data cleaning, is like the sniff test, right? If something doesn’t look right.”), spot checks (5/18) (P11:“ I take few data points and check, is it actually valid?”), and checking descriptive statistics (5/18) (P9: “I’m going to check the min, the max, the middle and see if that looks like the source data set”). Other verification techniques included running the LLM-produced code (4/18), looking through the code (2/18) (P18: “[…] then of course we go with it by checking line by line what it has done.”), checking against the raw data (3/18), prompting the LLM or a different LLM to check or run tests (3/18), or checking trusted (external) sources (2/18). Others reported asking the LLM to generate various types of charts for the same task (2/18), validating via online search (e.g. Google, GitHub)(1/18), and asking the LLM for explanations (1/18). Challenges with LLMs for Data Analytics While the majority of participant concerns (17/18) revolved around errors (especially hallucinations), several other concerns were raised, from minor quirks to large scale societal impacts. We analyzed and grouped challenges and concerns into five high-level categories. Common Pitfalls represent frequent, but often hidden, issues and impacts to the user for given Vis task(s). Societal Impacts represent potential and existing issues which have widespread impact across users and tasks. User Concerns represent broad or long-term concerns about potential negative impacts to the user rather than society at large or a specific task. Model-specific Quirks are noted model-specific behaviors which may or may not be seen as problems depending on the user and task. Lastly, System Limitations are barriers specific to a given LLM and subscription. In Table 2 we crosscut these challenges with applicable participant strategies. In the following subsections, we break down where vibe analysis challenges mirror similar concerns in vibe coding and where they diverge. We describe how aspects of data visualization have a unique impact on vibe analysis that differs from vibe coding. Challenges prevalent to both vibe analysis and vibe coding Most concerns focused on Common Pitfalls, especially hallucinations and errors. Participants described frequent types of LLM errors, including data ingest, library, filtering, and de-duplication. 11/18 described LLM hallucinations (P13: “And of course that the key part is [accuracy]. [An LLM] can definitely do it faster. it hasn’t just been accurate. It’s hallucinating a lot in my experiences so far.”; P17: “There are times that it even generates data points that are outside your data […] that is where you have to come in as a human in the loop.”). Other common pitfalls included a lack of consistency (6/18) (P4: “Sometimes it gives different answers to the same question and it makes it really hard to build reliable workflows because the consistency is not there,”), lack of Vis-specific guidelines (6/18) (P2: “I used to make a temperature map; it kind of gave me high temperature in blue and low temperature in red. And I think this is pretty, pretty misleading […]”), and the time and effort required to get to a finished product from what the LLM produced (4/18). Armed with this information, UI designers and LLM developers can devise interventions to guide users in each stage of the pipeline. Low Quality (Pipeline Phase: all). Issues in AI-generated code include subtle bugs, verbosity, security vulnerabilities, and hallucinated APIs [5]. Participants described similar issues specific to Vis, including low chart quality, verbosity, data and specification hallucinations, and data and chart errors during vibe analysis. P4: “the most common is when it produces a code that runs but gives some little bit of wrong results like grouping data incorrectly or using a default parameter that I wouldn’t have chosen. They usually look very fine at first glance, but then the output doesn’t usually like make sense.” P9: “…there have also been cases where it’s like, yeah, Vega-Lite will do this thing. And I’m like Vega-Lite explicitly does not do that thing. And you’re basically like arguing with this bot.” Technical Debt (Pipeline Phase: all). Vibe coding can lead to technical debt, unmaintainable code, and buggy or insecure products. It also complicates version control [12]. Our participants similarly hinted at consistency problems in Vis work, and especially a form of design debt, where visual design flaws, inconsistencies, and compromises pile up over time. This impacts analytic reproduceability, comparability, and consistency, since models will generate different responses even in uniform analytic tasks. In some cases this variation may not matter, but many analytic tasks do call for a consistent analytic and design approach, which, even with constraints cannot be guaranteed with LLMs. P17: “So when that happens, then you have …the first thing you generated is different from the second thing generated. That is some inconsistency.” P18: “But the problem that consists of the choice of too many, because then it keeps suggesting you can do this also, you can do that also. And then there are multiple ways of doing the same thing which confuses you.” For the Vis community, design debt has to be paid eventually through anything from fixing flaws ad hoc to auditing large scale analytics to complete overhauls. With the variety of Vis-specific tools and processes (see Visualization Tasks in Vibe Analytics subsection in Findings) and audience-specific factors (see External Factors subsection in Findings), Vis design debt can be more difficult to unwind and correct than traditional code-based technical debt. Switching between Vis tools like Tableau and Power BI, for example, means updating not just data flows, but Vis components as the availability, compatibility, and translation of design options differ between tools and may be far more rigid than code alone. Interpreting Intent (Pipeline Phase: all, especially critical to more complex, layered tasks in Planning & Ideation, Exploration). Vibe coding studies often point to the central impact of ambiguity handling on vibe coding [5]. Ge et al. note that even as 72% of software defects in production environments originate from misunderstood requirements, state-of-the-art Code LLMs generate code outputs in over 63% of ambiguous scenarios without seeking clarification [5]. Interpreting and contextualizing analytic intent, central to intelligent analytic systems [7], are analogous challenges in vibe analysis, where the effects of ambiguous requirements (natural language utterances are often underspecified) are compounded by Vis design-specific encoding language. P18: “I think the biggest concerns would be explaining the context of […] why we are making […] what is it required, and it misinterpreting every simple thing.” Education and Retention Impacts (Pipeline Phase: Data Collection & Prep, Exploration). While students learn to verify LLM-generated code, they struggle when asked to integrate their solutions or optimize performance [13]. Some of our participants who were also educators noted that while LLMs give users a shortcut, they do not learn the skills to design good visualizations. P14: “…the students could say, OK, I have to [use] chatGPT for that because I don’t understand you. I’m not sure it’s okay, but it’s a little bit sad.” Productivity Impacts (Pipeline Phase: Data Collection & Prep, Exploration). Programmers take less time to understand every line generated by the LLM and more time validating its output against their expectations and mental models [14]. For Vis, participants often note how LLMs speed up content creation, but LLM outputs take time to review and correct. This takes on additional significance when data work is involved, as users have to verify code as in vibe-coding, but also look at data handling and chart design to avoid issues. P12: “…you really have to pay close attention to to the [model] because at any moment in time, it starts inventing data, it starts adding data, and it sounds really good what it’s coming up with, but it’s not not factual, it’s not the truth.” Homogeneity (Pipeline Phase: Planning & Ideation, Exploration). A common challenge in vibe coding is the limited scope and diversity of the datasets used for training and evaluation [15]. Similarly in vibe analysis, models often revert to the same set of simple charts. In a field known for creativity and innovation, the impact on Vis is to constrain creativity. P3: “I found that very limiting in the sense that it almost started to direct my thinking process…” Context Window Limitations (Pipeline Phase: Exploration). To deal with context window limitations, vibe coders and vis practitioners sometimes start fresh sessions (see Strategies above). Vis practitioners noted the effect of a limited context window on their ability to get meaningful results beyond shorter interactions, which becomes a bigger problem the more complex the analytic requirements. Getting stuck on an increasingly unproductive thread can mean starting from scratch on a multi-layered analytic task, losing hours of work. P1 described a routine of getting LLM support for work in Tableau and having to switch from ChatGPT to Copilot and sometimes to Gemini. Each time, work would have to begin again by loading context and data, hoping to be done before the new model’s limit is reached. P3 described navigating between token limits in ChatGPT and time limits in Claude. Model-specific Quirks (Pipeline Phase: Review & Refine, Finishing Touches). Verbosity is common to both vibe coding and vibe analysis, though not discussed often in our interviews. This shows up as suboptimal or inefficient code, especially when practitioners use LLMs to set up sites to showcase their work. This added to the difficulty of accurately presenting analytic work and can slow progress or hamper how they share their work, since many Vis practitioners are not web development experts. P9: “I would end up with these like HTML pages in canvas that were like a few thousand lines long and every change would be like OK, we need to regenerate this whole like text. And that ended up just being slow.” Challenges more prevalent to vibe analysis than vibe coding Not trained for Vis (Pipeline Phase: Planning & Ideation, Exploration, Review & Refine). Participants noted several issues that expose the lack of Vis-specific training or guidance available to a model. LLMs tested against visualization benchmarks exhibit difficulties with data-dense and deceptive visualizations [19], and in practice our participants noted that LLMs regularly fall short on Vis tasks. P3: “So what Claude was doing is it was generating the grid from left to right even though I was instructing it again and again to, you know, generate the grid from top to bottom.” P7: “But especially for data visualization, I feel that we still haven’t reached the ideal, the phase to combine the power of LLM with the needs that we have when we are doing data visualization.” P10: “…table cartograms has actually been a […] on/off process for me that’s actually been going on for six months. It’s been more of a how can I actually prompt the LLM to generate this?” Hidden code (Pipeline Phase: Exploration, Review & Refine). The model’s output may not enable immediate or straightforward testing, especially when only the resulting chart is generated. In contrast, vibe coding is generally done either within a development environment where code can be immediately inspected and tested, or users have the option of copying code snippets directly into an interpreter. Vis practitioners must decide how and when to test aspects of the model’s code (which is often not automatically provided) along with the output chart(s). P4: “I think my biggest concerns are the accuracy and the tiny errors it can make because sometimes it can produce code or analysis that looks pretty much fine, but then it’s slightly wrong, and this is like really dangerous […] when you don’t catch it.” Though hidden code may be a challenge in vibe coding as well, in this case, the programmer is able to see and verify code by default in currently available LLM-based platforms, even when working through a user interface. In contrast, many who use LLMs for chart generation don’t inherently see, access, or interact with the code used to generate the chart; for image and chart generation, this code is not typically shared as a default. Secondly, in terms of industry use, software developers using vibe coding usually learn to verify the generated code. Meanwhile, some of our Vis participants do not code, instead working with tools like Excel and Tableau, and so could not verify a model’s code. Strategy Gaps As illustrated in Figure 2, we found that most strategies related to dealing with Common Pitfalls, Model-Specific Quirks, and System Limitations, categories described in the Challenges with LLMs for Data Analysis section. The practice of breaking tasks down into steps and providing more context or guidance were among the most cited strategies for these types of issues, along with starting again with a new chat. In contrast, Societal Impacts and User Concerns had few applicable strategies. Some societal impacts may not be completely mitigated – bias cannot be completely removed, there is no perfect privacy with connected content, and not all forms of uncertainty can be fully captured or calculated. Still, these and other societal concerns could be better acknowledged through clear disclosure by LLM developers, and at the user level when sharing LLM-generated results. This transparency would serve the public interest, users, and could help drive mitigation efforts. User concerns had by far the fewest applicable strategies, representing potential future work. While research to understand and prevent these issues has been conducted, the absence of strategies points to the possibility that developed solutions may be nascent, not well-known, and perhaps not broadly applicable. Discussion Here we draw on our interview results to provide recommendations for improving LLM-based Vis. Based on the results, we identify a need for verification support through tool integration, training and resources for practitioners, including best practices and pitfalls to avoid, and design considerations to prevent snowballing design debt. Just as the software development community is grappling with the impact of vibe coding, the Vis community will likewise need to navigate the stochastic nature of LLM-enabled data analysis and visualization. Recommendations for tool developers Build in verification processes. Although verification strategies are common among practitioners, they are far from standardized, and different users may have different needs depending on the nature of the data, task, and questions they are tackling. While no single verification technique is universally effective, we found that practitioners rely on combinations of (practical, yet anecdotal) checks and processes to strengthen their analyses and make current LLM tools more robust for data work. Almost half of participants noted verification using spot checks or descriptive statistics. Spot checks and multi-model checks provide opportunities to formalize what practitioners are currently doing ad hoc. Incorporating some linting-like processes can provide a more structured, standardized approach to spot checking both data and chart characteristics, while including a comparison across models can help surface both errors and new insights. Spot checking could be automated using more reliable rule-based techniques to complement LLM use. In addition, there is a key research opportunity to develop tools that increase transparency in the actions taken by an LLM on data, to save users the pain of spot checking and digging through code to determine whether and where the LLM made a mistake. Well-designed linting strategies could even nudge users towards noticing that they should verify LLM outputs. Incorporate constraints. P13’s example from Finishing Touches represents a good use case for how to provide effective LLM constraints either through prompts or other incorporated guidance. Every minor change to a part of their LLM-generated map should not risk an unwanted change elsewhere. Another consideration for incorporating LLMs in Vis tools is the issue of homogeneity, as described in the Challenges section. Developers in particular could consider ways to incorporate, rather than supplant, human creativity. As most of our participants noted their use of LLMs for ideation and visualization design, this represents a major space for innovation. One area for future work could center on how to incentivize and enable human creative input while focusing LLM constraints to mitigate hallucination. On the other hand, if left to models alone, the closed system creates an arbitrary limit on novel design. Domain expertise, design knowledge, and the development and testing of new techniques should be first class citizens in the development process, not secondary to expediency. Recommendations for the Vis Community Develop guidance. LLMs empower users to work beyond their established skillsets. Much like vibe coding, vibe analysis empowers people to quickly iterate through different Vis options, prototype dashboards, and produce other interactive data applications. But we caution that user training is critical. The amplifying power brought to bear on everyday Vis tasks could also be used to misinform and manipulate. To mitigate the risks of errors and other LLM pitfalls, we recommend the development of Vis-specific training modules to strengthen AI literacy, as well as verification guidance and processes designed specifically for the use of LLMs for Vis. Some examples include guidance about the specific Vis capabilities and limitations of LLMs, best practices and common pitfalls to watch out for, and resources like data and guided problem sets that users can practice with. The strategies described by our expert interviewees can serve as a starting point. Provide upskilling resources. Considering our participants’ concerns around job replacement, process changes, and the feeling by some of being compelled to use LLMs, more resources and new approaches are needed to help Vis practitioners navigate the process of incorporating LLMs in their workflows. As some of our interviewees noted, job impacts are perhaps inevitable and likely to come with trade-offs, but whether this includes more job loss or job shift is yet to be determined. Opportunities include formalization of training and verification processes, guidance on which aspects of the Vis pipeline are best suited for LLM use, best practices and automated tools for mitigating concerns around data privacy, and approaches to detect and prevent misleading visualization designs and visualization issues that models are known to miss. Training can focus on LLM benefits and how to use them, but should be balanced with a thorough understanding of limitations and risks. The development of practice sets that draw on realistic Vis tasks could be used to help users get comfortable with using LLMs and spotting issues. Reckon with design debt. While technical debt may be important whenever Vis projects rely on LLM-generated code, we identified a separate layer of design debt that should be equally concerning. Technical debt encompasses software development decisions that may not be immediately harmful, but left to accumulate can render code difficult or impossible to maintain or update [20, 12]. Design debt are visual design choices (in this case made by an LLM) that may serve an immediate need but can lead to scalability issues, difficulties updating or adding charts or chart features, or higher rates of bugs and errors [20], which in Vis may require more data analysis to identify. Design debt must be dealt with eventually, repaid in patches or overhauls [20], but unlike technical debt, design choices may be less modular than code alone, with license and application-specific limitations or requirements. Ideally, design debt can be prevented before it causes downstream problems by considering broader integration needs. For Vis, this could mean considering both system and audience implications before making choices about format, colors, and other visual designs (marks and channels). Questions about who will maintain and update data, choice of Vis products (e.g., dashboards, interactives), and hosting platforms should be answered early. Limitations of and compatibility between different tools or programming languages used on a project should be considered before switching costs get too high. Both P7 and P9 noted use of a Git repository for version control. Though not all Vis practitioners are comfortable using these platforms, some version of a repository specific to Vis projects would help practitioners track changes, revert to previous versions when needed, and allow for traceability and transparency. This type of Vis-focused provenance tracking could also support better sharing and attribution. Limitations and Future Work Participants represented various backgrounds, sectors, and roles, but there was a skew in country, age, race, and gender. They were not representative of all data workers, but information on how they use LLMs in their work is nonetheless valuable in understanding motivations, challenges, and practices. Participant use of LLMs and their various practices were self-reported and may reflect over- or under-reporting. Future research can extend this work to a broader sample, including data workers from other professions who may have lower visualization expertise and/or see data analytics as a secondary part of their role. Our study focused on LLMs due to their broad use, availability, and ability to handle a broad range of Vis tasks. Future work can also re-assess to see how findings evolve as the technology advances. Other opportunities for future work include identifying best-fit and user preferences for LLMs and traditional Vis tools, to guide how to split Vis functions in a hybrid process, and investigating design debt in Vis and how to best prevent or mitigate it. Conclusion We conclude with a call to the visualization community to think critically about where and how we apply LLMs and how we learn and train others to use them to present data. LLMs offer a tremendous opportunity to enable users to quickly and easily explore and visualize data. But this power has so far been applied without the requisite responsibility, and with inadequate guardrails. The next big question for the community may not be whether to use LLMs for Vis, as this may soon become a moot point, but instead how to effectively prepare users (from lay users to advanced practitioners) to use them effectively. In short, LLMs may speed up Vis work, but they may also increase verification burden, weaken skill formation, narrow design exploration, and create downstream maintenance and quality problems. For a community that values clear and accurate data representation, this presents a challenge, but also opportunities to innovate and adapt. We close with a quote about the importance of goal-driven exploration and experimentation, recognizing that while there are some aspects of data analysis and visualization that greatly benefit from the use of LLMs, there may be others that should not be offloaded: “Because learning does not consist only of knowing what we must or we can do but also of knowing what we could do and perhaps should not do.” - Umberto Eco ACKNOWLEDGMENT We are grateful to Jane Adams and Racquel Fygenson for their input. This material is based upon work supported by Northeastern University’s Khoury College of Computer Sciences and the National Science Foundation (NSF) CISE Graduate Fellowships under Grant No. 2313998. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. References [1] J. Yao, K. Ning, Z. Liu, M. Ning, Y. Liu, and L. Yuan (2024) LLM lies: hallucinations are not bugs, but features as adversarial examples. External Links: 2310.01469, Link, Document Cited by: Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners. [2] C. Qian, M. X. Liu, E. Reif, G. Simon, N. Hussein, N. Clement, J. Wexler, C. J. Cai, M. Terry, and M. Kahng (2024) The evolution of LLM adoption in industry data curation practices. arXiv. External Links: Link, Document, 2412.16089 [cs] Cited by: User Preferences and Adoption of LLMs, Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners. [3] S. D. Shaw and G. Nave (January 11, 2026) Thinking—fast, slow, and artificial: how AI is reshaping human reasoning and the rise of cognitive surrender. External Links: Link, Document Cited by: User Preferences and Adoption of LLMs, Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners. [4] D. Kosminsky, J. Walny, J. Vermeulen, S. Knudsen, W. Willett, and S. Carpendale (2019) Belief at first sight. Information Design Journal 25 (1), p. 43–55. External Links: Document, Link, ISSN 0142-5471 Cited by: User Preferences and Adoption of LLMs, Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners. [5] Y. Ge, L. Mei, Z. Duan, T. Li, Y. Zheng, Y. Wang, L. Wang, J. Yao, T. Liu, Y. Cai, B. Bi, F. Guo, J. Guo, S. Liu, and X. Cheng (2025) A survey of vibe coding with large language models. External Links: 2510.12399, Link Cited by: Vibe Coding, Vibe Coding, Figure 2, Figure 2, The Interaction Environment, The Model, Vibe Analysis, Vibe Analysis, Vibe Analysis, Vibe Analysis and the Changing Nature of Data Work, Challenges prevalent to both vibe analysis and vibe coding, Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners. [6] A. Crisan, B. Fiore-Gartland, and M. Tory (2021) Passing the data baton: a retrospective analysis on data science work and workers. IEEE Trans. Visualization and Computer Graphics 27 (2), p. 1860–1870. External Links: Document Cited by: User Preferences and Adoption of LLMs, Qualitative Analysis, Findings. [7] M. Tory and V. Setlur (2019) Do what I mean, not what I say! design considerations for supporting intent and context in analytical conversation. In 2019 IEEE Conf. Visual Analytics Science and Technology (VAST), Vancouver, BC, Canada, p. 93–103. External Links: ISBN 978-1-7281-2284-7, Link, Document Cited by: User Preferences and Adoption of LLMs, Challenges prevalent to both vibe analysis and vibe coding. [8] V. Schetinger, S. Di Bartolomeo, M. El-Assady, A. McNutt, M. Miller, J. P. A. Passos, and J. L. Adams (2023) Doom or deliciousness: challenges and opportunities for visualization in the age of generative models. Computer Graphics Forum 42 (3), p. 423–435. External Links: Document, Link, https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14841 Cited by: User Preferences and Adoption of LLMs. [9] Y. Chou, B. Jiang, Y. W. Chen, M. Weng, V. Jackson, T. Zimmermann, and J. A. Jones (2025) Building software by rolling the dice: a qualitative study of vibe coding. External Links: 2512.22418, Link Cited by: Vibe Coding, Vibe Coding, Visualization Goals. [10] B. Nolan (2026) Top engineers at anthropic, openai say AI now writes 100% of their code—with big implications for the future of software development jobs. External Links: Link Cited by: Vibe Coding, Vibe Analysis and the Changing Nature of Data Work. [11] K. Bindley and K. Blunt (25 Feb. 2026) Tech firms aren’t just encouraging their workers to use ai. they’re enforcing it.. Wall Street Journal. External Links: Link Cited by: Vibe Coding, Vibe Analysis and the Changing Nature of Data Work. [12] V. Pimenova, S. Fakhoury, C. Bird, M. Storey, and M. Endres (2025) Good vibrations? a qualitative study of co-creation, communication, flow, and trust in vibe coding. External Links: 2509.12491, Link Cited by: Vibe Coding, Challenges prevalent to both vibe analysis and vibe coding, Recommendations for the Vis Community. [13] F. Geng, A. Shah, H. Li, N. Mulla, S. Swanson, G. S. Raj, D. Zingaro, and L. Porter (2025) Exploring student-AI interactions in vibe coding. External Links: 2507.22614, Link Cited by: Vibe Coding, Challenges prevalent to both vibe analysis and vibe coding. [14] A. Sarkar and I. Drosos (2025) Vibe coding: programming through conversation with artificial intelligence. External Links: 2506.23253, Link Cited by: Vibe Coding, Challenges prevalent to both vibe analysis and vibe coding. [15] Umama, K. Usman Danyaro, M. Nasser, A. Zakari, S. Abdullahi, A. Khanzada, M. Muntasir Yakubu, and S. Shoaib (2025) LLM-based code generation: a systematic literature review with technical and demographic insights. IEEE Access 13 (), p. 194915–194939. External Links: Document Cited by: Vibe Coding, Challenges prevalent to both vibe analysis and vibe coding. [16] M. Naeem, W. Ozuem, K. Howell, and S. Ranfagni (2023) A step-by-step process of thematic analysis to develop a conceptual model in qualitative research. International Journal of Qualitative Methods 22, p. 1–18 (English). External Links: Document, ISSN 1609-4069 Cited by: Qualitative Analysis. [17] N. H. Riche, C. Hurter, N. Diakopoulos, and S. Carpendale (2018) Data-driven storytelling. A K Peters/CRC Press. Cited by: Figure 2, Figure 2, External Factors, Vibe Analysis, Findings. [18] A. McNutt, G. Kindlmann, and M. Correll (2020) Surfacing visualization mirages. In Proc. 2020 CHI Conf. Human Factors in Computing Systems, CHI ’20, New York, NY, USA, p. 1–16. External Links: ISBN 9781450367080, Link, Document Cited by: External Factors. [19] S. Pandey and A. Ottley (2025) Benchmarking visual language models on standardized visualization literacy tests. arXiv. External Links: Link, Document, 2503.16632 [cs] Cited by: Challenges more prevalent to vibe analysis than vibe coding. [20] J. P. Biazotto, D. Feitosa, P. Avgeriou, and E. Y. Nakagawa (2025) Automating technical debt management: insights from practitioner discussions in stack exchange. External Links: 2502.03153, Link Cited by: Recommendations for the Vis Community. IEEEbiography Shani C Spivak is a PhD candidate at Northeastern University and the primary contact of this paper (spivak.s@northeastern.edu). Her research lies at the intersection of Vis, AI ethics and governance, AI literacy and accessibility, and human-computer interaction. IEEEbiography Aditi Krishna is a PhD student at Northeastern University, Boston, Massachusetts, USA. Her research interests include human-AI interaction and responsible AI. Contact her at krishna.ad@northeastern.edu. IEEEbiography Mahsan Nourani is an Assistant Research Professor with Northeastern University, Portland, Maine, USA. Her research interests include human-centered AI and visual analytics. Nourani received her PhD in Computer Science from University of Florida. Contact her at m.nourani@northeastern.edu. IEEEbiography Melanie Tory is Professor of the Practice with Northeastern University, Oakland, California, USA. Her research interests include visualization and human-AI interaction. Tory received her PhD in Computer Science from Simon Fraser University. Contact her at m.tory@northeastern.edu.