Paper deep dive
Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool
Sanchita S. Kamath, Aziz N Zeidieh, Venkatesh Potluri, Sile O'Modhrain, Kenneth Perry, JooYoung Seo
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/14/2026, 1:53:32 AM
Summary
This paper presents an Experience-Based Co-Design (EBCD) study involving blind and low-vision (BLV) co-designers to develop an accessible, multi-modal, web-native 3D data visualization tool. The research addresses the gap in accessible 3D data exploration by translating tactile knowledge into digital interfaces, incorporating features like reference sonification, stereo/volumetric audio, and buffer aggregation to support analytic tasks such as orientation and peak finding.
Entities (5)
Relation Signals (3)
Sanchita S. Kamath â authored â Three Modalities, Two Design Probes, One Prototype, and No Vision
confidence 100% ¡ Sanchita S. Kamath... 2026. Three Modalities, Two Design Probes, One Prototype, and No Vision
3D Data Visualization Tool â targets â BLV
confidence 95% ¡ make 3D data visualizations accessible and learnable for BLV people
Experience-Based Co-Design â usedtodevelop â 3D Data Visualization Tool
confidence 95% ¡ we conducted an Experience-Based Co-Design (EBCD) ... to create an accessible, multi-modal, web-native visualization tool
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Three-dimensional (3D) data visualizations, such as surface plots, are vital in STEM fields from biomedical imaging to spectroscopy, yet remain largely inaccessible to blind and low-vision (BLV) people. To address this gap, we conducted an Experience-Based Co-Design with BLV co-designers with expertise in non-visual data representations to create an accessible, multi-modal, web-native visualization tool. Using a multi-phase methodology, our team of five BLV and one non-BLV researcher(s) participated in two iterative sessions, comparing a low-fidelity tactile probe with a high-fidelity digital prototype. This process produced a prototype with empirically grounded features, including reference sonification, stereo and volumetric audio, and configurable buffer aggregation, which our co-designers validated as improving analytic accuracy and learnability. In this study, we target core analytic tasks essential for non-visual 3D data exploration: orientation, landmark and peak finding, comparing local maxima versus global trends, gradient tracing, and identifying occluded or partially hidden features. Our work offers accessibility researchers and developers a co-design protocol for translating tactile knowledge to digital interfaces, concrete design guidance for future systems, and opportunities to extend accessible 3D visualization into embodied data environments.
Tags
Links
- Source: https://arxiv.org/abs/2604.09426v1
- Canonical: https://arxiv.org/abs/2604.09426v1
Trouble viewing inline? Open PDF directly â
Full Text
155,824 characters extracted from source content.
Expand or collapse full text
Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool Sanchita S. Kamath School of Information Sciences University of Illinois Urbana-Champaign Champaign, Illinois, USA ssk11@illinois.edu Aziz Zeidieh Informatics University of Illinois Urbana-Champaign Champaign, Illinois, USA azeidi2@illinois.edu Venkatesh Potluri School of Information University of Michigan Ann-Harbour, Michigan, USA potluriv@umich.edu Sile OâModhrain School of Information University of Michigan Ann-Harbour, Michigan, USA sileo@umich.edu Kenneth Perry American Printing House for the Blind Louisville, Kentucky, USA kperry@blinksoft.com JooYoung Seo School of Information Sciences University of Illinois Urbana-Champaign Champaign, Illinois, USA jseo1005@illinois.edu Figure 1: Iterative Prototyping based on the EBCD Framework ABSTRACT Three-dimensional (3D) data visualizations, such as surface plots, are vital in STEM fields from biomedical imaging to meteorology and spectroscopy, yet remain largely inaccessible to blind and low-vision (BLV) people. To address this gap, we conducted an Experience-Based Co-Design (EBCD) with BLV co-designers with expertise in non-visual data representations to create an accessible, multi-modal, web-native visualization tool. Using a multi-phase Please use nonacm option or ACM Engage class to enable C licenses This work is licensed under a Creative Commons Attribution 4.0 International License. CHI â26, April 13â17, 2026, Barcelona, Spain Š 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2278-3/2026/04 https://doi.org/10.1145/3772318.3791272 co-design methodology, our team of five BLV and one non-BLV researcher(s) participated in two iterative sessions, comparing a low-fidelity tactile probe with a high-fidelity digital prototype. This process produced a prototype with empirically grounded features, including reference sonification, stereo and volumetric audio, and configurable buffer aggregation, which our BLV co-designers vali- dated as improving analytic accuracy and learnability. In this study, we explicitly target core analytic tasks essential for non-visual 3D data exploration: 3D orientation, landmark and peak finding, com- paring local maxima versus global trends, gradient tracing, and identifying occluded or partially hidden features. Our work offers accessibility researchers and developers a co-design protocol for translating tactile knowledge to digital interfaces, concrete design arXiv:2604.09426v1 [cs.HC] 10 Apr 2026 CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. guidance for future systems, and opportunities to extend accessible 3D visualization into embodied data environments. Three-dimensional data visualizations, such as surface plots, are vital in STEM fields from biomedical imaging to meteorology and spectroscopy, yet remain largely inaccessible to blind and low- vision people. To address this gap, we conducted an Experience- Based Co-Design with BLV co-designers with expertise in non- visual data representations to create an accessible, multi-modal, web-native visualization tool. Using a multi-phase co-design method- ology, our team of five BLV and one non-BLV researcher(s) par- ticipated in two iterative sessions, comparing a low-fidelity tactile probe with a high-fidelity digital prototype. This process produced a prototype with empirically grounded features, including refer- ence sonification, stereo and volumetric audio, and configurable buffer aggregation, which our co-designers validated as improving analytic accuracy and learnability. In this study, we explicitly target core analytic tasks essential for non-visual 3D data exploration: 3D orientation, landmark and peak finding, comparing local maxima versus global trends, gradient tracing, and identifying occluded or partially hidden features. Our work offers accessibility researchers and developers a co-design protocol for translating tactile knowl- edge to digital interfaces, concrete design guidance for future sys- tems, and opportunities to extend accessible 3D visualization into embodied data environments. CCS CONCEPTS ⢠Human-centered computingâAccessibility design and evaluation methods; Accessibility technologies; Accessibility systems and tools; Empirical studies in accessibility. KEYWORDS accessibility, three-dimensional data visualizations, AI for data vi- sualization, multi-modality, embodied interaction ACM Reference Format: Sanchita S. Kamath, Aziz Zeidieh, Venkatesh Potluri, Sile OâModhrain, Kenneth Perry, and JooYoung Seo. 2026. Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI â26), April 13â 17, 2026, Barcelona, Spain. ACM, New York, NY, USA, 23 pages. https: //doi.org/10.1145/3772318.3791272 1 INTRODUCTION While there have been notable advances in producing accessible data visualizations [2â4], blind and low-vision (BLV) people remain largely excluded from engaging with three-dimensional (3D) data visualizations that represent data using three axes (X, Y, and Z) to create spatial models, such as 3D scatter plots or volumetric render- ing. Most accessibility efforts to date, have focused on audiotactile graphs [4,42] and two-dimensional representations [13]. The lack of accessible 3D data visualization tools is particularly pressing given that 3D visualizations are central to fields such as meteorology [81], biomedical imaging [108], VUV spectroscopy [51], geoscience [78], and computational fluid dynamics [34]; when inaccessible, they not only limit analytic rigor but also alienate BLV individuals from meaningful participation in knowledge production across these domains. Our target data type in this work is continuous height- field surfacesâscalar fields defined over a two-dimensional domain that produce smooth, continuous 3D cloth-like surfaces commonly found in scientific surface plots such as in VUV spectroscopy view- able in Figure 4. Focusing on these continuous surfaces allows us to support analytic tasks that depend on stable topological continuity. While accessibility for 2D charts has progressed through tactile graphics [20,53,72] and multi-modal, web-based interactions [39, 89,104], the spatial and complex nature of 3D data representa- tions such as surface and point plots continue to pose persistent barriers to non-visual access. Yet without access to these visualiza- tions, analysts risk missing essential insights in spatially complex datasets [11,21]. The leap from two-dimensional (2D) charts to interactive three-dimensional (3D) data reveals critical gaps in cur- rent accessibility research. First, most studies still concentrate on 2D charts, leaving a gap in methods for interactively exploring 3D surfaces and point clouds [49,50]. It is also important to clarify the scope of this claim within established visualization design practice, which generally cautions against projecting three-dimensional data onto two-dimensional displays due to perceptual distortions and the potential for misleading interpretations. In many cases, two- dimensional encodings such as color gradients, glyph size, or textual summaries can successfully convey three-variable correlations, and users are often able to infer relationships from such encodings. However, when analytic tasks rely on understanding continuous surface structure or spatial topology, as is common with heightfield- style representations that remain prevalent in practice, collapsing a three-dimensional topology into a flat, single-channel represen- tation can remove critical structural relationships. For BLV users in particular, TTS-only descriptions or reduced two-dimensional encodings may force the reconstruction of three-dimensional spa- tial logic mentally across modalities, increasing cognitive load and weakening the formation of coherent spatial mental models. Rather than asserting that three-dimensional representations are univer- sally superior, this work focuses on accessibility challenges that arise in contexts where three-dimensional surface representations are already in use and where preserving topological structure is central to the analytic task. Second, to the best of our knowledge, no studies have applied evidence-based methodologies for transferring the spatial knowl- edge gained from tactile prototypes into web-based digital envi- ronments for the specific challenge of 3D plots [94,96]. Third, prior systems often neglect the pedagogical scaffolding required for learnersâ transition from initial orientation to independent analytic reasoning [68]. Fourth, few studies provide fine-grained evidence on how specific non-visual modalities affect performance in 3D data visualization contexts; existing work is often limited to object recognition applications [75,84] or focuses on embodied interaction without exploring multi-modality in depth [8]. This paper introduces a methodology (inspired by the Experience- Based Co-Design (EBCD) framework [82]) and two design probes which were used to prompt the non-visual exploration of 3D scien- tific data. Our work was motivated by a foundational goal: to ensure that accessibility tools are developed with and by the BLV commu- nity, not just for them. Instead of positioning BLV individuals as participants in user studies framed by sighted researchers, rather than as integral collaborators from a projectâs inception [65]; this Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain study aims to flip that order. By adopting an EBCD approach [82], our work while proposed by a sighted researcher, is fundamentally designed with and inspired by BLV people. Unlike conventional co-design, which often focuses on collaborative ideation and proto- typing, EBCD emphasizes grounding design in lived experiences, typically through in-depth narrative accounts, storytelling, and shared reflection. This distinction was crucial for our project: rather than relying solely on structured design workshops, we began by gathering experiential narratives from BLV co-designers who have expert knowledge in data visualization, which then directly in- formed the framing of our design challenges and priorities. In this way, EBCD shaped not just how we collaborated, but also what we considered meaningful design outcomes. The methodology en- sured that our technical explorations were continually anchored in co-designersâ everyday practices and values, making the resulting tools not only functional but also genuinely usable and empower- ing. Within this process, our produced tactile probe (Section 4.1) served as the shared ground-truth reference against which all sub- sequent digital translation was calibrated, enabling consistent in- terpretation of spatial features across modalities. The significance of this work, therefore, is threefold: (1) it addresses the challenge of producing a transferable co-design protocol for translating tactile knowledge into effective digital interactions, (2) demonstrates a collaborative and inclusive research methodology - by adopting the EBCD Framework for HCI research on non-visual data access and representation, and (3) provides an evidence-based model for developing effective non-visual tools for 3D data exploration. By making surface visualizations accessible, we expand opportunities for BLV users to engage directly with spatial data, promoting fuller participation in STEM disciplines [46]. Our developed prototype, as evaluated by our ethnographic approach with expert data visu- alization researchers, helps demonstrate a successful translation of embodied, tactile knowledge into a well-known and well-adopted digital medium, providing non-visual features that were validated by our expert collaborators as improving their ability to orient, analyze, and accurately interpret 3D data. This gives our system a better position to help BLV people in producing spatial mental models of three-dimensional data, as compared to two-dimensional representations of the same data. The uniqueness of this work lies in demonstrating how one can transform a tactile learning experi- ence into an effective digital counterpart. Such an approach equips educators with concrete workflows for teaching 3D spatial reason- ing and provides developers with empirically grounded guidance for extending access to embodied or three-dimensional environ- ments. While our system can handle the input of closed-volume or molecular surface data, those structures currently appear as disjoint surfaces due to our coordinate grid handling algorithm, and thus do not behave equivalently; producing non-intuitive interaction, making any transfer only partial and not assumed. Hence, we shall not focus on them in this paper. The research problem at hand, therefore, is to make 3D data visualizations accessible and learnable for BLV people. Following are the research questions that direct this study: RQ1.How can spatial knowledge gained through tactile, low- fidelity prototypes be effectively transferred into high-fidelity digital experiences that support core 3D analytic tasks? RQ2.How can we (re)design specific non-visual features to em- power BLV usersâ ability to orient, analyze and accurately interpret 3D surfaces with confidence? RQ3.What mediums and interaction tools can most effectively translate a web-based digital prototype of three-dimensional data visualizations into embodied forms of engagement that preserve usersâ ability to execute the full set of analytic tasks? This scholarly work makes four key contributions - (1) it offers a co-design protocol that integrates tactile and digital probes, re- fined iteratively across two sessions with BLV collaborators; (2) it delivers a (re)designed high-fidelity prototype incorporating em- pirically grounded features such as reference sonification, stereo and volumetric audio cues, and configurable buffer aggregation; (3) it provides expert opinion on the impact of these features on analytic accuracy and user learnability in 3D data contexts; and (4) it contributes design ideas for the future extension to embodied 3D data visualization environments. 2 RELATED WORK To contextualize our contribution, we review the EBCD theoretical framework and map how it can be employed in HCI practices and list three areas of related work: adaptations of two-dimensional visu- alizations for BLV audiences, immersive and embodied approaches to data interaction, and emerging efforts in three-dimensional visu- alization. Together, these streams highlight both the progress made and the critical gaps that remain, underscoring the need for meth- ods that can center lived experience in tackling new accessibility challenges. 2.1 Theoretical Framework: Experience-Based Co-Design EBCD, originally developed in health services, is built upon itera- tive cycles of understanding lived experience, prioritizing issues, and (re)designing services with users and stakeholders [28,31,70]. According to EBCD practitioners, key stages include gathering staff and patient experiences, producing a âtrigger filmâ based on emo- tional touch-points, holding co-design events, forming working groups, and celebrating progress [30]. Although participatory and co-design approaches are common in HCI [87], EBCD is distin- guished by its systematic elevation of lived experience as central design material; not merely an input but the motivating, framing resource [27,69]. To the best of our knowledge, no prior HCI stud- ies have employed EBCD for accessible 3D data visualization; we adopt and adapt it (Table 1) to structure cycles of eliciting BLV collaboratorsâ tactile and narrative experiences, prioritizing their analytical challenges, and collaboratively redesigning non-visual 3D representations in a meaningful, grounded way. Compared to design studies, which often emphasize artifact cri- tique, visual aesthetics, and designer reflection [41], and more con- ventional co-design methods, which may solicit stakeholder input during ideation [48], EBCD offers a distinct advantage: it treats lived experience as the core design material [71]. In design studies, the fo- cus may lie on how form and function evolve through prototyping and critique, but without grounding in deep personal narrative [32]. In typical co-design, participants might guide requirements or sug- gest features, but their stories are often abstracted [74]. EBCD, by CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. Canonical EBCD StageOur Adaptation and Rationale Setting up the projectWe established our co-design team, clarified roles, and oriented BLV researchers to both the tactile and digital phases, ensuring a shared understanding and equal power in design. Gather staff / practitioner experiencesIn our context, âstaffâ maps to non-BLV researcher; we collected their perspectives on constraints and design possibilities, balancing with BLV lived experience. Gather patient / user experiences We ran low-fidelity and high-fidelity design probe sessions and narrative interviews to collect first-person accounts of exploring 3D surfaces non-visually. Trigger film (or tangible trigger)Instead of a video film, we used the tactile probe and semi-structured questioning to surface emotional and perceptual âtouch-pointsâ in how BLV collaborators explore surface features. First co-design event We brought BLV researchers together in our workshops, using prior experiential data to align on priorities and co-create interaction features through Session 1 and 2. Co-design working groupsWe iteratively developed our high-fidelity prototype (drawing from insights gathered during Session 1) and tested the final version with requested features (e.g., sonification and buffer aggregation) during Session 2. Celebration / review event We reserved the second half of Session 2 for reflection, where participants compared tactile and digital experiences, surfaced insights, and affirmed shared design ownership. ImplementationWe integrated the co-designed features into a web-native 3D visualization tool usable for 3D data analysis by BLV users. EvaluationWe performed iterative evaluation across both sessions where BLV collaborators used both the tactile and web-based versions, reflecting on accuracy, learnability, and fidelity. Table 1: Mapping from canonical EBCD stages (per standard health-services literature) to our adaptations in this work. contrast, privileges participantsâ emotional âtouch-pointsâ: their real, embodied encounters, and uses these to prioritize problems, spark design ideas (e.g., via trigger films/probes), and evaluate so- lutions. For our work, this is especially powerful: the way BLV individuals navigate and make sense of tactile spatial data is not just a usability issue but a form of domain knowledge. By cen- tering that experience, we ensure the resulting design is not only functional, but deeply aligned with how BLV users perceive, rea- son about, and interpret 3D data â leading to a more meaningful, effective, and empowering tool. 2.2Accessible Visualization in Two Dimensions Accessibility research has historically concentrated on two- dimen- sional charts, tactile graphics, and physical data representations. Research on haptic data visualization highlights how touch-based interfaces can support data access for BLV individuals by translat- ing charts, maps, and diagrams into tactile forms [13,20,35,53,72]. Physical visualization and data sculptures have been proposed as ed- ucational tools to represent abstract data tangibly, enabling learners to grasp complex relationships through embodied interaction [99]. Data physicalizations have been shown to support effective analysis of elevation data when compared with digital or VR alternatives, emphasizing the promise of haptic perception in learning spatial structures [36]. Complementing tactile approaches, sonification-based systems have demonstrated effectiveness in conveying data through audi- tory mappings. Zhao and Vande Moere[106]pioneered interactive sonification techniques for georeferenced data, while more recent work has integrated natural sound representations [39] and spatial audio cues [93] to enhance data comprehension. Electrotactile feed- back systems [46] and refreshable tactile displays combined with conversational agents [37,38,83] have expanded multimodal possi- bilities for chart comprehension. Haptic techniques such as Fan et al. [24]and broader sonification frameworks [6] have further enriched non-visual data access strategies. Screen-reader-based approaches, including tools like VoxLens [91], MAIDR [89], ChartA11y [104], AudioFunctions.Web [5], and Umwelt [110], have established frame- works for keyboard-driven navigation and semantic description of visualizations. Studies of screen-reader usersâ experiences [90] and rich screen-reader interaction designs [109] have illuminated the diverse needs and preferences of BLV individuals when accessing visual data. Broader surveys have documented the state of sonification - visualization integration [23], established design frameworks for accessible visualization [22,67], and articulated the need for so- ciotechnical perspectives that center disability justice [40,62]. Work on semantic levels of natural language description [63] and iden- tifying opportunities for accessible visualization [55] has further shaped our understanding of how to design for diverse sensory experiences. Additional research has explored sensory substitu- tion [16], non-visual visualization design [15], audio-based info- graphics [38], and declarative sonification grammars [54]. System- atic reviews of sonification in assistive systems [64] and efforts to reach broader audiences through visualization [56] provide com- prehensive overviews of the fieldâs trajectory. These works collectively demonstrate the potential of tactile, au- ditory, and multi-modal approaches but remain largely constrained to 2D or static contexts. Three lessons from this body of work directly informed our design approach. First, multi-modal redun- dancy: the principle that information should be encoded through multiple sensory channels simultaneously, guided our integration of sonification, spatial audio, and textual feedback to ensure ro- bust comprehension regardless of individual perceptual prefer- ences [23,39,46,83,89,110]. Second, pedagogical scaffolding: as demonstrated in systems that progressively introduce complex- ity [89,104,109], shaped our inclusion of overview-then-detail (whole to part strategy) features such as multi-perspective auto- play and jump-to-peak navigation. Third, progressive disclosure: the idea that systems should allow users to control the granular- ity of information they encounter [55,63,91], directly motivated our implementation of a switch between point and surface modes which enables users to toggle between aggregated âsquaresâ and point-by-point detail and the configurable buffer mechanism. How- ever these 2D methods fail for 3D analytics due to a collapsed topology (squashing three variable correlations into two channel representations), and the lack of occlusion or volumetric traver- sal. By grounding our 3D visualization prototype in established principles from 2D accessibility research (our multi-modal infras- tructure - is inspired by Seo et al. [89]and Seo et al. [88]), we extend their applicability to three-variable correlations and share their core commitment to user autonomy and multi-modal access. Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain Finally, A11yShape [105] is one of the first pieces of literature that allows BLV people to use AI-assist to create 3D models, how- ever it focuses on generative shape modeling through code-based construction (OpenSCAD with GPT-4o) and cross-representation highlighting of discrete geometric components, rather than the systematic, real-time exploration of continuous three-dimensional data visualizations. While A11yShape enables users to create and verify geometric forms through AI-generated descriptions and hi- erarchical abstractions, our work addresses the distinct challenge of interpreting pre-existing scientific datasets through features like reference sonification, spatial audio, configurable buffer aggrega- tion, and multi-perspective auto-play that support orientation, com- parison, and statistical reasoning within complex data landscapes common to STEM disciplines. 2.3 Immersive and Embodied Approaches Alongside tactile approaches, a growing body of research explores immersive and embodied interaction as alternative avenues for data access. Extended reality (XR) and embodied interaction research has highlighted the potential of immersive audio and haptic feedback for BLV users [60,61], with prototypes demonstrating embodied navigation and object labelling in 3D environments. Embodied in- teraction has been used to support visualization literacy, allowing learners to construct and manipulate visualizations through tangi- ble devices [47], embodied allegories for gesture design [14], and aesthetics-driven embodied installations [59]. These studies em- phasize the importance of bodily engagement and spatial presence in data interaction. In parallel, immersive environments such as VirtualDesk integrate embodied gestures and virtual workspaces to enhance comfort and analytic performance in 3D settings [100], while systematic reviews point to immersive analytics as a rapidly expanding field that seeks to unify spatial, embodied, and multi- sensory techniques [43]. Although immersive approaches expand the possibilities of non- visual access, they have primarily emphasized proof-of-concept VR or AR experiences for sighted users, rather than evidence-based accessibility for BLV audiences. This gap motivated our discussion of how embodied principles: particularly proprioception, physical movement, and spatial presence; might inform the design of more grounded, accessible interfaces. While our prototype is web-based and keyboard-driven, we deliberately engaged our co-designers in speculative discussions about translating the system into embodied forms (Section 6.4). These discussions, documented in Session 2, generated critical design insights about advanced haptics, mixed reality synchronization, and alternative physical input devices that both validate embodied approaches as a natural extension of our work and provide concrete pathways for future development be- yond screen-based interaction. 2.4 Gaps in Accessible 3D Data Visualization Despite progress in both tactile and immersive domains, the acces- sibility of analytic three-dimensional data visualizations remains underexplored. Existing research has developed methods for im- mersive exploration of 3D scatterplots, addressing challenges like occlusion and density perception through techniques such as Scap- tics and Highlight-Planes [76], or by investigating movement types and spatial abilities in VR-based scatterplot analysis [92]. Collabo- rative environments such as Shared Surfaces and Spaces highlight how groups interact with 2D and 3D visualizations in immersive contexts [57], while studies of device-input combinations stress the impact of controllers and display modalities on analytic accuracy [103]. Other work has explored VR systems such as VR-Viz [86], Un- charted Territoryâs design heuristics for VR data visualization [1], and web-based 3D visualization frameworks such as the Digital Li- brary of Mathematical Functions [101] and dbsliceâs infinite canvas approach [77]. These systems demonstrate the promise of 3D data interaction across scientific and educational domains but remain largely inaccessible to BLV users. Indeed, even when accessibility is considered, efforts are typically limited to object recognition tasks rather than analytic reasoning with continuous surfaces. Although physical tactile systems and AR offer powerful means of perceiving spatial information, they often do not scale well for analytic scenarios (Table 2). For instance, refreshable tactile displays remain expensive and low-resolution for continuous surfaces (as shown in dynamic tactile marker systems like Suzuki et al. [97]), while AR solutions for visual impairment frequently depend on co-located headsets or high-end devices, limiting deployment and web integration [17]. Shape-changing interfaces (e.g., pin-array displays) also struggle with cost, actuator density, and inability to dynamically render arbitrary, high-resolution scientific surfaces [58]. Our web-native, multimodal approach avoids these scalability bottlenecks by offering dynamically configurable, low-cost, high- resolution access to 3D surfaces via standard web technologies, enabling widespread and flexible analytic use without requiring any specialized hardware. Taken together, prior work underscores three trends: (1) acces- sibility efforts have focused on audiotactile and 2D modalities, (2) immersive and embodied interaction research demonstrates promis- ing but sighted-centered prototypes, and (3) existing 3D visualiza- tion systems emphasize technical innovation over inclusive de- sign. To the best of our knowledge, no prior research has applied evidence-based co-design to make three-dimensional data visual- izations accessible for BLV users by transferring knowledge from low-fidelity tactile models into high-fidelity, web-native multimodal experiences. This study addresses this gap by integrating tactile and digital probes, empirically evaluating non-visual features such as stereo sonification and volumetric cues, and offering concrete guidance for inclusive 3D visualization workflows. 3 CO-DESIGN PROCESS AND CONSIDERATIONS Having positioned Experience-Based Co-Design (EBCD) as our methodological foundation in Section 2.2, we now detail how we operationalized this framework through concrete design activities. Our adaptation of EBCDâs canonical stages (Table 1) unfolded across two major phases. First, we gathered lived experiences through low- fidelity (Phase 2, Thrust 1) and high-fidelity probes (Phase 1 and Phase 2, Thrust 1). Then, in Phase 2 (Thrust 3), we held co- design sessions where BLV collaborators prioritized interaction gaps and validated proposed features. Finally, we iteratively im- plemented solutions grounded in these experiential âtouch-pointsâ. This section documents the structure, composition, and progression CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. Criterion2D Web-based EncodingsPhysical Tactile DisplaysAugmented Reality (AR) Occlusion HandlingCannot multi-modally represent depth or hidden structures and remain limited to surface projection for 3D surfaces [26] Users must physically explore raised lines or pins, so occluded areas require manual navigation [107] Can simulate 3D depth with audio or visual layering, but hidden or overlapping features may still be difficult to disambiguate for non-visual users [66] Persistence Encoding via color, glyphs, or text is not appropriate for non- visual access due to high cognitive load [98] Physical relief or pins remain until manually reset, but reconfig- uration may be limited to pre-existing states [45] AR augmentations persist only while the device is held and tracked; tracking errors or device movement can break spatial consistency [33] Resolution for Continuous Surfaces Limited by the granularity of encoding (e.g., color bins, glyph size) [95] Limited by pin density or embossing precision; highly detailed surfaces are hard to render affordably [10] High potential: AR can render very fine-grained spatial structure digitally, but non-visual sensing is currently limited [18] Co-location RequirementUsers can access remotely via personal devices such as laptops and mobile phones [89, 104] User must have the tactile display physically present but do not require co-location with others AR devices (e.g., headset or phone) must be co-located with the physical space or digital model [29] CostVery low: software solutions are inexpensiveOften high: refreshable tactile displays or embossers are expen- sive to build or buy [73] High: AR headsets or development of specialized AR for accessi- bility can be costly [7] Web Integration Excellent: software and web-native systems are naturally inte- grated [89] Many tactile displays operate using specialized drivers and often require extensive troubleshooting Moderate to low: AR systems are often standalone applications and not deeply web-integrated [80] Table 2: Descriptive comparison of modalities for non-visual or multimodal 3D data access. of our collaborative inquiry, while Section 5 presents the empirical outcomes of these sessions, including the five design goals that emerged from co-designersâ narratives and the validated prototype features that these goals motivated. Our design methodology was rooted in collaborative autoethnog- raphy, an approach where researchers engage in a ârelational jour- neyâ to systematically study their own collective experiences [52]. This methodology positioned the design team, particularly its BLV members, as co-participants whose data visualization and research expertise was the very locus of the research. Instead of acting as objective instruments, their role was to engage in dialogic reflexiv- ity, where insights are co-constructed through shared narratives and mutual sense-making. We did not merely design for BLV users; rather, as a mixed-ability team, we systematically documented, ana- lyzed, and responded to our own lived experiences interacting with 3D data visualizations. Our reflections on moments of confusion, discovery, and frustration served as the primary data driving the iterative development of the prototype. 3.1 Methodological Justification and Design Rationale We deliberately chose this immersive, autoethnographic [19] EBCD approach [82] over conducting traditional user studies because our research goal was fundamentally generative, not evaluative. A traditional user study framework is optimized for assessing a pre- designed artifact against defined metrics; a process that would have required us to first build a prototype based on our own, likely flawed, assumptions about non-visual interaction. Such a process inevitably centers the researcherâs perspective and positions participants as subjects who merely react to a proposed solution. Given that our primary challenge was to translate a deeply embodied, tactile form of 3D data visualization interpretation and discovery into an entirely different digital and auditory modality, a traditional study would have failed to capture the nuanced, iterative process of discovery required. It could tell us if a feature worked, but not why it felt intuitive or disorienting. In contrast, we adopted an Experience-Based Co-Design (EBCD) approach [82], which framed our collaborators as co-designers with lived expertise rather than research subjects. As detailed in Section 2.1, EBCD treats lived experience as core design material, not merely background context. This methodological orientation shaped our process in several ways. First, our collaborators were en- gaged from the outset in identifying which challenges mattered and why, with their stories and reflections becoming the starting points for framing design problems grounded in lived realities rather than researcher assumptions. Second, their expertise guided evaluation of our tools not only in terms of usability, but also whether they contributed to autonomy, empowerment, and meaningful access, positioning them as co-judges of value rather than testers of func- tionality. As established in Section 2.1, EBCD distinguishes our work from conventional co-design [25,79] through its epistemological commitment: while co-design often begins from researcher-defined problem spaces and treats participation as a means of generating better solutions, EBCD foregrounds experiential grounding where design work emerges only after lived experiences are surfaced, shared, and co-interpreted, thus reconfiguring whose knowledge counts in defining the problem space. In our project, this meant resisting the urge to âtranslateâ existing visualization practices into accessible formats and instead reimagining what visualization could mean when defined through BLV collaboratorsâ perspectives. This shift was essential: it allowed our project to move beyond making visualizations technically accessible, toward exploring how visualization practices themselves might be transformed when co- shaped by BLV experience. Section 5 provides concrete examples of how co-designer narratives about orientation confusion and spatial reasoning directly motivated implemented features such as stereo panning, volumetric audio, and reference sonification, demonstrat- ing EBCDâs capacity to transform lived experience into grounded design decisions. The following subsections detail the composi- tion of the team whose narratives drove these design trajectories, the specific analytic tasks they addressed, and the structured pro- cess through which their expertise was systematically elicited and translated into prototype features. 3.2 Design Team Our research was conducted by an interdisciplinary, mixed-ability design team whose collective expertise and lived experiences were foundational to the co-design process. The team integrated sea- soned academic scholarship, practical software and hardware en- gineering, and a rich spectrum of visual abilities, ensuring that our technical explorations were deeply grounded in the nuances of non-visual interaction. The core academic group was composed of graduate researchers and faculty who drove the projectâs theoretical framing and it- erative development. Sanchita, a sighted Graduate Teaching and Research Assistant with a minor in Data Analytics, led the tech- nical implementation and interface development. She leveraged extensive prior experience creating 3D visualizations in MATLAB Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain and a strong understanding of statistical relationships to support the systemâs analytical foundations. Aziz, a Graduate Teaching and Research Assistant who is blind, provided continuous, real-time feedback from the perspective of a user with a foundational un- derstanding of data visualization. Aziz has experience reading and creating common plot types: including bar, line, box, scatter plots, histograms, and heatmaps; using visual, tactile, and sonified ap- proaches, and developed his 3D visualization knowledge through participation in this project. They were joined by Venkatesh, an Assistant Professor of Information with seven years of research experience in data visualization and programming tool accessibility, who contributed intermediate-to-advanced expertise shaped by his doctoral training and ongoing research in the field. Sile, a Professor of Information with 25 years of teaching experience, helped shape the cognitive aspects of this project. She regularly analyzes data in her own research and has contributed to preparing instructional materials through initiatives such as the Summer Institute. JooY- oung, an Assistant Professor of Information Sciences, brought over eight years of expert knowledge in creating accessible and multi- modal data visualization tools. His work includes the development of R packages for accessible visualization and deep expertise in sta- tistical concepts and research methods. This faculty trio guided two graduate researchers who were central to the prototypeâs creation. To bridge academic theory with real-world application, the team was expanded to include Ken, a Senior Software Engineer and R&D specialist in assistive technology. Ken contributed extensive exper- tise spanning both software and hardware, including the design of tactile graphics displays, paper tactile graphics, braille-to-digital pipelines, audioâtactile multimodal visualization systems, and data exploration workflows using tools such as pandas, Seaborn, and Matplotlib. He has also developed 3D tactile displays and collabo- rated with 3D printing workflows to represent STEM data, includ- ing exploratory work with haptic and tactile pin-array systems. He provided a critical perspective on the practical needs of blind and low-vision learners beyond the laboratory. Crucially, the teamâs strength was rooted in its diversity of lived experience. Five of the six members (JooYoung Seo, Sile OâModhrain, Venkatesh Potluri, Aziz Zeidieh, and Ken Perry) are blind or have low vision, with visual acuities represented in Table 3. This rich tapestry of per- spectives was the engine of our collaborative autoethnography, positioning BLV researchers as the primary research actors and ensuring that the design process was driven not by assumptions, but by a nuanced, collective understanding of non-visual data in- teraction. 3.3 Design Process Our inquiry was methodologically structured through a sequence of interconnected stages, inspired by the EBCD Framework [82]. We adapted its healthcare-centric model by translating the âpatient pathwayâ into the userâs journey of exploring a 3D visualization. Phase 1, (May to June 2025), initiated the project with a founda- tional ideation design cycle involving BLV and non-BLV collabo- rators (JooYoung, Aziz, and Sanchita), which produced an initial high-fidelity digital probe. The research then progressed to Phase 2 (July to August 2025), organized into three distinct thrusts to systematically refine this probe. Thrust 1 (first week of July 2025) Figure 2: Experience-Based Co-Design: Research Timeline involved the fabrication of a physical, low-fidelity probe to serve as an embodied reference. In Thrust 2 (July 2025), insights from the end of Phase 1 were integrated into the high-fidelity digital probe. Finally, Thrust 3 which expanded the collaborative team to include BLV co-designers (Sile, Venkatesh, Ken) and comprised two iterative co-design sessions lasting 90 minutes each: Session 1 (12 Aug 2025) centered on a comparative analysis of the experi- ence provided by two probes to identify interaction gaps (mostly situated around knowledge transfer from physical to digital proto- type), while Session 2 (22 Aug 2025) was dedicated to testing and validating new features that emerged from this analysis, culminat- ing in future directions that could provide the platform to observe embodied learning and interactions of users. 3.3.1Phase 1: Foundational Ideation and High-Fidelity Probe Development. Phase 1 was grounded in a mixed-ability co-design methodology, drawing upon the interdependence framework [9] to structure a sustained, two month long collaboration between two blind researchers (JooYoung, Aziz) and one non-BLV researcher (Sanchita) for creating an initial version of our high-fidelity design probe. In this initial phase, JooYoung and Aziz provided crucial design insight rooted in their lived experience, while Sanchita led the implementation and interface development. The process was highly iterative, characterized by daily collaboration with Aziz and weekly sessions with JooYoung, where each meeting involved direct evaluation of the evolving probe, the proposal of new features, and the identification of interaction pain points. This continuous feed- back loop allowed Sanchita to implement refinements in real-time. A central activity of this phase was the conceptual reframing of con- ventional visual affordances, such as grid overlays and highlighting, into non-visual strategies like wireframe traversal, axis-specific navigation, and the synchronized alignment of auditory, visual, CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. Table 3: Team membersâ visual acuity, onset of impairment, age, and roles. NameVisual AcuityAge of OnsetAgeOccupation SanchitaNot BLV; corrected to 20/20 with lensesNot applicable24Graduate Teaching and Research Assistant Aziz No vision in left eye; 20/2000 in right eyeCongenital28Graduate Teaching and Research Assistant VenkateshTotally BlindCongenital32Assistant Professor of Information SileSome light perceptionCongenital59Professor of Information KenTotally Blind2055Senior Software Engineer and R&D Specialist JooYoungNo light or shape perception in right eye, light and shape perception in left eye 1135Assistant Professor of Information Sciences and textual feedback. This foundational work culminated in the production of the initial high-fidelity digital design probe, which served as the empirically grounded starting point for the expanded research activities in Phase 2 - Thrust 2 and Thrust 3. 3.3.2Phase 2: Evidence-Based (Re)Design and Knowledge Transfer. The second phase of the research was designed to rigor- ously test, critique, and reconstruct the initial probe. The overar- ching goal of Phase 2 was to convert the high-fidelity probe into a functional prototype capable of facilitating effective knowledge transfer from a physical, embodied medium to a digital, multi-modal environment. This work was organized into three distinct but in- terconnected thrusts, with Sanchita, Aziz, and JooYoung leading the first two. Thrust 1 of Phase 2 was the creation of a physical, low-fidelity probe. This probe functioned not merely as a demonstrative tool but as a crucial epistemological anchor; it established a âground truthâ of embodied understanding, providing a shared sensory base- line against which the digital experience could be systematically evaluated. Thrust 2 of Phase 2 focused on implementing initial insights from our foundational work. Prior to engaging the broader group, Sanchita integrated the key design recommendations from JooYoung, which had been identified in Phase 1, into the digital probe. This act of translation ensured methodological continuity and established a starting point for the expanded sessions that was already grounded in expert user knowledge. Thrust 3, the most extensive component of Phase 2, was the execution of two iterative co-design sessions. (1)Session 1: This session was structured as a direct compara- tive analysis. JooYoung, Sile, Venkatesh, and Ken were invited to first explore the physical probe from Thrust 1, using man- ual and tactile inspection to construct a direct understanding of the dataâs form by feeling its gradients, slopes, and peaks. Subsequently, they transitioned to the high-fidelity digital probe from Thrust 2, using keyboard-only navigation and sonification to explore the same data structure. Session 1 concluded with a group discussion and a silent brainwriting exercise whose prompts were designed by Sanchita (read- able at Appendix Section A.1). This exercise produced a clear consensus on four major features required to bridge this experiential gap which we will talk about in Section 5. (2)Interim Development: During the interim period between Session 1 and 2, Aziz and Sanchita (posing themselves as co-developers - with Sanchita leading implementation and Aziz providing iterative feedback) implemented three of the four prioritized features. (3) Session 2: This session was bifurcated into two distinct parts. The first part was dedicated to the systematic testing and val- idation of three of the four prioritized features implemented after Session 1 (Section 5.3.1). The second part of the ses- sion shifted to future-oriented ideation, where collaborators engaged in a brainwriting exercise to brainstorm how the embodied knowledge from the physical prototype could be transferred into more immersive digital environments (Sec- tion 5.3.2). Session 2 concluded with a collaborative brain- writing and discussion exercise to generate speculative ideas that could generate an embodied 3D visualization learning experience. The prompts were designed by Sanchita (read- able at Appendix Section A.3). 3.3.3Analytic Tasks and Co-Design Prompts. In these Sessions, we explicitly targeted core analytic tasks essential for nonvisual 3D data exploration, tasks that are foundational to scientific reasoning across STEM disciplines. Our co-designers were prompted to en- gage with the following challenges during both probe exploration sessions and the brainwriting exercises: â˘3D Orientation: Establishing and maintaining awareness of position and direction within the three-dimensional coor- dinate space; prompts included âHow do you know where you are in the plot?â and âWhat cues help you understand the axes?â â˘Landmark and Peak Finding: Identifying local maxima, minima, and other salient features; prompts included âHow would you locate the highest point?â and âCan you find where the data changes most dramatically?â â˘Comparing Local Maxima versus Global Trends: Dis- tinguishing between isolated peaks and broader patterns; prompts included âHow do you differentiate a single peak from an overall rising trend?â and âWhat would help you compare multiple high points?â â˘Gradient Tracing: Following the direction and steepness of slopes across the surface; prompts included âCan you follow the path of steepest ascent?â and âHow would you trace a ridge or valley?â â˘Identifying Occluded or Partially Hidden Features: De- tecting regions obscured by overlapping data or structural complexity; prompts included âHow do you explore areas that might be hidden from one perspective?â and âWhat strategies help you uncover features that arenât immediately apparent such as those with similar Y values?â These tasks were not presented as formal test scenarios but rather as conversational prompts woven into our co-design sessions, en- couraging co-designers to articulate their exploration strategies, confusions, and moments of insight. The brainwriting exercises at Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain the end of Session 1 and Session 2 (Appendix Sections A.1 and A.3) further formalized these prompts, asking co-designers to reflect on which features supported or hindered each task. The complete set of task-based questions used throughout both sessions is documented in Appendix Sections A.2 and A.4, connecting the five analytic tasks directly to the specific prompts and evaluation criteria employed during probe exploration and feature validation. The brainwriting exercise in Session 1 produced a clear consensus on four major features, but the path from co-designer narratives to these imple- mented features merits closer examination. Two such examples illustrate this translation process. During probe exploration, JooY- oung articulated a fundamental analytic limitation: the difficulty of comparing different regions within a single plot. Where the physi- cal probe allowed simultaneous tactile comparison using multiple fingers, the digital interface forced sequential, memory-dependent exploration. Their narrative directly motivated the configurable buffer feature, implemented during the interim development period, which enables users to save a regionâs sonification and compare it against their current focus point. Similarly, when co-designers reported spatial disorientation, Ken and Sile proposed adding refer- ence sonifications. These narratives coalesced into the fixed origin sonification with replay mechanism, validated in Session 2. These trajectories exemplify how EBCD centered co-designersâ experien- tial knowledge to produce features addressing real, lived challenges rather than theoretically anticipated needs. 4 DESIGN PROBES 4.1 Low-Fidelity Design Probe The low-fidelity probe (created in Phase 2, Thrust 1 and employed in Phase 2, Thrust 3, Session 1) was constructed on an 11.7 x 16.5 inch, 3/16-inch thick polystyrene foam sheet, which formed a sturdy base and provided the co-designers with a clear orientation for the plot. We affixed thicker plastic tubes (from 19.69-inch balloon sticks - marked with tape to represent the ticks) along two perpendicular edges to represent the Z-axis (longer edge) and X-axis (shorter edge). A small peg holder was attached at the origin for the co-designers to insert a separate tube for the Y-axis, perpendicular to the base. To represent the raw data (5 points), an array of peg holders was attached across the foam board; into these, we inserted thinner plas- tic tubes cut to varying lengths corresponding to the height of each data point. Small balls of heavy-duty aluminum foil were placed on the tip of each tube to create a distinct tactile representation of the points. All elements, including the axes, were labeled using a 6-dot braille label maker for full accessibility. Finally, a 38 x 40 inch muslin cotton cloth was draped over the points to simulate the continuous mesh of the surface plot. Developed by Sanchita, this tactile artifact translated the abstract, visual topography of a 3D surface plot into a tangible landscape of peaks, valleys, and slopes (viewable in Figure 3). Figure 3: Low-Fidelity Prototype 4.2 High-Fidelity Design Probe Our high-fidelity design probe (created in Phase 2, Thrust 2 and employed in Phase 2 Thrust 3, Session 1 and 2) is a browser- based, multi-modal platform that evolved directly from our auto- ethnographic inquiry. To facilitate this, we made a deliberate method- ological choice to build a modular system architecture. This archi- tectural separation was the technical manifestation of our EBCD methodology; it was chosen specifically because it allowed us to be immediately responsive to collaborator feedback. By isolating concerns, we could rapidly prototype and integrate new, complex featuresâoften within a single layerâwithout destabilizing the en- tire application. This enabled us to translate the insights from our co-design sessions directly into functional system logic. The system is composed of five strictly hierarchical layers that communicate through an event-driven architecture using central- ized event constants (defined in EventConstants.js): â˘Data Layer (PlotData.js): Manages data ingestion, valida- tion, normalization, and statistical analysis. â˘Engine Layer (VisualizationEngine.js): Handles We- bGL rendering for points, surfaces, and contours using cus- tom GLSL shaders. â˘Accessibility Layer (NavigationController.js): Con- tains all non-visual interaction logic coordinating eleven controllers for sonification, text feedback, navigation, auto- play, jump-to-peak, drag-select, and review modes. â˘UI Layer (UIController.js): Manages visual interface components including menus, color schemes, axes rendering, and AI chat. â˘Application Layer (app.js): Serves as global coordina- tor, initializing dependencies and orchestrating cross-layer communication. Layer communication follows strict architectural constraints: components access only their own layer and lower layers through dependency injection, while cross-layer events propagate through a custom event bus. Representative event payloads includedisplay- mode-changed(notifying mode switches),drag-select-select ion-confirmed(communicating buffer boundaries), andautopl ay-state-changed(coordinating playback status). This architec- ture enabled us to implement complex features like buffer aggre- gation and jump-to-peak functionality within isolated controllers while maintaining system stability across iterative (re)designs. CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. Sonification Engine and Web Audio Implementation. The soni- fication engine (SonificationController.js) is implemented as a custom Web Audio API synthesis pipeline operating at the browserâs native sample rate (typically 48kHz). Audio synthesis occurs entirely client-side without external libraries, using the AudioContextinterface to create real-time parameter-mapped oscillators. The core signal path begins with oscillator genera- tion (createOscillator()), routes through envelope shaping (createGain()), applies spatial positioning via binaural panning (createStereoPanner()), and adds depth perception through con- volution reverb (createConvolver()). Timbre encoding employs three oscillator waveforms: sine for smooth low-frequency regions, triangle for mid-range transitions, and square for high-frequency accents; selected dynamically based on normalized X-axis position. The binaural setup maps X-coordinates to stereo pan values (-1.0 for leftmost to +1.0 for rightmost), creating horizontal spatial posi- tioning that requires headphone playback for accurate localization. Depth perception (Z-axis) is rendered through algorithmic reverb: we generate a 2.2-second impulse response buffer with exponen- tial decay (power 3.5) and apply variable wet/dry mixing (20-95% wet) scaled by normalized depth values. Pre-delay ranges from 10ms (foreground) to 90ms (background), with lowpass filtering (6500Hz to 2000Hz cutoff ) simulating distance-dependent damping. Round-trip latency from keystroke to audio output measures ap- proximately 15-30ms on modern browsers, below the threshold for perceived interaction delay. Visual Highlighting System and WebGL Implementation. The vi- sual highlighting system (HighlightController.js) provides high-contrast visual feedback for navigation and selection through direct manipulation of WebGL rendering buffers. The controller op- erates by maintaining state for three distinct highlighting contexts: single-point navigation (magenta highlight with 4x size multiplier), multi-selection regions (white highlighting for drag-selected ar- eas), and cursor positioning (dynamic yellow/white coloring based on selection state). The core highlighting mechanism integrates with the Engine LayerâsVisualizationEngine.jsthrough the createBuffers()pipeline: when a point or wireframe rectangle is marked for highlighting viasetHighlightedPoint(index)or setHighlightedWireframeRectangle(index), the controller triggers complete buffer regeneration, during which the render- ing engine queriesisPointHighlighted(index)for each data point to determine if enhanced visual properties should be applied. For point-mode visualization, highlighted vertices receive mod- ified RGBA color arrays (retrieved viagetHighlightColor()) and scaled point sizes (computed throughgetHighlightSize Multiplier()) before being uploaded to GPU vertex buffers us- ing WebGLâsgl.bufferData()withSTATIC_DRAWusage pat- tern. In surface mode, the highlighting system employs separate WebGL buffer objects (highlightPosition,highlightColor, highlightIndices) to isolate highlighted wireframe rectangles from the base mesh geometry. During rendering, these buffers are drawn with aggressive depth handling; disabling depth test- ing (gl.disable(gl.DEPTH_TEST)), applying polygon offset (gl.polygonOffset(-1.0,-1.0)), and forcing opaque rendering through the simpler line shader pipeline to ensure highlighted ele- ments render atop the base visualization regardless of y-ordering. This architecture maintains strict separation between highlighting logic (Accessibility Layer) and rendering implementation (Engine Layer), enabling the system to support simultaneous multi-selection highlighting, cursor feedback, and navigation focus without buffer conflicts. 4.2.1 Initial System: Before Session 1. The initial version of the probe was developed during and after the foundational work in Phase 1 (read more in Section 5.1), with features aimed at enhanc- ing user autonomy. To address the tedium of manual traversal, we implemented two key exploration strategies within the Accessibility Layer: â˘Multi-Perspective Auto-Play (AutoPlayController.js): This controller implements our âwhole-to-partâ navigation strat- egy by sequencing navigation commands and triggering corresponding sonification events, allowing users to receive auditory âscreenshotsâ of the data without exhaustive man- ual traversal. â˘Jump-to-Peak Function (JumpController.js): This feature sta- tistically analyzes the data (view paragraph below for a de- tailed explanation) to identify the most significant peaks and troughs, enabling rapid navigation between a surface plotâs most salient features with a simple J key press. In addition to the core visualization features, Sanchita implemented a multi-threaded AI chat assistant that leverages Gemini 2.5 Pro to support interactive data exploration. This feature integrates directly with the Visualization Engine, enabling users to query the underlying dataset or trends visible in the 3D plot. However, this feature was not extensively tested during Session 1 or Session 2, as our primary research focus remained on independent, non-visual data exploration and the transfer of knowledge between low-fidelity and high-fidelity design probes. A detailed description of the AI chat assistant implementation is provided in Appendix B. Jump-to-Peak Statistical Algorithm. The Jump-to-Peak feature (JumpController.js) identifies salient features through statistical outlier detection operating on wireframe rectangles. The algorithm: (1) extractsavgYvalues from all rectangles; (2) sorts byavgYto select top 20 positive and bottom 20 negative candidates; (3) re- moves spatial duplicates by(x,z,avgY)key; (4) calculates dataset statistics (mean,min,max,range); (5) computes 20% threshold = (maxâ min) Ă0.2; (6) tests proximity: if negatives are within threshold but positives are not, selects top 10 positive peaks only; if positives are within threshold but negatives are not, selects bottom 10 negative peaks only; otherwise selects top 10 of each; (7) sorts final peaks with positives descending and negatives ascending; (8) returns array of peak objects with sign and coordinates. When activated by pressing J in surface mode, the controller saves the current navigation position, detects peaks using this algorithm, and sequentially jumps through the identified features. Each jump updates the grid position to align with the peak rectangleâs center coordinates, highlights the rectangle visually, and plays a distinc- tive frequency-swept tone (400Hz base for negative peaks, 800Hz for positive peaks). Pressing J again cycles to the next peak, while Escape exits the mode and restores the saved navigation position. Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain 4.2.2System (Re)Design: After Session 1. Based on Sile, Venkatesh, Ken and JooYoungâs interaction with both low-fidelity and high- fidelity design probes, we postulated some design features to be implemented (read more in-depth in Section 5.2). To address the critical need for stable orientation, we enhanced the Sonification- Controller.js within the Accessibility Layer to provide a fixed ref- erence sound for the origin (0,0,0) by pressing the 0 key and a replay sonification mechanism of current point by pressing . key. The reference sound acts as a constant, while the replay function, bound to the . key, allowing a user to re-listen to their current lo- cationâs sonification without moving. This same controller is also responsible for encoding position and depth cues through stereo panning and volumetric spatial audio. As a user navigates along the X-axis, the sound pans between the left and right stereo chan- nels; movement along the Z-axis is conveyed through changes in volume, creating an intuitive auditory analog for proximity and distance. Finally, to balance analytic detail with cognitive load; an- other key finding from Session 1 (read in depth in Section 5.2), we developedDragSelectController.jsto implement configurable buffer playback as detailed below. The implementation spanned multiple layers: the Accessibility Layer managed selection logic and queried the Data Layer to retrieve buffered points or rectangles, calculated arithmetic means for aggregated mode, and organized sequential mode playback. TheHighlightController.js(Acces- sibility Layer) coordinated withUIController.js(UI Layer) to render visual feedback throughVisualizationEngine.js(En- gine Layer). This empowers users to fluidly shift between high-level summaries and detailed inspections of specific areas, streamlining the discovery process. Users could upload their own data as well, and all described multi-modal features are extended to the rendered plot. Buffer Aggregation and Playback Modes. Buffer selection (DragSe lectController.js ) implements a three-step rectangular region selection: users press D to enter selection mode, then press Enter to begin anchoring the selection start point at their current navigation position. They navigate using arrow keys to define the opposite cor- ner (visual highlighting also follows the expanding selection area), then press Enter again to confirm and store the region. The con- troller queries the Data Layer for all points or wireframe rectangles with coordinates falling within the defined bounds, storing them incurrentSelectionas a buffer array that persists until a new selection is made. Aggregation applies arithmetic mean calculation: for each selected item, we normalize coordinates (handling both point objects with x/y/z properties and rectangle objects with cen- terX/avgY/centerZ properties), sum all valid Y-values, and divide by count to produceaverageYValue. The aggregated playback mode (playable though the G-key) sonifies this single mean value for 1.0 second duration. In sequential (non-aggregated) mode, the buffer is organized by Z-index rows (front-to-back) with left-to- right X-coordinate sorting within each row. Each item plays for 0.3 seconds with 125ms inter-item silence, creating a systematic auditory sweep. Users toggle between modes by pressing G: the first press activates sequential playback, the second switches to aggregated, and the third returns to sequential. 5 FINDINGS Our findings are drawn directly from the auto-ethnographic data collected during the end of Phase 1 and the two co-design sessions in Phase 2, Thrust 3. 5.1 End of Phase 1: Allowing User Autonomy over Data Exploration Conversations and iterative testing with JooYoung revealed that empowering users with autonomy was the most critical factor for successful data exploration, consistent with prior findings in ac- cessible visualization that emphasize user-controlled interaction and progressive disclosure [55,63]. They expressed that a purely manual, point-by-point traversal of the data was not only tedious but also made it difficult to build a cohesive mental model of the 3D plot, echoing challenges reported in sequential non-visual explo- ration of complex data [90]. Their feedback consistently highlighted the need for tools that would allow them to control the scope and focus of their exploration, balancing high-level overviews with the ability to investigate specific features efficiently. This led to the development of our first two design goals aimed at granting users greater control over their interaction with the data. DG1. The prototype should provide multi-perspective auto- play to deliver auditory âsnapshotsâ of the dataset: Co- designers emphasized the need for rapid overviews of com- plex plots; auto-play from multiple perspectives was valued as a way to capture structural patterns across all three axes without exhaustive manual traversal, aligning with overview- first strategies in accessible visualization [89, 109]. DG2.The prototype should include a jump-to-peak func- tion to quickly locate salient features: JooYoung high- lighted the value of a dedicated command for automatically cycling between local maximum peak and trough values, en- abling efficient interrogation of surface plot trends without exhaustive searching, similar to landmark-based navigation approaches [91, 110]. 5.2 Phase 2, Session 1: Surfacing the Experiential Gap Session 1 directly addressed RQ1 by revealing a fundamental dis- connect between the tactile prototypeâs physical understanding and the digital versionâs abstract experience, a gap previously observed when translating tactile or embodied representations into screen- based systems [36,99]. Co-designers immediately surfaced multi- ple orientation and comparison challenges that mapped directly onto five essential analytic tasks: 3D orientation, gradient tracing, comparing local maxima versus global trends, landmark and peak finding, and identifying occluded features. Table 4 systematizes the prompts we used to elicit these experiences, the analytic tasks they addressed, the observed issues that emerged from co-designer narratives, and the design changes we subsequently implemented to support each task. A primary point of friction was the orientation of the coordinate system, which we anticipated from JooYoungâs experience in Phase 1. However, we wanted to gather insight from Sile, Venkatesh and Ken before we implemented features, so that we could learn from CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. PromptAnalytic TaskObserved IssueDesign Change Implemented âHow do you know where you are in the plot? What cues help you understand the axes?â 3D OrientationAxis convention misalignment: co-designers ex- pected Y-axis on board, Z-axis pointing out; dig- ital used Y-vertical, Z-depth. Ken: âI would ex- pect the y-axis to be on the board and the z- axis pointing out.â Venkatesh: âX-axis would be along the horizontal edge of the board, Y-axis would be along the vertical edge... and Z-axis would be the one popping.â Fixed reference sonification at origin (0,0,0) via 0-key; replay mechanism via .-key to re-hear current position (DG3). âCan you follow the path of steepest as- cent? How would you trace a ridge or val- ley?â Gradient TracingLack of spatial encoding for horizontal and depth dimensions. Venkatesh: âit would be nice to have it pan from left to right, and decrease the volume as itâs going forward.â Stereo panning for X-axis (left-right), volumet- ric audio (volume + reverb) for Z-axis depth (DG4). âWhat would help you compare multiple high points?â Comparing Local Max- ima vs. Global Trends Difficulty maintaining mental representation of previous positions during sequential explo- ration; physical probe allowed simultaneous tac- tile comparison. Configurable buffer: drag-select to save region, toggle between aggregated (mean) and sequen- tial (point-by-point) playback via G-key (DG5). âHow would you explore areas that might be hidden from one perspective?â Identifying Occluded Features Inter-plot comparison not supported; need to differentiate datasets by categorical variable. Ken: âWhat if I wanted to compare two sets of data... Is there any way to overlay them and switch between the two surfaces?â Sile: âif you gave the sound for Georgia a square wave in- stead of a sine wave.â Identified for future implementation: dual- dataset overlay with distinct timbres and key- binding toggle. Table 4: Session 1 evaluation: prompts, analytic tasks, observed issues, and implemented design changes. their expertise and follow the EBCD framework guidelines with all our collaborators [70]. Notably, this issue surfaced even before the co-designers interacted with our digital prototype; participants expressed expectations about axis orientation while experiencing the physical tactile probe, consistent with prior work on embodied spatial expectations [47]. For example, Ken remarked, âYeah, I find that kind of odd, too. I would expect the y-axis to be on the board and the z-axis pointing out.â Venkatesh shared a similar mental model: â...what I was expecting is X-axis would be along the hori- zontal edge of the board, Y-axis would be along the vertical edge... and Z-axis would be the one popping.â These comments referred specifically to their expectations for the physical device, but the same mental model carried over when they later interacted with the digital prototype, whose coordinate system followed standard visualization conventions (vertical Y, horizontal X, depth Z). This mismatch helped us identify the need to explicitly align coordinate orientations across both physical and digital components. The (re)design process was central to our methodology, repre- senting a direct, evidence-based response to the core challenges identified during our co-design sessions. The impetus for the (re)design was the experiential gap that collaborators surfaced when com- paring the low-fidelity tactile probe with the initial high-fidelity digital version, reflecting similar tensions observed in prior tactile- to-digital translation efforts [98]. While the tactile model afforded a rich, tactile understanding, the digital probe felt âabstractâ and âdisorientedâ, necessitating a fundamental reconstruction. This re- design involved implementing empirically grounded features to bridge this gap, including a fixed reference sonification for ori- entation, stereo and volumetric audio to encode position [39,93], and a configurable buffer playback to balance analytic detail with cognitive load. Conceptually, this (re)design process was our pri- mary method for beginning to translate tactile knowledge into a web-based digital form. It signifies a shift from creating a static artifact to engaging in a dynamic, collaborative refinement aimed at enhancing usersâ analytical agency and empowering them to independently interpret complex 3D data. Venkatesh proposed using spatial audio, suggesting, âit would be nice to have it pan from left to right, and decrease the volume as itâs going forward.â This idea of mapping spatial dimensions to audio properties was a recurring theme and formed the basis for our efforts to incorporate stereo and volumetric spatial audio to encode position and depth cues. The concept of a stable reference point also emerged as a critical need. Ken suggested, âYou could even mark a reference and then walk away from it on the surfaceâ. Sile expanded on this, proposing a dedicated key binding: âYou could have a reference just available on a key, where you would just ping itâ. This exchange was the genesis of our goal to provide a reference point and replay mechanism to restore orientation during exploration. A significant limitation of the initial digital probe was the difficulty of comparing different points or regions within a single plot. This led to the idea of a âcustom bufferâ. As JooYoung proposed, â...it might be instrumental, if we have a, like, custom buffer, so that, say, for example, we can save one specific segment in specific buffers... And then you can hear that buffer... and you can also replay your currently focused segmentâ. This concept of selecting and saving a region for later comparison formed the basis for our goal to implement configurable buffer playback to balance analytic detail and cognitive load. Beyond intra-plot comparison, the discussion surfaced the need for inter-plot comparison. Ken raised a critical use case: âWhat if I wanted to. . . Compare two sets of data, same surface types... Is there any way to overlay them and. . . Switch between the two surfaces?â This question identified a fourth key feature: the ability to compare two datasets differentiated by a categorical variable (e.g., precipitation in Texas vs. Georgia). The group brainstormed solutions, with Sile suggesting the use of distinct timbres to differ- entiate the two plots: âif you gave the sound for Georgia a square wave instead of a sine waveâ. This idea of layering two datasets in the same coordinate space and allowing a user to switch between them using a keybinding, with distinct auditory feedback for each, was identified as a crucial direction for future development, even Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain Figure 4: Prototype interface though it was not implemented before Session 2. Thus, our final three design goals were formulated: DG3. The prototype should provide a reference point and replay mechanism to restore orientation during explo- ration: Co-designers identified the fixed origin sonification and replay (.) as essential for regaining bearings after becom- ing disoriented in complex data spaces. DG4.The prototype should incorporate stereo and volumetric spatial audio to encode position and depth cues: Stereo panning and distance-based loudness was collectively de- cided upon as the most effective method to convey horizontal and vertical positioning within surface plots. DG5.The prototype should implement configurable buffer playback to balance analytic detail and cognitive load: The ability to toggle between aggregated and non-aggregated modes allowed co-designers to select their customized set of values to be replayed and compared; giving them the ability to adjust buffer density depending on task complexity and preference. 5.3 Phase 2, Session 2: Testing and Validating New Features Session 2 served two functions: validating the three features im- plemented after Session 1 and brainstorming future directions for embodied interfaces. The validation phase systematically tested each feature through task-specific prompts, while the brainstorm- ing phase generated speculative designs for translating the pro- totype beyond web-based interaction, in line with embodied and immersive accessibility research [43, 61]. 5.3.1Part 1: Testing of features. This validation phase addressed RQ2 by systematically testing whether each implemented feature successfully supported its target analytic tasks. Each feature was evaluated through specific task-based prompts, following task- centered evaluation practices in accessible visualization research [90]: orientation restoration for DG3, gradient tracing for DG4, and re- gion comparison for DG5. Table 5 summarizes the test prompts, task performance results, and refinements for each validated feature. The reference sonification and replay mechanism (DG3) received immediate validation. Co-designers found it essential for restoring orientation after extended navigation. Sileâs observation that the feature allowed both coincidental and sequential playback high- lighted its flexibility; users could hear the reference and current position simultaneously or separately depending on their analytic need. Venkateshâs positive assessment confirmed that the feature successfully addressed the spatial disorientation identified in Ses- sion 1. Spatial audio (DG4) yielded mixed results. Stereo panning ef- fectively encoded horizontal position (X-axis), with Sile noting its utility for tracing gradients. However, volume alone proved insuf- ficient for depth perception (Z-axis). Sile could not perceive the volumetric changes during navigation. JooYoungâs suggestion to augment volume with reverb emerged directly from this limitation. His proposal to combine amplitude decay with echo creates a multi- dimensional depth cue that reinforces distance perception through both loudness and spatial reflection. This refinement exemplifies the iterative validation process: co-designers not only identified failures but proposed empirically grounded solutions rooted in their perceptual expertise. The configurable buffer (DG5) validated as a powerful analytic concept but surfaced critical usability issues, consistent with prior findings that aggregation mechanisms require explicit feedback to support interpretation [63]. Venkatesh questioned the two-step selection initiation process, suggesting that the system should an- chor the selection at the userâs current position when D is pressed, rather than requiring a separate Enter confirmation. Kenâs question about the aggregated soundâs utility revealed that the featureâs purpose was not immediately apparent. Venkatesh clarified the in- tended use case: comparing aggregated values to individual points within the selected region. This narrative exposed a gap in the interaction design; the feature lacked sufficient feedback to com- municate its analytic function. Venkatesh further suggested that the system announce when users enter or exit the selected region during subsequent navigation, providing continuous spatial aware- ness of the bufferâs boundaries. These findings further refined our understanding of RQ2, demonstrating that empowering accurate task performance and confident interpretation requires not only implementing features but ensuring their interaction design clearly communicates how they support specific analytic workflows. Co-designers discovered they could select a region using the buffer, then invoke the AI assistant to ask targeted questions about only that subset of the data. This scoped query pattern aligns with recent work on coupling spatial selection with conversational anal- ysis [83]. This scoped query pattern emerged organically from Venkateshâs observation that âif Iâm looking at just this peak, I want to ask the AI about just this peak, not the whole dataset.â The buffer mechanism, originally designed for audio-based region comparison, thus revealed a second analytic function: constrain- ing the context window for natural language queries. This finding demonstrates how multimodal features can support complementary interaction modalitiesâspatial selection via keyboard navigation directly informs textual query scope, creating a tighter coupling between embodied exploration and AI-mediated analysis. Table 6 CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. FeatureTest PromptResult & Co-designer QuoteNext Steps / Refinements Reference Sonification & Re- play (DG3) âNavigate away from the origin and use the reference key to reorient yourself.â Success. Sile: âit does help, actually... hav- ing it separate like that, but allowing you to play either coincidentally or in sequence, thatâs really nice.â Venkatesh: âthat was the main observation I had, but I think itâs pretty cool that I get to hear the reference point.â None required; feature successfully validated for ori- entation restoration. Stereo & Volumetric Audio (DG4) âTrace a horizontal path and de- scribe the spatial cues you perceive.â Partial success. Stereo panning vali- dated. Sile: â(stereo panning) it actually has been very interesting.â Volume for depth too subtle. Sile: â(volume) I couldnât notice it that much.â Add reverb to depth encoding. JooYoung: âUsing vol- ume only may not be sufficient, so I suggest using echo sound (reverb)...if you combine volume plus, echo, then, it will be more pronounced.â Configurable Buffer (DG5)âSelect a region, compare its aggre- gated sound to individual points, then compare the region to a dif- ferent area.â Partial success. Concept validated, inter- action issues surfaced. Venkatesh: âwhy...I mean, just flip the order of actions, right? Like, first go to the point that you want to start selection from, and then when you press D, the backend knows.â Ken: âIâm wondering...How this is useful... what am I getting from that?â Redesign interaction: anchor selection at current po- sition on first D-press. Add boundary entry/exit an- nouncements. Venkatesh: âmy use case for suggesting this is comparing whatever you might hear in aggre- gate to individual points.â See Table 6 for prioritized fixes. Table 5: Session 2 feature validation: test prompts, results, and refinements. documents the prioritized issues surfaced during buffer interac- tion testing and the corresponding design refinements necessary to support both audio comparison and AI scoping workflows. 5.3.2Part 2: Discussed Future Directions. The brainwriting portion of Session 2 was primarily focussed on answering RQ3, generating a rich set of ideas for moving beyond the current web- based implementation. These ideas coalesced around three inter- connected themes that align with prior embodied and immersive accessibility research [47,60]: advanced haptics to restore persistent tactile feedback, mixed and augmented reality to merge physical and digital exploration spaces, and alternative physical interfaces to replace keyboard-centric navigation. Collectively, these propos- als represent co-designersâ vision for translating the intuitive, ex- ploratory nature of the tactile probe into digitally mediated forms that preserve, and potentially augment, its embodied affordances. (1)Advanced Haptics and Tactile Displays: Haptic feedback emerged as the most direct pathway for restoring the persis- tent tactile cues that grounded the physical prototype. Sile articulated hapticsâ strength: âreally good at dynamic feed- back,â proposing that force resistance could encode data den- sityâan aggregated data point âcould be heavier, or harder to press against, the more data it is aggregating.â This maps physical effort directly onto information density, creating an embodied metric for data complexity. Ken identified near- term implementation targets, advocating for existing devices like the Inverse3 Haptic Controller 1 or the Graphiti tactile display 2 as a âquickstep to trying a different method of âfeel- ingâ the graph.â Sile extended this vision to multi-line braille displays, imagining parallel multi-finger sensing that would allow simultaneous selection and comparison of multiple regionsâa shift from sequential keyboard navigation to par- allel tactile interaction. These proposals share a common 1 https://w.haply.co/inverse3 2 https://w.orbitresearch.com/product/graphiti-plus thread: leveraging haptic modalities to restore the simultane- ity and persistence lost in the transition from physical to digital. (2)Mixed and Augmented Reality: MR aims to merge the physical exploration space with digital data overlays. JooY- oung positioned MR as âcritical,â envisioning âbidirectional synchronization between the digital environments and phys- ical artifacts.â His three-step operationalization concretizes this vision: (a) construct a physical play mat demarcating the X-Z grid, (b) align the AR boundary with the matâs physical edges, and (c) sonify Y-values as pitch while users physi- cally traverse the mat. This design recasts the userâs body as an âembodied cursor,â directly mapping physical move- ment through space onto data navigation. The approach dis- solves the screen boundary, allowing co-designers to âwalk throughâ the dataset using proprioceptive and vestibular cuesâmodalities entirely absent from keyboard-based inter- action. This proposal directly extends the tactile prototypeâs affordances by preserving spatial exploration while augment- ing it with dynamic, real-time sonification. (3)Alternative Physical Interfaces: Beyond haptics and MR, co-designers proposed novel input devices to escape key- board constraints while remaining grounded in accessible hardware. Venkatesh suggested repurposing familiar con- trols: two-way trackballs, surface dials, laptop touchpads, or even the keyboard surface itself as a 2D positional sensor. These ideas prioritize learnability and availability, adapting existing assistive technology interaction patterns to 3D data navigation. Ken proposed a more elaborate system: a glove- mounted hand tracker controlled by CNC-style XYZ pulleys, providing kinesthetic feedback as users âsculptâ through the data space. This design borrows from haptic sculpture interfaces, mapping fine motor control onto precise data in- terrogation. Collectively, these speculative interfaces share a Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain PriorityUsability IssueCo-designer Quote / ObservationProposed Fix P1Two-step selection initiation breaks flow; users must navigate away, press D, navigate to start point, press Enter. Venkatesh: âwhy...I mean, just flip the order of actions, right? Like, first go to the point that you want to start selection from, and then when you press D, the backend knows.â Anchor selection at current position on first D-press; eliminate redundant Enter step. P2Lack of boundary awareness; users cannot tell when they re-enter or exit the selected region during subsequent navigation. Venkatesh: âit would be nice if, as youâre navigating around...it announces when you enter the region or when you leave the region.â Add audio announcement when cursor crosses buffer boundary; consider persis- tent background cue (e.g., subtle ambient tone) while inside region. P3Aggregated playback purpose unclear; co- designers unsure when mean value is useful versus sequential point-by-point replay. Ken: âIâm wondering...How this is useful... what am I getting from that?â Venkatesh clarified: âmy use case for suggesting this is comparing whatever you might hear in aggregate to individual points.â Provide contextual help (e.g., on-demand description of aggregation modes); add ex- ample prompts in tutorial demonstrating comparison workflow. P4No visual or audio confirmation when buffer selection is saved; uncertain whether system registered the region. Observed during testing: co-designers pressed G mul- tiple times, unsure if buffer was stored. Add distinct confirmation sound when buffer is saved; optionally announce re- gion dimensions (e.g., âBuffer saved: 6 by 7 regionâ). Table 6: Prioritized buffer usability issues and proposed fixes. rejection of traditional keyboard navigation in favor of con- trols that more closely mirror the physical prototypeâs direct- nessâwhether through rotational input, positional sensing, or kinesthetic manipulation. Each proposal answers RQ3 by translating a specific dimension of the tactile probeâs embodied affordances into digitally augmented form. 6 DISCUSSION Moving beyond a simple summary of outcomes, this section reflects on the insights from our iterative process. We interpret our findings by connecting them to existing academic literature and systems, focusing on the conceptual challenges of knowledge transfer from physical to digital forms, the nuanced role of non-visual features in empowering users, and the emerging pathways toward more physically-grounded data interaction [22, 44, 67]. 6.1 Limitations of Our Study Our studyâs methodological scope introduces five primary limita- tions that constrain the generalizability of our findings. First, expert sample bias: our six BLV co-designers all possess advanced tech- nical literacy, extensive experience with assistive technologies, and professional backgrounds in data analysis or accessibility research. While this auto-ethnographic approach provided invaluable, high- fidelity insights grounded in lived expertise, features that were intu- itive for this group may not be learnable for novice users unfamiliar with data visualization or advanced assistive technologies [90,109]. Our prototype reflects design priorities and interaction patterns that emerged from expert users who bring sophisticated mental models and analytic strategies to 3D data exploration, which may not align with the needs, preferences, or learning trajectories of BLV individuals with less prior exposure to data visualization, scientific computing, or spatial reasoning tasks. Second, limited dataset types: our evaluation was conducted using relatively clean, dense datasets (benzene spectroscopy, Gaussian surfaces, weather data). The effectiveness of our sonification and navigation strategies has not been tested on sparse, disjointed, or noisy datasets, where features like peak-jumping or aggregated buffering might behave differently [64]. Third, cross-sectional study design: we captured co-designersâ immediate reactions during two sessions but have not assessed longitudinal effects of using the prototype over time, where issues like auditory fatigue, habituation, or the development of new usage patterns might emerge [23]. Fourth, volumetric audio subtlety: as identified in Session 2, volume alone proved insufficient for depth perception (Z-axis), requiring augmentation with reverbâa refinement not yet implemented in the validated prototype [93]. Fifth, buffer UX issues: the configurable buffer surfaced critical usability barriers documented in Table 6, including two-step selection initiation, lack of boundary awareness, unclear aggregation purpose, and absent confirmation feedback. While the buffer concept validated successfully, its interaction design requires substantial refinement before deployment with non-expert users. 6.2Conceptual Contributions Derived from the Co-Design Study Our Experience-Based Co-Design process with expert BLV collabo- rators yielded three interrelated conceptual contributions that ex- tend beyond the specific implementation of our system. These con- tributions address longstanding gaps in accessible 3D data visual- ization by centering lived experience as design material [62,70,82]. â˘We developed a transferable co-design protocol for trans- lating tactile expertise into multimodal digital inter- action. By pairing a low-fidelity tactile probe with a high- fidelity digital prototype, we established a methodological framework that systematically elicits tactile knowledge [99] and translates it into non-visual digital affordances. This pro- tocol operationalizes EBCD principles [82] for HCI contexts where physical and digital modalities must interoperate. The tactile probe served as an epistemological anchor, provid- ing a shared sensory baseline against which digital expe- riences could be evaluated. Co-designersâ narratives about orientation confusion, spatial comparison challenges, and analytic workflows directly motivated implemented features CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. (reference sonification, configurable buffer, spatial audio), demonstrating how experiential âtouch-pointsâ [30] can be systematically converted into design requirements. This pro- tocol is transferable: future projects targeting other 3D data types (e.g., molecular structures, terrain models) can adapt our phased approach; tactile grounding, comparative evalua- tion, iterative refinement, to surface domain-specific analytic needs. ⢠We produced a validated prototype enabling indepen- dent non-visual 3D data exploration. Our system demon- strates that web-native, multimodal interaction can sup- port core analytic tasks (3D orientation, gradient tracing, landmark identification, region comparison) without requir- ing specialized hardware. Session 2 validation confirmed that features like reference sonification, stereo panning, and buffer aggregation improved co-designersâ ability to orient, analyze, and interpret 3D surfaces with confidence. Unlike prior systems that rely on expensive tactile displays [58,97] or sighted-centered VR [44], our web-based approach offers scalable, low-cost access to continuous height-field surfaces across scientific domains. The prototypeâs modular archi- tecture enabled rapid iteration during co-design sessions, allowing us to implement complex features (e.g., buffer aggre- gation with dual playback modes) within days of co-designer feedback. This responsiveness exemplifies how technical in- frastructure can support rather than constrain participatory design [87]. ⢠We established empirically grounded design principles for accessible visualization. Our findings converge on four principles that extend existing accessible visualization guidance [22,67] into three-dimensional contexts: (1) Multi- parameter redundancy; (2) Progressive disclosure; (3) Buffer- aggregation as analytic scoping; and (4) Persistent spatial reference. These principles, validated through co-designer testimony and artifact evaluation, offer concrete guidance for designers extending accessible visualization beyond 2D charts into volumetric, spatially continuous datasets [23]. 6.3 System-based Design Insights Drawing from both our prototype implementation and co-designer validation, we distill two complementary sets of insights: bound- ary conditions specific to our systemâs current instantiation, and design principles applicable to future accessible 3D visualization sys- tems. This dual framing, requested by co-designers during Session 2 debriefing, clarifies which design decisions are implementation- dependent versus conceptually transferable [41]. 6.3.1 Boundary Conditions and Prototype Specifics. Our systemâs validated configuration represents a constellation of design deci- sions that emerged iteratively through co-designer feedback and artifact evaluation. We document these boundary conditions to distinguish implementation-specific choices from transferable prin- ciples, clarifying the scope within which our findings remain em- pirically grounded. â˘Binaural audio engine: Our browser-native implementa- tion is built entirely on Web Audio API primitives operating at 48kHz sample rate, deliberately avoiding external syn- thesis libraries to ensure zero-latency compatibility across devices. The audio signal chain progresses sequentially from oscillator generation through envelope shaping, stereo pan- ning, and convolution reverb, creating a modular architec- ture that our co-designers found responsive during real-time exploration. This architectural choice prioritized immediate auditory feedback over sophisticated synthesis capabilities, which was validated when co-designers consistently praised the systemâs responsiveness during navigation. ⢠Sonification mappings: Spatial coordinates translated into perceptible audio through four concurrent encodings that emerged directly from Session 1 deliberations. Following Azizâs suggestion, horizontal position (X-axis) maps to stereo panning ranging from fully left (-1.0) to fully right (+1.0), leveraging usersâ innate spatial hearing capabilities. Vertical position (Y-axis, representing surface height) maps to pitch via a logarithmic scale spanning 200Hz to 800Hz, ensuring that equal perceptual differences correspond to proportional height changes [39]. Depth perception (Z-axis) proved most complex, ultimately requiring JooYoungâs proposed multi- parameter encoding that combines dynamic volume with frequency-dependent reverb: closer points play at higher amplitude (20% wet mix) with minimal pre-delay (10ms) and preserved high frequencies (6500Hz lowpass), while distant points grow quieter (95% wet mix), gain substan- tial echo (90ms pre-delay), and lose treble clarity (2000Hz lowpass). Additionally, waveform timbre variesâsine waves for low-frequency regions, triangle waves for mid-range, square waves for high-frequency areasâto provide redun- dant height information through timbral brightness. â˘Configurable buffer mechanism: JooYoungâs Session 1 brainwriting proposal established rectangular region selec- tion through a three-stage keyboard interaction: D-key ini- tiates selection mode, Enter anchors the starting corner at the current cursor position, arrow keys navigate to define the opposite corner, and a second Enter press confirms the bounded region. The buffer stores all grid rectangles or in- dividual data points falling within these bounds, enabling subsequent analytic operations. When users activate aggre- gation mode via G-key, the system computes the arithmetic mean of normalized Y-values within the buffered region and plays a sustained 1.0-second tone representing this summary statistic. Alternatively, sequential playback mode traverses each buffered item individually at 0.3-second intervals with 125ms inter-item silence, allowing detailed inspection. These timing parameters emerged through Azizâs iterative test- ing during artifact evaluation, balancing playback duration against cognitive processing demands. â˘Dataset density constraints: Our validation focused exclu- sively on dense, continuous surface datasets where neighbor- ing points maintain close spatial proximity: benzene VUV spectroscopy data (3,116 measurement points), synthetic Gaussian surfaces (2,500 points), and meteorological height fields (1,200 points). These dataset constraints have direct implications: peak-jumping functionality reliably identifies local maxima when surfaces exhibit smooth gradients, but Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain its behavior remains undefined for sparse point clouds or disjoint data distributions with large spatial voids. Similarly, buffer aggregation produces meaningful summary statistics when regions contain sufficient data density to yield repre- sentative means, a condition not guaranteed for irregular sampling patterns. We explicitly acknowledge this limitation in Section 6.1, recognizing that alternative dataset types may require modified algorithms or additional safeguards. ⢠Two-tier navigation structure: Navigation defaults to a 20Ă20 wireframe grid overlay where each rectangular cell aggregates the underlying surface region it bounds, reducing the initial exploration space from potentially thousands of individual points to 400 manageable zones. Aziz specifically requested this âsurface modeâ during Phase 1 co-design to mitigate cognitive load when first encountering unfamiliar datasets, allowing users to grasp overall topography before diving into granular detail. Users can toggle to point mode via 2-key when finer-grained analysis becomes necessary, shifting from regional overviews to vertex-by-vertex tra- versal. This structure mirrors cartographic conventions of zooming from overview maps to detailed street views [85]. â˘Spatial orientation support: Two complementary mech- anisms that Ken and Sile advocated during Session 1 was about reference sonification. It provides an absolute coordi- nate anchor: pressing 0-key triggers a fixed 300Hz sine tone representing the dataset origin (0,0,0) regardless of current cursor position, allowing users to recalibrate their mental spatial model at any moment. Complementing this, the re- play mechanism (. key) re-sonifies the userâs current posi- tion without causing navigation, enabling repeated listen- ing when interpreting ambiguous or complex local features. These orientation aids address the fundamental challenge of maintaining spatial awareness in non-visual environments where users cannot glance at coordinate axes as sighted analysts routinely do. These specifications collectively define the empirical scope of our co-design validation. We present them not as prescriptive re- quirements but as documented implementation choices that proved effective with our expert co-designer cohort exploring dense sur- face datasets via keyboard interaction. Different user populations, alternative dataset types, or novel deployment contexts may neces- sitate adjusted parameter ranges, modified interaction patterns, or architectural alternatives. 6.3.2 Design Principles. Beyond our systemâs specific configura- tion, our co-design process surfaced four transferable principles applicable to accessible 3D visualization systems regardless of im- plementation platform. â˘Multi-parameter redundancy for spatial encoding: En- coding each spatial dimension through multiple reinforcing modalities (e.g., X-axis via stereo panning and verbal coor- dinate announcements; Z-axis via volume and reverb and textual depth cues) ensures robust perception across diverse sensory preferences and abilities. Co-designers with varying visual acuities reported differential reliance on auditory vs. textual feedback, validating the necessity of parallel chan- nels [39,46]. Source: Co-designer testimony (Session 2); Sile noted reliance on text, Venkatesh prioritized audio. â˘Progressive disclosure through whole-to-part naviga- tion: Scaffolding users from high-level overviews (auto-play across multiple perspectives) to granular detail (point-by- point traversal) supports pedagogical transitions from ini- tial orientation to hypothesis-driven analysis. This mirrors established accessible visualization patterns [89,104] but ex- tends them into volumetric contexts. The ability to quickly âsnapshotâ a datasetâs topography before diving into specific features reduces cognitive load and accelerates mental model formation. Source: Co-designer testimony (Phase 1); JooYoung identified tedium of point-by-point exploration. â˘Buffer-aggregation as analytic scoping tool: Enabling users to define spatial regions of interest and toggle between summary statistics (mean) and sequential detail empowers two critical workflows: (1) comparing local maxima to sur- rounding context, and (2) scoping natural language queries to specific subsets of data. This principle extends beyond sonifi- cation; any multimodal system benefits from user-controlled granularity adjustments. The emergent use case of buffer- scoped AI queries (Section 5.3) demonstrates how spatial selection mechanisms can integrate with conversational in- terfaces. Source: Co-designer testimony (Session 2); Venkatesh articulated comparison use case, emergent AI scoping observed during testing. â˘Pedagogical scaffolding for tactile-to-digital transition: Designing digital features to replicate the cognitive func- tions (not literal affordances) of physical probes facilitates knowledge transfer. Our tactile probeâs persistent frame of reference became digital reference sonification; its multi- finger comparison became configurable buffer playback. This principle aligns with mixed-reality prototyping best prac- tices [94,96] but specifies a pathway for non-visual contexts: deconstruct physical affordances into core cognitive opera- tions, then implement digital analogs serving those opera- tions. Source: Co-designer testimony (Session 1 comparative analysis); conceptual mapping documented in Section 5.2. The principles aforementioned, grounded in co-designer narra- tives and validated through artifact evaluation, provide actionable guidance for future accessible 3D visualization systems. Unlike the boundary conditions in Section 6.3.1, these insights transcend spe- cific implementation choices and apply across platforms, modalities, and dataset types. 6.4 Exploratory Insights from Co-Designer Speculations Session 2âs brainwriting exercise (Section 5.3.2) produced spec- ulative extensions of the validated web-based prototype toward more embodied forms of 3D data exploration. These concepts were not implemented or evaluated; instead, they reflect co-designersâ expert intuitions about how embodied cognition [60] and proprio- ception [8] could support non-visual analysis beyond screen-based interaction. We present these ideas as exploratory insights that CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. outline possible future directions rather than validated design out- comes. 6.4.1Advanced Haptics as Persistent Tactile Grounding. Co-designers consistently identified haptic feedback as the most promising mech- anism for restoring a sense of physical persistence to data explo- ration. Mechanisms to encode data density through force resistance, allowing users to âfeelâ aggregated regions as heavier or harder to press against; maps analytic properties (e.g., density, gradients) onto physical effort, providing continuous, user-controlled cues rather than transient auditory signals. Co-designers also proposed large, multi-line braille displays to enable parallel, multi-finger comparison across regions of a dataset. These ideas build on prior work in touch-based accessible graphics [12] and shape-changing interfaces [58], but emphasize haptics as an analytic encoding rather than a visual surrogate. Future work could prototype task- specific haptic mappings (e.g., force gradients for slope, vibration for density) and compare their learnability to sonification. Source: Co-designer speculation (Session 2 brainwriting). 6.4.2Embodied Spatial Exploration Beyond the Screen. Mixed and augmented reality were discussed only as background inspiration for extending interaction beyond the screen, rather than as imple- mentation targets. Co-designers emphasized the value of leveraging whole-body proprioception, such as walking or reaching, to support spatial reasoning in 3D data. Related suggestions focused on spatial- ized audio, with data points rendered as a 3D soundscape that users could explore by moving through space. These ideas align with prior work on immersive and embodied analytics [44,61], but were not central to the prototypeâs design or evaluation. Instead, they highlight longer-term opportunities for coupling proprioception and spatial audio to support non-visual mental models of complex data. Source: Co-designer speculation (Session 2 brainwriting). 6.4.3 Alternative Physical Interfaces Beyond Keyboard Navigation. Co-designers also critiqued the mismatch between discrete key- board input and continuous 3D data spaces [102]. Arrow-key navi- gation enforces sequential, axis-aligned movement, increasing cog- nitive load when exploring continuous surfaces. Across proposals (Section 6.4.3), a shared principle emerged: input modalities should match the dimensionality and continuity of the data. Future work could compare navigation efficiency, learnability, and analytic ac- curacy across keyboards, analog controllers, and haptic devices, particularly for users with diverse motor abilities and assistive technology experience. Source: Co-designer speculation (Session 2 brainwriting). 6.5 Future Directions Our future work will move from foundational design to rigorous validation and conceptual expansion. The immediate priority is to conduct formal user studies to validate the usability and learn- ability of our current feature set with a broader, non-expert BLV population. Following this, we will pursue a research agenda fo- cused on enhancing the userâs control and perception. This includes investigating alternative physical input devices, such as joysticks and trackballs, to afford more fluid, multi-dimensional navigation that better aligns with the exploration of 3D space. We will also refine the sonification engine, exploring more perceptually salient audio cues, such as reverb, to create a richer and more intuitive representation of depth. Finally, we will expand the prototypeâs analytical scope by implementing features for more complex data scenarios, such as the comparative plot view, to study how non- visual users perform analysis across layered datasets. We will also attempt to prototype a spatially adaptive interface to examine how proprioception can transform data exploration from a navigational task into an immersive experience. 7 CONCLUSION Through an experience-based co-design process with BLV collabo- rators with expertise in non-visual data representations, our aim was to create a digital experience that preserves spatial understand- ing and enables independent multiperspectival exploration that is scalable into multiple fields irrespective of data types. Our re- fined prototype demonstrated how sonification, spatialized audio, and adaptive aggregation can support orientation, accuracy, and learnability in surface and point plots. Beyond the artifact, our work contributes (1) a transferable co-design protocol for trans- lating tactile expertise into digital interaction, and (2) concrete guidance for designing accessible analytic workflows around volu- metric data. At the same time, our findings are bounded by a small expert user group, a limited set of datasets, and a cross-sectional evaluation. Future work should expand evaluation with a broader BLV population, investigate alternative input modalities for more fluid navigation, refine sonification with perceptually salient cues, and extend analytic scope to comparative and embodied interfaces. Taken together, these directions chart a concrete, evidence-based path toward equitable access to 3D scientific visualization. ACKNOWLEDGMENTS This work was supported in part by JooYoung Seoâs Faculty Startup Fund, which provided funding for project materials, including low- fidelity prototypes and shipping costs. We would also like to ex- tend our sincere gratitude to Tad Schroeder, an overall superhero, day-jobbing as the Assistant Director of Facilities at the School of Information Sciences, University of Illinois Urbana-Champaign for his invaluable support in coordinating the shipping logistics for our low-fidelity prototype. REFERENCES [1]Knight Lab Studio [n. d.]. Uncharted Territory: Diving in to Data Visualization in Virtual Reality. Knight Lab Studio.https://studio.knightlab.com/r esults/exploring-data-visualization-in-vr/uncharted-territory- datavis-vr/ [2]Ali Abdolrahmani, Kevin M. Storer, Antony Rishin Mukkath Roy, Ravi Kuber, and Stacy M. Branham. 2020. Blind Leading the Sighted: Drawing Design Insights from Blind Users towards More Productivity-oriented Voice Interfaces. ACM Transactions on Accessible Computing 12, 4 (Jan. 2020), 18:1â18:35.https: //doi.org/10.1145/3368426 [3]Nancy E Adams. 2015. Bloomâs taxonomy of cognitive learning objectives. Journal of the Medical Library Association : JMLA 103, 3 (July 2015), 152â153. https://doi.org/10.3163/1536-5050.103.3.010 [4]Dragan Ahmetovic, Cristian Bernareggi, JoĂŁo Guerreiro, Sergio Mascetti, and Anna Capietto. [n. d.]. AudioFunctions.Web: Multimodal Exploration of Math- ematical Function Graphs. In Proceedings of the 16th International Web for All Conference (New York, NY, USA, 2019-05-13) (W4A â19). Association for Com- puting Machinery, 1â10. https://doi.org/10.1145/3315002.3317560 [5]Dragan Ahmetovic, Cristian Bernareggi, JoĂŁo Guerreiro, Sergio Mascetti, and Anna Capietto. 2019. AudioFunctions.web: Multimodal Exploration of Math- ematical Function Graphs. In Proceedings of the 16th International Web for All Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain Conference (W4A â19). Association for Computing Machinery, New York, NY, USA, 1â10. https://doi.org/10.1145/3315002.3317560 [6]Safinah Ali, Laya Muralidharan, Felicia Alfieri, Monali Agrawal, and Jacob Jorgensen. 2020. Sonify: Making Visual Graphs Accessible. In Human Interaction and Emerging Technologies, Tareq Ahram, Redha Taiar, Serge Colson, and Arnaud Choplin (Eds.). Springer International Publishing, Cham, 454â459.https: //doi.org/10.1007/978-3-030-25629-6_70 [7]Ahmed Amer and Phillip Peralez. 2014. Affordable altered perspectives: Making augmented and virtual reality technology accessible. In IEEE Global Humanitar- ian Technology Conference (GHTC 2014). 603â608.https://doi.org/10.110 9/GHTC.2014.6970345 [8]Robert Ball and Chris North. 2007. Realizing embodied interaction for visual analytics through large displays. Computers & Graphics 31, 3 (June 2007), 380â 400. https://doi.org/10.1016/j.cag.2007.01.029 [9]Cynthia L. Bennett, Erin Brady, and Stacy M. Branham. 2018. Interdependence as a Frame for Assistive Technology Research and Design. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â18). Association for Computing Machinery, New York, NY, USA, 161â 173. https://doi.org/10.1145/3234695.3236348 [10]Tigmanshu Bhatnagar, Albert Higgins, Nicolai Marquardt, Mark Miodownik, and Catherine Holloway. 2023. Analysis of Product Architectures of Pin Array Technologies for Tactile Displays. Database of Pin Array Technologies included for the Analysis of Product Architectures of Pin Array Technologies for Tactile Displays 7, ISS (Nov. 2023), 432:135â432:155. https://doi.org/10.1145/3626468 [11]Ivy Bui, Arunabh Bhattacharya, Si Hui Wong, Harinder R. Singh, and Arpit Agarwal. 2021. Role of Three-Dimensional Visualization Modalities in Medical Education. Frontiers in Pediatrics 9 (Dec. 2021), 760363.https://doi.org/10 .3389/fped.2021.760363 [12] Matthew Butler, Leona M Holloway, Samuel Reinders, Cagatay Goncu, and Kim Marriott. [n. d.]. Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2021-05-07) (CHI â21). Association for Computing Machinery, 1â15.https: //doi.org/10.1145/3411764.3445207 [13] Matthew Butler, Leona M Holloway, Samuel Reinders, Cagatay Goncu, and Kim Marriott. 2021. Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI â21). Association for Computing Machinery, New York, NY, USA, 1â15.https://doi.org/10.114 5/3411764.3445207 [14] Francesco Cafaro. [n. d.]. Using Embodied Allegories to Design Gesture Suites for Human-Data Interaction. In Proceedings of the 2012 ACM Conference on Ubiq- uitous Computing (New York, NY, USA, 2012-09-05) (UbiComp â12). Association for Computing Machinery, 560â563.https://doi.org/10.1145/2370216. 2370309 [15] Jinho Choi, Sanghun Jung, Deok Gun Park, Jaegul Choo, and Niklas Elmqvist. 2019.Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization. Computer Graphics Forum 38, 3 (2019), 249â260.h t t p s : / / d o i . o r g / 1 0 . 1 1 1 1 / c g f . 1 3 6 8 6_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13686. [16] Pramod Chundury, Biswaksen Patnaik, Yasmin Reyazuddin, Christine Tang, Jonathan Lazar, and Niklas Elmqvist. 2022. Towards Understanding Sensory Substitution for Accessible Visualization: An Interview Study. IEEE transactions on visualization and computer graphics 28, 1 (Jan. 2022), 1084â1094.https: //doi.org/10.1109/TVCG.2021.3114829 [17] James M. Coughlan and Joshua Miele. 2017. AR4VI: AR as an Accessibility Tool for People with Visual Impairments. ... IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct). IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) 2017 (Oct. 2017), 288â292. https://doi.org/10.1109/ISMAR-Adjunct.2017.89 [18]Chris Creed, Maadh Al-Kalbani, Arthur Theil, Sayan Sarcar, and Ian Williams. 2024. Inclusive Augmented and Virtual Reality: A Research Agenda. In- ternational Journal of HumanâComputer Interaction 40, 20 (Oct. 2024), 6200â 6219.https://doi.org/10.1080/10447318.2023.2247614_eprint: https://doi.org/10.1080/10447318.2023.2247614. [19]Caitlin de Villiers. 2023. Embodied Knowledge in 4IR-Oriented Design Practice â Autoethnographic Approaches to Experiential Futuredirected Ways of Knowing and Learning in Selected Case Studies - ProQuest. Ph. D. Dissertation. University of Johannesburg, South Africa.https://w.proquest.com/docview/322 4565125 [20] Yehor Dzhurynskyi, Volodymyr Mayik, and Lyudmyla Mayik. 2024. Enhancing Accessibility: Automated Tactile Graphics Generation for Individuals with Visual Impairments. Computation 12, 12 (Dec. 2024), 251.https://doi.org/10.339 0/computation12120251 [21]Salla Eilola, Kaisa Jaalama, Petri Kangassalo, Pilvi Nummi, Aija Staffans, and Nora Fagerholm. 2023. 3D visualisations for communicative urban and landscape planning: What systematic mapping of academic literature can tell us of their potential? Landscape and Urban Planning 234 (June 2023), 104716.https: //doi.org/10.1016/j.landurbplan.2023.104716 [22]Frank Elavsky, Cynthia Bennett, and Dominik Moritz. 2022. How accessible is my visualization? Evaluating visualization accessibility with Chartability. Computer Graphics Forum 41, 3 (2022), 57â70.https://doi.org/10.1111/cgf.14522 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14522. [23]Kajetan Enge, Elias Elmquist, Valentina Caiola, Niklas RĂśnnberg, Alexander Rind, Michael Iber, Sara Lenzi, Fangfei Lan, Robert HĂśldrich, and Wolfgang Aigner. 2024. Open Your Ears and Take a Look: A State-of-the-Art Report on the Integration of Sonification and Visualization. Computer Graphics Forum 43, 3 (June 2024), e15114.https://doi.org/10.1111/cgf.15114arXiv:2402.16558 [cs]. [24]Danyang Fan, Alexa Fay Siu, Wing-Sum Adrienne Law, Raymond Ruihong Zhen, Sile OâModhrain, and Sean Follmer. 2022. Slide-Tone and Tilt-Tone: 1- DOF Haptic Techniques for Conveying Shape Characteristics of Graphs to Blind Users. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI â22). Association for Computing Machinery, New York, NY, USA, 1â19. https://doi.org/10.1145/3491102.3517790 [25] Danyang Fan, Olivia Tomassetti, Aya Mouallem, Gene S-H Kim, Shloke Nirav Pa- tel, Saehui Hwang, Patricia Leader, Danielle Sugrue, Tristen Chen, Darren Reese Ou, Victor R Lee, Lakshmi Balasubramanian, Hariharan Subramonyam, Sile OâModhrain, and Sean Follmer. 2025. Promoting Comprehension and Engage- ment in Introductory Data and Statistics for Blind and Low-Vision Students: A Co-Design Study. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI â25). Association for Computing Machinery, New York, NY, USA, 1â20. https://doi.org/10.1145/3706598.3713333 [26] Anis Farshian, Markus GĂśtz, Gabriele Cavallaro, Charlotte Debus, Matthias NieĂner, JĂłn Atli Benediktsson, and Achim Streit. 2023. Deep-Learning-Based 3-D Surface ReconstructionâA Survey. Proc. IEEE 111, 11 (Nov. 2023), 1464â1501. https://doi.org/10.1109/JPROC.2023.3321433 [27] Emilie Francis-Auton, Colleen Cheek, Elizabeth Austin, Natalia Ransolin, Lieke Richardson, Mariam Safi, Nematullah Hayba, Luke Testa, Reema Harrison, Jef- frey Braithwaite, and Robyn Clay-Williams. 2024. Exploring and Understanding the âExperienceâ in Experience-Based Codesign: A State-of-The-Art Review. International Journal of Qualitative Methods 23 (Jan. 2024), 16094069241235563. https://doi.org/10.1177/16094069241235563 [28]Beth Fylan, Justine Tomlinson, David K. Raynor, and Jonathan Silcock. 2021. Using experience-based co-design with patients, carers and healthcare pro- fessionals to develop theory-based interventions for safer medicines use. Re- search in Social and Administrative Pharmacy 17, 12 (Dec. 2021), 2127â2135. https://doi.org/10.1016/j.sapharm.2021.06.004 [29] David Geary, Jon Francombe, Kristian Hentschel, and Damian Murphy. 2022. Design Dimensions of Co-Located Multi-Device Audio Experiences. Applied Sciences 12, 15 (Jan. 2022), 7512. https://doi.org/10.3390/app12157512 [30] Nils Graber, Nina Canova, Denise Bryant-Lukosius, Glenn Robert, Blanca Navarro-Rodrigo, Lionel Trueb, George Coukos, Manuela Eicher, Tourane Corbière, and Sara Colomer-Lahiguera. 2024.Reflections on the oppor- tunities and challenges of applying experience-based co-design (EBCD) to phase 1 clinical trials in oncology. Health Expectations 27, 4 (2024), e14068.h t t p s : / / d o i . o r g / 1 0 . 1 1 1 1 / h e x . 1 4 0 6 8_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/hex.14068. [31] Theresa Green, Ann Bonner, Laisa Teleni, Natalie Bradford, Louise Purtell, Clint Douglas, Patsy Yates, Margaret MacAndrew, Hai Yen Dao, and Raymond Javan Chan. 2020. Use and reporting of experience-based codesign studies in the healthcare setting: a systematic review. BMJ Quality & Safety 29, 1 (Jan. 2020), 64â76. https://doi.org/10.1136/bmjqs-2019-009570 [32] Silvia Grimaldi, Steven Fokkinga, and Ioana Ocnarescu. 2013. Narratives in design: a study of the types, applications and functions of narratives in design practice. In Proceedings of the 6th International Conference on Designing Plea- surable Products and Interfaces. ACM, Newcastle upon Tyne United Kingdom, 201â210. https://doi.org/10.1145/2513506.2513528 [33]Paul Grimm, Wolfgang Broll, Rigo Herold, Johannes Hummel, and Rolf Kruse. 2022. VR/AR Input Devices and Tracking. In Virtual and Augmented Reality (VR/AR): Foundations and Methods of Extended Realities (XR), Ralf Doerner, Wolfgang Broll, Paul Grimm, and Bernhard Jung (Eds.). Springer International Publishing, Cham, 107â148.https://doi.org/10.1007/978-3-030-79062- 2_4 [34]N. C. Harte, D. Obrist, M. Versluis, E. Groot Jebbink, M. Caversaccio, W. Wimmer, and G. Lajoinie. 2024. Second order and transverse flow visualization through three-dimensional particle image velocimetry in millimetric ducts. Experimental Thermal and Fluid Science 159 (Dec. 2024), 111296.https://doi.org/10.101 6/j.expthermflusci.2024.111296 [35]Tingying He, Maggie McCracken, Daniel Hajas, Sarah Creem-Regehr, and Alexander Lex. 2025. Using Tactile Charts to Support Comprehension and Learning of Complex Visualizations for Blind and Low-Vision Individuals. https://doi.org/10.48550/arXiv.2507.21462 arXiv:2507.21462 [cs]. [36]Bridger Herman, Cullen D. Jackson, and Daniel F. Keefe. [n. d.]. Touching the Ground: Evaluating the Effectiveness of Data Physicalizations for Spatial Data Analysis Tasks. 31, 1 ([n. d.]), 875â885.https://doi.org/10.1109/TVCG.2 CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. 024.3456377 [37]Leona Holloway, Swamy Ananthanarayan, Matthew Butler, Madhuka Thisuri De Silva, Kirsten Ellis, Cagatay Goncu, Kate Stephens, and Kim Marriott. 2022. Animations at Your Fingertips: Using a Refreshable Tactile Display to Convey Motion Graphics for People who are Blind or have Low Vision. In Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â22). Association for Computing Machinery, New York, NY, USA, 1â16. https://doi.org/10.1145/3517428.3544797 [38] Leona M Holloway, Cagatay Goncu, Alon Ilsar, Matthew Butler, and Kim Mar- riott. 2022. Infosonics: Accessible Infographics for People who are Blind using Sonification and Voice. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI â22). Association for Computing Machinery, New York, NY, USA, 1â13. https://doi.org/10.1145/3491102.3517465 [39]Md Naimul Hoque, Md Ehtesham-Ul-Haque, Niklas Elmqvist, and Syed Masum Billah. 2023. Accessible Data Representation with Natural Sound. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI â23). Association for Computing Machinery, New York, NY, USA, 1â19.https: //doi.org/10.1145/3544548.3581087 [40] Stacy Hsueh, Beatrice Vincenzi, Akshata Murdeshwar, and Marianela Ciolfi Fe- lice. 2023. Cripping Data Visualizations: Crip Technoscience as a Critical Lens for Designing Digital Access. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â23). As- sociation for Computing Machinery, New York, NY, USA, 1â16.https: //doi.org/10.1145/3597638.3608427 [41]Janet Yi-Ching Huang, Stephan Wensveen, and Mathias Funk. 2023. Experiential speculation in vision-based AI design education: Designing conventional and progressive AI futures. (2023). https://doi.org/10.57698/V17I2.01 [42] Ying Ying Huang, Jonas Moll, Eva-Lotta Sallnäs, and Yngve Sundblad. 2012. Auditory feedback in haptic collaborative interfaces. International Journal of Human-Computer Studies 70, 4 (April 2012), 257â270.https://doi.org/10.1 016/j.ijhcs.2011.11.006 [43] Najwa Ayuni Jamaludin, Farhan Mohamed, Vei Siang Chan, Mohd Shahrizal Sunar, Ali Selamat, Ondrej Krejcar, and Andres Iglesias. [n. d.]. Answering Why and When?: A Systematic Literature Review of Application Scenarios and Evaluation for Immersive Data Visualization Analytics. 25, 1 ([n. d.]), 1â29. https://doi.org/10.4018/JCIT.323799 [44] Najwa Ayuni Jamaludin, Farhan Mohamed, Vei Siang Chan, Mohd Shahrizal Sunar, Ali Selamat, Ondrej Krejcar, and Andres Iglesias. 2023. Answering Why and When?: A Systematic Literature Review of Application Scenarios and Evaluation for Immersive Data Visualization Analytics. Journal of Cases on Information Technology (JCIT) 25, 1 (Jan. 2023), 1â29.https://doi.org/10.4 018/JCIT.323799 [45]Seungwoo Je, Hyunseung Lim, Kongpyung Moon, Shan-Yuan Teng, Jas Brooks, Pedro Lopes, and Andrea Bianchi. 2021. Elevate: A Walkable Pin-Array for Large Shape-Changing Terrains. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI â21). Association for Computing Machinery, New York, NY, USA, 1â11. https://doi.org/10.1145/3411764.3445454 [46]Chutian Jiang, Yinan Fan, Junan Xie, Emily Kuang, Kaihao Zhang, and Ming- ming Fan. 2024. Designing Unobtrusive Modulated Electrotactile Feedback on Fingertip Edge to Assist Blind and Low Vision (BLV) People in Comprehending Charts. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI â24). Association for Computing Machinery, New York, NY, USA, 1â20. https://doi.org/10.1145/3613904.3642546 [47] Elise Johnson, S. Sandra Bae, and Ellen Yi-Luen Do. [n. d.]. Supporting Data Visualization Literacy through Embodied Interactions. In Proceedings of the 15th Conference on Creativity and Cognition (New York, NY, USA, 2023-06-19) (C&C â23). Association for Computing Machinery, 346â348.https://doi.org/10.1 145/3591196.3596607 [48]Peter Jones. 2018. Contexts of Co-creation: Designing with System Stakeholders. In Systemic Design, Peter Jones and Kyoichi Kijima (Eds.). Vol. 8. Springer Japan, Tokyo, 3â52.https://doi.org/10.1007/978-4-431-55639-8_1Series Title: Translational Systems Sciences. [49]Shakila Cherise S Joyner, Amalia Riegelhuth, Kathleen Garrity, Yea-Seul Kim, and Nam Wook Kim. 2022. Visualization Accessibility in the Wild: Challenges Faced by Visualization Designers. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI â22). Association for Computing Machinery, New York, NY, USA, 1â19. https://doi.org/10.1145/3491102.3517630 [50]Sanchita S. Kamath, Aziz N. Zeidieh, and JooYoung Seo. 2025. Explore, Listen, Inspect: Supporting Multimodal Interaction with 3D Surface and Point Data Visu- alizations.https://doi.org/10.1145/3663547.3759765arXiv:2508.08554 [cs]. [51]Alexander S. Kaplitz and Kevin A. Schug. 2023.Gas chromatogra- phyâvacuum ultraviolet spectroscopy in petroleum and fuel analysis. Analytical Science Advances 4, 5-6 (2023), 220â231.h t t p s : / / d o i . o r g / 1 0 . 1 0 0 2 / a n s a . 2 0 2 3 0 0 0 25_eprint: https://chemistry- europe.onlinelibrary.wiley.com/doi/pdf/10.1002/ansa.202300025. [52]Tiffany Karalis Noel, Aiko Minematsu, and Nikki Bosca. 2023. Collective Autoethnography as a Transformative Narrative Methodology. International Journal of Qualitative Methods 22 (Oct. 2023), 16094069231203944.https: //doi.org/10.1177/16094069231203944 [53]Adnan Khan, Alireza Choubineh, Mai A. Shaaban, Abbas Akkasi, and Majid Komeili. 2025. TactileNet: Bridging the Accessibility Gap with AI-Generated Tactile Graphics for Individuals with Vision Impairment.https://doi.org/ 10.48550/arXiv.2504.04722 arXiv:2504.04722 [cs]. [54] Hyeok Kim, Yea-Seul Kim, and Jessica Hullman. 2024. Erie: A Declarative Grammar for Data Sonification. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1â19.https://doi.org/10.1145/3613904.36 42442 arXiv:2402.00156 [cs]. [55]N. W. Kim, S. C. Joyner, A. Riegelhuth, and Y. Kim. 2021. Accessible Visualiza- tion: Design Space, Opportunities, and Challenges. Computer Graphics Forum 40, 3 (2021), 173â188.https://doi.org/10.1111/cgf.14298_eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14298. [56]Bongshin Lee, Eun Kyoung Choe, Petra Isenberg, Kim Marriott, and John Stasko. 2020. Reaching Broader Audiences With Data Visualization. IEEE Computer Graphics and Applications 40, 2 (March 2020), 82â90.https://doi.org/10.1 109/MCG.2020.2968244 [57] Benjamin Lee, Xiaoyun Hu, Maxime Cordeil, Arnaud Prouzeau, Bernhard Jenny, and Tim Dwyer. [n. d.]. Shared Surfaces and Spaces: Collaborative Data Vi- sualisation in a Co-located Immersive Environment. 27, 2 ([n. d.]), 1171â1181. https://doi.org/10.1109/TVCG.2020.3030450 [58]Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2015. Shape Displays: Spatial Interaction with Dynamic Physical Form. IEEE Computer Graphics and Applications 35, 5 (Sept. 2015), 5â11.https://doi.org/10.110 9/MCG.2015.111 [59] Richard Qi Li. [n. d.]. Taoist Data Visualization: An Embodied Aesthetic Approach to Data Visualization through Gesture-Based Technology. 34, 1 ([n. d.]), 67â91.https://hta.ac/ojs/htr/article/view/taoist-data- visualization [60] Qiuyu Liu, Jingwen Pan, and Gang Li. 2020. Research of Immersive Geospa- tial Data Visualization based on Embodied Cognition. In 2020 International Conference on Innovation Design and Digital Technology (ICIDDT). 358â362. https://doi.org/10.1109/ICIDDT52279.2020.00071 [61] Richen Liu, Min Gao, Lijun Wang, Xiaohan Wang, Yuzhe Xiang, Aolin Zhang, Jiazhi Xia, Yi Chen, and Siming Chen. 2022. Interactive Extended Reality Tech- niques in Information Visualization. IEEE Transactions on Human-Machine Systems 52, 6 (Dec. 2022), 1338â1351.https://doi.org/10.1109/THMS.202 2.3211317 [62] Alan Lundgard, Crystal Lee, and Arvind Satyanarayan. 2019. Sociotechnical Considerations for Accessible Visualization Design.https://doi.org/10.4 8550/arXiv.1909.05118 arXiv:1909.05118 [cs]. [63]Alan Lundgard and Arvind Satyanarayan. 2021. Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content.ht tps://doi.org/10.1109/TVCG.2021.3114770/ arXiv:2110.04406 [cs]. [64] Andrei LÄpus , teanu, Anca Morar, Alin Moldoveanu, Maria-Anca BÄlut , oiu, and Florica Moldoveanu. 2024. A review of sonification solutions in assistive systems for visually impaired people. Disability and Rehabilitation. Assistive Technology 19, 8 (Nov. 2024), 2818â2833.https://doi.org/10.1080/17483107.2024. 2326590 [65] Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, and Leah Findlater. 2021. What Do We Mean by âAccessibility Researchâ? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI â21). Association for Computing Machinery, New York, NY, USA, Article 371, 18 pages.https://doi.org/10.1145/3411 764.3445412 [66]Alexander Marquardt. 2023. Multisensory Guidance Under Sensory Constraints in Augmented Reality. Ph. D. Dissertation. Universitaet Bremen, Germany.https: //w.proquest.com/docview/3226986651?pq-origsite=gscholar&from openview=true&sourcetype=Dissertations%20&%20Theses [67]Kim Marriott, Bongshin Lee, Matthew Butler, Ed Cutrell, Kirsten Ellis, Cagatay Goncu, Marti Hearst, Kathleen McCoy, and Danielle Albers Szafir. 2021. Inclusive data visualization for people with disabilities: a call to action. interactions 28, 3 (April 2021), 47â51. https://doi.org/10.1145/3457875 [68]Roberto Martinez-Maldonado, Abelardo Pardo, Negin Mirriahi, Kalina Yacef, Judy Kay, and Andrew Clayphan. 2015. LATUX: an iterative workflow for designing, validating, and deploying learning analytics visualizations. Journal of Learning Analytics 2, 3 (2015), 9â39.https://doi.org/10.18608/jla.2 015.23.3 [69] Mille Nabsen Marwaa, Susanne Guidetti, Charlotte Ytterberg, and Hanne Kaae Kristensen. 2023. Using experience-based co-design to develop mobile/tablet applications to support a person-centred and empowering stroke rehabilitation. Research Involvement and Engagement 9, 1 (Aug. 2023), 69.https://doi.org/ 10.1186/s40900-023-00472-z [70]Claire Morley, Kim Jose, Sonj E. Hall, Kelly Shaw, Deirdre McGowan, Martina Wyss, and Tania Winzenberg. 2024. Evidence-informed, experience-based co- design: a novel framework integrating research evidence and lived experience Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain in priority-setting and co-design of health services. BMJ Open 14, 8 (Aug. 2024), e084620. https://doi.org/10.1136/bmjopen-2024-084620 [71]Claire Morley, Kim Jose, Sonj E Hall, Kelly Shaw, Deirdre McGowan, Mar- tina Wyss, and Tania Winzenberg. 2024. Evidence-informed, experience- based co-design: a novel framework integrating research evidence and lived experience in priority-setting and co-design of health services. BMJ Open 14, 8 (2024).https://doi.org/10.1136/bmjopen- 2024- 084620 arXiv:https://bmjopen.bmj.com/content/14/8/e084620.full.pdf [72] Omar Moured, Sara Alzalabny, Thorsten Schwarz, Bastian Rapp, and Rainer Stiefelhagen. 2023. Accessible Document Layout: An Interface for 2D Tac- tile Displays. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA â23). Association for Com- puting Machinery, New York, NY, USA, 265â271.https://doi.org/10.114 5/3594806.3594811 [73]Mukhriddin Mukhiddinov and Soon-Young Kim. 2021. A Systematic Literature Review on the Automatic Creation of Tactile Graphics for the Blind and Visually Impaired. Processes 9, 10 (Oct. 2021), 1726.https://doi.org/10.3390/pr91 01726 [74] Nicola OâBrien, Ben Heaven, Gemma Teal, Elizabeth H Evans, Claire Cleland, Suzanne Moffatt, Falko F Sniehotta, Martin White, John C Mathers, and Paula Moynihan. 2016. Integrating Evidence From Systematic Reviews, Qualitative Research, and Expert Knowledge Using Co-Design Techniques to Develop a Web-Based Intervention for People in the Retirement Transition. Journal of Medical Internet Research 18, 8 (Aug. 2016), e210.https://doi.org/10.219 6/jmir.5790 [75] Zoe J. Oliver, Filipe Cristino, Mark V. Roberts, Alan J. Pegna, and E. Charles Leek. 2018. Stereo Viewing Modulates Three-Dimensional Shape Processing During Object Recognition: A High-Density ERP Study. Journal of Experimental Psychology. Human Perception and Performance 44, 4 (April 2018), 518â534. https://doi.org/10.1037/xhp0000444 [76]Arnaud Prouzeau, Maxime Cordeil, Clement Robin, Barrett Ens, Bruce H. Thomas, and Tim Dwyer. [n. d.]. Scaptics and Highlight-Planes: Immersive Interaction Techniques for Finding Occluded Features in 3D Scatterplots. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow Scotland Uk, 2019-05-02). ACM, 1â12.https://doi.org/10.1145/ 3290605.3300555 [77] Graham Pullan. [n. d.]. Visualization of Large Datasets of Simulations - An Infinite Canvas Approach. In AIAA SCITECH 2025 Forum. American Institute of Aeronautics and Astronautics.https://arc.aiaa.org/doi/abs/10.2514/6 .2025-1576 [78] Min Qi, Bao-lin Zhang, Guang-he Liang, Jie Wang, and Xin-ping Cai. 2007. 3D Modeling and Visualization of Geology Volume based on Geophysical Field Data. Data Science Journal 6, 0 (Oct. 2007).https://doi.org/10.2481/dsj.6.S652 [79] Lauren Race, Chancey Fleet, Danielle Montour, Lindsay Yazzolino, Marco Sals- iccia, Claire Ferrari, Sheri Wells-Jensen, and Amy Hurst. 2023. Designing While Blind: Nonvisual Tools and Inclusive Workflows for Tactile Graphic Creation. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â23). Association for Computing Machinery, New York, NY, USA, 1â8. https://doi.org/10.1145/3597638.3614546 [80]Agis Abhi Rafdhi. 2024. Integrating Web Technologies with Augmented Reality. International Journal of Research and Applied Technology (INJURATECH) 4, 2 (Dec. 2024), 239â252.https://ojs.unikom.ac.id/index.php/injuratec h/article/view/16431 [81] M. Rautenhaus, M. Kern, A. Schäfler, and R. Westermann. 2015.Three- dimensional visualization of ensemble weather forecasts â Part 1: The visualiza- tion tool Met.3D (version 1.0). Geoscientific Model Development 8, 7 (July 2015), 2329â2353. https://doi.org/10.5194/gmd-8-2329-2015 [82]David K. Raynor, Hanif Ismail, Alison Blenkinsopp, Beth Fylan, Gerry Armitage, and Jonathan Silcock. 2020. Experience-based co-designâAdapting the method for a researcher-initiated study in a multi-site setting. Health Expectations : An International Journal of Public Participation in Health Care and Health Policy 23, 3 (June 2020), 562â570. https://doi.org/10.1111/hex.13028 [83]Samuel Reinders, Matthew Butler, Ingrid Zukerman, Bongshin Lee, Lizhen Qu, and Kim Marriott. 2025. When Refreshable Tactile Displays Meet Conversational Agents: Investigating Accessible Data Presentation and Analysis with Touch and Speech. IEEE Transactions on Visualization and Computer Graphics 31, 1 (Jan. 2025), 864â874.https://doi.org/10.1109/TVCG.2024.3456358 arXiv:2408.04806 [cs]. [84] David Rojas, Bill Kapralos, Andrew Hogue, Karen Collins, Lennart Nacke, Sayra Cristancho, Cristina Conati, and Adam Dubrowski. 2013. The effect of sound on visual fidelity perception in stereoscopic 3-D. IEEE transactions on cybernetics 43, 6 (Dec. 2013), 1572â1583. https://doi.org/10.1109/TCYB.2013.2269712 [85] Robert E. Roth. 2021. Cartographic Design as Visual Storytelling: Synthesis and Review of Map-Based Narratives, Genres, and Tropes. The Cartographic Journal 58, 1 (Jan. 2021), 83â114.https://doi.org/10.1080/00087041.2019.1633 103 _eprint: https://doi.org/10.1080/00087041.2019.1633103. [86]Mustafa Saifee. [n. d.]. VR-Viz: Visualization System for Data Visualization in VR. ([n. d.]). https://aaltodoc.aalto.fi/handle/123456789/36113 [87]Elizabeth B.-N. Sanders and Pieter Jan Stappers. 2008.Co-creation and the new landscapes of design. CoDesign 4, 1 (March 2008), 5â18.ht t p s : / / d o i . o r g / 1 0 . 1 0 8 0 / 1 5 7 1 0 8 8 0 7 0 1 8 7 5 0 6 8_eprint: https://doi.org/10.1080/15710880701875068. [88]JooYoung Seo, Sanchita S. Kamath, Aziz Zeidieh, Saairam Venkatesh, and Sean McCurry. 2024. MAIDR Meets AI: Exploring Multimodal LLM-Based Data Visu- alization Interpretation by and with Blind and Low-Vision Users. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessi- bility (ASSETS â24). Association for Computing Machinery, New York, NY, USA, 1â31. https://doi.org/10.1145/3663548.3675660 [89]JooYoung Seo, Yilin Xia, Bongshin Lee, Sean Mccurry, and Yu Jun Yam. 2024. MAIDR: Making Statistical Visualizations Accessible with Multimodal Data Representation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI â24). Association for Computing Machinery, New York, NY, USA, 1â22. https://doi.org/10.1145/3613904.3642730 [90]Ather Sharif, Sanjana Shivani Chintalapati, Jacob O. Wobbrock, and Katharina Reinecke. 2021. Understanding Screen-Reader Usersâ Experiences with Online Data Visualizations. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â21). Association for Com- puting Machinery, New York, NY, USA, 1â16.https://doi.org/10.1145/34 41852.3471202 [91] Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, and Ja- cob O. Wobbrock. 2022. VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI â22). Association for Computing Machinery, New York, NY, USA, 1â19.https://doi.org/10.1145/3491102. 3517431 [92] Mark Simpson, Jiayan Zhao, and Alexander Klippel. [n. d.]. Take a Walk: Eval- uating Movement Types for Data Visualization in Immersive Virtual Reality. ([n. d.]). [93]Alexa F. Siu, Mike Sinclair, Robert Kovacs, Eyal Ofek, Christian Holz, and Edward Cutrell. 2020. Virtual Reality Without Vision: A Haptic and Auditory White Cane to Navigate Complex Virtual Worlds. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI â20). Association for Computing Machinery, New York, NY, USA, 1â13.https://doi.org/10.114 5/3313831.3376353 [94] Chris Snider, Aman Kukreja, Christopher Michael Jason Cox, James Gopsill, and Lee Kent. 2024. Mixed reality prototyping: a framework to characterise simultaneous physical/virtual prototyping. Proceedings of the Design Society 4 (May 2024), 775â784. https://doi.org/10.1017/pds.2024.80 [95] Hyemi Song, Sai Gopinath, and Zhicheng Liu. 2025. Intents, Techniques, and Components: a Unified Analysis of Interaction Authoring Tasks in Data Visual- ization.https://doi.org/10.48550/arXiv.2409.01399arXiv:2409.01399 [cs]. [96]Maximilian Speicher, Katy Lewis, and Michael Nebeling. 2021. Designers, the Stage Is Yours! Medium-Fidelity Prototyping of Augmented & Virtual Reality Interfaces with 360theater. Proc. ACM Hum.-Comput. Interact. 5, EICS (May 2021), 205:1â205:25. https://doi.org/10.1145/3461727 [97] Ryo Suzuki, Abigale Stangl, Mark D. Gross, and Tom Yeh. 2017. FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile Markers.https://doi.or g/10.48550/arXiv.1708.03783 arXiv:1708.03783 [cs]. [98] Anders Høyland Syvertsen. 2022. Tangible Scalar Fields. Masterâs thesis. The University of Bergen.https://bora.uib.no/bora-xmlui/handle/11250/3 004277 Accepted: 2022-07-08T23:39:01Z. [99] Andrew Vande Moere and Stephanie Patel. [n. d.]. The Physical Visualization of Information: Designing Data Sculptures in an Educational Context. In Visual Information Communication (Boston, MA, 2010), Mao Lin Huang, Quang Vinh Nguyen, and Kang Zhang (Eds.). Springer US, 1â23.https://doi.org/10.1 007/978-1-4419-0312-9_1 [100]J. A. Wagner Filho, C.m.d.s. Freitas, and L. Nedel. [n. d.]. VirtualDesk: A Com- fortable and Efficient Immersive Information Visualization Approach. 37, 3 ([n. d.]), 415â426. https://doi.org/10.1111/cgf.13430 [101]Qiming Wang and Bonita Saunders. [n. d.]. Web-Based 3D Visualization in a Digital Library of Mathematical Functions. In Proceedings of the Tenth In- ternational Conference on 3D Web Technology (New York, NY, USA, 2005-03- 29) (Web3D â05). Association for Computing Machinery, 151â157.https: //doi.org/10.1145/1050491.1050513 [102]Qiming Wang and Bonita Saunders. 2005. Web-based 3D visualization in a digital library of mathematical functions. In Proceedings of the tenth international confer- ence on 3D Web technology (Web3D â05). Association for Computing Machinery, New York, NY, USA, 151â157.https://doi.org/10.1145/1050491.1050513 [103]Xiyao Wang, Lonni Besançon, Mehdi Ammi, and Tobias Isenberg. [n. d.]. Understanding Differences between Combinations of 2D and 3D Input and Output Devices for 3D Data Visualization. 163 ([n. d.]), 102820.https: //doi.org/10.1016/j.ijhcs.2022.102820 [104]Zhuohao Zhang, John R Thompson, Aditi Shah, Manish Agrawal, Alper Sarikaya, Jacob O. Wobbrock, Edward Cutrell, and Bongshin Lee. 2024. ChartA11y: De- signing Accessible Touch Experiences of Visualizations with Blind Smartphone CHI â26, April 13â17, 2026, Barcelona, SpainKamath et al. Users. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â24). Association for Computing Machinery, New York, NY, USA, 1â15. https://doi.org/10.1145/3663548.3675611 [105] Zhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, and Liang He. 2025. A11yShape: AI-Assisted 3-D Modeling for Blind and Low-Vision Programmers. In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS â25). Association for Computing Machinery, New York, NY, USA, 1â20.https://doi.org/10 .1145/3663547.3746362 [106]Jack Zhao and Andrew Vande Moere. 2008. Embodiment in data sculpture: a model of the physical visualization of information. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts (DIMEA â08). Association for Computing Machinery, New York, NY, USA, 343â 350. https://doi.org/10.1145/1413634.1413696 [107]Kaixing Zhao, Sandra Bardot, Marcos Serrano, Mathieu Simonnet, Bernard Ori- ola, and Christophe Jouffrais. 2021. Tactile Fixations: A Behavioral Marker on How People with Visual Impairments Explore Raised-line Graphics. In Pro- ceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI â21). Association for Computing Machinery, New York, NY, USA, 1â12. https://doi.org/10.1145/3411764.3445578 [108] Liang Zhou, Mengjie Fan, Charles Hansen, Chris R. Johnson, and Daniel Weiskopf. 2022. A Review of Three-Dimensional Medical Image Visualiza- tion. Health Data Science 2022 (April 2022), 9840519.https://doi.org/10.3 4133/2022/9840519 [109] Jonathan Zong, Crystal Lee, Alan Lundgard, JiWoong Jang, Daniel Hajas, and Arvind Satyanarayan. 2022. Rich Screen Reader Experiences for Accessible Data Visualization.https://doi.org/10.48550/arXiv.2205.04917 arXiv:2205.04917 [cs]. [110] Jonathan Zong, Isabella Pedraza Pineros, Mengzhu (Katie) Chen, Daniel Hajas, and Arvind Satyanarayan. 2024. Umwelt: Accessible Structured Editing of Multi- Modal Data Representations. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI â24). Association for Computing Machinery, New York, NY, USA, 1â20. https://doi.org/10.1145/3613904.3641996 A BRAINWRITING SESSION PROMPTS AND TASK-BASED QUESTIONS A.1 Session 1: Brainwriting Prompts (1) Which elements of the tactile experience feel essential for understanding the surface? This question was designed to focus the team on the core affordances of the physical prototype, identifying what made it effective for building a spatial mental model. (2)How might these translate into a digital interface? This prompts a direct translation of the previously identified es- sential traces of the physical prototype into potential digital features, bridging the gap between the physical and virtual. (3)What features or feedback would help reproduce the tactile clarity in a web interface?The aim was to focus on the specific quality of âclarityâ, encouraging co-designers to think about the fidelity and type of feedback needed to make the digital experience as unambiguous as the physical one. (4)What risks do we face if these features arenât included? This prompt helped us consider what would be lost in a poor translation. Our aim was to brainstorm the most critical design requirements by forcing a consideration of failure modes. A.2 Session 1: Extended Task-Based Questions Throughout Session 1âs probe exploration and brainwriting exer- cise, co-designers engaged with the following task-based questions, which were presented conversationally to elicit their exploration strategies, confusions, and insights: 3D Orientation Tasks. â˘How do you know where you are in the plot when using the tactile probe versus the digital prototype? ⢠What cues help you understand the axes in each version? ⢠When you lose your orientation, what helps you recover your bearings? â˘How would you differentiate between the X, Y, and Z axes nonvisually? Landmark and Peak Finding Tasks. ⢠How would you locate the highest point on the surface using the tactile probe? ⢠How would you match this and locate the highest point using the digital prototype? ⢠Can you find where the data changes most dramatically? What strategies do you use? ⢠What features or feedback would help you quickly jump to salient points of interest? Comparing Local Maxima versus Global Trends. ⢠How do you differentiate a single peak from an overall rising trend in the tactile probe? â˘What would help you compare multiple high points in the digital interface (key-binding or continuous sonification)? ⢠When exploring the surface, how do you maintain awareness of the overall shape while examining local features? â˘What feedback mechanisms would support comparison be- tween different regions? Gradient Tracing Tasks. â˘Can you follow the path of steepest ascent on the physical probe? How? â˘How would you trace a ridge or valley in the digital version? â˘What auditory or tactile cues would best indicate the direc- tion and steepness of slopes? â˘How do you distinguish between gradual slopes and sharp cliffs? Identifying Occluded or Partially Hidden Features (for those with Residual Vision). â˘How do you explore areas that might be hidden from one perspective on the tactile probe? â˘What strategies help you uncover features that arenât imme- diately apparent in the digital interface? â˘How would you know if there are features "behind" or "un- derneath" other parts of the surface? ⢠What feedback would indicate overlapping or complex spa- tial relationships? A.3 Session 2: Brainwriting Prompts (1) Which tools, techniques, or mediums can help preserve embodied cognition digitally? This question was intention- ally broad, using the term âembodied cognitionâ to encourage the team to think beyond simple keyboard interactions and consider how physical movement and presence could be integrated. (2)How can sound, text or haptics replicate the spatial clar- ity of the physical prototype - shifting from a web-based prototype to a spatially adaptive interface to support Experience-Based Co-Design of a Multi-modal 3D Data Visualization ToolCHI â26, April 13â17, 2026, Barcelona, Spain embodied cognition? This prompt focuses on the specific modalities (sound, text, haptics) and the goal of creating a âspatially adaptive interfaceâ, pushing the team to ideate on how the system could respond to a userâs physical context. (3) Whatâs the most feasible and understandable step to move toward these interfaces? After the expansive ideation prompted by the first two questions, the idea was to ground the discussion in pragmatism, asking the team to identify a concrete, actionable first step toward their more ambitious, long-term visions. A.4 Session 2: Extended Task-Based Questions During Session 2âs feature validation phase, co-designers were asked to evaluate the newly implemented features through the following task-based questions: Reference Sonification and Orientation (DG3). ⢠Does the fixed origin sound help you maintain orientation during exploration? â˘How useful is the replay mechanism when you become dis- oriented? â˘Can you use the reference point to navigate back to previ- ously explored regions? â˘What additional reference points or landmarks would be helpful? Spatial Audio for Position and Depth (DG4). â˘How effectively does stereo panning convey horizontal posi- tion (X-axis)? â˘Can you perceive depth (Z-axis) through volume changes alone? â˘What makes spatial audio cues more or less salient during exploration? â˘How would additional audio cues (e.g., reverb, echo) enhance depth perception? Configurable Buffer for Comparison (DG5). ⢠How intuitive is the buffer selection process? â˘Does the aggregated playback help you understand regional patterns? â˘Can you effectively compare a saved buffer region to your current focus point? â˘What improvements would make the buffer more useful for analytical tasks? â˘How might the buffer support focused queries to the AI assistant? Multi-Perspective Auto-Play and Jump-to-Peak (DG1, DG2). ⢠How do the auto-play overviews help you build an initial mental model? â˘Does jump-to-peak functionality help you efficiently locate salient features? â˘What additional automated exploration strategies would be valuable? ⢠How do these features balance efficiency with user control? Future Embodied Interface Concepts. â˘What physical input devices would best support 3D naviga- tion? ⢠How could haptic feedback enhance your understanding of data density or slope? ⢠In a mixed reality environment, how would you use your body to explore the data space? ⢠Whatâs the most important first step toward creating a spa- tially adaptive interface? B AI CHAT ASSISTANT IMPLEMENTATION The multi-threaded AI chat assistant (viewable in Figure 5) lever- ages Gemini 2.5 Pro to support interactive data exploration. Since the WebGL plot supports cursor-based interactivity, a screenshot of the userâs current view was passed to the model alongside the raw dataset used by the VisualizationEngine.js from the Engine Layer. This allows the model to provide contextually relevant explana- tions or respond to targeted questions about specific data points and regions. To ensure robustness, a system of static responses was also implemented, enabling the assistant to provide fallback answers based on pre-computed dataset statistics if the live model connection fails. For users with access to the system repository, if they chose to run it locally rather than through the web, the tool supported integration with Ollama, allowing queries to be directed to a userâs local language model, including fine-tuned versions. The assistant was designed to automatically detect the most recent available model and establish a connection to facilitate seamless responses. Additionally, four example prompts were provided to guide users in framing queries to the assistant. Figure 5: AI Chat Interface