From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains

From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains XIAOHAN PENG, LISN Université Paris-Saclay, CNRS, Inria, France SOTIRIS PILIOURAS, LISN Université Paris-Saclay, CNRS, Inria, France CARL ABOU SAADA NUJAIM, LISN Université Paris-Saclay, CNRS, Inria, France Analyzing creative activity traces requires capturing activity at appropriate granularity and interpreting it in ways that reflect the structure of creative practice. However, existing approaches record state changes without preserving the intent or relationships that define higher-level creative moves. This decoupling manifests differently across domains: GenAI tools lose non-linear exploration structure, visualization authoring obscures representational intent, and programmatic environments flatten interaction boundaries. We present three complementary approaches: a node-based interface for stateful GenAI artifact management, a vocabulary of visual cues as higher-level creative moves in visualization authoring, and a programming model that embeds semantic histories directly into interaction state. CCS Concepts:• Human-centered computing→Interactive systems and tools;• Applied computing→Arts and humanities. Additional Key Words and Phrases: Creativity Support Tools, Human-AI Interaction, Design Practice, Visualization Authoring, Version Control, History Management, Collaborative Editing ACM Reference Format: Xiaohan Peng, Sotiris Piliouras, and Carl Abou Saada Nujaim. 2026. From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains. In Herding CATs: Making Sense of Creative Activity Traces Workshop of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain. ACM, New York, NY, USA, 5 pages. 1 Introduction Analyzing creative activity traces requires capturing activity at appropriate levels of granularity and interpreting them in ways that reflect the structure of creative practice. Existing approaches such as version histories, interaction logs, or collaborative records tend to record state changes without preserving the higher-level intent or semantic context that give those changes meaning. Traces thus remain structurally decoupled from the creative decisions they result from across creative domains and tool paradigms: losing non-linear exploration structure in GenAI tools, Authors’ Contact Information: Xiaohan Peng, LISN Université Paris-Saclay, CNRS, Inria , Orsay, France, xiaohan.peng@inria.fr; Sotiris Piliouras, LISN Université Paris-Saclay, CNRS, Inria , Orsay, France, sotirios.piliouras@inria.fr; Carl Abou Saada Nujaim, LISN Université Paris-Saclay, CNRS, Inria, Paris, France, carl@lisn.fr. This work is licensed under a Creative Commons Attribution 4.0 International License. © 2026 Copyright held by the owner/author(s). Manuscript submitted to ACM Manuscript submitted to ACM1 arXiv:2603.07184v1 [cs.HC] 7 Mar 2026 2Xiaohan Peng, Sotiris Piliouras, and Carl Abou Saada Nujaim obscuring representational intent in visualization authoring, and reducing meaningful actions to low-level operations in programming environments. In this paper, we present a multi-faceted approach to documenting and interpreting creative traces in GenAI, visualization, and programming contexts. First, we introduce a node-based interface design that treats generative artifacts as stateful, manipulable units, enabling designers to preserve alternative versions, branch explorations, and link semantic attributes to visual content. Second, we examine how 3D authoring traces such as material appearance, perspective, deformation, and dynamics can be mapped to higher-level “creative moves” that reflect representation and framing choices in expressive 3D visualizations. Finally, we propose a programming model that embeds semantic histories directly into interaction state, supporting both single-user and collaborative environments. 2 Documenting GenAI Creative Traces: Tracking Interactive Generative Visual Artifacts across Scales While prior creative history management systems introduce git-style version control, semantic grouping [2], and node-based workflows [1], they are primarily oriented toward capturing discrete, linear iterations and state transitions as the system evolves [11], or often focus on relatively simple graphic elements [6] rather than higher-abstraction, context-dependent and hierarchically structured design assets. We argue for more visual, dynamically applicable and retrievable, and non-linear histories tailored to iterative, GenAI-supported design, particularly as GenAI images become a new class of design assets. Fig. 1. DesignTrace: a GenAI design tool for exploring alternatives and tracking design progress. Designers can extract or create semantic attributes then apply them to visuals, branch multiple alternatives under shared semantic categories, make localized edits inside individual canvas node while navigating editing history and different versions. 2.1 Stateful Nodes for Dynamic Visual Interaction and Documentation Existing node-based systems typically treat nodes as parametric operators in a data-flow pipeline. However, in Deisng- Trace [4], we explored nodes as interactive and stateful artifacts placed on a freeform canvas, each containing an editable image canvas and a local history of generation and editing actions. They can be independently resized, collapsed, duplicated, and spatially arranged to externalize design states as discrete, manipulable units. This structure enables Manuscript submitted to ACM From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains3 designers to keep alternatives visible, compare them side by side, branch manually or intelligently with semantics, and reuse prior states without clutter or overlap. By linking semantic attributes to visual content and preserving branching histories, Nodes support non-linear exploration while documenting the evolving rationale behind design decisions. 2.2 Multi-Scale GenAI Traces Combine Commands, Artifact States, and Exploration Paths Current chat-based GenAI tools’ logs can grow long easily since they weigh minor edits and major design shifts equally, while canvas-based ones can get cluttered without proper design on the prompt-image relationship. DesignTrace supports multi-scale trace management. Each node maintains a local history of edits, while persistent, reusable nodes represent intermediate states that can be spatially arranged and revisited. By connecting these nodes into single or multiple exploration paths, the system provides an overview of design progression while preserving flexibility for branching and reuse. This structure enables designers to trace minor temporal-proximal changes, but also trace “design versions,” while keeping a broader overview of different exploration paths. 3 Visualizing Creative Traces: Representation and Framing Choices in Expressive 3D Visualization Authoring 3D creators iterate through intertwined decisions about geometry, materials, lighting, camera/viewpoint, and motion expressive in 3D visualization authoring. Yet creative activity traces are typically captured as sequences of parametric edits but not the higher-level representational intent behind those creative choices. Seemingly small edits can correspond to high-level representation and framing choices with substantial novelty, such as scattering objects in 3D space to communicate accumulation of microplastics in the ocean [8] or using particle motion simulations to convey air pollution [7]. Reconstructing creative intent from parametric edits is challenging because most 3D software is representation- agnostic, and logs lack the semantic and representational structure needed to relate actions to visualization goals, especially for unconventional representations. We propose that interpreting creativity traces in expressive 3D visualiza- tion can benefit from an intermediate vocabulary that translates tool actions into representation-level “creative moves”. Our ongoing work on physically-inspired visualization [5] provides a toolkit for this translation by describing (i) the visual elements that compose a scene, (i) the physically-inspired properties being manipulated, and (i) their semantic relationship to the data variables and theme (such as literal, iconic, or symbolic relationships). In this workshop, we want to explore how this structured design space of material-based visual representation choices can be used to make sense of creative activity traces in expressive 3D visualization authoring, supporting the analysis of 3D exploration trajectories and informing future creativity-support tools. 4 Programming Creative Traces: Embedding Semantic Histories in Interaction State Digital creative activity traces—ranging from changing the color of a shape to generating a new design alternative in a GenAI tool—remain difficult to analyze because they are typically captured as external logs rather than produced by the system’s interaction model. Even in collaborative environments based on Conflict-Free Replicated Data Types [9] (CRDTs), histories record operational changes but not the structure of user activity, requiring developers to manually reconstruct intent by grouping operations into higher-level transactions. Inspired by Object-Oriented Drawing [10], we introduce a model for embedding histories directly in application state and take a first step toward representing semantics in both single-user and collaborative environments. Manuscript submitted to ACM 4Xiaohan Peng, Sotiris Piliouras, and Carl Abou Saada Nujaim Fig. 2. (1) A reactive signal records value changes in a persistent history. (2) A semantic action groups a segment of history changes recorded between the start and end of an interaction, such as a drag. Modern front-end frameworks increasingly rely on reactive signals [3], primitives that propagate state changes to dependent computations, such as functions that re-render part of the interface. As signals typically expose only the current value of a variable, we augment them with persistent histories (figure 2.1) that record all past values together with relevant metadata (e.g. timestamps). Developers can therefore visualize, query, and replay prior states without implementing custom instrumentation. We further group state changes according to interaction boundaries (e.g., from drag start to drag end), producing explicit action blocks (figure 2.2) that provide contextual information about user activity. This constitutes a first step toward semantic preservation by reconstructing meaningful user actions from state evolution. In CRDT systems such as Yjs 1 , all value changes are reliably shared between collaborators and can be explicitly grouped into transactions, with undo/redo functionalities operating over these updates. We build on this mechanism by recording a semantic action as described above along with each transaction. As a result, the shared history contains not only what changed, but also what action was performed, allowing collaborators to follow meaningful steps instead of a stream of low-level edits. 5 Conclusion This paper examined the challenge of capturing creative activity traces that reflect the structure of creative practice rather than mere state changes. Across three domains, we showed how this structural decoupling manifests differently and requires distinct approaches: stateful nodes that preserve non-linear exploration and semantic branching in GenAI design; a vocabulary of 3D design cues that surfaces representational intent in expressive visualization authoring; and a programming model that embeds semantic action boundaries directly into interaction state. These contributions together suggest that meaningful trace capture is less a logging problem than a design problem—one that requires rethinking how tools expose the decisions underlying creative activity. 1 https://yjs.dev Manuscript submitted to ACM From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains5 References [1] Tyler Angert, Miroslav Suzara, Jenny Han, Christopher Pondoc, and Hariharan Subramonyam. 2023. Spellburst: A Node-based Interface for Exploratory Creative Coding with Natural Language Prompts. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST ’23). Association for Computing Machinery, New York, NY, USA, 1–22. doi:10.1145/3586183.3606719 [2]Hsiang-Ting Chen, Li-Yi Wei, Björn Hartmann, and Maneesh Agrawala. 2016. Data-driven adaptive history for image editing. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (Redmond, Washington) (I3D ’16). Association for Computing Machinery, New York, NY, USA, 103–111. doi:10.1145/2856400.2856417 [3] Bjarno Oeyen, Joeri De Koster, and Wolfgang De Meuter. 2024. Reactive Programming without Functions. The Art, Science, and Engineering of Programming 8, 3 (Feb. 2024). doi:10.22152/programming-journal.org/2024/8/11 [4]Xiaohan Peng, Debanjana Haldar, Janin Koch, and Wendy E. Mackay. 2026. DesignTrace: Exploring, Iterating and Tracking Design Alternatives with GenAI. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (Barcelona, Spain) (CHI ’26). Association for Computing Machinery, New York, NY, USA. doi:10.1145/3772318.3791036 [5]Sotiris Piliouras, Pierre Dragicevic, Michel Beaudouin-Lafon, and Theophanis Tsandilas. 2025. Incorporating 3D-Rendered Materials in Visualization. In EUROVIS 2025-27th Eurographics Conference on Visualization. The Eurographics Association. [6]Eric Rawn, Jingyi Li, Eric Paulos, and Sarah E. Chasins. 2023. Understanding Version Control as Material Interaction with Quickpose. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–18. doi:10.1145/3544548.3581394 [7]Real World Visuals. [n. d.]. New York City’s greenhouse gas emissions as one-ton spheres of carbon dioxide gas. https://w.youtube.com/watch? v=DtqSIplGXOA. Accessed: 2026-02-21. [8] Science Communication Lab. [n. d.]. Explore the Ocean. https://scicom-lab.com/project/explore-the-ocean. Accessed: 2026-02-21. [9]Marc Shapiro, Nuno Preguiça, Carlos Baquero, and Marek Zawirski. 2011. Conflict-free replicated data types. In Proceedings of the 13th international conference on Stabilization, safety, and security of distributed systems (S’11). Springer-Verlag, Berlin, Heidelberg, 386–400. [10]Haijun Xia, Bruno Araujo, Tovi Grossman, and Daniel Wigdor. 2016. Object-Oriented Drawing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4610–4621. doi:10.1145/2858036.2858075 [11] Weitao You, Yinyu Lu, Zirui Ma, Nan Li, Mingxu Zhou, Xue Zhao, Pei Chen, and Lingyun Sun. 2025. DesignManager: An Agent-Powered Copilot for Designers to Integrate AI Design Tools into Creative Workflows. ACM Trans. Graph. 44, 4, Article 35 (July 2025), 26 pages. doi:10.1145/3730919 Manuscript submitted to ACM