Paper deep dive
DraftTrace: A Multi-View Analytics Environment for AI-Integrated Writing
Divyansh Chandarana, Sandipan De, Vivek Gupta
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Generative AI has changed how students produce writing assignments. The final artifact is no longer sufficient to understand the process through which it was produced. We introduce DraftTrace, a writing environment that jointly captures three complementary views of writing: the final product, the writing process and interactions with an integrated AI-assistant. DraftTrace reconstructs how a document develops over time and organizes these signals into submission, longitudinal, and class-level analytics for instructors. We deployed DraftTrace in a graduate NLP course with 81 students and compared their sessions with LLM-generated responses entered by automated tools and with copy-typed responses. While product measures distinguish differences in text formulation, process measures distinguish differences in how text is entered. Considering both views together helps characterize cases such as copy-typing. Interaction traces show that students use the assistant differently across stages of writing: to clarify the question at an early stage and to verify answers at a later stage. A preliminary instructor survey highlights the importance of multi-view writing analytics and their interpretability.
Tags
Links
- Source: https://arxiv.org/abs/2609.36544v1
- Canonical: https://arxiv.org/abs/2609.36544v1
Trouble viewing inline? Open PDF directly →
Full Text
25,707 characters extracted from source content.
Expand or collapse full text
DRAFTTRACE: A Multi-View Analytics Environment for AI-Integrated Writing Divyansh Chandarana * Sandipan De * Vivek Gupta Arizona State University DemoÅ Video dchanda1, sandipan, vgupt140@asu.edu Abstract Generative AI has changed how students pro- duce writing assignments. The final artifact is no longer sufficient to understand the process through which it was produced. We introduce DRAFTTRACE, a writing environment that jointly captures three complementary views of writing: the final product, the writing process and interactions with an integrated AI-assistant. DRAFTTRACE reconstructs how a document develops over time and organizes these signals into submission-, longitudinal-, and class-level analytics for instructors. We deployed DRAFT- TRACE in a graduate NLP course with 81 stu- dents and compared their sessions with LLM- generated responses entered by automated tools and with copy-typed responses. While product measures distinguish differences in text formu- lation, process measures distinguish differences in how text is entered. Considering both views together helps characterize cases such as copy- typing. Interaction traces show that students use the assistant differently across stages of writing: to clarify the question at an early stage and to verify answers at a later stage. A pre- liminary instructor survey highlights the impor- tance of multi-view writing analytics and their interpretability. 1 Introduction Generative AI (GenAI) has changed not only the artifacts students submit as part of written assign- ments, but also the processes through which those artifacts are produced. Students can use AI assis- tance at different stages of an assignment: to un- derstand a concept, brainstorm ideas, construct an outline, revise existing writing, co-write portions of a response, or generate a complete response. Traditionally, although students draw on textbooks, scholarly literature, online resources, or other ex- ternal materials, producing a coherent response * Contributed equally. generally required them to engage with the mate- rial. GenAI has substantially changed this rela- tionship between the final product and students’ engagement, and has made it increasingly difficult for instructors to determine whether and how the writing activity contributed to student learning. Prior work has studied the writing process by capturing measures like writing speed, pauses, bursts, revisions and deletion events during text production. These measures have been used to investigate their relationships with underlying cog- nitive processes. However, such behavioral signals do not map uniquely to specific cognitive activi- ties; for example a pause can be associated with planning, reflection, or revision. GenAI introduces an additional source of ambiguity into these ob- servations. A pause may now also correspond to interaction with AI system. These changes call for a reconsideration of how writing activities are characterized, measured and interpreted in AI-mediated settings. We argue that understanding of writing requires considering three complementary views: the product view, which characterizes the final artifact; the process view, which captures how the artifact develops over time; and the interaction view, which captures how the student engages with the AI during that process. Existing tools provide different subset of these ca- pabilities, including final product analysis, revision histories, and integrated AI assistance. However, these capabilities are treated as separate signals rather than a complementary views of the same writing activity. To support these integrated perspective, we in- troduce DRAFTTRACE a writing environment that jointly capture the writing process, characterize the resulting product, and records students’ inter- actions with an integrated AI assistant. DRAFT- TRACE organizes these complementary sources of information into instructor-facing views at multiple levels of granularity. 1 arXiv:2609.36544v1 [cs.CL] 29 Sep 2026 2 Background and Related Work 2.1 Writing Analytics Writing analytics has used characteristics of com- pleted text for applications ranging from automated essay scoring to automated feedback generation. Ke and Ng (2019) survey product-level measures spanning style, relevance, organization, cohesion, and coherence, among other dimensions of writing quality. More recently, Pande et al. (2026) develop a pipeline that utilizes stylometric analytics along- side a large language model (LLM) to generate feedback for student writing at scale. Writing research has also used keystroke events to study the writing process. Guo et al. (2018) show that longer writing time and shorter and less vari- able within-word keystroke intervals are associated with higher essay scores. This finding is consis- tent with the view that fluency in lower-level tran- scription processes may free cognitive resources for higher-level composition. Schaller et al. (2026) further demonstrate that keystroke-derived features can provide predictive signals of essay quality dur- ing the early stages of composition, before suffi- cient textual content is available for conventional product-based scoring. At the same time, Babalola et al. (2026) find that many writing-process fea- tures exhibit variability across tasks and contexts, including grade level, academic proficiency, and school setting. 2.2 AI-Mediated Writing Studies of AI-mediated writing have examined how differences in AI access and interaction behavior relate to students’ writing processes. While Chris- tenson et al. (2026) measure how ownership percep- tion of the student changes based on the frequency of LLM access, Park et al. (2026) categorize the types of LLM use and examine their association with student performance. Complementing these studies Yang et al. (2025) examine how students process AI-generated assistance, distinguishing be- tween suggestions that are rejected, accepted with modifications, or accepted unchanged. Collectively, these findings highlight substantial variation in how student engage with and use AI during writing. Recent work has started to combine these per- spectives. He et al. (2025) combine process and product information to improve writing assessment. In AI assisted writing, Chen et al. incorporate interaction traces to characterize how students in- tegrate AI into their writing. While these studies combine subsets of signals for specific analytical tasks, collectively they suggest the value of consid- ering product, process, and interaction together to provide a comprehensive view of student writing. 3DRAFTTRACE The design of DRAFTTRACE is guided by four goals that determine what information is captured, how assistance is provided, how resulting informa- tion is presented and how the platform fits within existing educational workflows. Integrated view of writing. Process, product, and interaction information is captured and aligned as complementary views of the writing. Configurable AI support. Rather than treating AI access as allowed or forbidden, instructors can configure the level of AI assistance according to the assignment and learning objectives. Longitudinal and class-level analytics.Writing analytics are organized to compare a student’s sub- missions over time and with broader class-level patterns. Low-friction access and integration. The writ- ing environment operates independently or inte- grates with existing Learning Management Sys- tems to allow students and instructors to use the platform within their normal workflows without requiring separate procedures. Existing writing and educational platforms pro- vide different subsets of capabilities to these design goals. Table 1 compares DRAFTTRACE with repre- sentative systems across these capabilities. 3.1 System Architecture DRAFTTRACE is a web application with a React front end, Express back end, and a PostgreSQL store (Figure 1).The editor is built on Tip- Tap/ProseMirror. The browser and server share the same document schema and analysis modules, so the writing playback shown to instructors and the metrics computed on the server are derived from identical logic. Process capture.DRAFTTRACE records each change to the document as an event with a times- tamp, associated text, and its input channel: typing, paste, drag-and-drop, undo/redo, or insertion from the integrated assistant. Switching away from the editor is also logged. Complete document snap- shots are stored periodically to bound replay cost. 2 St udent Brow ser Writing Editor (TipTap) Local Storage (IndexDB) Records Editing Events Inst ruc tor Brow ser Assignment Creation Submission View s Cadd y HTTPS Edge (LTS Termination, Routing SSE Passthrough) DraftTrac e App Ser ver Express API, React (Ingestion, Access Control, Integration - OpenRouter, LMS) DataStore (PostgreSQL) (Events, Snapsots Chat, Submissions Metrics) Analytics Services (Document Reconstruction, Process and Product Metrices, Reply, Replay, Report Generation) AI Ser v ic es (AskAI) Context Management, Interaction Capture LTI Int rgrat ion (Optional) Assignment, Roster Synchonization HTTPS J SON + AI STREAM S HTTPS Repor ts and Expor ts Figure 1: Overview of the DRAFTTRACE system architecture for capturing, reconstructing, and analyzing student writing and AI interactions. SystemProc.Prod.Interact.Assist.Config.PlaybackAnalyticsEditor DraftTrace✓ Turnitin Clarity ✓–✓–✓ Documark✓–✓ Grammarly Authorship ✓–✓ Draftback✓–✓ GPTZero–✓– Pangram–✓– Table 1: Comparison of capabilities across represen- tative writing platforms. Proc.: process view; Prod.: product view; Interact.: interaction view; Assist.: inte- grated AI assistant; Config.: configurable AI support; Editor: integrated writing environment. To avoid data loss on unreliable classroom net- works, events are queued in the browser’s In- dexedDB and then uploaded small batches. Each event carries a unique sequence number, making retries idempotent. The events are discarded locally only after the server acknowledges them. For timed assignments, deadlines are enforced on the server, and incomplete sessions are submitted automati- cally when time expires. Product reconstruction and provenance.After submission, DRAFTTRACE replays the recorded events to rebuild the document and labels each char- acter by how it entered: typed, inserted from the built-in assistant, pasted with or without a cited source, or unknown. These labels are summarized per paragraph and for the whole submission, along- side process measures such as bursts, pauses, and revisions. Interaction capture. The integrated assistant is proxied through the server rather than called from the browser. Each student prompt is persisted. Re- sponses are streamed to the student and saved to the session transcript which is visible to the instruc- tor. Text inserted from the assistant is tagged at the transaction level, linking the interaction view to the process and product views. Instructors can enable or disable the assistant per assignment, and a sys- 3 tem prompt instructs it to support understanding rather than write answers. 3.2 Instructor Interface The instructor interface supports two primary work- flows - to create quizzes and to review the resulting analytics. Instructors can configure the questions, instructions, time and length constraints (Figure 2). For each submission, DRAFTTRACE summarizes writing-process measures (Figure 3) and provides a chronological timeline of the writing session (Fig- ure 4). Additional class-level and longitudinal views are provided in Appendix A. Figure 2: Instructor interface for creating an assignment. 3.3 Student Interface The student interface provides an environment for completing the assignment task. Student compose their responses directly in the editor (Figure 5). When enabled by instructor, students can access the integrated AI assistant from within the same environment (Figure 6). Figure 3: Writing-process view summarizing measures captured during an individual writing session. 4 Case Study 4.1 Data Collection We collected writing traces under three settings that vary how the written response is formulated and entered into the writing environment: classroom writing, automated writing, and copy-typing. Classroom Writing. We used DRAFTTRACE during a writing activity in graduate-level Natu- ral Language Processing course and collected re- sponses from 81 students who were present during the session. At the beginning of the activity, stu- dents were briefly introduced to the writing envi- ronment and its available features. Students were given 15 minutes to answer two related questions based on the materials covered in a previous lecture. They were allowed to consult lecture slides and use the AI assistant integrated within DRAFTTRACE, 4 Figure 4: Activity timeline showing the chronological development of an individual writing session. but were instructed not to use external AI tools. The first question asked the students to select a prompting technique and explain it in their own words and provide examples of its use. The second question asked the students to identify a scenario in which the selected technique might not work well and explain why. While all students followed the same general task structure, they could select different prompting techniques and construct their own examples, allowing variation in the content of the responses. Automated Writing.We collected an additional 81 writing sessions in which response to the same questions were generated through LLM. The gen- erated responses were entered into DRAFTTRACE using automated typing mechanism that approxi- mates human-like text entry. Copy-Typing. To examine writing traces when text formulation and text entry are separated, we collected additional eleven sessions in which partic- ipants were allowed to use provided responses, ex- Figure 5: Writing environment for completing the as- signed task. Figure 6: Integrated AI assistant. ternal AI tools, or other resources while answering the same questions. Participants were instructed to manually type text into DRAFTTRACE. 5 Class.Auto.Copy (n=81) (n=81) (n=11) Product view Flesch Reading Ease52.838.439.9 Mean word length (chars)4.675.185.40 Mean sentence length (chars)130.50163.17147.50 Process view Typing speed (wpm)27.549.033.0 Revisions / 100 chars12.52.75.4 Left the editor (count)1400 Active / elapsed time0.771.001.00 Table 2: Median values per setting. Class.: classroom writing; Auto.: automated writing; Copy: copy-typing. 4.2 Analysis We compare the three settings using a small set of measures from the process and product views. From the process view we use typing speed (words per minute), revisions per 100 typed characters, the number of times the student left the editor, and the fraction of elapsed time spent actively writing. From the product view we use Flesch Reading Ease and mean word length, computed on the concate- nated answers to both questions. The product view reflects who formulated the text. Responses formulated by an LLM were harder to read and used longer words than class- room responses, regardless of how they were en- tered. Median Flesch Reading Ease was 52.8 for classroom responses, compared with 38.4 for auto- mated and 39.9 for copy-typed responses, and me- dian word length rose from 4.67 to 5.18 and 5.40 characters. Automated and copy-typed responses had nearly identical product measures. This is ex- pected as the text originates from an LLM, the final artifact looks similar irrespective of how it was entered. The process view reflects how the text was en- tered. Automated entry left a regular trace: a median typing speed of 49 wpm, 2.7 revisions per 100 characters, and no idle time or departures from the editor. For copy-typing we observe similar be- havior: they neither paused nor left the editor as the text was already formulated. However, the par- ticipants typed more slowly (33 wpm) and revised twice as often (5.4 per 100 characters). Classroom writers, by contrast, revised most (12.5 per 100 characters), left the editor a median of 14 times, and were actively writing for 77% of the session. Q1Q2Q3Q4 Prompts11172126 Students’ first prompt105710 Table 3: Assistant use by session quarter for the 32 students who used the integrated assistant. Neither view is sufficient alone. Copy-typing illustrates why the views must be read together. Its product resembles automated writing, while its process resembles human typing without the pauses and editor departures seen in the classroom. Identifying such a session requires combining both views, the integrated perspective DRAFTTRACE is designed to provide. The interaction view reflects when and why stu- dents sought help. In the classroom setting, 32 of 81 students used the integrated assistant. In total, they have issued 75 prompts (median 2 per student; 15 students asked only once). We divided each stu- dent’s session into four equal quarters and assigned each prompt to a quarter based on its timestamp. Distribution of the requests across these segments is shown in Table 3 To examine how students used the assistant, we categorized their interactions into four types: clari- fication about the question, verification of a drafted answer, direct help (requesting an answer or exam- ple), and collaboration (an extended exchange that develops the response over several turns). Students whose first prompt came in the first quarter sought clarification (6 of 10) or direct help (4 of 10). Stu- dents who first used the assistant in the last quarter almost always sought verification of their answers (9 of 10). Collaboration appeared only among stu- dents who issued three or more prompts (7 of 13). Students who issued only one or two prompts gen- erally used the assistant for either direct help or verification. These patterns show that students use the assis- tant differently at different stages of the writing process. DRAFTTRACE aligns the interaction view with the process timeline and provides additional context for interpreting these patterns. 5 Instructor Survey To complement the classroom study, we conducted a survey 1 to understand instructors’ current prac- tices, their perceptions of AI-integrated writing 1 https://forms.gle/jV5jeqQnwTnve7m28 6 environments, and their considerations regarding the collection and analysis of student writing data. The survey targets educators involved in teaching, assisting, or grading student work across different academic levels and disciplines. Findings. •Current practices provide limited confidence. Nine of the ten respondents reported encoun- tering suspected inappropriate AI use at least sometimes, while only one reported being very confident in their current process for in- vestigating such cases. This contrast suggests a gap between the frequency with which in- structors encounter questions about how stu- dent work was produced and their confidence in the information currently available to inves- tigate those questions. • Writing support and visibility were valued across multiple dimensions. Writing-process visibility and final-submission analysis were each rated as very valuable by 7 of 10 respon- dents, while instructor-controlled AI access, AI-interaction visibility, and writing-process playback were each rated very valuable by 6 respondents. •Acceptance of information collection was largely conditional. While 6 of 10 respon- dents considered copy/paste activity and AI- interaction history generally acceptable to col- lect, writing/editing activity, detailed typing patterns, and AI processing of student writing were more often viewed as acceptable only under limited circumstances. • Accuracy and interpretability were identified as important considerations. Inaccurate or misleading results could discourage adoption for 9 of 10 respondents, while 4 of 10 indi- cated that a lack of transparency about how such systems work could affect adoption. 6 Discussion While DRAFTTRACE is designed as a writing ana- lytics tool, we also view it as a research apparatus for studying how writing changes in AI-integrated environments. By collecting aligned product, pro- cess, and interaction information, the environment enables researchers to observe not only what stu- dents produce, but how their writing and use of assistance evolve during an assignment and across assignments. Beyond observation, DRAFTTRACE provides an environment for adaptive experimen- tation. As the AI assistance is configured per as- signment, researchers can vary its availability and behavior. This enables controlled studies of when assistance should be provided and what form it should take. As the system already observes writ- ing and interaction patterns, it could be extended to adapt the level or form of assistance based on these observations, a direction we plan to explore in future work 7 Limitations DRAFTTRACE captures activity occurring within its writing environment and cannot observe all ex- ternal resources or activities that may contribute to a response. While the captured process, product, and interaction signals provide complementary in- formation, they do not capture every aspect of the writing process. Additionally, our case study is lim- ited to a single graduate-level course and a short writing activity. The writing patterns may vary across tasks, student populations, and instructional settings. 8 Ethical Considerations DRAFTTRACE collects fine-grained information about students’ writing behavior and AI interac- tions beyond what is contained in a conventional final submission. Students should therefore be in- formed about what information is collected, how it is processed, and who has access to it. Such data, particularly writing traces and AI conversations, should be treated as sensitive educational data with appropriate access and retention controls. References Damilola Babalola, Naga Buddarapu, Piotr Mitros, Paul Deane, and Collin Lynch. 2026. Stability and contex- tual sensitivity of keystroke process features in lon- gitudinal student writing portfolios. In Proceedings of the 19th International Conference on Educational Data Mining, pages 256–266, Seoul, Republic of Ko- rea. International Educational Data Mining Society. Nuo Chen, Wanying Zhong, Kejie Shen, and Yizhou Fan. Beyond the chat window: A trace-based posi- tioning analysis of student-genai interactions in aca- demic writing. In Artificial Intelligence in Education, pages 640–649, Cham. Springer Nature Switzerland. 7 Julia Christenson, Karin de Langis, Shirley Anugrah Hayati, and Dongyeop Kang. 2026. Effects of vary- ing LLM access on essay writing behavior. In Pro- ceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), pages 685–701, San Diego, California, USA. Association for Computational Linguistics. Hongwen Guo, Paul D. Deane, Peter W. van Rijn, Mo Zhang, and Randy E. Bennett. 2018. Modeling basic writing processes from keystroke logs. Journal of Educational Measurement, 55(2):194–216. Xinyun He, Qi Shu, Mo Zhang, Wei Huang, Han Zhao, and Mengxiao Zhu. 2025. Beyond final products: Multi-dimensional essay scoring using keystroke logs and deep learning. In Proceedings of the 15th Inter- national Learning Analytics and Knowledge Confer- ence, LAK ’25, page 601–610, New York, NY, USA. Association for Computing Machinery. Zixuan Ke and Vincent Ng. 2019. Automated essay scoring: A survey of the state of the art. In IJCAI, volume 19, pages 6300–6308. Stuti Pande, Yige Song, Kamila Misiejuk, Sonsoles López-Pernas, Mohammed Saqr, and Eduardo A. Oliveira. 2026. Profiling writing skills at scale: A hy- brid stylometry-llm pipeline for formative feedback. In Proceedings of the Thirteenth ACM Conference on Learning @ Scale, L@S ’26, page 491–495, New York, NY, USA. Association for Computing Machin- ery. Minju Park, Ivan Orozco Vasquez, and Cristina Conati. 2026. Characterizing students’ llm usage behaviors and their association with learning in critical thinking tasks. In Proceedings of the 19th International Con- ference on Educational Data Mining, pages 57–66, Seoul, Republic of Korea. International Educational Data Mining Society. Nils-Jonathan Schaller, Daniel Mora Melanchthon, Thorben Jansen, Olaf Köller, and Andrea Horbach. 2026. KEYSCORE — keystroke-enhanced auto- mated essay scoring. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), pages 347– 357, San Diego, California, USA. Association for Computational Linguistics. Kaixun Yang, Mladen Rakovi ́ c, Zhiping Liang, Lixi- ang Yan, Zijie Zeng, Yizhou Fan, Dragan Gaševi ́ c, and Guanliang Chen. 2025. Modifying ai, enhancing essays: How active engagement with generative ai boosts writing quality. In Proceedings of the 15th In- ternational Learning Analytics and Knowledge Con- ference, LAK ’25, page 568–578, New York, NY, USA. Association for Computing Machinery. A Interfaces Figure 7 presents additional instructor-facing ana- lytics in DRAFTTRACE, including an overall sum- mary of an individual submission, a longitudinal (a) Overall analytics for an individual submission. (b) Longitudinal view of a student’s writing across sub- missions. (c) Comparison of an individual submission with class- level patterns. Figure 7: Additional instructor-facing analytics in DRAFTTRACE: (a) overall analytics for an individual submission, (b) longitudinal analysis of a student’s writ- ing across submissions, and (c) comparison with class- level writing patterns. view of the student’s writing across submissions, and a class-level comparison for the assignment. 8