Paper deep dive
Can LLMs Test Terminal User Interfaces?
Chao Peng, Ruida Hu, Ajitha Rajan, Tegawendé F Bissyandé, Jacques Klein, Cuiyun Gao
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tool tuicov at this https URL and the testing framework tuibot at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.03743v1
- Canonical: https://arxiv.org/abs/2608.03743v1
Trouble viewing inline? Open PDF directly â
Full Text
58,082 characters extracted from source content.
Expand or collapse full text
Can LLMs Test Terminal User Interfaces? Chao Peng â University of Edinburgh United Kingdom chao.peng@acm.org Ruida Hu * Harbin Institute of Technology Shenzhen, China 200111107@stu.hit.edu.cn Ajitha Rajan University of Edinburgh United Kingdom arajan@exseed.ed.ac.uk TegawendĂ© F. BissyandĂ© University of Luxembourg Luxembourg tegawende.bissyande@uni.lu Jacques Klein University of Luxembourg Luxembourg jacques.klein@uni.lu Cuiyun Gao Harbin Institute of Technology Shenzhen, China gaocuiyun@hit.edu.cn Abstract Terminal User Interfaces (TUIs) combine the stateful, screen-oriented behaviour of GUIs with terminal deployment and are now common in developer tools. Yet they lack a dedicated testing methodology. We survey 197 real-world TUI applications: only 12% of test code exercises the interface, and 45% of those tests never send input, checking a static frame instead. We turn these applications into a headless benchmark spanning ratatui/Rust, bubbletea/Go, textual/Python, and ink/TypeScript, packaging each as an instrumented Docker image. We record line and widget coverage where reliable, rendered terminal states, and crashes. Under equal wall-clock budgets, we compare four frontier LLMs with random exploration. No model dominates. Random is a strong time-budgeted baseline, but its crash advantage comes from higher throughput: per interaction, LLM guidance is more efficient and uniquely reaches input-gated faults. Automatically deriving launch inputs yields the largest practical gain, enabling applications that otherwise never start. Line coverage poorly predicts crash discovery, weakening it as a proxy for test effectiveness. Automated TUI testing is feasible but far from solved, and honest baselines matter more than model choice. We release the coverage tooltuicovathttps:// github.com/tui-testing/tuicovand the testing frameworktuibotathttps://github.com/ tui-testing/tuibot. Testing loop: rendered TUIâ tuibotâ generate actionâ exercise TUIâČ until the budget is exhausted. Post-run: tuicov measures executionâ coverage and crash report. Figure 1: Control flow of the released tuibot and tuicov toolchain. â Chao Peng and Ruida Hu are co-first authors and contributed equally. Preprint. arXiv:2608.03743v1 [cs.SE] 4 Aug 2026 Command-Line Interface (CLI) Stateless transducer metaphor. Raw byte stream Terminal User Interface (TUI) Screen-oriented state management Trapped in a terminal. Graphical User Interface (GUI) Object-Oriented and Event-Driven Metaphor (accessibility tree). stdin input stdout output graphics engine DOM tree accessibility tree Figure 2: A technical comparison of the three interface families. The CLI is a stateless transducer (argv/stdinin,stdoutout); the GUI exposes state and events as addressable objects through an accessibility tree; the TUI exposes neither, presenting only a character grid rendered into a pseudo- terminal driven by a raw byte stream. 1 Introduction Terminal User Interface (TUI) applications combine properties of command-line (CLI) and graphical (GUI) software without belonging to either. Like CLI tools they run inside a terminal emulator and are launched from a shell. Like GUI applications they present a stateful, screen-oriented interface with windows, panes, focus management, modal dialogs, and keyboard- and mouse-driven navigation. This class is large and growing fast. File managers, system monitors, database clients, git frontends, and, most recently, LLM coding agents are routinely shipped as TUIs, built atop frameworks such as ratatui [15], bubbletea [3], textual [21], and ink [5]. Despite their prevalence, TUI applications have no dedicated testing methodology, and crashes in TUI software remain both common and uninvestigated. Existing research targets either CLI programs, treated as pure input/output transducers whose correctness is judged by exit codes and stdout, or GUI programs, driven through accessibility trees and pixel-level widget hierarchies [11,12,20]. Neither model transfers to a TUI. CLI techniques discard all stateful, screen-oriented behaviour, which is the very layer where TUI bugs manifest. GUI techniques assume rich accessibility metadata, namely widget identifiers, roles, and bounding boxes, that a character-grid terminal does not expose. There is no DOM, no accessibility tree, and no coordinate system beyond a two-dimensional array of cells rendered into a pseudo-terminal. A TUIâs entire observable state is a character grid, and its behaviour is driven by raw key, mouse, and resize events. A comparison of the three interfaces is shown in Figure 2. No existing fuzzing, exploration, or test-generation technique was designed for that interaction model. The consequence is a practical gap. TUI applications crash, users file bug reports, and no automated technique exists to find, reproduce, or prevent those failures. Two questions motivate this paper. First, how well are TUIs tested today? We surveyed the test suites of 197 real-world TUI applications and found the interface layer doubly under-tested: only about 12% of all test code exercises the terminal/widget layer at all, and of the tests that do, nearly half never send a single input event (Section 2). Second, do the LLM-driven exploration and test-generation techniques now reshaping CLI and GUI testing [6,8,17] transfer to TUIs? Our results suggest they do not transfer cleanly. The relationship between code coverage and crash detection in TUI applications diverges from the pattern observed in CLI and GUI domains, where higher coverage tends to predict more faults found [18,19,23]. In TUIs, coverage and fault-finding decouple. That finding undermines the standard practice of using line coverage as a proxy for test effectiveness and motivates a TUI-specific coverage metric that aligns with crash discovery. We focus on crash bugs. Silent logic errors and performance regressions matter, but they require ground truth that does not yet exist for TUIs, whereas crashes are unambiguous, automatically detectable, and, as our bug corpus shows, a substantial and underexplored failure mode in practice. 2 We make the following contributions: âą An open, multi-language TUI benchmark. 197 real-world applications spanning the four dominant frameworks and languages (ratatui/Rust, bubbletea/Go, textual/Python, ink/Type- Script), each packaged as a per-instance, instrumented Docker image that builds and runs headlessly (Section 3). âąA kill-resilient instrumentation toolkit.tuicov 2 is a language-agnostic coverage tool that records both native line coverage and an experimental widget-coverage signal from an interactive application that must be terminated rather than allowed to exit (Section 3). âąAn empirical study and open-source test framework.tuibot 3 implements the frontier- LLM and exploration strategies evaluated on this benchmark, answering three research questions about model capability, technique comparison, and the coverage-fault relationship (Sections 4-5). Our findings counsel caution about the anticipated benefits of LLM-based TUI testing. No single model achieves consistent superiority across applications. A model-free random baseline remains competitive throughout. The principal determinant of practical effectiveness proves to be the correct derivation of launch inputs, without which input-dependent applications cannot be exercised at all. Most consequentially, conventional line coverage is a poor predictor of crash discovery in TUI software, which reinforces the case for a domain-specific coverage criterion. We releasetuicovand tuibot as open-source tools to support TUI testing as an independent research area. 2 Motivation Why TUIs need their own testing approach. To understand why existing testing techniques fall short for TUIs, consider a concrete behaviour from a file-manager TUI: pressingjmoves the cursor down, and pressingEnteropens the highlighted entry. This single, familiar interaction exposes a fundamental mismatch with both CLI and GUI testing models, as illustrated in Figure 2. In the CLI model, the interaction cannot be expressed at all. CLI testing treats a program as a pure input/output transducer: a test fixesargvandstdin, capturesstdout/stderrand an exit code, and compares the result against an expected value. A sequence such as âmove cursor down, then open entryâ has no representation in this model. It unfolds across time against a persistent, on-screen state that a single-invocation transducer cannot capture. In the GUI model, the same behaviour is straightforwardly testable, because a graphical application exposes an accessibility tree of named, queryable objects. A test driver locates the file list by its accessibility role, reads itsselectedRowproperty, synthesises a keypress or mouse click to advance the selection, and then queriesrow.isSelectedor awaits anactivatedevent. State and events are first-class, named entities that the harness can inspect and subscribe to directly. In the TUI model, neither of these handles exists. The entire observable state of a TUI is a two- dimensional grid of characters rendered into a pseudo-terminal; input is the raw byte stream"j ". There is no queryable property for âwhich row is highlightedâ, that information is encoded implicitly as the single line drawn with an inverted SGR (Select Graphic Rendition) attribute somewhere within anH Ă Wgrid of cells. To verify that the cursor moved, a test must write the bytes into the pseudo-terminal (PTY), wait for the screen to repaint, scan the full cell grid for the styled run, and decode the file path from its glyphs. To verify that an entry was opened, it must diff the entire frame against a prior state. There is no widget to query and no event to await, with only bytes in and a repainted screen out. This is precisely the layer that existing CLI- and GUI-oriented techniques bypass, and the layer that any automated TUI tester must be capable of driving directly. Figure 3 reinforces this point with a richer example: issuing an authenticatedPOSTof a JSON user record, realised across all three interface families. TUIs are poorly tested in practice. TUI application frameworks provide testing harness and devel- opers can write TUI-level tests with it. Figure 4 shows a representative example fromtwig, a JSON- 2 https://github.com/tui-testing/tuicov 3 https://github.com/tui-testing/tuibot 3 (a) CLI (curl). (b) GUI (Postman). (c) TUI (Posting). Figure 3: The same task (issuing an authenticatedPOSTto/userswith a JSON body) across the three interface families. The CLI (a) encodes the whole request as oneargv/stdininvocation; the GUI (b) exposes named fields and buttons as addressable objects; the TUI (c) renders the same fields as a character grid in a terminal, driven entirely by keystrokes. 4 explorer TUI in our benchmark, written against textualâsPilotdriver. The harness boots the app inside a headless pseudo-terminal (run_test), synthesises keystrokes (pilot.press), and, crucially, lets the test query live widget objects (query_one,screen.focused,OptionList.highlighted) instead of decoding glyphs from the screen buffer. This is exactly the âmove cursor, then assert which row is highlightedâ interaction from our running example, made expressible only because the framework re-exposes the named state and events that the terminal itself discards. Every TUI ecosystem ships some version of this affordance (ratatuiâsTestBackend, bubbleteaâsteatest, inkâs ink-testing-library); the question is how much developers actually use it. async with TwigApp(SAMPLE).run_test() as pilot: nav = app.query_one("ColumnNavigator") # type a path into the search bar and jump to it await pilot.press("/") await pilot.press(*â.regions["us-east-1"].vpcs[0]â) await pilot.press("enter") # assert on widget state, not on screen glyphs assert "col-" in str(app.screen.focused.id) # search "available", then step matches with n / N await pilot.press("/"); await pilot.press(*"available") await pilot.press("enter") first = nav_selected(nav) await pilot.press("n") # next match assert nav_selected(nav) != first await pilot.press("N") # previous match assert nav_selected(nav) == first Figure 4:A TUI-level test fromtwigusing textualâsPilotharness (condensed from tests/test_integration.py). The harness drives the app through keystrokes and asserts on queryable widget state, which is the affordance that makes âpress a key, check which row is high- lighted" testable. To establish that this gap has real-world consequences, we surveyed the test suites of all 197 bench- mark applications, classifying every test file (excluding vendored and dependency code) as either TUI-level (exercising the terminal or widget layer via the frameworkâs TUI test harness) or non-TUI, in the spirit of prior test-suite effectiveness studies [7]. Classification was LLM-assisted and verified by manual inspection; all counts are computed deterministically. Most applications barely test their interface. Of the 197 applications surveyed, only 76 (38.6%) include any TUI-level tests; 35.0% have tests that never touch the UI layer at all; and 26.4% ship no tests whatsoever. As shown in Table 1, only approximately 12% of all test code exercises the TUI itself. The remainder targets parsers, configuration, data layers, and business logic. Even textual, the ecosystem with the most mature TUI testing infrastructure, allocates only a quarter of its test files to the UI. The Median app share column computes each applicationâs own TUI-test fraction and takes the median across applications, describing the typical app rather than the aggregate. The three can diverge sharply, which is precisely why we report all of them: ink, for instance, shows 18.3% of files but a median of 0%, because its pooled figure is inflated by a single exhaustively tested outlier (gemini-cli) while the median ink application has no TUI test at all. Table 1: Fraction of test effort targeting the TUI layer, by framework. FrameworkApps w/ TUI testTUI files / allMedian app share ratatui32%8.9%7.5% bubbletea43%10.0%6.2% textual41%25.4%15.6% ink43%18.3%0.0% All38.6%12.3%â The same gap at the test-case level. Table 2 re-counts at the granularity of individual test cases (test functions). The 785 TUI-level test files resolve to 8,353 distinct TUI-level test cases (10.6 test cases per file in average) , of which only 5,213 (62%) are interactive. The TUI tests that do exist are shallow. Among the 785 TUI-level test files found across the 76 applications that include them, only 55% (430 test files) are genuinely interactive, that is, they feed key, mouse, or resize events and assert on the resulting state. A further 24% are render-only tests that draw a single static frame and diff the buffer, and the remaining 21% are snapshot-only tests 5 Table 2: TUI-level testing at the granularity of individual test cases (test functions), by framework. Interactive cases belong to a test that sends at least one key, mouse, or resize event. FrameworkTUI test casesInteractiveInteractive % ratatui1,37651137% bubbletea3,8202,84174% textual1,5831,35786% ink1,57450432% All8,3535,21362% Study design and measurement scope Corpus mining 649 awesome-tuis 601 GitHub repos 4 target frameworks Build filter 257 apps attempted 197 runnable images 23% attrition reported Experiment matrix 4 settings 4 LLMs + random 2,561 scheduled jobs Completed run 2,290 executed 600 s per run same PTY harness Coverage scope Line: Rust/Python usable TypeScript sparse; Go unavailable Widget: experimental pilot Crash oracle Rendered-screen classifier filters usage exits, SIGINT, harness noise Reported faults 1,033 raw crash events 179 valid TUI faults 45 affected apps Figure 5: Overview of the completed study design and the measurement scope used in the paper. The benchmark starts from the mined TUI corpus, filters to 197 headlessly runnable applications, runs the four exploration settings under the same PTY harness, and reports only measurements that are valid in the completed run. that compare against a golden frame without sending any input. In total, 45% of TUI test files never send a single input event. Sixteen of the 76 applications (21%), including widely used tools such asbottom,bandwhich,kmon, andtelevision, have TUI test suites that are composed entirely of static renders or snapshots. The interactive tests are also thin. The median interactive file uses only nine input actions, and 11% use just one or two. A file with one or two input actions typically presses a single key and checks a single outcome, exercising none of the multi-step navigation, focus transitions, modal dialogs, or edge-case key sequences that characterize real use. The TUI interface layer is therefore doubly under-tested: it receives a small fraction of overall test effort, and the majority of that effort is static. This is the gap that an automated TUI exploration approach is positioned to fill. 3 Benchmark and Infrastructure Figure 5 gives a roadmap of the benchmark and the measurement scope used throughout the paper. The top row traces the construction and execution pipeline: we mine a corpus of real TUIs and resolve their frameworks, filter to the applications that build and run headless as per-instance instrumented images, schedule the four-setting experiment matrix across the model panel, and execute it. The bottom row records what the completed run can measure: the cross-language coverage scope, the rendered-screen crash oracle, and the resulting fault counts. 3.1 Frameworks and Applications To ground our choice of frameworks in evidence rather than intuition, we first mined every TUI listed inawesome-tuis 4 , an actively maintained collection of TUI applications with approximately 20,000 GitHub stars. In this collection, there are 649 projects across 13 categories. We fetched each repositoryâs evidence files (README,pyproject.toml,Cargo.toml,package.json,go.mod, 4 https://github.com/rothgar/awesome-tuis 6 . . . ), and asked three LLMs (Claude-Sonnet-4.5, Claude-Opus-4.7, GPT-5.4) to identify each projectâs underlying TUI library from that evidence. The three models agreed on the same library for 76% of resolvable projects and never once named three different libraries, giving us confidence in the resulting tally. The result is a near-monolithic leader within each language: of projects whose library could be identified, Rust overwhelmingly uses ratatui (78%), Go uses bubbletea (the clear plurality), Python uses textual, and TypeScript uses ink. We therefore target exactly these four frameworks, one per mainstream language, as the dominant choice each ecosystem has converged on. From these ecosystems we assemble a suite of real-world applications spanning a wide popularity range, from 100k-star coding agents (codex, gemini-cli) down to niche hobby tools. 3.2 Per-Instance Instrumented Images As there is no existing unified tool to measure how thoroughly a TUIâs interface is exercised across all four target languages and frameworks, we implementtuicov 5 , an open-source, language-agnostic coverage toolkit that underpins the benchmark and the measurements throughout this paper. Given an applicationâs source,tuicovperforms a four-stage instrumentâbuildârunâreport pipeline. It first parses the source and statically detects widget sites, the source locations where the frameworkâs UI components (e.g., ratatui, bubbletea, textual, and ink widgets) are constructed or rendered, and injects a lightweight runtime probe at each one. The instrumented app is then built under each languageâs native line-coverage facility (cargo llvm-covfor Rust,go tool covdatafor Go, coverage.py for Python, andc8for TypeScript). At runtime the probe appends one record per widget hit to a log, while the native facility records executed lines. After a driven session, a language- agnostic reporter merges the two raw sources into a single unified report that pairs native line coverage with TUI-specific widget coverage per file. This dual signal (ordinary code coverage alongside an interface-aware one) is what lets us reason about interface exploration rather than mere code execution. The benchmark is realized as per-instance,tuicov-instrumented Docker images that run in a plain headless Linux environment. A single Dockerfile family (one per language) clones the app at the target commit, applies instrumentation in place, builds under native coverage when the runtime supports it, and ships a runnable image. Of 257 apps attempted at the most-recent commit, 197 produced a runnable image. A subset of arbi- trary apps do not build or run headless, whether from missing system dependencies, display/daemon requirements, or compile failures. 3.3 Execution Harness and Coverage Instrumentation Each run is driven bytuibot 6 , our open-source automated TUI testing framework, through a pseudo- terminal at a fixed terminal size. The harness waits for the rendered grid to stabilize after each action, records the character-grid state, and sends key, mouse, and resize events as raw terminal input. The same harness backs all four exploration settings we study: (1) random (model-free input as a baseline), (2) llm-guided (an LLM picks each action at runtime), (3) llm-guided-derived (llm-guided seeded with automatically derived launch inputs), and (4) llm-generated (an LLM reads the source and emits test scenarios that steer the explorer), defined in detail in Section 4. A key subtlety distinguishes TUI testing from ordinary unit testing. The exploration harness termi- nates each session when the time budget is used up, but most native coverage tools only flush on a clean exit. This makes line coverage unavailable for Go as it does not support extracting line coverage when the app is terminated by external signals. 3.4 A Content-Aware Crash Oracle A consequential lesson from driving 197 apps is that âprocess exited non-zeroâ is not a usable crash oracle for TUIs. Across the capability run, 1,033 crash events were recorded, but only 179 (17.3%) are valid TUI-level faults after our examination. The remaining 82% are noise of a few distinct kinds (Section 5). These include an app printing a usage message and exiting because it needed required arguments, a graceful response to Ctrl+C, adocker runname clash, a PTY ioctl 5 https://github.com/tui-testing/tuicov 6 https://github.com/tui-testing/tuibot 7 failure, and, as the single largest category, a coverage-runtime emit failure that turns a clean exit into a non-zero one. An exit-code oracle counts all of these as crashes. We therefore classify each crash event on its rendered terminal screen rather than its exit code. A content-aware classifier scans the post-termination grid for a rendered traceback, an on-screen run- time exception, a Rust/Go panic, or a fatal signal (SIGSEGV/SIGABRT), and hard-excludes harness artifacts (coverage-emit failures, docker races, PTY errors, SIGKILL-at-budget, usage/âhelpexits). Events with no decisive on-screen signal are adjudicated individually: two of the authors indepen- dently labeled all 233 ambiguous crash events as valid TUI-level faults, harness or environment noise, or undecidable. The annotators reached 93% raw agreement, and the remaining disagreements were resolved through discussion until consensus. Per-app crash saturation, the fraction of an appâs runs that crash, together with the median number of steps to the crash, further separates input-invariant startup and environment failures (high saturation, crash at step 1) from interaction-reached faults (mid saturation, many steps). A direct implication for future TUI fault studies is that we adopt a rendered-screen oracle rather than counting raw exits. 4 Research Questions and Study Design We investigate three research questions. Across all RQs we evaluate on TUI applications spanning multiple languages and frameworks (Rust/ratatui, Go/bubbletea, Python/textual, TypeScript/ink) to assess generalization, and all techniques drive each app through a pseudo-terminal, in the manner of end-to-end terminal testing tools [13], running through the same instrumented-container harness so their coverage and crash measurements are directly comparable. RQ1: How do frontier LLMs perform on testing TUI applications? We compare four frontier LLMs (Claude-Opus-4.8,GPT-5.5,Gemini-3.5-Flash, andDeepSeek-V4-Pro) using a fixed LLM-guided exploration strategy. We measure code coverage achieved, the crashes uncovered, and the token cost each model incurs, with budgets equalized in wall-clock time per session so every model drives each subject app for the same exploration window regardless of its latency or pricing. Because the budget is fixed in time rather than in steps, a modelâs per-step throughput, namely how many actions it can issue before the clock runs out, is itself part of what RQ1 measures. RQ2: How do random exploration, LLM-guided exploration, and LLM-based test generation compare? We compare four settings: (1) random exploration as a model-free baseline, (2) LLM- guided runtime exploration, (3) LLM-guided exploration augmented with automatically derived launch inputs, and (4) LLM-based test generation from source code. We measure coverage and crashes, again with budgets equalized in wall-clock time per session, and, for the crash comparison, report effectiveness normalized by interaction (valid crashes per unit of input) rather than by raw count, because a time-equalized budget gives each setting a very different number of steps. RQ3: Does code coverage correlate with fault-finding in TUI applications? We analyze the relationship between code coverage and the number of valid faults discovered across sessions and applications, using the adjudicated crash set (Section 5) as observed fault evidence and accounting for the session-truncation confound that a crash introduces by ending an episode. 4.1 Techniques (Settings) The comparison axis comprises four settings. (1) random: Model-free random input, which serves as the baseline and is analogous to random/monkey testing in the GUI domain [4, 11, 20]. (2) LLM-guided: An LLM selects each action at runtime taking the current terminal screen (characters shown in terminal) as input. (3) LLM-guided-derived: The LLM-guided setting augmented with an LLM that derives CLI arguments and input fixtures so that input-dependent applications launch. (4) LLM-generated: An LLM reads source code and emits natural-language test scenarios that subsequently steer the LLM-guided explorer. 8 Each LLM setting runs across the model panel; random is model-independent. To account for the stochasticity of both random and LLM-guided exploration, every configuration is repeated three times per app, and all reported metrics are averaged over the three runs. Each run receives a 600 s wall-clock budget, equalized across settings. Exploration relaunches the app on exit/crash until the budget is spent, so coverage and crashes accumulate across episodes. To isolate each arm, the LLM settings employ no random fallback. A failed LLM call is retried and then fails the run rather than degrading silently to random, so that every step attributed to an LLM arm constitutes a genuine model decision. 4.2 Metrics We record three signals per run. Line coverage and widget coverage are harvested from the unified report after each session (Section 3), but the results scope each signal to the runtimes where the completed run produced valid data. For crashes we do not report raw exit-non-zero counts, because, as Section 5 shows, those are dominated by environment and harness noise. We instead classify every crash event on its post-termination terminal screen into valid TUI-level faults and noise, and we report valid crashes both per run and per unit of interaction (per 1,000 steps), together with the count of unique(app, fault)signatures. For every LLM run we additionally record token usage (input tokens, output tokens, and number of model calls), so that effectiveness can be weighed against cost. 5 Results Our study comprises a unified run of 197 applications under four settings and four models, with every configuration repeated three times per app and all reported numbers averaged over the three runs. Before answering the RQs, we scope the measurements. Crash results use the adjudicated valid-fault set rather than raw exit codes. Line coverage is reported for Rust, Python, and TypeScript, not for Go due to framework limitations as discussed in Section 3. 5.1 RQ1: No model dominates, and capability is decoupled from cost Across the model panel, no single LLM leads on every metric. Within each settingĂlanguage cell, the best model on line coverage is rarely the best on crashes, and the four models are close on unique-fault discovery (Gemini 27, Claude 24, GPT-5.5 18, and DeepSeek 16 unique valid faults). The same non-dominance holds on coverage: broken down by model (Table 5),Gemini-3.5-Flash andGPT-5.5lead line coverage in every setting while the costliest model,Claude-Opus-4.8, tops none. What separates the models sharply is token cost, not capability (Table 3). Under the same 600 s budget, the models differ by an order of magnitude in tokens consumed and in how many steps they manage to issue.Claude-Opus-4.8spendsâŒ150k tokens/run overâŒ53 model calls, while GPT-5.5reaches a comparable outcome withâŒ40k tokens, andDeepSeek-V4-Prowith onlyâŒ24k. Because the budget is fixed in time, slow or verbose models take fewer steps before the clock expires, which, as RQ3 shows, is the dominant driver of measured effectiveness. Table 3: Per-model cost under the equalized 600 s budget (LLM settings, weighted by runs) ModelTokens/runCalls/runTok/call Claude-Opus-4.8150,28952.72,853 Gemini-3.5-Flash98,14651.21,916 GPT-5.539,77717.52,274 DeepSeek-V4-Pro23,50513.01,813 5.2 RQ2: Coverage and crash-finding by setting Coverage. Table 4 summarizes per-app union line coverage (the union of lines covered across an appâs runs) on the usable line-coverage subset, together with per-run token cost and steps. Within that scope,LLM-guided-derivedleads on line coverage (30.4% vs. 26â28% for the other arms). The gain is concentrated in getting input-hungry apps to start at all. Plain arms reach 0% on these apps that exit immediately without arguments or fixtures, and input derivation rescues them. Smarter per-step 9 reasoning alone (LLM-guided vs. random) yields little additional coverage. Randomâs 28.4% is competitive with both non-derived LLM arms despite issuing only model-free input. The determining factor is input derivation, not action selection. Table 4: Coverage and cost by setting. Line % is per-app union coverage on the usable line-coverage subset; tokens are per LLM run. SettingLine %Tokens/runSteps random28.4â784 LLM-guided26.473,00535 LLM-guided-derived30.475,98836 LLM-generated26.694,37130 Coverage by model.Table 5 breaks the same per-app union coverage down by model within each LLM setting, alongside the per-run token cost. Two patterns hold across all three settings. First, capability does not track cost:Gemini-3.5-FlashandGPT-5.5reach the highest line coverage, yetGPT-5.5does so at roughly a quarter ofClaude-Opus-4.8âs token spend andClaudenever leads a single cell despite being the most expensive. Second, input derivation helps every model: each of the four gains 3â8 points moving fromLLM-guidedtoLLM-guided-derived, confirming that the derived-input effect is a property of the setting rather than of one strong model. Widget coverage. Alongside line coverage we collected the interface-level widget-coverage. There, widget coverage is flat across settings: 33.3% (random), 34.2% (LLM-guided), 37.1% (LLM-guided-derived), and 35.1% (LLM-generated). The ordering echoes line coverage, with input derivation improving coverage up a few points and action-selection strategy otherwise making little difference. Table 5: Line coverage by model within each LLM setting. Line % is per-app union coverage on the usable line-coverage subset; tokens are per run. The best model in each setting is bold. SettingModelLine %Tokens/run LLM-guidedClaude-Opus-4.825.2130,162 Gemini-3.5-Flash28.291,149 GPT-5.528.532,967 DeepSeek-V4-Pro23.226,521 LLM-guided-derivedClaude-Opus-4.832.1140,445 Gemini-3.5-Flash33.791,376 GPT-5.530.735,323 DeepSeek-V4-Pro26.125,989 LLM-generatedClaude-Opus-4.825.8169,283 Gemini-3.5-Flash29.6102,757 GPT-5.527.652,045 DeepSeek-V4-Pro22.813,149 Raw crash counts are 82% noise. Driving 197 apps produced 1,033 crash events, but a content- aware classifier over the post-termination terminal screen, followed by per-case adjudication of every ambiguous exit (Section 3), reduces these to 179 valid TUI-level faults (17.3%) over 45 apps (Table 6). 7 Noticeable crash sources include apps that printed a usage message and never entered their TUI (156) and graceful Ctrl+C exits (105). The ambiguous middle contained no reservoir of real bugs. Of 233 undecidable exits, adjudication identified only 4 genuine faults and 218 startup/environment failures. Exit-non-zero is not a crash oracle for this domain. Random wins per run, guidance wins per keystroke. On valid faults, random has the high- est per-run yield (15.2%), 8 ahead ofLLM-guided-derived(9.0%),LLM-guided(8.1%), and 7 Five events remain undecidable as they exited without any error message or traceback. 8 Per-run yield is the fraction of runs (one run = one(app, setting, model, seed)session under the fixed 600 s budget) that surface at least one valid fault; a run that crashes several times still counts once. Randomâs 15.2% is 30 of its 197 runs. 10 Table 6: Crash-event triage (1,033 events) BucketEventsShare Valid TUI-level fault17917.3% Noise (Go cov-emit, usage exit, SIGINT, . . . )84982.2% Undecidable (Non-crash fault)50.5% Raw exits are mostly noise 1,033 raw crash events V alid TUI faults: 179 Noise: 849 Undecidable: 5 Figure 6: Most raw non-zero exits are not TUI faults. The rendered-screen oracle reduces 1,033 raw crash events to 179 valid TUI-level faults. LLM-generated(4.1%). This lead is, however, a pure throughput artifact. Normalizing by inter- action reverses the ranking completely (Table 7). Per 1,000 steps, the guided LLMs areâŒ13Ămore crash-efficient than random. Random attains the higher per-run yield only because it issuesâŒ24Ă more inputs per run. It spends the entire 600 s emitting keystrokes (median 253 steps), whereas the LLM arms spend most of their wall-clock time awaiting API round-trips and take only a dozen steps. Random is not a better explorer. It is a faster one. Table 7: Valid crashes per run vs. per unit of interaction SettingMed. stepsYield/runValid / 1k steps random25315.2%0.19 LLM-guided128.1%2.41 LLM-guided-derived139.0%2.57 LLM-generated64.1%1.42 11 In addition, truncating every arm to a matched step budget, atk †12steps (the LLM armsâ median)LLM-guided(5.3%) andLLM-guided-derived(6.4%) both exceed random (3.0%), which overtakes only once granted its full-throughput keystroke budget. This confirms that exploration assisted by the LLM wins per keystroke. Throughput vs. crash efficiency 0 1 2 3 51050100250 random llm-guided-derived llm-generated Median steps per run (log scale) Valid faults per 1,000 steps Circle area encodes per-run yield llm-guided Figure 7: The crash result is a throughputâefficiency trade-off. Random issues far more inputs within the fixed time budget, while LLM-guided settings find valid faults much more efficiently per interaction. Three factors explain the picture: (i) Crashes are shallow. The median valid crash is reached in just 5 steps (115 of 179 within 15 steps), so a few hundred blind keystrokes stumble onto shallow bugs without guidance, and the guidance premium would only show on deep, state-gated bugs, of which this corpus has few. (i) The strategies are complementary, not ranked. Some crashing apps are reached only by an LLM arm (15 apps; e.g. a Rust panic intwitch-tuithat fires only after completing an interactive OAuth wizard, and faults inposting,frogmouth, andBagelsthat need a filled form or a valid query), while 10 are random-only (e.g. akmonsegfault reached by brute volume). (i) LLM attrition shrinks the denominator, not the hit-rate. 71â83 runs per LLM setting never executed (API failures, timeouts), andLLM-generatedquits within 5 steps in half of its non-crash runs (premature âtask doneâ), so its 4.1% understates the explorer. Under a fixed time budget with shallow targets, random finds the most crashes, but per unit of interaction guidance is an order of magnitude more efficient and uniquely reaches input-gated bugs. The right design is therefore a hybrid that spends LLM calls to unlock state and random throughput to storm it. 5.3 Fault characteristics The 179 valid faults collapse to 47 unique fault signatures over 45 affected applications. Python has the highest fault density (20.8 per 100 runs, vs. Rust 6.5, TS 8.6, Go 2.2), largely because Textual renders clean, attributable tracebacks (ScreenStackError,MarkupError,AttributeError). Its failures are legible, not necessarily more frequent. Rust contributes the most distinct panics and the only fatal signals (two SIGSEGVs), the highest-severity crashes in the set. Faults cluster on a 12 few primitive triggers, namely Enter (37), ArrowDown (14),?(14), Tab (11), and Escape (11), with the help overlay (?) and field submission (Enter) disproportionately fatal. Opening a help pane or committing a field is where unhandled state lives. 5.4 RQ3: Coverage decouples from fault-finding The settingLLM-guidedfound many crashes yet does not lead on line coverage, and random leads on neither coverage nor per-step crashes yet attains the highest per-run crash yield. Within the usable line-coverage subset, the coverageâfault relationship is therefore essentially flat, contrary to the conventional expectation that more line coverage should imply more faults found. This pattern is consistent with a session-truncation confound, in which a crash terminates the episode and caps the coverage that can accrue afterward, so that the runs which expose faults are penalized on coverage. Together with the observation that valid TUI crashes are shallow (median depth 5 steps) and broadly reachable, this implies that line coverage is a poor proxy for crash-finding in this TUI setting. The result echoes prior cautions that coverage and test effectiveness need not align [7,14] and motivates a TUI-specific criterion keyed on reachable interactive states rather than executed lines. 6 Discussion Terminal user interfaces sit at the heart of widely used software, from coding assistants and system- management tools to database and network clients, yet to the best of our knowledge their automated testing has received almost no systematic study. We therefore take the first step toward closing that gap, and our results point to several directions for TUI testing research. The central lesson is that the testing problem is not only an action-selection problem. It is also a benchmark-design, measurement, launch-configuration, and oracle-design problem. Metric design is an important research topic. The completed run shows that raw exit counts and raw coverage numbers are easy to misread. Most non-zero exits are not valid TUI faults, and coverage availability differs sharply across runtime instrumentation paths, and even obtaining reliable line coverage for Go took substantial engineering effort. This difficulty is structural: whereas mobile GUI testing targets just two platforms (Android and iOS) with a small set of languages, TUI development is fragmented across many frameworks written in different languages, so a cross-framework study must solve instrumentation and measurement anew for each language rather than once. Future TUI benchmarks should therefore treat measurement validity as a first-class design goal: report valid faults rather than raw exits, and normalize crash-finding by interaction as well as by run. Crash-finding is a tractable first target. This paper focuses on failures that are visible from the terminal session: tracebacks, panics, fatal signals, and abnormal terminations with rendered evidence. That scope gives the field a concrete starting point. It makes automated triage possible and exposes real faults in widely used applications. The next step is to expand the oracle beyond crashes, toward visual misrenderings, incorrect state transitions, performance stalls, and usability defects. Doing so will require richer ground truth than exit status or stack traces, likely combining replayable interaction logs, screen-state differencing, and application-specific assertions. Hybrid exploration can be a practical path. Under a fixed wall-clock budget, random exploration remains strong because it emits many more inputs than an LLM-guided policy. At the same time, LLM guidance is much more efficient per interaction and reaches input-gated faults that random input misses. This argues against treating random and LLM-guided testing as mutually exclusive baselines. A practical TUI tester should use LLMs where semantic reasoning matters: deriving launch arguments, creating fixtures, filling forms, and unlocking states. Once the application is in a meaningful state, high-throughput exploration can still be valuable for stressing shallow or brittle transitions. Launch derivation deserves more attention. A recurring source of low coverage and false crashes is not poor navigation but failure to start the TUI in a meaningful mode. Many applications require files, command-line arguments, credentials, databases, or setup state before their interface appears. Thellm-guided-derivedsetting shows that deriving these launch inputs can matter more than choosing the next keypress. Future work should study launch configuration as its own subproblem, with benchmarks that measure whether a testing system can infer required fixtures and enter the interactive UI before exploration begins. 13 Multimodal models are a natural next step. Our LLM-guided arms reason from textual rep- resentations of the terminal state. Modern multimodal models could instead consume rendered screenshots directly, potentially using layout, styling, alignment, and visual salience that are lost or flattened in a text-grid representation. Screenshot-based policies may be especially useful for TUIs whose state is encoded through color, highlighting, tables, charts, or spatial grouping. They should, however, be evaluated against the same baselines used here: a fixed wall-clock budget, per-interaction normalization, random throughput, and a content-aware crash oracle. The benchmark is a starting point. The 197 applications in this study form a practical, runnable, cross-framework slice of modern TUIs. They are not a complete census of terminal software. Future benchmark extensions should add other ecosystems, include applications that need controlled external services or accounts, and revive the historical-crash benchmark once buggy versions can be rebuilt and replayed systematically. The current benchmark establishes the measurement foundation on which those broader fault and correctness studies can build. 7 Threats to Validity Construct validity. Our main construct validity includes coverage, interaction efficiency, and valid TUI-level crashes. Line coverage is an imperfect proxy for TUI behaviour because many terminal- state changes occur through framework rendering code or data-dependent UI state that may not map cleanly to newly executed application lines. Widget coverage is closer to the interface layer but the order of widgets being executed may matter more. Crash validity is also non-trivial: raw non-zero exits include usage errors, graceful interrupts, harness failures, and coverage artifacts. We mitigate this by classifying crashes from the rendered terminal screen and manually adjudicating ambiguous cases, with two of the authors independently labeling each ambiguous event and resolving disagreements by discussion until they reach agreement, but some valid faults may still be missed if they leave no recognizable on-screen evidence. Internal validity. The fixed wall-clock budget introduces a throughput confound. Random explo- ration issues hundreds of inputs per run, while LLM-guided exploration often issues only a dozen because of LLM reasoning time. We address this by reporting both per-run yield and valid faults per 1,000 steps. External validity. The benchmark targets four dominant frameworks: ratatui, bubbletea, textual, and ink. Results may not transfer to TUIs built with other frameworks and programming languages, and custom terminal engines. The corpus is also limited to apps that build and run headlessly in Docker. Applications that require live network services, user credentials, hardware, desktop integration, or unusual terminal features are underrepresented and hard to be included. 8 Related Work Testing terminal user interfaces. Despite the prevalence of TUI applications, automated testing of the terminal interface itself has received almost no systematic study. The testing support that exists is framework-native and practitioner-built rather than the subject of published research. Each major framework ships a unit-level harness that renders the interface into an in-memory buffer and compares it against a stored frame, including the ratatuiTestBackend[15], the bubbletea teatest model harness [3], the textual run_test Pilot API [21], and the ink-testing-library lastFrameprobe [5]. End-to-end drivers that operate a real pseudo-terminal exist at the tooling level [13]. These harnesses share two limitations that our study documents quantitatively. They are exercised by hand-written tests whose coverage of the interface is shallow (Section 2), and they assume a clean process exit, which an interactive application under exploration does not provide (Section 3). No prior work establishes a cross-framework benchmark, an instrumentation method that recovers coverage from a terminated session, or an empirical account of how automated techniques perform on TUIs. This paper supplies all three, and in doing so treats TUI testing as a research problem in its own right rather than a downstream application of GUI or CLI methods. Automated GUI exploration. The body of work nearest to our exploration component is automated testing of graphical applications, where a driver issues input events and observes the resulting interface state. Random and model-free strategies set the baseline. Dynodroid generates input events through a guided observe-select-execute loop [11], and the random âmonkeyâ tester remains the 14 reference point that more elaborate techniques are measured against. Choudhary et al. ran exactly that comparison and found that none of the surveyed academic tools reliably beat random exploration on coverage [4], a result that anticipates our own. Later work moved to model-based and search-based drivers, with Stoat building a stochastic model of the interface [20], Sapienz casting test generation as multi-objective search [12], and reinforcement learning treating exploration as a sequential decision problem [16]. Every one of these techniques relies on the addressability a GUI provides, namely a widget tree with stable identifiers, roles, and bounding boxes against which an action can be aimed and a result queried. A TUI exposes none of this. Its observable state is anH Ă Wgrid of styled character cells, and its input is a raw byte stream, so the perception and action interfaces these methods assume do not exist. We retain the random baseline these papers established and ask, for the first time, how it and its learned competitors behave when the only observation is a rendered screen. LLM-driven testing. A second line of work replaces hand-built exploration policies with large language models. For mobile GUIs, Liu et al. prompt an LLM to choose functionality-aware actions during testing [8] and to synthesise unusual text inputs that trigger crashes [9]. For library and unit testing, LLMs have been used as zero-shot fuzzers of deep-learning frameworks [6] and as generators of unit test suites [17]. These studies report that LLM guidance improves over weaker baselines on their respective targets, but each operates where the model can read structured artefacts such as an accessibility tree, a method signature, or library documentation. None addresses an application whose entire interface is a character grid driven by raw key events, and none reports the throughput cost that a per-step LLM call imposes under a fixed wall-clock budget. We evaluate LLM-guided exploration, LLM-derived launch inputs, and LLM-generated test scenarios on TUIs under a time-equalised budget, and we find that the per-keystroke advantage of guidance is real but is repeatedly outrun by the sheer throughput of random input, a trade-off prior LLM-testing work does not surface. Coverage and fault-finding. Whether code coverage predicts a test suiteâs ability to find faults is a long-running question, and the evidence is mixed even within a single domain. Inozemtseva and Holmes report that coverage is not strongly correlated with suite effectiveness once suite size is controlled [7], and Papadakis et al. qualify the use of mutants as fault proxies at scale [14]. In GUI testing specifically, coverage and fault detection have been found to move together more often [19,23], and coverage continues to serve as the default proxy in fuzzer benchmarking [2] and fault localisation [18]. Our results place TUIs on the sceptical side of this debate and explain why. A crash truncates the exploration session that found it, capping the coverage that run can accrue, so the very executions that expose faults are penalised on coverage. The decoupling we observe is therefore not noise but a structural property of interactive, crash-terminated testing, which is precisely the regime a TUI tester operates in, and it motivates a coverage criterion keyed on reachable interactive states rather than executed lines. Empirical âhow far are weâ studies. Our paper follows the empirical tradition that measures the real gap between a techniqueâs promise and its delivered behaviour rather than proposing a new tool. Choudhary et al. asked whether automated Android input generation had arrived and answered with a sober comparison against random [4]; Lukasczyk et al. subjected automated unit-test generation for Python to the same scrutiny [10]; and Tian et al. assessed how far a frontier LLM actually goes as a programming assistant [22]. Each of these studies derives its value from honest baselines, careful confound analysis, and an open artefact, and recent guidance for LLM-based software-engineering experiments codifies exactly these practices [1]. No such study exists for TUIs, because there has been no benchmark on which to run one. We provide that benchmark, hold the LLM arms to a model-free random baseline and a content-aware crash oracle, and report where automated TUI testing stands today. 9 Conclusion Terminal User Interfaces are a large and growing class of software that is poorly tested in practice and lacks a dedicated testing methodology. We presented the first open, multi-language empirical study of automated TUI testing, over a benchmark of 197 headlessly-runnable, coverage-instrumented applications across the four dominant frameworks, driven by four frontier LLMs and a random baseline under an equalized wall-clock budget. No single model dominates, and capability is decoupled from cost. The largest gain comes from automatically deriving launch inputs, not from smarter per-step reasoning; and while random looks 15 competitive under a time budget, per interaction LLM guidance is an order of magnitude more efficient and uniquely reaches input-gated faults, arguing for hybrid strategies. Measurement is itself a research problem: raw exits are 82% noise, and once an oracle isolates the 179 valid faults, line coverage proves a poor proxy for crash-finding. Automated TUI testing is feasible but far from solved, and we hope this study establishes it as a research area in its own right. We release the cross-language coverage tooltuicovathttps://github.com/tui-testing/tuicovand the automated TUI testing framework tuibot at https://github.com/tui-testing/tuibot. References [1] Sebastian Baltes, Florian Angermeir, Chetan Arora, Marvin Muñoz BarĂłn, Chunyang Chen, Lukas Böhme, Fabio Calefato, Neil A. Ernst, Davide Falessi, Brian Fitzgerald, Davide Fucci, Marcos Kalinowski, Stefano Lambiase, Daniel Russo, Mircea Lungu, Lutz Prechelt, Paul Ralph, Christoph Treude, and Stefan Wagner. Guidelines for empirical studies in software engineering involving large language models. arXiv preprint arXiv:2508.15503, 2025. [2]Marcel Böhme, LĂĄszlĂł Szekeres, and Jonathan Metzman. On the reliability of coverage- based fuzzer benchmarking. In Proceedings of the 44th International Conference on Software Engineering (ICSE), pages 1621â1633. ACM, 2022. doi: 10.1145/3510003.3510230. [3] Charm.Bubble tea:A powerful little TUI framework.https://github.com/ charmbracelet/bubbletea, 2024. Accessed: 2026-06-24. [4]Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. Automated test input generation for Android: Are we there yet? In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 429â440. IEEE, 2015. doi: 10.1109/ASE.2015.89. [5] Vadim Demedes. Ink: React for interactive command-line apps.https://github.com/ vadimdemedes/ink, 2024. Accessed: 2026-06-24. [6]Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 423â435. ACM, 2023. doi: 10.1145/3597926.3598067. [7] Laura Inozemtseva and Reid Holmes. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th International Conference on Software Engineering (ICSE), pages 435â445. ACM, 2014. doi: 10.1145/2568225.2568271. [8]Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. Make LLM a testing expert: Bringing human-like interaction to mobile GUI testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), pages 1â13. ACM, 2024. doi: 10.1145/3597503.3639180. [9]Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Zhilin Tian, Yuekai Huang, Jun Hu, and Qing Wang. Testing the limits: Unusual text inputs generation for mobile app crash detection with large language model. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE), pages 1â12. ACM, 2024. doi: 10.1145/3597503.3639118. [10]Stephan Lukasczyk, Florian KroiĂ, and Gordon Fraser. An empirical study of automated unit test generation for Python. Empirical Software Engineering, 28(2), 2023. doi: 10.1007/ s10664-022-10248-w. [11]Aravind Machiry, Rohan Tahiliani, and Mayur Naik. Dynodroid: An input generation system for Android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pages 224â234. ACM, 2013. doi: 10.1145/2491411.2491450. [12] Ke Mao, Mark Harman, and Yue Jia. Sapienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA), pages 94â105. ACM, 2016. doi: 10.1145/2931037.2931054. 16 [13]Microsoft. TUI Test: End-to-end terminal testing framework.https://github.com/ microsoft/tui-test, 2024. Accessed: 2026-06-24. [14]Mike Papadakis, Donghwan Shin, Shin Yoo, and Doo-Hwan Bae. Are mutation scores correlated with real fault detection? a large scale empirical study on the relationship between mutants and real faults. In Proceedings of the 40th International Conference on Software Engineering (ICSE), pages 537â548. ACM, 2018. doi: 10.1145/3180155.3180183. [15]Ratatui Developers. Ratatui: A Rust library to build rich terminal user interfaces.https: //ratatui.rs, 2024. Accessed: 2026-06-24. [16]Andrea Romdhana, Alessio Merlo, Mariano Ceccato, and Paolo Tonella. Deep reinforcement learning for black-box testing of Android apps. ACM Transactions on Software Engineering and Methodology, 31(4):1â29, 2022. doi: 10.1145/3502868. [17] Max SchĂ€fer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 50(1):85â105, 2024. doi: 10.1109/TSE.2023.3334955. [18]AndrĂ© Silva, Matias Martinez, Benjamin Danglot, Davide Ginelli, and Martin Monperrus. FLACOCO: Fault localization for Java based on industry-grade coverage. arXiv preprint arXiv:2111.12513, 2021. [19]Jaymie Strecker and Atif Memon. Relationships between test suites, faults, and fault detection in GUI testing. In 2008 1st International Conference on Software Testing, Verification, and Validation (ICST), pages 12â21. IEEE, 2008. doi: 10.1109/ICST.2008.26. [20]Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), pages 245â256. ACM, 2017. doi: 10.1145/3106237.3106298. [21]Textualize. Textual: A Rapid Application Development framework for Python.https: //textual.textualize.io, 2024. Accessed: 2026-06-24. [22]Haoye Tian, Weiqi Lu, Tsz On Li, Xunzhu Tang, Shing-Chi Cheung, Jacques Klein, and TegawendĂ© F. BissyandĂ©. Is ChatGPT the ultimate programming assistant â how far is it? arXiv preprint arXiv:2304.11938, 2023. [23] Xun Yuan, Myra B. Cohen, and Atif M. Memon. GUI interaction testing: Incorporating event context. IEEE Transactions on Software Engineering, 37(4):559â574, 2011. doi: 10.1109/TSE. 2010.50. 17