Paper deep dive
CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
Marco De Luca, Tiziano Santilli, Domenico Amalfitano, Anna Rita Fasolino, Patrizio Pelliccione
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/10/2026, 4:29:01 AM
Summary
CIAO (Code In Architecture Out) is an automated process that leverages Large Language Models (LLMs) to generate system-level architectural documentation from GitHub repositories. By utilizing a template derived from ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model, CIAO produces structured, developer-oriented documentation. Evaluation with 22 developers indicates that the generated documentation is perceived as valuable, comprehensible, and accurate, though it faces challenges in diagram quality and high-level context modeling.
Entities (6)
Relation Signals (3)
CIAO â usestemplate â ISO/IEC/IEEE 42010
confidence 100% ¡ The process... produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010
CIAO â usestool â REPOMIX
confidence 100% ¡ The repository is converted into a single textual artifact using REPOMIX
CIAO â usesmodel â GPT-5
confidence 95% ¡ Based on these observations, GPT-5 was selected as the default model in CIAO
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local artifacts rather than producing coherent, system-level architectural descriptions. This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models. The process, called CIAO (Code In Architecture Out), defines an LLM-based workflow that takes a repository as input and produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010, SEI Views \& Beyond, and the C4 model. The resulting documentation can be directly added to the target repository. We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to. The evaluation shows that developers generally perceive the produced documentation as valuable, comprehensible, and broadly accurate with respect to the source code, while also highlighting limitations in diagram quality, high-level context modeling, and deployment views. We also assessed the operational cost of the process, finding that generating a complete architectural document requires only a few minutes and is inexpensive to run. Overall, the results indicate that a structured, standards-oriented approach can effectively guide LLMs in producing system-level architectural documentation that is both usable and cost-effective.
Tags
Links
- Source: https://arxiv.org/abs/2604.08293v1
- Canonical: https://arxiv.org/abs/2604.08293v1
Trouble viewing inline? Open PDF directly â
Full Text
70,049 characters extracted from source content.
Expand or collapse full text
CIAO - CODE IN ARCHITECTURE OUT - Automated Software Architecture Documentation with Large Language Models Marco De Luca University of Naples Federico I Naples, Italy marco.deluca2@unina.it Tiziano Santilli University of Southern Denmark Odense, Denmark tisa@mmmi.sdu.dk Domenico Amalfitano University of Naples Federico I Naples, Italy domenico.amalfitano@unina.it Anna Rita Fasolino University of Naples Federico I Naples, Italy fasolino@unina.it Patrizio Pelliccione Gran Sasso Science Institute LâAquila, Italy patrizio.pelliccione@gssi.it AbstractâSoftware architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local artifacts rather than producing coherent, system-level architectural descriptions. This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models. The process, called CIAO (Code In Architecture Out), defines an LLM-based work- flow that takes a repository as input and produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model. The resulting documentation can be directly added to the target repository. We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to. The evaluation shows that developers generally perceive the produced documentation as valuable, comprehensible, and broadly accurate with respect to the source code, while also highlighting limitations in diagram quality, high-level context modeling, and deployment views. We also assessed the operational cost of the process, finding that generating a complete architectural document requires only a few minutes and is inexpensive to run. Overall, the results indicate that a structured, standards-oriented approach can effectively guide LLMs in producing system-level architectural documentation that is both usable and cost-effective. Index TermsâSoftware Architecture Documentation, Archi- tecture Recovery, Large Language Models, ChatGPT, GitHub Repositories, Automated Documentation I. INTRODUCTION Software architecture documentation plays a central role in supporting system comprehension, communication, and long-term evolution. When documentation is missing, out- dated, or inconsistent with the implementation, developers struggle to understand system decomposition, responsibili- ties, and dependencies, often leading to architectural drift, erosion, and ultimately architectural technical debt [1]â[4]. Despite its recognized importance, architectural documenta- tion is frequently incomplete or informal in industrial and open-source projects, where time pressure and code-centric practices make its production difficult to sustain. Established standards provide guidance on how architectural information should be documented. ISO/IEC/IEEE 42010 defines core concepts and the relationship between architecture descriptions and stakeholder concerns, emphasizing that documentation should explicitly address the needs of its intended readers [5]. SEIâs Views & Beyond framework promotes a view-based, stakeholder-oriented approach in which different views capture complementary architectural structures [6]. In line with these principles, we focus on system-level architectural documenta- tion aimed at developers who require a consolidated, high-level understanding of a repository. Recent years have seen the rapid adoption of Large Lan- guage Models (LLMs) in software engineering, with growing evidence of their usefulness in tasks such as program com- prehension, code summarization, documentation generation, and automated repair [7]â[10]. Industrial analyses, such as GitHubâs Octoverse report, similarly highlight the increasing reliance on LLM-based assistants to help developers nav- igate large codebases [11]. Research has explored LLMs for generating fine-grained documentation artifacts, such as API descriptions [12], code explanations [13], test-case sum- maries [14], and requirement-like statements extracted from source code [15]. Other works explored LLMs for supporting higher-level reasoning, such as extracting UML diagrams, identifying design patterns, or reconstructing domain models from code [16]â[19]. These contributions highlight the po- tential of LLMs to support documentation and architecture recovery tasks. However, most approaches focus on specific artifacts or narrow tasks. The use of LLMs to produce system-level architectural documentation directly from reposi- tories, following established documentation standards, remains largely unexplored. arXiv:2604.08293v1 [cs.SE] 9 Apr 2026 This paper addresses this gap by introducing CIAO (Code In Architecture Out), a structured process for automatically generating system-level architectural documentation from GitHub repositories using LLMs. CIAO defines a workflow that takes a repository as input and produces an architectural documentation following a template, which has been defined by taking inspiration from ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model [20]. The final output is an architec- tural documentation, which can be directly integrated into the target repository as a README file. Our study evaluates CIAO with 22 developers who analyzed the documentation gener- ated for repositories they contributed to. The questionnaire addresses perceived value (RQ1), comprehensibility (RQ2), accuracy (RQ3), and identified limitations (RQ4). We also measure the generation time and computational cost (RQ5). The main contributions of this work are as follows: ⢠A standards-oriented template for system-level architec- tural documentation, based on ISO/IEC/IEEE 42010, SEI Views & Beyond, and the C4 model. ⢠A structured LLM-based workflow that generates archi- tectural documentation directly from GitHub repositories. ⢠An open-source prototype implementing the proposed workflow, capable of producing ready-to-use architectural documentation that can be directly integrated into the target repository. The remainder of this paper is organized as follows. Sec- tion I reviews related work. Section I presents our doc- umentation process. Section IV describes the experimental evaluation, and Section V reports the results. Section VI discusses the threats to validity. Finally, Section VII concludes the paper and outlines future work. The supplementary material, which includes the CIAO implementation, the generated documentation, and the full survey structure and results, is available at the following link: https://doi.org/10.5281/zenodo.18710540 I. RELATED WORK Reverse engineering comprises techniques for reconstruct- ing the structure, behavior, and design intent of software sys- tems from low-level artifacts such as source code and execu- tion traces [21]. It is particularly valuable when documentation is missing or outdated, helping developers regain understand- ing of system decomposition, component relationships, and design decisions [22]. A central branch of this field is Software Architecture Recovery (SAR), which focuses on rebuilding the architectural organization of a system by identifying its main components, dependencies, and structural patterns [23], [24]. SAR is particularly relevant when architectural drift or erosion causes the implemented architecture to diverge from the intended one [1], [2], [4], [25]â[27]. When these inconsistencies are not addressed in architectural descriptions, they become architectural debt [3]. Existing SAR approaches include static analysis, dynamic analysis, and techniques using ranking, clustering, or machine learning to infer architectural boundaries [28]â[30]. Across these techniques, the goal is to provide maintainers with a coherent architectural view that is otherwise implicit in large codebases. The rapid adoption of LLMs in software engineering has led to multiple documentation-related approaches that vary in input artifacts, abstraction level, and produced outputs. LLMs have been used for summarization and automated repair [9], [10], [31]â[33], with prompt-driven interaction enabling flexible, natural-language guidance [34]. A first group of studies uses source code as input to derive natural- language documentation or higher-level descriptions. Applica- tions include extracting REST API endpoints for microservice systems [12], generating context-aware explanations during program comprehension [13], and producing requirement-like statements through the AUTOREQGEN pipeline [15]. Other contributions focus on code fragments, such as intent-oriented summaries [35] or explanations of legacy languages [36], and on test artifacts, where LLMs generate concise summaries to support navigation [14]. Recent studies have investigated whether LLMs can support higher-level reasoning, such as identifying design patterns, generating UML diagrams, or reconstructing domain models [17], [19], [37]. Hybrid ap- proaches combine static analysis with LLM reasoning to recover structural views [16], while other studies compare LLM-based extraction with traditional MDRE techniques [38]. MDRE-LLM leverages RAG techniques to recover domain models at different granularities [18]. Additional work has explored architectural reasoning from requirements or code in microservice settings [39]â[41]. LLMs have also been used to generate textual descriptions from UML Use Case Diagrams [42] or to support modeling tasks in educational contexts [43]. A third line of work explores how LLMs can enrich or review existing documentation. Examples include enhancing API documentation with usage examples [44], tailoring documentation to different stakeholders [45], and supporting documentation review processes [46]. A recent contribution closely related to our work is CodeDocs-GenAI [47], which uses RAG and LLM-based summarization to generate README documentation for Git repositories. Like CIAO, it analyses the entire repository, including its structure and source code, to generate structured documentation. However, its focus remains on general-purpose project overviews rather than system-level architectural docu- mentation grounded in established standards and guidelines. Overall, prior work shows that LLMs have been applied to many documentation-related tasks, mostly targeting specific artifacts such as APIs, code snippets, tests, requirements, or individual diagrams. By contrast, their use for full architecture recovery and end-to-end architectural documentation directly from source code remains largely unexplored. Our work ad- dresses this gap by operating at a higher level of granularity: starting from a repository, we generate comprehensive archi- tectural documentation across multiple views, stakeholders, and concerns within a unified template, extending the role of LLMs from local documentation support to system-level architectural documentation. I. THE DOCUMENTATION PROCESS In this section, we present the automated documentation process implemented in CIAO, which generates system-level architectural documentation from GitHub repositories using an LLM. The process is built around a system-level architectural template that is lightweight enough to be provided as input to the LLM while remaining consistent with established stan- dards and practitioner guidelines. Section I-A introduces the template and its expert validation, Section I-B describes the overall workflow, Section I-C details the prompt-engineering strategy, and Section I-D discusses the selection of the underlying LLM. A. The Proposed Template Software architecture documentation is commonly described as view-based [48]: the information to be presented and the appropriate level of abstraction should depend on stakeholdersâ concerns and the architectural perspectives to be supported. In this work, we focus on system-level architectural documen- tation aimed at developers who need a consolidated, high- level understanding of a repository to work with and evolve the system. The template must therefore remain aligned with established architectural practices while being lightweight and regular enough to be used as input to an LLM. Template Design. To determine which architectural infor- mation should be included and how it should be organized and presented, we adopted an expert-driven iterative design process. Three experts participated in this phase: one soft- ware engineering researcher with ten years of experience in architecture and documentation, and two industry software architects with five years of experience each. Through three separate focus groups, we elicited the architectural elements they deemed essential at system level. The sessions converged on a common set of needs: a clear definition of system scope and external context, a representation of deployable units and their responsibilities, an internal structural view of modules and their relationships, explicit links to code artifacts, and attention to cross-cutting concerns and quality-related rationale. The resulting synthesis was consolidated by two authors and validated by a third. These needs were then organized into a coherent structuredrawingonthreewell-establishedsources. ISO/IEC/IEEE 42010 [49] guided the inclusion of concepts such as system scope, stakeholder concerns, external interactions, and architectural rationale. SEIâs Views & Beyond [48] informed the separation between structural, behavioral, and deployment perspectives. The C4 model [50], aligned with the Architecture-as-Code paradigm [26], provided a four-level hierarchy, Context (L1), Container (L2), Component (L3), and Code (L4), that is widely used for structuring developer-oriented documentation [51]. A revised draft was then discussed in a second round of focus groups, and a final joint session with all experts was used to resolve remaining ambiguities and validate the structure. The resulting template comprises eight sections, each ad- dressing a distinct architectural concern and drawing consis- tently on ISO/IEC/IEEE 42010, SEIâs Views & Beyond, and the four abstraction levels of the C4 model (Context, Container, Component, Code): 1) System Overview. Provides an entry point to the archi- tectural description by summarizing the systemâs purpose, scope, and main responsibilities. It establishes the system- of-interest, in line with ISO 42010, and offers a concep- tual anchor for the architectural views that follow, helping readers quickly situate the repository before moving to more technical details. 2) Architectural Context. Characterizes the systemâs ex- ternal environment by identifying actors, interacting sys- tems, APIs, and data sources. By clarifying boundaries and external dependencies, this section supports ISOâs emphasis on contextualizing the architecture and corre- sponds to the C4 Level 1 Context view. It provides the background necessary to understand integration points and the role of the system within a broader ecosystem. 3) Containers. Describes the systemâs logical runtime or- ganization, following the notion of âcontainersâ in the C4 model, namely, the applications and data stores that must be running for the system to operate. For each container, the documentation summarizes responsibilities, exposed interfaces, key technologies, and interaction pat- terns. This section offers a coarse-grained view of how major building blocks collaborate at runtime, aligning with SEIâs Component-and-Connector perspective and ISOâs runtime-oriented concerns. 4) Components. Presents the internal logical structure of the system by identifying key modules, packages, or classes and the structural relationships among them. This sec- tion provides a technology-independent representation of architectural organization, complementing the container perspective by revealing how domain responsibilities are grouped and how subsystems interact. It aligns with SEIâs Module viewtype and corresponds to C4 Level 3. 5) Code-Level. Connects architectural abstractions to their concrete implementation by mapping components to source-code artifacts. It identifies relevant directories, files, entry points, and recurring design or architectural patterns. By making the realization of architectural ele- ments explicit, this section supports traceability to im- plementation as recommended by ISO 42010 and corre- sponds to C4 Level 4. 6) Cross-Cutting Concerns. Summarizes concerns that in- fluence multiple parts of the system, such as security, configuration, logging, testing, and monitoring, and de- scribes how they manifest in the codebase. These aspects are critical to understanding system behavior beyond structural decomposition and align with SEIâs âbeyond- the-viewsâ guidance and ISOâs focus on capturing stake- holder concerns that transcend individual elements. 7) Quality Attributes and Rationale. Highlights the qual- ity attributes (e.g., performance, maintainability, scala- bility, security) supported by the implementation and synthesizes the rationale inferred from observable design choices. This section implements ISO 42010âs recom- mendation to document architectural rationale and sup- ports readers in understanding why certain architectural decisions were made. 8) Deployment. Characterizes the systemâs operational in- frastructure by describing deployment artifacts (e.g., Dockerfiles and configuration files), execution environ- ments, storage and compute nodes, and their relation- ships. It shows how software elements map to physical, virtual, or containerized resources, aligning with the SEI Allocation perspective and C4 deployment practices, and clarifies how the system runs in practice and how runtime responsibilities are distributed. B. CIAO Workflow CIAO is implemented as a Python-based tool that automati- cally generates system-level architectural documentation from GitHub repositories using LLMs. Figure 1 summarizes the end-to-end workflow. The process takes two primary inputs: (i) the target GitHub repository, which provides the source artifacts to be analyzed, and (i) the documentation template, which defines the structure of the architectural document, the goal of each section, and the global writing guidelines. Based on these inputs, CIAO performs four automated steps: repository flattening, prompt generation, section generation, and final assembly with diagram rendering. Fig. 1. Overview of the CIAO automated documentation workflow. Repository Flattening: The repository is converted into a sin- gle textual artifact using REPOMIX [52], which aggregates the projectâs source code into one AI-friendly file. REPOMIX can be configured to apply different filters; in our case, we remove comments and exclude files or directories that do not contain source code (e.g., binaries, build outputs, large test datasets, generated documentation), while retaining essential configura- tion artifacts (such as Dockerfile, container-orchestration descriptors, and dependency manifests like package.json or pom.xml). We also enable the option to enrich the output with a textual description of the repository structure. The resulting Flattened Repository representation captures both the projectâs folder layout and the curated code base, and serves as input for the subsequent steps. Prompt Generation: For each documentation section, CIAO builds, in parallel, a composite prompt consisting of two parts: (i) a fixed global prompt, shared across all sections, and (i) a section-specific prompt, derived from the Documentation Template. Moreover, the Flattened Repository representation is appended to each prompt to ensure code-grounded generation. LLM-based Section Generation: For each prompt generated in the previous step, CIAO submits it to the LLM, processing all sections in parallel. This step produces the set of Generated Sections, each corresponding to a specific part of the template and adhering to its prescribed structure and constraints. Documentation Assembly: The Generated Sections produced in the previous step are assembled into a single document, referred to as the Intermediate Documentation. This yields a complete system-level architectural description that follows the structure defined by the template. Diagram Rendering: Since LLM-generated diagrams are pro- vided in textual PlantUML format [53], CIAO renders them as images and replaces the textual definitions accordingly. Applied to the Intermediate Documentation, this step produces the final Generated Documentation, in which all diagrams are available in visual form. C. Prompt Engineering CIAO employs a structured prompt-engineering strategy to maximize architectural accuracy, limit hallucinations, and ensure consistency across sections. Each prompt consists of (i) a global prompt, shared across all sections, which defines the LLMâs role, target audience, writing style, and grounding requirements; and (i) a section-specific prompt, instantiated from the documentation template and tailored to the goal and expected artifacts of that section. This separation reflects the fact that architectural documentation comprises heterogeneous elements that require different abstraction levels and extraction strategies. This design follows the task-decomposition prompt- ing strategy proposed by Liu et al. [54], guiding the LLM to address a sequence of smaller, section-specific subtasks rather than generating the entire documentation in a single step. CIAO further follows Liu et al.âs taxonomy [54] by adopt- ing both profile and instruction prompting, specifying who the model should act as and how it should perform the task. Along the profile dimension, role prompting [55] positions the model as a âMeticulous Software Architectâ, helping to stabilize tone, terminology, and stylistic coherence across sections. Additionally, light motivational cues [56] encourage careful reasoning. On the instruction side, prompts specify evidence- grounded generation requirements and explicitly forbid in- venting architectural elements not present in the repository. Selective few-shot examples [57] (e.g., small Markdown or PlantUML skeletons) provide structural scaffolding that guides formatting and level of detail. This combination supports co- herent, template-aligned, and verifiable architectural documen- tation. The complete prompt is available in the supplementary material. D. Model Selection To select the LLM used in CIAO, we conducted an exploratory pilot study comparing four state-of-the-art mod- els: GPT-5, Claude Sonnet 4.5, Gemini 2.5, and Mistral Large 2. For three representative repositories provided by developers familiar with the projects, we gen- erated documentation with each model and discussed the outputs in dedicated focus groups. The evaluation considered (i) accuracy of architectural elements and diagrams, (i) con- sistency of terminology and relationships across sections, (i) the presence of hallucinated or speculative content, and (iv) adherence to the template. Across models, GPT-5 consis- tently produced the most accurate, stable, and template-aligned documentation, particularly in sections requiring multi-level structural reasoning. It also yielded more syntactically correct PLANTUML diagrams and fewer speculative elements. Based on these observations, GPT-5 was selected as the default model in CIAO, although the tool remains configurable to support alternative costâquality trade-offs. IV. EXPERIMENTAL EVALUATION The goal of this study is to evaluate the effectiveness of the proposed LLM-based process in generating system-level architectural documentation directly from source code while adhering to our standards-oriented template. We assess the per- ceived value, comprehensibility, and accuracy of the generated documentation, as well as the limitations and missing aspects identified by practitioners. To this end, we conducted a survey- based evaluation in which software developers provided real- world repositories and reviewed the documentation produced by our tool. A direct comparison with related approaches was not feasible, as the only closely related work, i.e., CodeDocs- GenAI [47], does not provide an openly accessible implemen- tation. The validation aims to answer the following Research Questions (RQs): RQ1: Do developers perceive the documentation as valuable enough to be integrated into their own projects? RQ2: To what extent is the documentation comprehensible? RQ3: To what extent is the documentation accurate with respect to the systemâs source code? RQ4: What limitations or missing aspects do developers iden- tify in the documentation? RQ5: What are the costs of generating the documentation? Metrics. To answer our research questions, we rely on two complementary sets of measurements. For RQ1âRQ4, we base our evaluation on the metrics derived from a questionnaire that we designed for this study. The questionnaire includes both Likert-scale items and open-ended questions, and the corresponding items for each research question are described in the following sections. For RQ5, we assess the cost of the documentation-generation process by measuring two quantita- tive indicators: (i) the time required to generate the complete architectural documentation for each repository, and (i) the financial cost associated with the modelâs API usage. These measurements allow us to evaluate both the technical and economic impact of the process. Questionnaire Design. To address our research questions, we constructed a questionnaire composed of closed-ended items measured on a 5-point Likert scale ( 1=stronglydisagree(SD), 2=disagree(D),3=neutral(N),4=agree(A),5=strongly agree(SA)) and open-ended questions designed to elicit richer qualitative insights. The introductory section collects background information about each participant and their relationship with the evalu- ated repository. Specifically, it records job role, application domain (e.g., web application, automotive, healthcare), prior familiarity with the codebase, and whether the participant had previously authored or maintained any repository documenta- tion (e.g., READMEs, architectural descriptions, wiki pages, ADRs). When applicable, participants briefly described the type of documentation they had produced.This information helped contextualize their assessments and interpret perception differences across levels of experience and prior knowledge. The second part of the questionnaire is organized around the four research questions. Table I summarizes all items, including identifiers (Q1âQ30), wording, and type. Perceived value (RQ1) is examined through three Likert-scale items and one open-ended question (Q1âQ4), probing the usefulness and perceived contribution of the generated documentation. Comprehensibility (RQ2) is evaluated through four Likert- scale items and one open-ended question (Q5âQ9) focusing on clarity, structure, terminology, and redundancy. Accuracy and consistency (RQ3) are assessed using the largest set of items (Q10âQ27), which combine global questions on architectural correspondence, correctness of responsibilities and relation- ships, and internal consistency with section-level items aligned with the proposed template (e.g., Containers, Components, Code-Level, Use Case Diagram, and Deployment Diagram), followed by an open-ended prompt for additional comments. Limitations and missing aspects (RQ4) are captured through three open-ended questions (Q28âQ30) targeting structural or conceptual weaknesses, missing information, and potential improvements. The questionnaire was implemented in Google Forms, which was used to collect all participantsâ responses. Participants and Repositories. Participants were recruited through convenience sampling from the authorsâ professional and academic networks. A total of 22 developers agreed to take part in the study by contributing one of their software repositories and completing the survey after reviewing the automatically generated documentation. All participants had prior software development experience and were familiar with the repositories they submitted, ensuring knowledgeable and informed evaluations. The sample consists of 15 researchers, 5 PhD students, and 2 software developers. Most respondents reported substantial development experience: 17 indicated more than five years of practice, while only two reported one year or less. Participants also expressed strong familiarity with software architecture concepts, such as UML, architectural styles, and architectural patterns, with an average self-assessed score of 4.45/5. Regarding their involvement with the ana- lyzed repositories, 15 participants stated that they had authored or maintained documentation (e.g., READMEs, architecture descriptions, or user guides). Their free-text responses refer- enced onboarding materials, installation guides, usage docu- mentation, and high-level architectural overviews. Overall, the sample reflects practitioners and researchers with solid soft- TABLE I OVERVIEW OF ANALYZED REPOSITORIES WITH LANGUAGES AND CODE SIZE. RepoDomainLang. (LOC %)FilesLOC Group-Key-Phemapn [58]IoTC (17.0%), C++ (83.0%)91061 pyALS-RF-tmr [59]Machine LearningC (0.9%), C++ (0.3%), CMake (1.6%), JSON (0.7%), Python (95.8%), Shell (0.7%)859270 sa-bsn [60]HealthcareC (12.6%), C++ (75.5%), CMake (4.1%), Python (4.5%), Shell (0.5%), XML (2.9%)1759877 ocean-lib [61]Machine LearningPython (80%), YAML (20%)331208 robethichor [62]Robotic SystemsCMake (1.7%), Dockerfile (1.6%), JSON (3.7%), Python (88.1%), XML (4.9%)28697 Openjob [63]Web ApplicationDockerfile (0.6%), Java (65.3%), XML (32.3%), YAML (1.9%)1465386 DSP-Frontend [64]CybersecurityCSS (10.4%), JS (88.7%), XML (0.9%)447 238951 DSP-Backend [65]CybersecurityJS (100%)386870 QuFI [66]Quantum computing Python (100.0%)8851 BoMoDT [67]Digital TwinsCSS (0.1%), JSON (0.9%), Python (9.7%), XML (89.0%), YAML (0.4%)5550928 m2dt [68]Digital TwinsJSON (0.2%), Python (86.1%), XML (13.7%)364040 GymportalService [69]Web ApplicationJava (97.7%), XML (2.3%)352458 pyicub [70]Healthcare Robotics JSON (31.2%), Python (64.3%), Shell (2.3%), XML (1.2%), YAML (0.9%)1238441 treeco [71]Machine LearningPython (100.0%)616959 TestingRobotChallenge-T4 [72] Web ApplicationBatch (8%), Java (71%), Shell (11%), XML (7%), YAML (3%)481660 rse2025 [73]Web ApplicationJS (1.2%), YAML (98.8%)481 write2audiobook [74]Accessibility toolsPython (94.1%), YAML (5.9%)121080 BF-JITcompiled [75]JIT CompilerC (6.3%), C++ (93.7%)3668 visualnovelpyarcade [76]Visual novel engineJSON (7.0%), Python (93.0%)181269 SAM-CyFra [77]Network SecurityINI (0.1%), Makefile (0.1%), Python (91.5%), SQL (7.5%), XML (0.9%)1319796 SCASS [78]CybersecurityDockerfile (1.8%), JSON (33.8%), JS (0.2%), Makefile (0.1%), Perl (6.9%), Prolog (9.6%), Python (15.1%), Shell (1.5%), YAML (31.1%) 585088 OSINT [79]CybersecurityCSS (1.0%), Dockerfile (0.4%), Go (3.6%), JSON (71.6%), Java (7.0%), JS (0.7%), Python (11.7%), Shell (1.1%), TypeScript (1.2%), XML (0.3%), YAML (1.4%) 17223999 ware engineering backgrounds and architectural knowledge. Many of them have direct responsibility for the documentation of the evaluated repositories. Table I summarizes the 22 repositories contributed by participants, reporting for each project the main implementation languages, number of source files, and non-comment lines of code (LOC). The repositories exhibit substantial variability in size and technology stack: the number of files ranges from 3 to 447, and LOC from 81 to 238,951. The sample includes code bases predominantly writ- ten in Python, Java, C/C++, or JavaScript, often complemented by configuration and infrastructure artifacts such as YAML, JSON, XML, Dockerfiles, and shell scripts. This diversity indicates that the generated documentation was evaluated on both small and relatively large, multi-language repositories. Survey Execution and Data Analysis. Each participant provided a software repository of their choice, which was processed by our tool to generate the full architectural docu- mentation. Participants were then asked to review the gen- erated documentation and complete the questionnaire. No training phase was required, as the evaluation focused on the documentation itself rather than on the tool. Responses were collected individually. For the closed-ended questions, we grouped answers according to the Likert-scale values and computed their frequencies to describe the overall distribution of participantsâ perceptions. For the open-ended questions, we performed a thematic analysis [80] following standard quali- tative research practices: two authors independently coded all textual answers, iteratively identifying and refining recurring themes related to strengths, weaknesses, and missing aspects of the generated documentation. Discrepancies between coders were discussed and resolved with the involvement of a third author, leading to a consolidated set of themes and ensuring consistency in the interpretation. V. RESULTS In this section, we present the results of our empirical study by addressing each of the five research questions (RQ1âRQ5). For RQ1âRQ4, we report the quantitative findings from the questionnaire together with the qualitative insights derived from the open-ended responses, providing a comprehensive view of how developers evaluated the generated architectural documentation. For RQ5, we describe the time and cost required to automatically generate the documentation, char- acterizing the practicality of the proposed process. A. Answer to RQ1 Figure 2 shows the distribution of responses to Q1âQ4, indicating an overall positive perception of the LLM-generated architectural documentation. Q1, which probes whether the documentation provides valuable architectural insights, re- ceived predominantly positive ratings, with A(11) and SA(4), while a smaller group selected N(5) or D(2). A similar tendency appears in Q2, where respondents again expressed strong agreement, with A(9) and SA(8) accompanied by limited neutrality N(4) or disagreement D(1). This suggests that the documentation generally supports comprehension of the systemâs architectural structure. Perceptions of maturity (Q3) were more varied yet remained broadly positive, with A(9) and SA(5). A subset of respondents selected N(3) or D(5), indicating that some additional refinement or domain- specific adjustments may be needed before full adoption. The open-ended responses further contextualize these find- ings by showing which sections developers found most valu- able. The most recurrent theme was the usefulness of diagrams (13 mentions), which were appreciated for making structural relationships easier to understand, as reflected in comments such as âDiagrams [. . . ] visually clarify complex informa- tionâ and âClass Diagrams [. . . ] make it easier to navigate dependenciesâ. TABLE I OVERVIEW OF THE QUESTIONNAIRE STRUCTURE, QUESTION TYPES, AND MAPPING TO QUESTIONNAIRE PARTS. LEGEND: L = 5-POINT LIKERT SCALE; O = OPEN-ENDED QUESTION. IDQuestionType Part 1 â Perceived value (RQ1) Q1The LLM-generated architectural documentation provides valuable architectural insights about the system.L Q2The LLM-generated documentation is useful for understanding and maintaining the systemâs structure and dependencies.L Q3The LLM-generated documentation is valuable enough that I would consider using it in my own project.L Q4Which sections or features of the LLM-generated documentation were most or least valuable to you?O Part 2 â Comprehensibility (RQ2) Q5The LLM-generated architectural documentation is clear, well-structured, and easy to follow.L Q6The LLM-generated architectural documentation uses appropriate terminology for software architecture.L Q7The LLM-generated architectural documentation contains excessive redundancy or unnecessary information.L Q8The explanations within the documentation are sufficiently detailed.L Q9Which parts of the documentation were most or least comprehensible to you?O Part 3 â Accuracy and consistency (RQ3) Q10 The LLM-generated documentation accurately reflects the architecture implemented in the repository.L Q11 The documentation is internally consistent across sections and diagrams.L Q12 The content is complete and does not omit key architectural elements.L Q13 The relationships and dependencies among components are well capturedL Q14 The component responsibilities are correctly described in the documentationL Q15 I would use the documentation as a reliable reference for the system architecture.L Q16 Section 1: System Overview: The purpose, scope, and key features described are correctly derived from the repository content.L Q17 Section 2 Architectural Context: The external systems, APIs, data sources, and actors are correctly derived from the repository content.L Q18 Section 2.1 Use Case Diagram: The diagram correctly represents actors and use cases derived from the repository content.L Q19 Section 3 Containers: The described containers, technologies, and communication protocols match the repository content.L Q20 Section 3.1 Component Diagram: The diagram correctly reflects the containers and externals as found in the repository.L Q21 Section 4 Components: The identified components and interactions correspond to code-level relationships (imports, calls, dependencies).L Q22 Section 5 Code-Level: The entry points, key modules, and patterns correspond to actual code structure (no speculative elements).L Q23 Section 5.1 Code-Level Diagram: The diagram includes only elements and relationships explicitly present in the repositoryL Q24 Section 6 Cross-Cutting Concerns: The concerns listed (e.g., security, logging, configuration) are correctly represented.L Q25 Section 7 Quality Attributes & Rationale: The quality attributes and rationales are correctly represented (no speculative elements).L Q26 Section 8 Deployment: The deployment information (nodes, environments) accurately reflects the repositoryâs infrastructure.L Q27 Section 8.1 Deployment Diagram: The diagram includes only infrastructure elements and connections declared in the repository.L Part 4 â Limitations and missing aspects (RQ4) Q28 Please describe any inconsistencies, inaccuracies, or unclear parts you noticed in the documentation. Which sections need improvement?O Q29 What important architectural information is missing or insufficiently represented in the LLM-generated documentation?O Q30 What improvements or changes would make the LLM-generated documentation more useful and trustworthy for your project?O Q1 Q2 Q3 2 (9%) 1 (5%) 5 (23%) 5 (23%) 4 (18%) 3 (14%) 11 (50%) 9 (41%) 9 (41%) 4 (18%) 8 (36%) 5 (23%) SD (1)D (2)N (3)A (4)SA (5) Fig. 2. Distribution of participantsâ Likert-scale ratings for RQ1 A second prominent theme relates to the value attributed to the Components section (9 mentions), which several par- ticipants described as essential for understanding the systemâs internal organization and for supporting modification tasks. As noted in the responses, âWhen I need to find which components I need to modify, I would check it here firstâ and â[It] highlighted aspects of the codebase that I was not aware of.â Other sections were mentioned less frequently but still appreciated. The Deployment (3 mentions) was valued for clarifying runtime and execution contexts, as reflected in âThe deployment representation clearly identified dependenciesâ. Similarly, Cross-Cutting Concerns and Quality Attributes (4 mentions) were recognized for surfacing system-wide behav- iors and architectural considerations, with comments such as âCross-Cutting and Quality Attributes [. . . ] the list of APIsâ. Based on this evidence, the perceived value of the generated documentation can be summarized as follows: RQ1 Answer Developers generally evaluated the LLM-generated docu- mentation positively, finding it valuable for understanding system structure and, in many cases, mature enough for integration into their projects. Architectural diagrams, code- level details, and component views emerged as the most appreciated elements. B. Answer to RQ2 Figure 3 shows that the overall perception of comprehen- sibility is positive. Clarity and organization (Q5) received predominantly favorable evaluations, with A(7) and SA(4), while N(8) and a small number of negative ratings D(3) indi- cate that some parts of the documentation required additional effort to follow. Terminology (Q6) was viewed very positively, with A(9) and SA(7), and only limited N(4) or D(2). These results suggest that the architectural vocabulary used across the various sections, ranging from containers and components to code-level elements, was generally considered appropriate and consistent with standard architectural practice. Opinions on redundancy (Q7) were more evenly distributed. Several respondents noted the presence of repetitive content A(6), SA(4), while many remained neutral N(7), and fewer expressed disagreement D(4), SD(1). Overall, redundancy ap- pears present but not sufficiently prominent to hinder compre- hension. With respect to the level of detail (Q8), participants expressed positive perceptions, with A(12) and SA(6), and only limited N(3) or D(1). The documentation was considered Q5 Q6 Q7 Q8 1 (5%) 3 (14%) 2 (9%) 4 (18%) 1 (5%) 8 (36%) 4 (18%) 7 (32%) 3 (14%) 7 (32%) 9 (41%) 6 (27%) 12 (55%) 4 (18%) 7 (32%) 4 (18%) 6 (27%) SD (1)D (2)N (3)A (4)SA (5) Fig. 3. Distribution of participantsâ Likert-scale ratings for RQ2 sufficiently detailed to support understanding of architectural relationships, responsibilities, and interactions across the dif- ferent abstraction levels represented in the template. The open-ended responses highlight which parts of the documentation participants perceived as most comprehensible. The most frequently mentioned theme is the overall clarity of the documentation (4 mentions), with respondents noting that âcomprehensibility is good with all partsâ and âthe entire documentation is easily readable and understandableâ. A second recurrent theme concerns the clarity of high-level architectural sections (3 mentions), where system-level de- scriptions such as the Architectural Context and Containers were described as particularly easy to follow, as illustrated by comments like âSection 2 was the most comprehensibleâ and âSection 3 is the most comprehensible [. . . ] provides valuable informationâ. Among less frequent themes, some respondents pointed to Cross-Cutting Concerns and Quality Attributes (2 mentions) as especially clear, for example âCross-Cutting Concernsâ and âQuality Attributes and Rationaleâ. Finally, module-to-code and function mappings (1 mention) were also singled out as understandable, as in âModule-to-Code Map- ping and Key Functionsâ. Overall, these observations indicate that participants generally perceived the documentation as comprehensible, particularly in its high-level narrative sections and in those parts that provide explicit structural mappings. Overall, these quantitative patterns and qualitative remarks can be summarized as follows: RQ2 Answer Developers generally find the documentation comprehensi- ble: clarity, organization, terminology, and level of detail all received predominantly positive ratings, with only limited disagreement. Open-ended feedback reinforces this view, highlighting the overall readability of the documentation and, in particular, the clarity of high-level architectural sections. C. Answer to RQ3 The responses to Q10âQ15, summarized in Figure 4, assess several dimensions of accuracy, including architectural cor- respondence, internal consistency, completeness, correctness of relationships and responsibilities, and perceived reliability. Architectural correspondence (Q10) received largely positive ratings, with A(8) and SA(8), while smaller groups selected N(4) or D(2). Internal consistency (Q11) showed a similar trend, with A(10) and SA(7), and only limited N(2), D(2), or SD(1). Completeness (Q12) also followed this pattern, with SA(9) and A(5), whereas N(4) and D(4) indicate that some omissions were noticed but did not dominate evaluations. Items Q13âQ15 confirm this trend: correctness of relationships (Q13) and responsibilities (Q14) was generally recognized, and perceived reliability (Q15), supported by A(8) and SA(3), suggests that many participants viewed the documentation as a usable architectural reference. A more detailed view of section-level accuracy is provided by Q16âQ27 and summarized in Figure 5. Sections closely grounded in observable code evidence received the strongest agreement. The Components section (Q21), describing the sys- temâs internal logical structure, achieved the highest positive ratings with SA(10) and A(9). Cross-Cutting Concerns (Q24) showed a similarly strong pattern, with SA(8) and A(11). High levels of agreement also emerged for the Component Diagram (Q20) and the Containers section (Q19) (18 and 17 positive responses, respectively). The Code-Level section (Q22), capturing the concrete implementation of architectural elements in the source code, was likewise positively assessed, with 17 participants selecting A or SA. Collectively, these results indicate that sections reflecting architectural structure or explicit codebase artifacts were consistently perceived as ac- curate, whereas a smaller set of sections received more mixed evaluations. The System Overview (Q16) and Architectural Context (Q17) accumulated the highest numbers of negative ratings, D(4) and D(3), suggesting that high-level descriptions were more prone to inaccuracies or omissions. The Use Case Diagram (Q18) also showed weaker agreement, with several negative responses (D(3) and SD(1)). Additional structural diagrams also drew some negative feedback: the Code-Level Diagram (Q23) received D(4), while the Deployment Diagram (Q27) received D(1) and SD(1), indicating occasional mis- matches with the underlying implementation or configuration. Q10 Q11 Q12 Q13 Q14 Q15 1 (5%) 2 (9%) 2 (9%) 4 (18%) 5 (23%) 1 (5%) 6 (27%) 4 (18%) 2 (9%) 4 (18%) 1 (5%) 3 (14%) 5 (23%) 8 (36%) 10 (45%) 5 (23%) 5 (23%) 10 (45%) 8 (36%) 8 (36%) 7 (32%) 9 (41%) 11 (50%) 8 (36%) 3 (14%) SD (1)D (2)N (3)A (4)SA (5) Fig. 4. Distribution of participantsâ Likert-scale ratings for RQ3 - General Part Based on this combination of high-level and section-level assessments, the answer to RQ3 can be expressed as follows: RQ3 Answer Developers perceived the LLM-generated documentation as accurate, indicating that it reflects the implemented architec- ture, is consistent across sections, and captures key struc- tural elements. Accuracy was rated highest for code-related sections (Components, Containers, Code-Level), while more interpretive views (System Overview, Architectural Context, Use Case) received more mixed feedback. Q16 Q17 Q18 Q19 Q20 Q21 Q22 Q23 Q24 Q25 Q26 Q27 1 (5%) 1 (5%) 1 (5%) 4 (18%) 3 (14%) 3 (14%) 1 (5%) 1 (5%) 4 (18%) 1 (5%) 1 (5%) 3 (14%) 2 (9%) 6 (27%) 5 (23%) 3 (14%) 3 (14%) 4 (18%) 3 (14%) 3 (14%) 5 (23%) 7 (32%) 4 (18%) 8 (36%) 8 (36%) 7 (32%) 7 (32%) 8 (36%) 9 (41%) 8 (36%) 6 (27%) 11 (50%) 6 (27%) 8 (36%) 9 (41%) 7 (32%) 8 (36%) 5 (23%) 10 (45%) 10 (45%) 10 (45%) 9 (41%) 9 (41%) 8 (36%) 10 (45%) 7 (32%) 7 (32%) SD (1)D (2)N (3)A (4)SA (5) Fig. 5. Distribution of participantsâ Likert-scale ratings for RQ3 - Section Part D. Answer to RQ4 Coding of the open-ended answers identified eight limitation categories, summarized in Table I. Beyond Q28âQ30, we also reviewed the open-ended responses from the other RQs to capture additional limitations or missing aspects. The most prominent category was Diagram Errors (32 occurrences). Participants frequently noted incomplete or misleading dia- grams, mentioning for example âthe class diagram is truncated and omits some of the classesâ, all the diagrams should be reviewed, and should be better presented [. . . ] it always seems a little too artificialâ, and that some diagrams do not provide correct and useful informationâ. These remarks reinforce that diagrammatic views were perceived as the most fragile artifacts. A second major theme relates to deployment issues (14 occurrences), particularly in runtime and infrastruc- ture views. Reported problems included unclear relationships, such as âDeployment Diagram [. . . ] the relation between components is not clearâ, as well as incorrect or confusing arrows, for example Deployment (Local Workstation) has few arrows that should not existâ. Other responses indicated that the deployment view should specify not only the production environment but also include an artifact repository to man- age dependenciesâ or that more details on the Deployment Diagram would be usefulâ. Less frequent but still relevant themes include Inconsistencies across sections (6 occurrences) and Missing Information (8 occurrences). Some participants observed that âthe overall components are captured, however [. . . ] wrapping of some of them is not well capturedâ or noted that the LLM needs to be guided to avoid hallucinations TABLE I LIMITATION CATEGORIES FROM OPEN-ENDED RESPONSES. CategoryOccurrences Diagram errors32 Deployment issues14 Improvement suggestions13 Missing information8 Inconsistencies6 Redundancy1 and inconsistenciesâ. Missing content was also highlighted, such as the class diagram is truncated and omits some of the classesâ, it misses the use case tablesâ, or that respondents lack an in-depth description of this componentâ. Finally, several responses offered general Improvement Suggestions (13 occurrences), including âdouble-check with existing doc- umentation if presentâ, human inputs would surely enhance the quality of produced documentationâ, and requests for a shorter and better structured output to help users navigate it more easilyâ. Redundancy was mentioned only once, with one participant noting that all sections are complete, but some content is repetitiveâ. In summary, the themes emerging from the open-ended responses converge toward the following answer to RQ4: RQ4 Answer Developers primarily point to problems in the diagrams, which are often described as incomplete, unclear, or in- accurate. Deployment views are the second most common source of issues, with respondents noting missing details or confusing relationships. Other limitations include occasional inconsistencies and missing information, along with sugges- tions to improve structure, reduce verbosity, and complement the generated output with human review. E. Answer to RQ5 Table IV reports the minimum, maximum, and average values of execution time and monetary cost for generating architectural documentation across the 22 analyzed reposito- ries. Generation times ranged from 1m 50s to 4m 25.85s, with an average of approximately 3m 1s, showing that a complete architectural description can be produced in only a few minutes. Monetary costs ranged from $0.35 to $2.48, with an average of $1.19 per repository, showing that an entire architecture document can be generated for roughly one to two dollars in API usage. These results suggest that the process is both time-efficient and economically lightweight compared to manual documentation effort. TABLE IV COSTS OF GENERATING THE DOCUMENTATION MetricMinMaxMean Generation Time1m 50s4m 25.85s3m 0.90s Financial API Cost$0.35$2.48$1.19 Overall, these measurements of time and monetary cost can be summarized in the following answer: RQ5 Answer Generating complete architectural documentation for a repos- itory takes about three minutes on average, at an average API cost of $1.19, making the process time-efficient and cost- effective compared to manual authoring. F. Additional Practitioner Interest Beyond the formal evaluation, three participants expressed interest in using CIAO to document internal closed-source projects in their organizations, suggesting perceived practical value beyond the study setting. For example, one participant working on a safety-critical railway software component sub- ject to strict regulations requested CIAO-generated documen- tation to support compliance activities, where architectural descriptions must be explicitly traceable to the source code. The participant found the structured representation of contain- ers, components, and their dependencies particularly useful for SDLC activities such as test planning and execution. Another participant, responsible for an internal tool used for penetration testing of edgeâcloud infrastructures, faced challenges due to significant misalignment between the sys- temâs architecture and its implementation, an instance of architectural drift. Since the codebase had evolved faster than the documentation, CIAO was used to reconstruct an updated system-level description and restore architectural clarity. A third participant, a software developer working in a company that provides digital services for public adminis- trations, highlighted the absence of structured architectural documentation in several internal systems. They employed CIAO to generate system-level descriptions aimed at facili- tating onboarding and improving knowledge transfer within their development team. Taken together, these accounts illustrate early practitioner interest in deploying CIAO beyond open-source settings, particularly in contexts where regulatory compliance, architec- tural alignment, or team onboarding make structured system- level documentation especially valuable. VI. THREATS TO VALIDITY In this section, we discuss the main threats to validity of our study and the mitigation measures adopted to address them. Internal Validity. A potential threat lies in the subjective nature of the evaluation. Participants assessed documentation generated for repositories they had personally developed or maintained. While this familiarity reduces the risk of misin- terpreting the underlying architecture, it may also introduce positive or negative bias in judging accuracy, completeness, or usefulness. To mitigate this threat, the questionnaire combined Likert-scale items with open-ended questions that asked par- ticipants to justify their ratings through concrete observations, thus reducing reliance on overall impressions. Construct Validity. This study evaluates developersâ percep- tions of value, clarity, and accuracy rather than objective archi- tectural correctness. The results also depend on the prompting strategy and the documentation template adopted in CIAO. An additional threat concerns the template itself, which was iteratively designed with a limited number of experts and may therefore cover only a restricted range of architectural concerns or shape how they are represented. To mitigate this, the questionnaire explicitly asked participants to assess the clarity, completeness, and usefulness of each section, providing an indirect validation of the templateâs suitability for architecture- level analysis. External Validity. The study involved 22 participants, mostly from academic settings, and a diverse but not exhaustive set of repositories. This limits the generalizability of the findings to other developer populations, software domains, and LLM configurations. To partly mitigate this threat, we included repositories varying in size, language, and domain, although broader studies, especially with industrial participants, are needed to further assess the applicability of the results. Conclusion Validity. The thematic analysis of the open-ended responses may introduce interpretive bias. To mitigate this threat, two authors independently coded all responses and resolved disagreements with a third author, following estab- lished qualitative analysis practices to improve consistency and reliability in theme identification. VII. CONCLUSIONS AND FUTURE WORK This paper introduced CIAO, a structured process for automatically generating system-level architectural documen- tation from complete GitHub repositories using LLMs. CIAO combines a standards-oriented template with an LLM-based workflow grounded in ISO/IEC/IEEE 42010, SEIâs Views & Beyond, and the C4 model. Our empirical study with 22 developers suggests that the generated documentation is generally perceived as useful, understandable, and consistent with the architecture of the analyzed systems. Participants particularly appreciated the nar- rative sections and views closely linked to source code, while the main limitations concerned diagrammatic and deployment- related artifacts. The qualitative analysis also indicates that most inaccuracies are localized rather than structural, making the documentation a potentially useful architectural reference. In addition, the results show that the process is practically feasible, with manageable generation time and cost when using GPT-5. Future work will focus on improving the reliability of diagrammatic views, which emerged as the weakest aspect of the generated documentation. Possible directions include in- tegrating static or dynamic analysis and retrieval mechanisms tailored to architectural artifacts. We also plan to extend the evaluation to more system types, larger repositories, industrial contexts, and broader groups of developers. Finally, future studies may investigate how CIAO can support documentation and maintenance workflows, and whether human-in-the-loop refinement can further improve documentation quality and adoption. ACKNOWLEDGMENT This work was supported by the Italian Ministry of Research, under the complementary actions to the NRRP âFit4MedRob - Fit for Medical Roboticsâ Grant (# PNC0000007). REFERENCES [1] D. Garlanetal., âArchitectural mismatch: Why reuse is still so hard,â IEEESoftware, vol. 26, no. 4, p. 66â69, July 2009. [2] R. Verdecchiaetal., âArchitectural technical debt: A grounded theory,â inSoftwareArchitecture:14thEuropeanConference,ECSA2020, LâAquila,Italy,September14â18,2020,Proceedings.Berlin, Hei- delberg: Springer-Verlag, 2020. [3] P. Kruchtenetal., âTechnical debt: From metaphor to theory and practice,âIEEESoftw., vol. 29, no. 6, p. 18â21, Nov. 2012. [Online]. Available: https://doi.org/10.1109/MS.2012.167 [4] D. M. Leetal., âAn empirical study of architectural decay in open- source software,â in2018IEEEInternationalConferenceonSoftware Architecture(ICSA), 2018. [5] âISO/IEC/IEEE 42010:2011 systems and software engineeringâ architecture description,â International Organization for Standardization, 2011. [6] P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, P. Merson, and J. Stafford,DocumentingSoftwareArchitectures:Views andBeyond. Addison-Wesley, 2010. [7] Z. Fan, R. Liu, X. Chen, and H. Yu, âLarge language models for software engineering: A systematic literature review,âACMComputingSurveys, 2024. [8] F. Zhang, Q. Kang, and D. Yu, âA survey on large language models for software engineering,âarXivpreprintarXiv:2308.11679, 2023. [9] J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, âA survey on large language models for code generation,â 2024. [10] M. Jin etal., âInferfix: End-to-end program repair with llms,â in Proceedingsofthe31stACMJointEuropeanSoftwareEngineering ConferenceandSymposiumontheFoundationsofSoftware Engineering, 2023. [11] GitHub, âThe state of open source and ai: Github octoverse report,â https://octoverse.github.com/, 2023. [12] O. Chaplia and H. Klym, âExtracting rest api endpoints from mi- croservices using llm agents,â in202414thInternationalConference onDependableSystems,ServicesandTechnologies(DESSERT), 2024, p. 1â7. [13] D. Nam etal., âUsing an llm to help with code understanding,â in ProceedingsoftheIEEE/ACM46thInternationalConference onSoftwareEngineering, ser. ICSE â24.New York, NY, USA: Association for Computing Machinery, 2024. [Online]. Available: https://doi.org/10.1145/3597503.3639187 [14] N. Djajadietal., âUsing large language models to generate concise and understandable test case summaries,â in2025IEEE/ACM33rd InternationalConferenceonProgramComprehension(ICPC), 2025, p. 322â326. [15] R. Xuetal.,AutoReqGen:APipelineApproachforAutomated RequirementsGenerationfromSourceCode, 07 2025, p. 444â455. [16] H. A. Siala and K. Lano, âUsing large language models to extract uml class diagrams from java programs,â in20258thInternational ConferenceonSoftwareandSystemEngineering(ICoSSE), 2025, p. 70â74. [17] J. Mirandaetal., âTowards an in-context llm-based approach for automating the definition of model views,â 2024. [18] A. Boronat and J. Mustafa, âMdre-llm: A tool for analyzing and applying llms in software reverse engineering,â in 2025IEEEInternational ConferenceonSoftwareAnalysis,EvolutionandReengineering (SANER), 2025, p. 850â854. [19] K. Chenetal., âAutomated domain modeling with large language models: A comparative study,â in 2023ACM/IEEE26thInternational ConferenceonModelDrivenEngineeringLanguagesandSystems (MODELS), 2023. [20] S. Brown, TheC4ModelforVisualisingSoftwareArchitecture. Leanpub, 2018. [Online]. Available: https://c4model.com [21] M. L. Nelson, âA survey of reverse engineering and program compre- hension,â ArXiv, vol. abs/cs/0503068, 2005. [22] C. Stringfellowetal., âComparison of software architecture reverse engineering methods,â InformationandSoftwareTechnology, vol. 48, no. 7, p. 484â497, 2006. [23] A. Telea etal., âAn open visualization toolkit for reverse architecting,â in Proceedings10thInternationalWorkshoponProgramComprehension, 2002. [24] J. Garciaetal., âA comparative analysis of software architecture recovery techniques,â in201328thIEEE/ACMInternationalConference onAutomatedSoftwareEngineering(ASE), 2013, p. 486â496. [25] âControlling software architecture erosion: A survey,âJournalof SystemsandSoftware, 2012. [26] A. Bucaionietal., âArchitecture as code,â in2025IEEE22nd InternationalConferenceonSoftwareArchitecture(ICSA). IEEE, 2025, p. 187â198. [27] R. Wohlrabetal., âImproving the consistency and usefulness of architec- ture descriptions: Guidelines for architects,â in2019IEEEInternational ConferenceonSoftwareArchitecture(ICSA), 2019, p. 151â160. [28] T. Syst Ě a, âStatic and dynamic reverse engineering techniques for java software systems,â 2000. [29] M. Pinzgeretal., âRevealer: a lexical pattern matcher for architecture recovery,â inNinthWorkingConferenceonReverseEngineering,2002. Proceedings., 2002, p. 170â178. [30] I. S ̧ ora, âHelping program comprehension of large software systems by identifying their most important classes,â in EvaluationofNovel ApproachestoSoftwareEngineering, L. A. Maciaszek and J. Filipe, Eds. Cham: Springer International Publishing, 2016, p. 122â140. [31] T. Ahmed etal., âAutomatic semantic augmentation of language model prompts (for code summarization),â in ProceedingsoftheIEEE/ACM 46thInternationalConferenceonSoftwareEngineering, 2024. [32] L. Zhang etal., âEvaluating large language models for real-world vulnerability repair in c/c++ code,â in Proceedingsofthe10thACM InternationalWorkshoponSecurityandPrivacyAnalytics, 2024. [33] M. Nejjar, L. Zacharias, F. Stiehle, and I. Weber, âLlms for science: Usage for code generation and data analysis,â JournalofSoftware: EvolutionandProcess, 2023. [34] P. Liuetal., âPre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,âACMComput. Surv., vol. 55, no. 9, Jan. 2023. [35] Y. S. Nugrohoetal., âUncovering intention through llm-driven code snippet description generation,â in2025InternationalConferenceon SmartComputing,IoTandMachineLearning(SIML), 2025, p. 1â6. [36] C. Diggs,etal., âLeveraging llms for legacy code modernization: Evalu- ation of llm-generated documentation,â in2025IEEE/ACMInternational WorkshoponLargeLanguageModelsforCode(LLM4Code), 2025, p. 177â184. [37] D. Amalfitano, M. D. Luca, T. Santilli, P. Pelliccione, and A. R. Fasolino, âAutomated software architecture design recovery from source code using llms,â inSoftwareArchitecture.Cham: Springer Nature Switzerland, 2026, p. 73â89. [38] H. Siala and K. Lano, âA comparison of large language mod- els and model-driven reverse engineering for reverse engineering,â Frontiers in Computer Science, vol. 7, 2025, publisher Copyright: Copyright Š 2025 Siala and Lano. [39] Q. Caballeroetal., âEvaluating chatgptâs proficiency in understanding and answering microservice architecture queries using source code insights,âSNComputerScience, 2024. [40] B. Gustrowskyetal., âUsing generative artificial intelligence for sug- gesting software architecture patterns from requirements,â inIntelligent SystemsandApplications, 2024. [41] J. Jahic and A. Sami, â State of Practice: LLMs in Software Engineering and Software Architecture ,â in 2024IEEE21stInternationalConference onSoftwareArchitectureCompanion(ICSA-C), 2024. [42] L. Naimi,etal., âAutomating software documentation: Employing llms for precise use case description,âProcediaComputerScience, vol. 246, p. 1346â1354, 2024, 28th International Conference on Knowledge Based and Intelligent information and Engineering Systems (KES 2024). [Online]. Available: https://w.sciencedirect. com/science/article/pii/S1877050924026176 [43] G. Garaccione etal., âEvaluating large language models in exercises of uml use case diagrams modeling,â in2025IEEE/ACMInternational WorkshoponNaturalLanguage-BasedSoftwareEngineering(NLBSE), 2025, p. 41â44. [44] A. Naghshzan, AutomatingAPIDocumentationwithLLMs:A BERTopicApproach.New York, NY, USA: Association for Computing Machinery, 2025, p. 1013â1015. [Online]. Available: https://doi.org/10.1145/3696630.3728615 [45] S. Bala etal.,ChatGPTforTailoringSoftwareDocumentationfor ManagersandDevelopers, 01 2025, p. 103â109. [46] K. A. Demir, etal., âGenerative ai efficiency and effectiveness in software project documentation review process,â in2025International ConferenceonArtificialIntelligence,Computer,DataSciencesand Applications(ACDSA), 2025, p. 1â7. [47] A. Emerson, T. Meehan, M. Rogers, W. Cowen, and C. Darabos, âCodedocs: Genai to generate documentation from git repositories,â inPracticeandExperienceinAdvancedResearchComputing2025: ThePowerofCollaboration, ser. PEARC â25.New York, NY, USA: Association for Computing Machinery, 2025. [Online]. Available: https://doi.org/10.1145/3708035.3736102 [48] P. Clements, D. Garlan, R. Little, R. Nord, and J. Stafford, âDocument- ing software architectures: views and beyond,â in25thInternational ConferenceonSoftwareEngineering,2003.Proceedings., 2003, p. 740â741. [49]Software,systemsandenterpriseâArchitecturedescription, International Standard ISO/IEC/IEEE 42 010:2022, November 2022. [Online]. Available: https://w.iso.org/standard/74393.html [50] https://c4model.info/\#app. [51] R. Jongeling, N. J. Strøm, L. P. T. Nissen, M. Kitchen, and J. Carlson, âAdopting the c4 model for lightweight architecture modeling - an experience report,â inSoftwareEngineeringandAdvancedApplications, D. Taibi and D. Smite, Eds. Cham: Springer Nature Switzerland, 2026, p. 393â409. [52] https://repomix.com. [53] https://plantuml.com/. [54] Y. Y. Liu, Z. Zheng, F. Zhang, and et al., âA comprehensive taxonomy of prompt engineering techniques for large language models,â Front. Comput.Sci., vol. 20, p. 2003601, 2026. [Online]. Available: https://doi.org/10.1007/s11704-025-50058-z [55] A. Kong, S. Zhao, H. Chen, Q. Li, Y. Qin, R. Sun, X. Zhou, E. Wang, and X. Dong, âBetter zero-shot reasoning with role-play prompting,â in Proceedingsofthe2024ConferenceoftheNorth AmericanChapteroftheAssociationforComputationalLinguistics: HumanLanguageTechnologies(Volume1:LongPapers), K. Duh, H. Gomez, and S. Bethard, Eds.Mexico City, Mexico: Association for Computational Linguistics, Jun. 2024, p. 4099â4113. [Online]. Available: https://aclanthology.org/2024.naacl-long.228/ [56] C. Li, J. Wang, K. Zhu, Y. Zhang, W. Hou, J. Lian, and X. Xie, âLarge language models understand and can be enhanced by emotional stimuli,â 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID: 260126019 [57] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, âLanguage models are few-shot learners,â inAdvancesinNeuralInformationProcessing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.Curran Associates, Inc., 2020, p. 1877â 1901. [Online]. Available: https://proceedings.neurips.c/paperfiles/ paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf [58] https://github.com/AntonioEmmanuele/Group-Key-Phemap. [59] https://github.com/SalvatoreBarone/pyALS-RF/tree/tmr. [60] https://github.com/rdinizcal/sa-bsn. [61] https://github.com/SaraPettinari/ocean-lib. [62] https://github.com/gianlucafilippone/robethichor. [63] https://github.com/gianlucafilippone/springboot-microservices-development. [64] https://github.com/DockerSecurityPlayground/DSP/tree/master/public. [65] https://github.com/DockerSecurityPlayground/DSP/tree/master/app. [66] https://github.com/QuTAM/QuFI. [67] https://github.com/alessandrasomma28/BoMoDT. [68] https://github.com/alessandrasomma28/m2dt. [69] https://github.com/AposVinc/GymportalService. [70] https://github.com/s4hri/pyicub. [71] https://github.com/francescodaghero/treeco. [72] https://github.com/Testing-Game-SAD-2023/A13/tree/main/T4/ gamerepo. [73] https://github.com/ykris152/rse2025-priv. [74] https://github.com/deangelisdf/write2audiobook. [75] https://github.com/deangelisdf/BF-JITcompiled. [76] https://github.com/deangelisdf/visual novelpyarcade. [77] https://github.com/VSecLab/SAM-CyFra. [78] https://github.com/NS-unina/SCASS. [79] https://github.com/NS-unina/OSINT-Collector. [80] V. Braun and V. Clarke, âUsing thematic analysis in psychology,â QualitativeResearchinPsychology, vol. 3, p. 77â101, 01 2006.