Paper deep dive
Studying, Identifying, and Fixing Hidden Technical Debt in AI-Intensive Cyber-Physical Systems
Beena
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 8/5/2026, 3:38:36 AM
Summary
This PhD thesis proposes a framework for identifying, characterizing, and fixing hidden technical debt (TD) in AI-intensive Cyber-Physical Systems (AI-CPS). The research involves three phases: empirical characterization of AI-CPS TD via repository mining and developer interviews, development of automated classification and mitigation methods, and the creation of an agentic AI-based governance tool for continuous TD monitoring and repayment.
Entities (8)
Relation Signals (6)
AI-CPS â exhibits â Technical Debt
confidence 95% ¡ AI-CPS can exhibit technical debt (TD) that is peculiar and potentially more challenging than that of conventional systems.
Technical Debt â hassubtype â Self-Admitted Technical Debt
confidence 90% ¡ A widely studied form of TD is self-admitted TD (SATD)
Agentic AI â supports â TD Governance
confidence 90% ¡ we plan to develop and validate an automated tool that supports agentic AI solutions to monitor, govern, and repay AI-CPS TD.
InnoGuard â supports â PhD Thesis
confidence 85% ¡ The PhD work is part of the InnoGuard... Marie SkĹodowska-Curie Doctoral Network
Hugging Face â hostedby â AI Models
confidence 80% ¡ mining Hugging Face and GitHub repositories to explore... pre-trained models
GitHub â hostedby â AI-CPS Repositories
confidence 80% ¡ mining Hugging Face and GitHub repositories... AI-CPS repositories
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Artificial Intelligence (AI) components are increasingly pervasive in several software systems, including Cyber-Physical Systems (CPSs). AI-CPS are used in several domains, including autonomous vehicles, industry, home automation, robotics, and healthcare. Being composed of hardware, AI components, and conventional modules, AI-CPS can exhibit technical debt (TD) that is peculiar and potentially more challenging than that of conventional systems. This thesis aims to characterize AI-CPS TD and propose approaches for its identification and repair. In a first phase, we characterize AI-CPS TD by analyzing AI ecosystems and AI-CPS repositories, as well as interviewing developers. Based on the acquired knowledge, we define approaches to identify and mitigate such TD. Finally, we plan to develop and validate an automated tool that supports agentic AI solutions to monitor, govern, and repay AI-CPS TD.
Tags
Links
- Source: https://arxiv.org/abs/2608.02638v1
- Canonical: https://arxiv.org/abs/2608.02638v1
Trouble viewing inline? Open PDF directly â
Full Text
22,348 characters extracted from source content.
Expand or collapse full text
Studying, Identifying, and Fixing Hidden Technical Debt in AI-Intensive Cyber-Physical Systems Beena Abstract Artificial Intelligence (AI) components are increasingly pervasive in several software systems, including Cyber-Physical Systems (CPSs). AI-CPS are used in several domains, including autonomous vehicles, industry, home automation, robotics, and healthcare. Being composed of hardware, AI components, and conventional modules, AI-CPS can exhibit technical debt (TD) that is peculiar and potentially more challenging than that of conventional systems. This thesis aims to characterize AI-CPS TD and propose approaches for its identification and repair. In a first phase, we characterize AI-CPS TD by analyzing AI ecosystems and AI-CPS repositories, as well as interviewing developers. Based on the acquired knowledge, we define approaches to identify and mitigate such TD. Finally, we plan to develop and validate an automated tool that supports agentic AI solutions to monitor, govern, and repay AI-CPS TD. I Introduction Nowadays, several kinds of software systems are governed by Artificial Intelligence (AI)-intensive components. Among others, such systems include Cyber-Physical Systems (CPSs), i.e., systems composed of both software and hardware/mechanical components. Given the complex interactions among software, firmware, and hardware/mechanical components, previous research has shown that CPSs require specialized development processes [22]. This includes, for example, the use of digital twins, tailored continuous integration and delivery (CI/CD) [23], and testing [11, 19]. Modern CPSs also include AI components. Examples of AI-intensive CPSs (AI-CPSs) include autonomous vehicles, industrial or home automation systems, robotic systems, and healthcare automation systems. The AI components themselves have been shown to be responsible for specific types of technical debt (TD) [13]. We conjecture that the interaction between CPS components and AI components may lead to specific challenges, for example, regarding how the AI autonomously enacts a CPS hardware/mechanical component, the extent to which this interaction accounts for real-time hardware requirements, includes the presence of suitable safeguards, and, last but not least, accounts for legal/regulatory requirements. This PhD project aims to address this gap by answering the following question: What kind of specific technical debt occurs in the development of AI-CPSs, and how can it be automatically repaid? The PhD work is part of the InnoGuard: âHybrid and Generative Intelligence for Trustworthy Autonomous Cyber-Physical Systemsâ Marie SkĹodowska-Curie Doctoral Network111https://innoguard.eu/, and it is currently at the beginning of its second year. The research will follow a multi-stage approach. The project begins with mining Hugging Face and GitHub repositories and a practitioner interview study on the real-world use of AI pre-trained components in software (particularly CPS) automation. The outcome of this first phase will be a taxonomy characterizing AI-CPS TD, developersâ challenges in AI-CPS development, and current TD-repairing strategies. The empirical characterization serves as a foundation for automated TD classification and mitigation recommendations. Lastly, we plan to develop agentic AI-based governance for AI-CPS TD continuous identification and classification, monitoring, effect estimation, and repair. The predicted outcome of this research is, first, a systematic understanding of TD in AI-CPSs, its introduction, root causes, and current mitigation strategies. In a second stage, the thesis will provide AI-CPS developers with a set of approaches for automated TD management, from detection to repayment. Moreover, as indicated by previous work, CPS developmentâand AI-CPS developmentâmay require suitable and diverse development skills, the thesisâ outcome will aid âdemocratizingâ AI-CPS development and maintenance. I Background And Related Work âTechnical debtâ (TD) refers to the long-term maintenance cost initiated by short-term design, implementation, or performance decisions in software systems [7, 8, 21]. Earlier research often focused on source code TD, but subsequent studies have shown that TD can also affect architecture, testing, documentation, and maintenance artifacts [4, 15]. A widely studied form of TD is self-admitted TD (SATD), where developers explicitly acknowledge suboptimal solutions through comments or other software artifacts. Prior work has proposed several techniques for detecting and classifying SATD using natural language processing and machine learning [10, 18]. In AI-intensive systems, TD becomes broader and more difficult to localize. Besides source code, debt may arise from other artifacts on external platforms or frameworks [13, 16, 20]. On top of that, AI-CPSs introduce additional concerns related to the interaction between software intelligence and the physical world, such as sensor robustness, safety constraints, simulation fidelity, calibration, middleware workarounds, and runtime behavior [6, 14]. These forms of debt are not yet fully captured by existing taxonomies. Some research proposed approaches for TD classification, including recent LLM-based approaches [1, 9], with less focus on mitigation and governance. Some other work has investigated how TD can be automatically ârepaidâ. Specifically, previous approaches used either deep learning techniques [2], fine-tuned pretrained models [12], or, more recently, LLMs [17] to repay TD in conventional software. While prior work provides strong foundations for TD detection, classification, and automated repayment, we conjecture that AI-CPSs warrant a distinct characterization and approach. In AI-CPSs, TD emerges from the complex interactions among hardware/mechanical components, firmware, conventional software, and AI models. This leads to new TD categories, but also requires TD repayment across heterogeneous artifact types. Figure 1: Overview of the proposed PhD research approach I Methodology We first provide an overview of the PhD work methodology. Then, for each step, we detail the proposed methodological or technical solutions. I-A Methodology Overview The methodology overview is depicted in Fig. 1. The research consists of three phases. In a first phase, we leverage a mixed-method approach to gather empirical knowledge about AI-CPS TD. We mine Hugging Face and GitHub repositories to explore (i) how pre-trained models are chosen, integrated, grown, and maintained, and (i) the types of TD that developers self-admit in AI-CPSs. Also, we interview and survey developers to understand their adoption and integration processes for ML models, in particular open-weight models, in software systems, especially in AI-CPSs. This will allow us to produce a taxonomy of AI-CPS-specific TD and identify challenges in AI model development and integration in CPSs. In a second phase, based on the output of the first phase, we will develop methods for identifying hidden TD and devise strategies for mitigating each type of debt. The final phase focuses on agentic AI-based governance. This stage analyzes how the classification and mitigation elements can be integrated into an autonomous governance framework for continuous TD management and repayment. I-B Proposed Solutions I-B1 Phase 1 - Repository Mining of AI Model Ecosystems The first phase of the project is to gather empirical knowledge about AI-CPS-specific TD. First, we perform an empirical repository mining of AI model ecosystems and AI-CPS repositories. On the one hand, we are studying the AI model development process by collecting and analyzing information from Hugging Face models and their linked GitHub repositories. The study evaluates repository links, commits, contributors, file changes, documentation updates, metadata changes, issues, and commit synchronization. On the other hand, we are mining AI-CPS repositories to identify and characterize AI-CPS-specific TD, distinguishing AI-CPS-specific instances from those related to conventional and ML-specific TD. Examples of AI-CPS-specific TD concern the relations between software intelligence and the physical world, i.e., sensing, actuation, motion control, planning, calibration, localization, trajectory generation, physical modeling, safety constraints, robot-environment interaction, simulation fidelity, coordinate transformations, and CPS-specific runtime behavior. To complement the mining studies and to understand in depth the experiences that are not directly visible in repository data, we are conducting interview-based studies with practitioners. We investigate how pre-trained AI models are selected, integrated, adapted, deployed, and maintained in real-world software systems from the perspectives of engineers and researchers, and how they work with them. We also investigate the challenges they face in developing AI-intensive systems, particularly AI-CPS. The first outcome of Phase 1 will be a taxonomy of AI-CPS-specific TD categories, complementing those previously developed for conventional systems [3] and ML systems [13]. Alongside the taxonomy, we will compile a catalog of the main challenges developers encounter in AI-CPS development, along with possible mitigations and TD repayment strategies. I-B2 Phase 2 - Methods for Automated Classification and Empirically Grounded Mitigation The second phase develops methods to automate identification, classification, and mitigation of âhiddenâ AI-CPS TD, i.e., TD that has not been admitted by developers. Concerning the TD identification and classification, we develop an agentic-based approach that takes as input the AI-CPS taxonomy and the taxonomies of conventional and ML-specific TD definitions. The approach will leverage metrics and static analysis warnings extracted from the code, alongside the code itself, as well as hardware-related information mined from the repository and ML model information. For the latter, the approach will extract and categorize information from model cards, building on prior literature on model card analysis [5]. The elicited AI-CPS TD taxonomy will serve as the main reference framework. More specifically, the identification approach will analyze multiple sources of repository evidence rather than relying solely on explicit TD comments, including CPS-specific evidence such as sensor and actuator descriptions, middleware settings, simulation files, calibration parameters, safety constraints, deployment scripts, and resource-related information. From the ML side, it will analyze model cards, training and inference scripts, datasets, preprocessing pipelines, and hyperparameters. By combining these sources, we will identify indicators of hidden TD across software, AI/ML, and CPS components. Mitigation strategies will be empirically identified from the two phases above. We will analyze remediation strategies to observe how developers address debt-related problems in repositories. We will also use practitioner feedback to understand how developers handle challenges in model selection, integration, compatibility, documentation, deployment, reliability, and maintenance. These sources will be used to map debt categories to possible challenges and remediation actions. I-B3 Phase 3 - Agentic AI-based TD governance The last phase of the PhD work integrates the outputs of the previous phases into an agentic AI-based governance approach for continuous TD management in AI-CPSs. The purpose is not only to detect TD once, but also to support constant monitoring, classification, impact assessment, and mitigation recommendations as AI-intensive repositories evolve. Figure 2: Overview of the Agentic AI-based system As shown in Fig. 2, the agentic AI system will be composed of: ⢠A TD identification agent, which identifies the hidden TD and classifies it along the taxonomy; ⢠A TD solution recommender that proposes a remediation solution and, wherever possible, is able to enact it. ⢠A challenge and impact assessment agent will maintain information on the potential effects of the TD in terms of maintainability, reliability, deployability, and other CPS-specific effects. ⢠A Knowledge base of empirically acquired evidence, consisting of the AI-CPS TD taxonomy, developersâ challenges, and adopted TD remediation solutions. ⢠Specialized analyzers that mine information from the AI-CPS repositories, and, specifically, analyzers for AI models, firmware, hardware specifications/configurations, and AI-intensive source code. ⢠Retrievers that mine data from the Internet, e.g., from model/data hubs such as Hugging Face or Kaggle, from software repositories, and from hardware data sheets. ⢠An orchestrator, which will handle the entire workflow: monitoring repository changes, invoking analyzers, assigning tasks to specialized agents, verifying output completeness, and producing a final governance report. Notably, the agentic AI solution will be semi-automated, i.e., able to interact with the developer through the IDE. Human feedback will be used to validate classifications and remediation suggestions. This feedback can later be used to refine the taxonomy, improve prompts, update remediation strategies, and make the governance process more reliable. We plan to empirically validate the agentic AI solution initially through smaller studies in academic settings, e.g., by letting students use it in their projects. Then, we will evaluate the Innoguard projectsâ demonstrators, which involve a humanoid robot and a Leo Rover. This scenario provides a realistic AI-CPS setting that includes sensing, actuation, robot coordination, navigation, runtime monitoring, safety constraints, middleware, and hardware assumptions. It will allow us to assess whether the agentic governance approach can identify AI-CPS-specific debt across software, models, configurations, simulations, and hardware-related artifacts, and whether its impact assessments and remediation suggestions are useful to developers involved. IV Expected Contributions The PhD work research aims to grow the understanding and management of TD in AI-intensive systems through the following contributions: ⢠An empirical characterization of AI model ecosystem evolution and adoption, resulting from the analysis of ML model GitHub and Hugging Face repositories and from the interview study. ⢠A taxonomy of TD in AI-CPS: The PhD will produce a taxonomy of AI-CPS-specific TD, complementing those for conventional and ML-intensive systems. ⢠Practitionersâ perspectives and challenges on the integration of AI models in software systems and, in particular, in AI-CPS. ⢠Automated techniques for AI-CPS TD identification, classification, and remediation: While leveraging knowledge acquired in identifying and repaying conventional TD, the proposed techniques will bring specific identification strategies that look (and fix) into diverse artifacts beyond code, and account for the AI-CPS specific TD defined in our taxonomy. ⢠An agentic AI-based AI-CPS TD governance framework: The framework will use agentic AI to support repository monitoring, TD identification, classification, impact assessment, and the generation of mitigation recommendations. . V Current status and dissemination plan Although the PhD work is in its early stages, two initial results have already been obtained. Empirical study of Hugging Face and GitHub.at We completed an empirical study on the development and evolution of ML models, which looked at their Hugging Face and GitHub repositories. Our results show that fewer than 15% of the most-downloaded Hugging Face models have a development repository on GitHub. We also observed that model cards typically undergo only minor updates, limiting their role as live documentation for ML models. Initial AI-CPS TD taxonomy construction. By analyzing SATD in repositories of 15 open-source AI-CPSs, we started creating a taxonomy of AI-CPS-specific TD. This includes, for example, sensor robustness, simulation simplification, safety, middleware or framework workaround, and model integration TD. We plan to submit this work to a software engineering conference in the early fall. Interviews with practitioners are halfway through being conducted, and we plan to submit an article to a software engineering conference by the fall. For the forthcoming research, the dissemination will mainly target software engineering conferences and journals, next year for the TD classification and remediation, and during the final year (a journal) for the agentic AI infrastructure. VI Acknowledgments This work is supported by the InnoGuard Marie SkĹodowska-Curie Doctoral Network (Grant Agreement No. 101169233) References [1] D. Albuquerque, E. GuimarĂŁes, G. Tonin, P. RodrĂguezs, M. Perkusich, H. Almeida, A. Perkusich, and F. Chagas (2022) Managing technical debt using intelligent techniques-a systematic mapping study. IEEE Transactions on Software Engineering 49 (4), p. 2202â2220. Cited by: §I. [2] A. Alhefdhi, H. K. Dam, and A. Ghose (2024) Towards automating self-admitted technical debt repayment. Information and Software Technology 167, p. 107376. External Links: ISSN 0950-5849 Cited by: §I. [3] G. Bavota and B. Russo (2016) A large-scale empirical study on self-admitted technical debt. In Proceedings of the 13th international conference on mining software repositories, p. 315â326. Cited by: §I-B1. [4] T. Besker, A. Martini, and J. Bosch (2018) Managing architectural technical debt: a unified model and systematic literature review. Journal of Systems and Software 135, p. 1â16. Cited by: §I. [5] A. Bhat, A. Coursey, G. Hu, S. Li, N. Nahar, S. Zhou, C. Kästner, and J. L. C. Guo (2023) Aspirations and practice of ML model documentation: moving the needle with nudging and traceability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, Hamburg, Germany, April 23-28, 2023, p. 749:1â749:17. Cited by: §I-B2. [6] J. Bogner, R. Verdecchia, and I. Gerostathopoulos (2021) Characterizing technical debt and antipatterns in ai-based systems: a systematic mapping study. In 2021 IEEE/ACM International Conference on Technical Debt (TechDebt), p. 64â73. Cited by: §I. [7] N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim, P. Kruchten, E. Lim, A. MacCormack, R. Nord, I. Ozkaya, et al. (2010) Managing technical debt in software-reliant systems. In Proceedings of the FSE/SDP workshop on Future of software engineering research, p. 47â52. Cited by: §I. [8] P. Kruchten, R. L. Nord, and I. Ozkaya (2012) Technical debt: from metaphor to theory and practice. Ieee software 29 (6), p. 18â21. Cited by: §I. [9] Y. Li, M. Soliman, and P. Avgeriou (2023) Automatic identification of self-admitted technical debt from four different sources. Empirical Software Engineering 28 (3), p. 65. Cited by: §I. [10] E. d. S. Maldonado and E. Shihab (2015) Detecting and quantifying different types of self-admitted technical debt. In 2015 IEEE 7Th international workshop on managing technical debt (MTD), p. 9â15. Cited by: §I. [11] C. Mandrioli, S. Y. Shin, D. Bianculli, and L. Briand (2025) Testing cps with design assumptions-based metamorphic relations and genetic programming. IEEE Transactions on Software Engineering 51 (6), p. 1666â1684. External Links: Document Cited by: §I. [12] A. Mastropaolo, M. Di Penta, and G. Bavota (2023) Towards automatically addressing self-admitted technical debt: how far are we?. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), Vol. , p. 585â597. External Links: Document Cited by: §I. [13] F. Pepe, F. Zampetti, A. Mastropaolo, G. Bavota, and M. Di Penta (2024) A taxonomy of self-admitted technical debt in deep learning systems. In 2024 IEEE international conference on software maintenance and evolution (ICSME), p. 388â399. Cited by: §I, §I, §I-B1. [14] G. Recupito, F. Pecorelli, G. Catolino, V. Lenarduzzi, D. Taibi, D. Di Nucci, and F. Palomba (2024) Technical debt in ai-enabled systems: on the prevalence, severity, impact, and management strategies for code and architecture. Journal of Systems and Software 216, p. 112151. Cited by: §I. [15] N. Rios, M. G. de Mendonça Neto, and R. O. SpĂnola (2018) A tertiary study on technical debt: types, management strategies, research trends, and base information for practitioners. Information and Software Technology 102, p. 117â145. Cited by: §I. [16] D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J. Crespo, and D. Dennison (2015) Hidden technical debt in machine learning systems. Advances in neural information processing systems 28. Cited by: §I. [17] M. S. Sheikhaei, Y. Tian, S. Wang, and B. Xu (2026-02) A large-scale empirical evaluation of llms for automated self-admitted technical debt repayment. ACM Trans. Softw. Eng. Methodol.. Note: Just Accepted External Links: ISSN 1049-331X, Link, Document Cited by: §I. [18] G. Sierra, E. Shihab, and Y. Kamei (2019) A survey of self-admitted technical debt. Journal of Systems and Software 152, p. 70â82. Cited by: §I. [19] R. J. Somers, J. A. Douthwaite, D. J. Wagg, N. Walkinshaw, and R. M. Hierons (2023) Digital-twin-based testing for cyber-physical systems: A systematic literature review. Inf. Softw. Technol. 156, p. 107145. Cited by: §I. [20] Y. Tang, R. Khatchadourian, M. Bagherzadeh, R. Singh, A. Stewart, and A. Raja (2021) An empirical study of refactorings and technical debt in machine learning systems. In 2021 IEEE/ACM 43rd international conference on software engineering (ICSE), p. 238â250. Cited by: §I. [21] E. Tom, A. Aurum, and R. Vidgen (2013) An exploration of technical debt. Journal of Systems and Software 86 (6), p. 1498â1516. Cited by: §I. [22] M. TĂśrngren and U. Sellgren (2018) Complexity challenges in development of cyber-physical systems. In Principles of Modeling - Essays Dedicated to Edward A. Lee on the Occasion of His 60th Birthday, Lecture Notes in Computer Science, Vol. 10760, p. 478â503. Cited by: §I. [23] F. Zampetti, D. A. Tamburri, S. Panichella, A. Panichella, G. Canfora, and M. Di Penta (2023) Continuous integration and delivery practices for cyber-physical systems: an interview-based study. ACM Trans. Softw. Eng. Methodol. 32 (3), p. 73:1â73:44. Cited by: §I.