Paper deep dive
Can an LLM Detect Instances of Microservice Infrastructure Patterns?
Carlos Eduardo Duarte, Neil B. Harrison, Filipe Figueiredo Correia, Ademar Aguiar, PavlĂna Gonçalves
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/26/2026, 1:44:15 AM
Summary
The paper introduces MicroPAD, a language-agnostic tool utilizing GPT-5 nano to detect microservice infrastructure architectural patterns in software repositories. By analyzing 190 human-annotated GitHub repositories, the authors evaluate the tool's performance, finding that detection accuracy varies based on pattern prevalence and the distinctiveness of associated artifacts. The study provides a new dataset and insights into how LLMs can be effectively applied to architectural pattern recognition.
Entities (5)
Relation Signals (3)
MicroPAD â utilizes â GPT 5 nano
confidence 100% ¡ This tool utilizes GPT 5 nano to identify architectural patterns
MicroPAD â detects â Microservice Infrastructure Patterns
confidence 95% ¡ MicroPAD was capable of detecting pattern instances across multiple languages
Chris Richardson â authored â Microservice Infrastructure Patterns
confidence 90% ¡ microservice infrastructure patterns, a subset of microservice architecture patterns defined in Chris Richardsonâs pattern catalog
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Architectural patterns are frequently found in various software artifacts. The wide variety of patterns and their implementations makes detection challenging with current tools, especially since they often only support detecting patterns in artifacts written in a single language. Large Language Models (LLMs), trained on a diverse range of software artifacts and knowledge, might overcome the limitations of existing approaches. However, their true effectiveness and the factors influencing their performance have not yet been thoroughly examined. To better understand this, we developed MicroPAD. This tool utilizes GPT 5 nano to identify architectural patterns in software artifacts written in any language, based on natural-language pattern descriptions. We used MicroPAD to evaluate an LLM's ability to detect instances of architectural patterns, particularly infrastructure-related microservice patterns. To accomplish this, we selected a set of GitHub repositories and contacted their top contributors to create a new, human-annotated dataset of 190 repositories containing microservice architectural patterns. The results show that MicroPAD was capable of detecting pattern instances across multiple languages and artifact types. The detection performance varied across patterns (F1 scores ranging from 0.09 to 0.70), specifically in relation to their prevalence and the distinctiveness of the artifacts through which they manifest. We also found that patterns associated with recognizable, dominant artifacts were detected more reliably. Whether these findings generalize to other LLMs and tools is a promising direction for future research.
Tags
Links
- Source: https://arxiv.org/abs/2603.23073v1
- Canonical: https://arxiv.org/abs/2603.23073v1
Trouble viewing inline? Open PDF directly â
Full Text
41,929 characters extracted from source content.
Expand or collapse full text
Can an LLM Detect Instances of Microservice Infrastructure Patterns? Carlos Eduardo Duarte â , Neil B. Harrison â , Filipe Figueiredo Correia â , Ademar Aguiar â and Pavl Ě Äąna Gonc ̧alves ⥠â INESC TEC, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal â Department of Computer Science, Utah Valley University, Utah, USA ⥠INESC TEC, Porto, Portugal carlos.e.duarte, filipe.correia, ademar.aguiar, pavlina.goncalves@inesctec.pt, neil.harrison@uvu.edu AbstractâArchitectural patterns are frequently found in var- ious software artifacts. The wide variety of patterns and their implementations makes detection challenging with current tools, especially since they often only support detecting patterns in artifacts written in a single language. Large Language Mod- els (LLMs), trained on a diverse range of software artifacts and knowledge, might overcome the limitations of existing ap- proaches. However, their true effectiveness and the factors influ- encing their performance have not yet been thoroughly examined. To better understand this, we developed MicroPAD. This tool utilizes GPT 5 nano to identify architectural patterns in software artifacts written in any language, based on natural-language pattern descriptions. We used MicroPAD to evaluate an LLMâs ability to detect instances of architectural patterns, particularly infrastructure-related microservice patterns. To accomplish this, we selected a set of GitHub repositories and contacted their top contributors to create a new, human-annotated dataset of 190 repositories containing microservice architectural patterns. The results show that MicroPAD was capable of detecting pattern instances across multiple languages and artifact types. The detec- tion performance varied across patterns (F1 scores ranging from 0.09 to 0.70), specifically in relation to their prevalence and the distinctiveness of the artifacts through which they manifest. We also found that patterns associated with recognizable, dominant artifacts were detected more reliably. Whether these findings generalize to other LLMs and tools is a promising direction for future research. Index Termsâarchitectural patterns, microservices, large lan- guage models, pattern detection I. INTRODUCTION Software patterns are reusable solutions to recurring prob- lems in a given context [1]. Patterns can play a significant role in helping practitioners understand how programs work, especially considering that software maintainers spend, on average, around 58% of their time understanding the soft- ware [2]. Notably, preserving the knowledge obtained from architectural patterns present in the codebase is essential, as it empowers practitioners to make better decisions related to the systemâs quality attributes [3], which in turn influence the systemâs overall success. Nonetheless, practitioners often fail to maintain software documentation [4]. This work is co-financed by Component 5 - Capitalization and Business Innovation, integrated in the Resilience Dimension of the Recovery and Resilience Plan within the scope of the Recovery and Resilience Mechanism (MRR) of the European Union (EU), framed in the Next Generation EU, for the period 2021 - 2026, within project HfPT, with reference 41. To preserve this knowledge, researchers have proposed several automated tools to identify instances of architectural patterns in software artifacts. However, this task is not simple for several reasons. First, practitioners might not be aware that these patterns exist. Second, they may struggle to identify instances of the patterns across the software artifacts. Third, architectural pattern instances can vary significantly, and their presence can span multiple different artifacts [5], making detection more challenging. Moreover, existing approaches for detecting architectural patterns are notably limited in several aspects. For example, many existing tools can detect pattern instances only in a single language. Since projects contain artifacts in multiple languages, and architectural patterns can span throughout these artifacts, detection tools may miss many patterns. Another lim- itation is that the detection mechanisms of existing approaches are specific to the patterns they can detect. Given the wide variety of architectural patterns and the scarcity of available detection tools, there is a need for approaches that enable users to easily detect the specific patterns they require, whether by extending existing tools or creating new ones. LLMs may address these issues. They have been trained on vast amounts of source code, learning the syntax and semantics of numerous languages while observing many instances of architectural patterns. An LLM-based pattern detector should be able to analyze artifacts in multiple languages without requiring language-specific parsers. However, software repos- itories are typically too large to process in a single LLM call. Additionally, analyzing every file individually would be prohibitively expensive due to API costs. Consequently, we needed a structured approach to select relevant files and aggregate evidence. To explore this further, we developed MicroPAD, an auto- mated tool that uses GPT 5 nano to identify architectural pat- terns from natural-language descriptions. MicroPAD wraps the LLM in a multi-step pipeline that addresses these challenges. Thus, our findings reflect MicroPADâs overall performance rather than the LLMâs contribution in isolation. We focused on microservice infrastructure patterns, a subset of microservice architecture patterns defined in Chris Richardsonâs pattern cat- alog [6], given the prevalence of microservice architectures in modern software systems and the extensive research interest in arXiv:2603.23073v1 [cs.SE] 24 Mar 2026 this area. Our evaluation across 190 open-source repositories, validated by repository contributors, revealed insights about how pattern characteristics influence detection performance. The main contributions of this article are the following: ⢠MicroPAD, a language-agnostic detection tool requiring only natural language pattern descriptions and repository artifacts as input; ⢠Novel insights on detection performance for microservice architectural patterns, including which pattern character- istics influence detection results; ⢠A human-annotated dataset of 190 repositories with 47 microservice architectural patterns labeled by repository contributors as present or absent in their systems. These contributions can serve as a basis for further research on LLM-based architectural pattern detection and microservice architectures in general. I. RELATED WORK Researchers have proposed various tools for architectural pattern detection using Heuristics, Machine Learning (ML), and LLMs. Table I summarizes these approaches, detailing their detection techniques, supported languages, dataset sizes, and performance metrics (precision (P), recall (R), accuracy (A), and F1 score (F1)). A. Approaches Based On Heuristics Various tools detect instances of architectural patterns by looking, for example, for the presence of particular keywords or calculating code metrics related to each specific pattern [5], [7], [9]. Daniel et al. [9] present one such tool, which is capable of detecting several microservice architectural patterns. To detect the patterns, the approach analyzes SQL scripts and calculates pattern-related metrics, such as the number of operations exposed by a microservice. Daoudi et al. [7] detect MODEL-VIEW-WHATEVER (MVW) patterns (which include patterns such as MODEL- VIEW-CONTROLLER (MVC), MODEL-VIEW-VIEWMODEL (MVVM), and MODEL-VIEW-PRESENTER (MVP)) in Dalvik bytecode using structural and event-based heuristics. This approach is difficult to extend, as supporting new patterns requires manually defining new heuristics. Haitzer and Zdun [5] overcome single-language limita- tions by using UML and a Domain-Specific Language (DSL) for pattern detection. However, the approach is semi-automatic and constrained by the DSLâs capabilities and learning curve. B. Approaches Based On ML ML approaches [10]â[12], [15] require training models on pattern-specific datasets, making it costly to extend them to new patterns. Komolov et al. [12] use nine ML models to detect MVW patterns in Java programs based on code metrics. However, extending the tool to new languages or patterns requires costly and time-consuming retraining. Chekhaba et al. [11] use several ML classifiers to detect MVW patterns in Java Android apps. Like Komolov et al., this approach is limited to a single language and requires retraining for expansion. Milhem et al. [10] extend Archie [16] to detect five archi- tectural patterns in Java using ML. Their approach functions by identifying pattern-related terms directly in the source code. C. Approaches Based On LLMs LLMs are significantly impacting how we develop soft- ware [17]. Therefore, it is natural that researchers have begun experimenting with them to identify instances of architectural patterns in software artifacts. Rukmono et al. [14] employ GPT 4 to detect the LAYERED ARCHITECTURE pattern via prompting. The approach is rigid, as the prompts are tailored to a single pattern, offering limited insight into general architectural detection capabilities. J Ě anki and Bilicki [13] use rule-based prompting to de- tect MVW patterns in Angular/TypeScript projects. How- ever, defining these rules is labor-intensive, and the approach remains limited to a single language despite using natural language definitions. Overall, existing approaches for detecting architectural pat- tern instances have limitations with respect to the languages they can identify patterns in and the extent to which their implementation depends on the patterns they target. Table I shows that most contributions do not disclose any detection performance metrics for the approaches they present, such as F1 scores, making it difficult to compare the detection performance of the different tools. Finally, although practitioners are beginning to explore the use of LLMs to detect architectural patterns, the effectiveness of these models in identifying them and their numerous vari- ations remains unclear and warrants further research. In par- ticular, existing LLM-based approaches are limited to specific patterns or languages and require custom prompt engineering. I. RESEARCH QUESTIONS Our work employs MicroPAD to automatically detect a set of microservice infrastructure patterns. This evaluation also provides initial insights into how an LLM can detect these patterns. Our research aims to address two key research questions: RQ1. What is the detection performance of a language- agnostic, LLM-based approach to detect microservice infras- tructure pattern instances in software artifacts? We measured precision, recall, accuracy, and F1 score for each of the nine patterns from Richardsonâs catalog. Noticing that detection performance varied by pattern, we investigated how it is influenced by the specific file types provided to the LLM. Thus, we formulated the following research question: RQ2. How do the artifacts in software repositories impact the ability of an LLM-based approach to detect microservice infrastructure pattern instances? We explore these questions using the MicroPAD tool with GPT 5 nano, which will be described in the next section. WorkTechnique PatternsLanguageSample SizePRAF1 Daoudi et al. [7]HeuristicsMVWBytecode1000.860.910.900.88 Haitzer and Zdun [8]HeuristicsAny------ Daniel et al. [9]HeuristicsDATABASE PER SERVICE, SINGLE SER- VICE PER HOST, API COMPOSITION, ASYN- CHRONOUS MESSAGE, COMMAND-QUERY RESPONSIBILITY SEGREGATION SQL----- Milhem et al. [10]MLBROKER,OBSERVER/PUBLISH-SUBSCRIBE, LAYERED ARCHITECTURE, PIPES AND FIL- TERS, SHARED REPOSITORY Java2---- Chekhaba et al. [11]MLMVWJava265---0.56-0.68 Komolov et al. [12]MLMVWJava5,9730.830.830.830.83 J Ě anki and Bilicki [13]LLMsMVWTypeScript18,830--0.90- Rukmono et al. [14]LLMsLAYERED ARCHITECTUREJava10.710.69-0.69 TABLE I: A summary of existing tools for detecting architectural patterns. IV. METHODOLOGY To answer these research questions, we first developed a tool to detect instances of architectural patterns in an automated manner using an LLM. We then obtained a set of GitHub repositories and analyzed them with the detection tool. To establish ground truth for evaluating our tool, we conducted a questionnaire survey with the top contributors of those repositories to identify which patterns they contained. Finally, we compared the LLM-based approach to the ground truth. The tool, human-annotated dataset, and other materials from the study are available in an empirical package [18]. A. MicroPAD Software repositories are typically too large to analyze in a single LLM call, so MicroPAD uses a structured pipeline to select relevant files and aggregate evidence across them. MicroPAD is a custom Python pipeline that orchestrates LLM calls through a multi-step process specifically designed for pattern detection. It supports various LLMs and Small Lan- guage Models (SLMs) to identify pattern instances in software artifacts, taking a codebase and natural-language descriptions of desired patterns as input. The tool detects pattern instances across languages by processing various textual artifacts, in- cluding natural language and source code, without requiring language-specific parsers. Users can configure several parameters, such as selecting the language model to use, with options including OpenAI LLMs or SLMs available through the Ollama platform. Additionally, users can specify the number of files to analyze for each pattern and adjust the temperature settings, if supported by the model. New patterns can be added via natural language descriptions without retraining, and the tool provides reasoning traces showing which files contributed to each decision. We have configured the tool with sensible defaults, as described in the next section. MicroPAD builds on an earlier prototype [19] that detected patterns using only Infrastructure-as-Code artifacts. Our ap- proach has four main components: Pattern Profile Generator, Database Seeder, Repository Parser, and Pattern Detector. The Pattern Profile Generator is responsible for creating Pat- tern Profiles. These are files that contain valuable information to identify pattern instances. The Database Seeder generates and stores embeddings of pattern examples from the Pattern Profiles in a vector database. The Repository Parser filters files likely to contain instances of microservice patterns. Fi- nally, the Pattern Detector identifies pattern instances within the filtered artifacts. Figure 1 shows MicroPADâs main components and how they interact. The tool executes in three phases: Creating Pattern Profiles, Determining File Paths to Analyze, and Detecting Patterns. Arrows indicate interactions between components, with colors marking each phase: red for Creating Pattern Profiles, blue for Determining File Paths to Analyze, and green for Detecting Patterns. 1) Creating Pattern Profiles: MicroPAD detects patterns using Pattern Profiles. Specifically, these are YAML files containing the patternâs name, description, glob patterns for file path matching (e.g., **/src/*.py), and example instances. The LLM uses these glob patterns along with the pattern description to identify relevant files at runtime. To introduce a new pattern, users provide a pattern name, description, and a link to its catalog entry, and MicroPAD uses GPT 5 to generate the Pattern Profile. We manually inspected the generated profiles and verified that glob patterns and examples were reasonable for each pattern. We chose the following patterns from Richardsonâs cat- alog: SINGLE SERVICE INSTANCE PER HOST, MULTIPLE SERVICE INSTANCES PER HOST, SERVICE INSTANCE PER CONTAINER, SERVICE MESH, SERVICE INSTANCE PER VM, SERVICE DEPLOYMENT PLATFORM, SERVER-SIDE SERVICE DISCOVERY, 3RD PARTY REGISTRATION, and SERVICE REGISTRY. Finally, the Database Seeder creates embeddings from the example instances in the Pattern Profiles and stores them in the ChromaDB vector database. 2) Determining File Paths to Analyze: The Repository Parser scans the repository and generates its file tree, which may be trimmed if too large. It is then sent to the LLM along Fig. 1: An overview of MicroPADâs main components and interactions. with the Pattern Profiles. The LLM returns a filtered list of file paths it believes are likely to contain instances of the pattern, together with a confidence score. This information is used in the next phase to detect pattern instances in the filtered artifacts. 3) Detecting Patterns: The Pattern Detector performs pattern detection in four steps. First, during the Planning Step , the tool devises a plan to de- tect each pattern. Second, during the File Prioritization Step , the tool determines which artifacts to analyze to minimize costs. Files are selected based on the similarity between Pattern Profile example embeddings and artifact embeddings, the presence of keywords in the artifacts, and the LLMâs confidence scores that were calculated during the file tree analysis. Third,thePatternDetectormovestothe Investigation Step . During this step, the tool selects a subset of files with the highest confidence scores for analysis by the LLM. It also sends keywords relevant for detecting the pattern, a repository summary, the detection plan, a pattern description, and an example instance. The LLM analyzes these files to determine which contain evidence of the pattern, offering reasoning, supporting code snippets, and additional relevant information. It also gives a confidence level based on the actual content of the files. Fourth, the Deliberation Step begins. The Pattern Detector reviews summaries of the evidence collected earlier. This includes file paths, confidence scores, and explanations for why patterns may be present. This step is vital as architectural patterns often span multiple files, making individual evidence insufficient. After analysis, the Pattern Detector assigns a final confidence score from 0 to 10 for each identified pat- tern, requiring a score that exceeds a predefined threshold to confirm its presence. B. Experiment Configuration We configured the tool with default parameters to recognize real-world patterns, balancing time and cost. For the experi- ment, we used GPT 5 nano, accessed through OpenAIâs API, known for its reasoning capabilities in coding and multi-step planning [20]. The tool processed the top 20 files with the highest confidence scores and truncated any files over 50,000 characters to comply with OpenAIâs rate limits, while also keeping costs and execution time under control. For artifact prioritization, we assigned a 70% weight to the LLMâs file-tree analysis to understand how the LLM detects patterns. The remaining 30% was initially intended to be split equally between keyword matching and embedding similarity. However, due to the limited examples in the Pattern Profiles, we allocated 20% to keyword matching and 10% to embedding similarity, using multiples of 10 for simplicity. We set the confidence score threshold at 5/10 and limited LLM explanations during the Deliberation Step to 220 char- acters. These and other configuration details are available in the empirical package [18]. C. Repository Selection To evaluate the tool, we assembled a set of repositories containing examples of microservice infrastructure patterns. Using the APIs of both GitHub and GitHub Archive [21], we identified all repositories with activity on September 30, 2025, between 00:00 and 01:00 GMT, and on October 15, 2025, between 00:00 and 01:00 GMT. We required each repository to have at least ten stars, be active for at least six months, weigh between 100 KB and 100 MB, contain at least three artifacts that matched the glob patterns present in Pattern Profiles, have at least five commits in the past three months, and have a minimum of two contributors. D. Questionnaire Survey To establish ground truth and verify whether the chosen repositories contain the patterns we aimed to identify, we contacted the top two contributors in each repository, based on their activity over the last 100 commits, and invited them to participate in a survey. The survey was delivered through our institutionâs LimeSurvey platform. To comply with applicable data protection regulations, we ensured that no contributor was contacted more than once. Although our study focuses on nine microservice infras- tructure patterns, we asked practitioners about the presence of 47 patterns from Richardsonâs catalog. In each pattern identification question section, practitioners had access to a brief description of each pattern. We collected several demographic data points from the participants, such as their country of residence. Additionally, we asked them how many years of experience they had in professional software development, the approximate number of years they had used microservices in software projects, how familiar they were with microservice pattern concepts, their self-assessed competence in developing microservice- based systems, and their understanding of the software system. E. Data Analysis To answer RQ1, the contributions from respondents, along with the results of MicroPAD, were used to calculate confusion metrics and derive additional statistics, specifically precision, recall, accuracy, and the F1 score. To answer RQ2, we examined how MicroPAD interacts with the artifacts present in the repositories. To better under- stand which files are most frequently involved in MicroPADâs detection decisions, we introduced the File Dominance In- dex (FDI). The FDI is a post-hoc metric computed from MicroPADâs processing logs across all 190 repositories and measures how often MicroPAD analyzes a file compared to the average file. We define it as follows: FDI i = c i Îź = c i ¡ N T where c i is the occurrence count of file i, N is the total number of unique filenames analyzed for the pattern, T = P N j=1 c j is the total file occurrences, and Îź = T/N is the average occurrence count per unique filename. V. RESULTS We cloned microservice infrastructure repositories, con- ducted a questionnaire survey, and analyzed the data to answer our research questions. A. RQ1. Pattern Detection Performance The first research question was addressed by comparing the pattern-detection results from our tool with those from repository contributors in the questionnaire survey, which are part of the dataset we built. To the best of our knowledge, no published human-annotated dataset of repositories containing instances of microservice architectural patterns, as defined in Richardsonâs catalog, exists to date. 1) Dataset Participant Characterization: We received 208 responses from 4,082 contacted contributors across 48 coun- tries. Most participants had over 8 years of experience. Furthermore, 78.4% understood microservice patterns, 54.8% could build them, and 85.6% were highly familiar with the repositories they contributed to. 2) Dataset Repository Characterization: The participantsâ 190 annotated repositories form our final dataset. Spanning 35 primary languages (mostly TypeScript, Python, Rust, Go, and C#), the repositories vary widely in size, popularity, and age, as detailed in Table I. MetricMinMaxMeanMedianSD Age (years)0.715.24.83.83.7 Contributors1.08.01.71.01.2 Size (MB)0.195.722.913.825.4 Stars10.078,162.03,132.0352.08,223.0 TABLE I: Characterization of the 190 analyzed repositories. As detailed in Table I (PV column), pattern prevalence, according to the contributors, varied significantly, ranging from 10% to 35%. 3) Pattern Detection: The confusion matrix in Table IV en- ables us to evaluate MicroPADâs ability to recognize microser- vice infrastructure architectural patterns. It details the number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). MicroPADâs detection performance varies significantly by pattern (see Table I), with F1 scores ranging from 0.70 for SERVICE INSTANCE PER CONTAINER to 0.09 for 3RD PARTY REGISTRATION. There is a strong positive correlation (r = 0.74) between the F1 score and pattern prevalence, showing that MicroPAD effectively detects common patterns such as SERVICE INSTANCE PER CONTAINER and SERVICE DEPLOY- MENT PLATFORM, while less prevalent patterns remain more challenging to detect. Overall, MicroPAD achieved 79.5% accuracy, 49.6% precision, 32.8% recall, and an F1 score of 39.5%. We investigate the factors behind this variation in Section VI. B. RQ2. Impact of Repository Artifacts on Detection In RQ2, we further examined how the files analyzed by MicroPAD relate to the overall detection scores for each pattern. Table V shows the top three most frequently analyzed files per pattern, ranked by maximum FDI. Specific patterns have dominant files that are analyzed significantly more often than average, where some files PatternPVPRAF1Max FDI SERVICE INSTANCE PER CONTAINER 0.350.680.710.780.70 54.59 SINGLE SERVICE INSTANCE PER HOST 0.320.540.110.68 0.19 19.61 MULTIPLE SERVICE INSTANCES PER HOST 0.300.440.500.660.47 48.32 SERVICE DEPLOYMENT PLATFORM 0.24 0.400.430.710.42 79.12 SERVICE INSTANCE PER VM 0.170.75 0.09 0.840.16 9.15 SERVICE REGISTRY 0.14 0.29 0.070.840.12 3.67 SERVICE MESH 0.110.400.100.880.15 5.42 3RD PARTY REGISTRATION 0.110.330.05 0.890.09 1.93 SERVER-SIDE SERVICE DISCOVERY 0.100.270.220.870.24 11.56 TABLE I: Various pattern-related metrics, including: prevalence of each pattern in the repositories, according to the repository contributors (PV); the toolâs precision (P), recall (R), accuracy (A), and F1 score (F1), per pattern; and the maximum file dominance index (Max FDI) value, per pattern. Predicted PositivePredicted Negative Actual Positive114 (TP)234 (FN) Actual Negative116 (FP)1,246 (TN) TABLE IV: Confusion matrix describing MicroPADâs perfor- mance. are analyzed 50 to 80 times more frequently. We found a strong positive correlation (r = 0.83) between the F1 scores and the maximum FDI values for each pattern, meaning that patterns with more prominent artifacts are easier to identify. Conversely, patterns such as 3RD PARTY REGISTRATION and SERVICE REGISTRY lack any dominant file, with maximum FDI values below 4, and correspond to the lowest F1 scores. VI. DISCUSSION Our results show that MicroPADâs detection performance with GPT 5 nano varies significantly across patterns. As a tool designed to make pattern detection accessible through natural language descriptions alone, MicroPAD combines engineer- ing decisions (file prioritization, Pattern Profiles, detection threshold) with LLM capabilities (semantic analysis, pattern recognition). Our findings, therefore, reflect the tool as a whole rather than the LLM in isolation. A. Pattern Prevalence Correlates With Detection Success As shown in RQ1, there is a strong correlation between pattern prevalence and detection success. More frequently occurring patterns, such as SERVICE INSTANCE PER CON- TAINER, are identified more effectively, whereas patterns like 3RD PARTY REGISTRATION and SERVICE REGISTRY, which have low prevalence and poor F1 scores, are identified less effectively. One possible explanation is the representativeness of the training data, as common patterns may be better represented in the repositories on which the LLM was trained. Moreover, prevalence alone does not explain performance: MULTIPLE SERVICE INSTANCES PER HOST has similar prevalence to Pattern / FileCountFDI Service deployment platform Makefile12079.12 main.yml4429.01 tsconfig.json4026.37 Service instance per container Dockerfile8954.59 HEAD3219.63 README.md2917.79 Multiple service instances per host Makefile6648.32 release.yaml2719.77 Dockerfile2417.57 Single Service Instance per Host versions.tf3319.61 platformvariables.tf3219.02 project.tf3219.02 Server-side service discovery init.py1411.56 service.yaml1310.73 README.md119.08 Service instance per VM 2024-07-01.xml2059.15 2023-09-01.xml1968.75 2024-10-01-preview.xml1938.61 Service mesh mod.rs75.42 deployment.yaml43.10 ingress.yaml43.10 Service registry mod.rs43.67 plugin.ex32.75 Cargo.toml32.75 3rd Party Registration docker-compose.yaml21.93 externalregistry.cpp21.93 servicediscovery.md10.96 TABLE V: Top three dominant files per pattern (FDI). SERVICE INSTANCE PER CONTAINER but a much lower F1 score, indicating that other factors also affect detection. B. Artifact Distinctiveness Predicts Detection Success The Pearson Correlation Coefficient between the F1 score and FDI indicates a strong positive relationship, revealing that patterns with higher file dominance are detected more successfully. The top-performing patterns are associated with popular technologies like Docker, Terraform, and Kubernetes, which reinforces their strong association in practice. However, high file dominance does not guarantee detection success, as seen with the SERVICE DEPLOYMENT PLATFORM pattern. Despite Makefiles having the highest FDI, this pattern achieves only a modest F1 score. Makefiles are ubiquitous across repositories, regardless of whether a pattern is present, making them a poor discriminator. This indicates that artifact distinctiveness, not just frequency, matters for detection per- formance. Conversely, patterns like SERVICE REGISTRY typically manifest through technology-specific artifacts such as Consul or etcd configurations (see Table V). Yet MicroPAD analyzed generic files like mod.rs and Cargo.toml instead, suggesting that poor file selection also contributes to low detection performance. C. Comparison With Existing Approaches Direct comparison with existing tools (listed in Table I) is difficult, as only three report F1 scores alongside sample sizes. Moreover, these tools typically detect patterns in a single language, focus on MVW patterns with predictable structures, and require substantial effort to extend. MicroPAD supports multiple languages, targets patterns with high implementation variability, and can be extended by adding a new Pattern Profile without retraining or developing new heuristics. The difference in pattern complexity and scope means that F1 scores are not directly comparable across these approaches. D. Practical Implications For practitioners, MicroPAD is most reliable for patterns with well-defined artifacts such as containers and deployment platforms. Results for patterns with low FDI should be verified manually. MicroPAD is available in the replication package and can be used to explore unfamiliar codebases that lack architecture documentation or to support the creation of such documentation for existing systems. For researchers, the Pat- tern Profile mechanism enables extending MicroPAD to other pattern catalogs, and FDI may serve as a predictor of which patterns are amenable to automated detection. VII. THREATS TO VALIDITY We identified several threats to validity and limitations that warrant explicit disclosure, along with the steps we took to minimize them. A. Construct Validity Detection threshold. Different thresholds can yield varying results. We chose 5/10 as the midpoint of the confidence scale to avoid biasing detection toward either over- or under- detection. Preliminary tests with values above 5 confirmed this choice, as they yielded a greater imbalance between precision and recall. A full sensitivity analysis is left for future work. Patterns are subjective constructs. Practitioners and LLMs may interpret the same pattern definition differently. This is an inherent challenge of architectural pattern detection rather than a limitation of our study, but it does introduce measurement variability. The large sample sizes across repos- itories, participants, and patterns help absorb this variability. Dataset Imbalance. Our ground truth dataset is skewed toward true negatives, artificially inflating baseline accuracy. We mitigate this evaluation threat by relying on precision, recall, and F1 scores instead. B. Internal Validity Pattern Profile quality. Pattern Profiles were generated by GPT 5 and manually inspected to verify that glob patterns and examples were reasonable for each pattern. Profile quality may still vary across patterns, as some patterns admit many implementation variations that a limited set of examples cannot fully cover. However, the impact is partially limited, as pattern examples influence only 10% of the final decision through embedding-based file prioritization. Information scoping. MicroPAD limits the information sent to the LLM. This is done by truncating files exceeding 50,000 characters, trimming file trees for large repositories, and analyzing only the top 20 files per pattern. These decisions trade completeness for feasibility, since each LLM call incurs API costs. In practice, most infrastructure-relevant files (Dock- erfiles, YAML) fall well below the truncation limit, and the file prioritization step compensates for tree trimming by scoring files based on content-level keyword matching and embedding similarity. File selection accuracy. We did not separately validate the accuracy of MicroPADâs file selection step. The strong correlation between FDI and detection success (r = 0.83) suggests that file selection performs well for patterns with distinctive artifacts. For patterns without such artifacts, file selection is inherently more challenging regardless of the approach, and improving it is a direction for future work. C. External Validity Limited pattern selection. Resource constraints limited our scope to nine infrastructure patterns from Richardsonâs catalog. These patterns span diverse implementation strategies, from container deployment to service discovery and registra- tion, providing meaningful variation in detection difficulty. Generalizability. The 190 analyzed repositories are open- source projects from GitHub; industrial or private repositories may differ. Additionally, results characterize MicroPAD with GPT 5 nano specifically, and we intentionally scope our claims to this tool rather than to LLMs in general. Whether these findings extend to other LLMs, tools, or agentic approaches remains an open question for future research. We make all data and code available to enable such comparative studies. D. Conclusion Validity LLM non-determinism. Since GPT 5 nano does not allow setting its temperature to zero, we ensured reproducibility by using predefined seeds, logging exact model versions and timestamps, and open-sourcing all code and prompts. VIII. CONCLUSION AND FUTURE WORK In this article, we presented three main contributions. First, an extensible LLM-based tool that automates the detection of microservice architectural patterns, paving the way for broader software pattern detection tools. Second, we provide key insights into the automated detection of microservice infrastructure patterns, informing future LLM research and identifying characteristics that make patterns detectable. Fi- nally, we introduce a novel dataset of repositories containing microservice architectural patterns to assist practitioners in studying microservice architectures. Several directions for future work emerge from these find- ings. Evaluating the approach with additional LLMs would clarify whether the performance patterns we observed are model-specific or reflect broader challenges in architectural pattern detection. Investigating which architectural character- istics make patterns easier or harder to identify could guide both tool development and pattern catalog design. Further improvements to file selection strategies, optimization of the detection threshold through sensitivity analysis, and the exten- sion to additional patterns from Richardsonâs catalog are also promising directions. Finally, exploring whether richer pattern examples, smaller language models, or fine-tuned LLMs can improve detection performance remains an open question. IX. EMPIRICAL PACKAGE The annotated data set and MicroPADâs source code are available for replication and further studies on Zenodo [18] and GitHub [22]. To respect the privacy of all involved users, all data containing personal user information has either been anonymized or withheld. ACKNOWLEDGMENT We sincerely thank all survey participants for their insights. We acknowledge the use of Claude 4.5 Sonnet for code refac- toring and documentation, and Grammarly for proofreading the manuscript. REFERENCES [1] E. Gamma, R. Helm, R. Johnson, and J. M. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software.Addison-Wesley Professional, 11 1994. [2] X. Xia, L. Bao, D. Lo, Z. Xing, A. E. Hassan, and S. Li, âMeasuring Program Comprehension: A Large-Scale Field Study with Professionals,â IEEE Transactions on Software Engineering, vol. 44, no. 10, p. 951â976, 10 2018. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/7997917 [3] N. B. Harrison and P. Avgeriou, âLeveraging Architecture Patterns to Satisfy Quality Attributes,â Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4758 LNCS, p. 263â270, 2007. [Online]. Available: https://link.springer.com/chapter/10.1007/978-3-540- 75132 -8 21 [4] T. C. Lethbridge, J. Singer, and A. Forward, âHow Software Engineers Use Documentation: The State of the Practice,â IEEE Software, vol. 20, no. 6, p. 35â39, 11 2003. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/1241364 [5] T. Haitzer and U. Zdun, âSemi-automatic architectural pattern identification and documentation using architectural primitives,â Journal of Systems and Software, vol. 102, p. 35â57, 4 2015. [Online]. Available: https://w.sciencedirect.com/science/article/pii/S0164121 214002921 [6] âWhat are microservices?â [Online]. Available: https://microservices.io/ [7] A. Daoudi, N. Moha, G. ElBoussaidi, and S. Kpodjedo, âAn exploratory study of MVC-based architectural patterns in android apps,â Proceedings of the ACM Symposium on Applied Computing, vol. Part F147772, p. 1711â1720, 2019. [Online]. Available: /doi/pdf/10.1145/3297280.3297447?download=true [8] U. Zdun, âSystematic pattern selection using pattern language grammars and design space analysis,â Software - Practice and Experience, vol. 37, no. 9, p. 983â1016, 7 2007. [Online]. Available: /doi/pdf/10.1002/spe.799https://onlinelibrary.wiley.com/doi/abs/10.100 2/spe.799https://onlinelibrary.wiley.com/doi/10.1002/spe.799 [9] J. Daniel, E. Guerra, T. Rosa, and A. Goldman, âTowards the Detection of Microservice Patterns Based on Metrics,â Proceedings - 2023 49th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2023, p. 132â139, 2023. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10371525 [10] H. Milhem, âExtraction of Architectural Patterns from Frameworks and Modeling their Contributions to Qualities,â 2019. [Online]. Available: https://dl.acm.org/doi/pdf/10.5555/3492252.3492255 [11] C. Chekhaba, H. Rebatchi, G. Elboussaidi, N. Moha, and S. Kpodjedo, âCoach: Classification-based architectural patterns detection in Android apps,â Proceedings of the ACM Symposium on Applied Computing, vol. 10, p. 1429â1438, 3 2021. [Online]. Available: https: //dl.acm.org/doi/pdf/10.1145/3412841.3442018 [12] S. Komolov, G. Dlamini, S. Megha, and M. Mazzara, âTowards Predicting Architectural Design Patterns: A Machine Learning Approach,â Computers 2022, Vol. 11, Page 151, vol. 11, no. 10, p. 151, 10 2022. [Online]. Available: https://w.mdpi.com/2073-431X/ 11/10/151/htmhttps://w.mdpi.com/2073-431X/11/10/151 [13] Z. R. J Ě anki and V. Bilicki, âRule-Based Architectural Design Pattern Recognition with GPT Models,â Electronics (Switzerland), vol. 12, no. 15, p. 3364, 8 2023. [Online]. Available: https://w.mdpi.com/207 9-9292/12/15/3364/htmhttps://w.mdpi.com/2079-9292/12/15/3364 [14] S. A. Rukmono, L. Ochoa, and M. R. Chaudron, âDeductive Software Architecture Recovery via Chain-of-thought Prompting,â Proceedings - International Conference on Software Engineering, p. 92â96, 5 2024. [Online]. Available: https://dl.acm.org/doi/pdf/10.1145/3639476.36397 76 [15] A. Zakurdaeva, M. Weiss, and S. Muegge, âDetecting architectural integrity violation patterns using machine learning,â Proceedings of the ACM Symposium on Applied Computing, p. 1480â1487, 3 2020. [Online]. Available: /doi/pdf/10.1145/3341105.3374008?download=true [16] M. Mirakhorli, A. Fakhry, A. Grechko, M. Wieloch, and J. Cleland- Huang, âArchie: A tool for detecting, monitoring, and preserving architecturally significant code,â Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering, vol. 16- 21-November-2014, p. 739â742, 11 2014. [Online]. Available: /doi/pdf/10.1145/2635868.2661671?download=true [17] X. Hou, Y. Zhao, Z. Yang, D. Lo, J. Grundy, Y. Liu, K. Wang, L. Li, X. Luo, and H. Wang, âLarge Language Models for Software Engineering: A Systematic Literature Review,â ACM Transactions on Software Engineering and Methodology, vol. 33, no. 8, p. 79, 2024. [Online]. Available: https://doi.org/10.1145/3695988 [18] âMicroPAD Empirical Package.â [Online]. Available: https://zenodo.o rg/records/18638563 [19] C. E. Duarte, âAutomated Microservice Pattern Instance Detection Using Infrastructure-as-Code Artifacts and Large Language Models,â Proceedings - 2025 IEEE 22nd International Conference on Software Architecture, ICSA-C 2025, p. 161â166, 2025. [Online]. Available: https://ieeexplore.ieee.org/document/11014940 [20] âReasoning models - OpenAI API.â [Online]. Available: https: //platform.openai.com/docs/guides/reasoning [21] âGH Archive.â [Online]. Available: https://w.gharchive.org/ [22] âceduarte31/micropad: Empirical package for MicroPAD.â [Online]. Available: https://github.com/ceduarte31/micropad